Genesys CTI User Forum

Genesys CTI User Forum => Genesys CTI Technical Discussion => Topic started by: PFCCWA on March 25, 2013, 07:34:07 PM

Title: Resource Manager NLB HA behaviour
Post by: PFCCWA on March 25, 2013, 07:34:07 PM
Hello,
We operate a resource manager HA set up, with windows 2003 nlb cluster.
I am trying to establish if the behavior when restarting the primary and backup rm's is correct..

So rm is running in primary mode and active, with member id = 2, and host nlb port 5060 enabled.
rm_backup is running in primary mode (but not active), member id = 1, and host nlb port 5060 is disabled.
Cluster section set up as per gvp 8.1 deployment guide.
Both install path bin folders contain nlb.bat and init.bat which have been configured as per instruction.

If we stop both resource managers, then start rm first then rm_backup - the nlb port for rm host is disabled, and nlb port for rm_backup enabled.  This now means rm is active, but rm_backup nlb port is enabled.  We have issue here where all available ports are blocked after usage in outbound.
Is this correct behaviour after start up?
If we start rm_backup first, then rm, the rm host nlb port is enabled (rm_backup nlb port disabled).  everything works as expected with this.
Similarly if we only start rm_backup, its host nlb port is enabled and everything works correctly.
Our expectation is whatever order we start the resource managers, the rm port is always enabled.
Running on RM version 8.1.5 and outbound voip solution (ms 8.1.5, sip 8.1, ocs 8.1).

thanks.
Title: Re: Resource Manager NLB HA behaviour
Post by: Kubig on March 26, 2013, 07:26:02 AM
I know that the doc says other description, but according to my experience the mymemberid works in inverted mode, ie. primary must have id=1, backup id=2.
Title: Re: Resource Manager NLB HA behaviour
Post by: bublepaw on March 26, 2013, 09:18:41 AM
I agree with Kubig, I always set id to 1 on primary. In addition I would recommend looking and new feature in SIPS 8.1.1 which provides active/active support for RM without any scripts or NLB stuff. One comment for now it works best on UDP
Title: Re: Resource Manager NLB HA behaviour
Post by: PFCCWA on March 26, 2013, 09:57:59 AM
is it still possible to use Windows NLB with an active-active RM set up? or even recommended?
Im assuming the virtual ip remains open on both hosts nlb cluster in this instance..
Ive had a look at the gvp deployment guide which seems to follow 3 steps, the third being an external load balancer set up.

thanks.
Title: Re: Resource Manager NLB HA behaviour
Post by: bublepaw on March 26, 2013, 02:47:59 PM
In active/active setup NLB is no longer need (actually it is not supported) on RM side. SIPS "pings" each local ip address of RM and based on result of the ping decides how to route calls. In addition when both ip addreses are available SIPS distributes requests in round-robin fashion. This configuration is described in more details in SIPS dep guide 8.1.1
Title: Re: Resource Manager NLB HA behaviour
Post by: PFCCWA on March 28, 2013, 04:42:12 PM
I have found the issue i raised on this post is caused by an incorrect NLB.bat script.
If the virtual IP is 177.00.000.3
resource manager is member id=2
resource manager backup is member id=1
host priority of resource manager host in nlb=1
host priority of resource manager backup host in nlb=2
Assuming nlb password is the default.
sip port of rm=5060

what should the script be?

REM Update the value below with the virtual IP value for any Windows platform
set virtual_ip=138.120.84.204


REM If running on Windows 2K8, update the below values with the second IP address (non-virtual-ip) of the interface associated with the NLB interface.
REM Otherwise, if running on windows 2003, leave the below as-is (i.e., 1 and 2)
set private_ip_member1=1
set private_ip_member2=2

REM If running on Windows 2003, update the below value with the wlbs password configured
REM Otherwise if running on Windows 2008, leave the below field blank (i.e., set wlbs_password=)
set wlbs_password=/PASSW root1

REM Set this to node ID defined by cluster.mymemberid in RM MF options - eg. 1 for NLB host ID 1, and 2 for NLB host ID 2
set mymemberid=1

REM If running on 64-bit OS, then the wlbs needs to be reference directly from sysnative directory
IF EXIST %SystemRoot%\SysWOW64 set NLBPATH=%SystemRoot%\sysnative\

%NLBPATH%wlbs start

if %1 equ enable (

                if %2 equ 1 (

                               
                                %NLBPATH%wlbs enable 9999 %virtual_ip%:%private_ip_member1% %wlbs_password%
                %NLBPATH%wlbs disable 9999 %virtual_ip%:%private_ip_member2% %wlbs_password%
                                %NLBPATH%wlbs enable 9999 %virtual_ip%:%private_ip_member1% %wlbs_password%

                )

                if %2 equ 2 (

                                %NLBPATH%wlbs enable 9999 %virtual_ip%:%private_ip_member2% %wlbs_password%
                %NLBPATH%wlbs disable 9999 %virtual_ip%:%private_ip_member1% %wlbs_password%
                                %NLBPATH%wlbs enable 9999 %virtual_ip%:%private_ip_member2% %wlbs_password%

                                if ERRORLEVEL 1 exit /B 0
                                exit %ERRORLEVEL%
                )
) else (

REM ER260413563 - Disable only if it's this node.  If disabling other node that cannot be reached, it may get delayed for very long time.
REM              When NLB.bat disable <other node> is issued, it is accompanied by NLB.bat enable <this node>, and when enabling this node it will disable the other node
                if %2 equ %mymemberid% (
                                if %2 equ 1 (
               
                                                %NLBPATH%wlbs %1 9999 %virtual_ip%:%private_ip_member1% %wlbs_password%
                                )
                                if %2 equ 2 (
                               
                                                %NLBPATH%wlbs %1 9999 %virtual_ip%:%private_ip_member2% %wlbs_password%
                                )
                )
)

if ERRORLEVEL 1 exit /B 0
exit %ERRORLEVEL%

I understand what needs updated for most but am confused on the arguments under the if statements..

thanks.
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on June 26, 2013, 06:12:00 AM
[quote author=bublepaw link=topic=7736.msg33488#msg33488 date=1364309279]
In active/active setup NLB is no longer need (actually it is not supported) on RM side. SIPS "pings" each local ip address of RM and based on result of the ping decides how to route calls. In addition when both ip addreses are available SIPS distributes requests in round-robin fashion. This configuration is described in more details in SIPS dep guide 8.1.1
[/quote]
Hi,bublepaw
was surprised to read this. Are you sure that  active/active RM setup does't need external load balancer? I'm re-read all available latest version of deployment guide for SIP and GVP but don't found any way to use active/active w/o ext LB. Can you point on that exactly doc you mean as  "SIPS dep guide 8.1.1" ?
WBR Timur
Title: Re: Resource Manager NLB HA behaviour
Post by: Kubig on June 26, 2013, 07:25:16 AM
You can build HA on RM via load-balancing on SIP server. It is described in last deployment guide and HA dep guide for SIP.
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on June 26, 2013, 11:11:46 AM
[quote author=Kubig link=topic=7736.msg34342#msg34342 date=1372231516]
You can build HA on RM via load-balancing on SIP server. It is described in last deployment guide and HA dep guide for SIP.
[/quote]
Kubig,
i'll ask again - Can you point on that exactly doc you mean as  "last deployment guide and HA dep guide for SIP" ?
I had just checking the:
"Framework 8.1 SIP Server High-Availability Deployment Guide" Document Version: 81fr_dep-sip_ha_10-2011_v8.1.001.04
"Framework 8.1 SIP Server Deployment Guide" Document Version: 81fr_dep-sip_05-2013_v8.1.101.05
"Genesys Voice Platform 8.1 Deployment Guide" Document Version: 81gvp_dep_12-2012_v8.1.601.00
"Voice Platform Solution 8.0 Integration Guide" Document Version: 80gvp_ig-vps_11-2008_v8.0.001.06

All of those doc's did't contain any active-active scheme for RM w/o ext. LB. 
Title: Re: Resource Manager NLB HA behaviour
Post by: bublepaw on June 26, 2013, 11:15:23 AM
Search for Active-Active Resource Managers in SIPS dep guide. If You decide to go this way please use latest SIPS. There are two ways to configure it - using ip addresses or using DNS name. DNS name is preferred option ( less configuration on RM/MCP side).

http://genesyslab.info/wiki/index.php/Special:Repository/81fr_dep-sip.pdf?id=2e30d00a-05d6-4c84-a539-eb7ddcbde5f4 page 322
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on June 27, 2013, 07:38:05 AM
[quote author=bublepaw link=topic=7736.msg34347#msg34347 date=1372245323]
Search for Active-Active Resource Managers in SIPS dep guide. If You decide to go this way please use latest SIPS. There are two ways to configure it - using ip addresses or using DNS name. DNS name is preferred option ( less configuration on RM/MCP side).

http://genesyslab.info/wiki/index.php/Special:Repository/81fr_dep-sip.pdf?id=2e30d00a-05d6-4c84-a539-eb7ddcbde5f4 page 322
[/quote]
Hi,bublepaw
fanx for detailed answer.
WBR Tim
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on July 01, 2013, 07:59:01 AM
Hi All,
another question about Active-Active scheme for RM. We all know how unstable current version of RM and MCP. So, I'll try to reach we maximum availability and utilization of GVP services. Well, the next question which i now have - can we configure two MCP to work with AA RM pair as one RG r we should configure one MCP for one RM and second one for second RM? The problem in that , how i can see:
1) then we create RG  - we must choice the primary RM. So if we have two primary RM  - we must create independent RG for each RM ?
2) then we configure MCP for work with RM - we must use RM ip-address in in several parameters(for example, routeset in sip section). So if we have RM without the shared IP - we must use independent MCP with independent settings for each RM?

On the other hand...it is just a change of paradigm - with Active-StandBy RM we use RG for sharing and load-balance the calls traffic between MCP pair. Now we use the SIP for sharing and load-balace the calls traffic between RM-MCP pair, right?

WBR Tim
Title: Re: Resource Manager NLB HA behaviour
Post by: Kubig on July 01, 2013, 08:20:45 AM
You are not right, the working mechanism with LRGs are still same. You have not to change routeset in sip section,you have to only change configuration of TG,Trunk and VoIP objects (from RM IP address to "::msml" - it is described in dep guide very well).
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on July 01, 2013, 08:51:44 AM
Kubig,
look at gvp dep guide. in sections "Integrating Application Objects with Resource Manager", we have mention about routeset option in sip and vrmrecorder sections. We need to set this options for call recording solution works. and this options use the RM IP. it's a little confused for me now , coz i still can't build complete test zone.
SIP Dep guide did't cover this options. 
WBR Timur
Title: Re: Resource Manager NLB HA behaviour
Post by: Kubig on July 01, 2013, 08:54:48 AM
I was confused too as you, but after test and deployment to production system I can say, it works. Genesys can change these options dynamically and internally. From my experience is this mode more stable then active-standby with bonding.
Title: Re: Resource Manager NLB HA behaviour
Post by: bublepaw on July 01, 2013, 02:55:18 PM
routeset option in sip and vrmrecorder section should contain FQDN of active-active cluster - in such case switchover between two RM works without any further configuration. For active-active RM based only on ip addresses You have to configure transport.staticroutelist option which should contain both RM ip separated by comma.
Title: Re: Resource Manager NLB HA behaviour
Post by: Kubig on July 01, 2013, 04:54:23 PM
So, for active-active RM cluster based on IP address list only is necessary to set transport.staticroutelist to comma separated IP addresses of each RM in cluster and in routeset (sip,vrmrecorder) one of these IP's or can be empty? I have always deployed and configured with both options. And all worked fine (announcement,vxml and rec services).

Timur has right that the function is not described very well in actually doc and some parts is missing.
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on July 01, 2013, 05:50:11 PM
[quote author=bublepaw link=topic=7736.msg34379#msg34379 date=1372690518]
routeset option in sip and vrmrecorder section should contain FQDN of active-active cluster
[/quote]
WOW?! Wait a second! That's a "FQDN of active-active cluster" ?! In active-active cluster we didnt' have any shared IP or FQDN, each RM have independent IP and FQDN name!
Title: Re: Resource Manager NLB HA behaviour
Post by: bublepaw on July 01, 2013, 05:55:11 PM
That is why I mentioned that there are 2 ways of deploying active-active cluster:

- each RM has it's own ip address and in all configurations we enter both ip addresses where appropriate (contact-list in SIPS, routset and staticroutelist)
- each RM has it's own ip address but we reference RM pair using FQDN which resolves to both ip addreses. We use it in routeset and contact options

Another difference in using DNS is fact that we can use different port for each instance while with ip addresses we need to use same port for both instances of RM.
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on July 01, 2013, 08:52:08 PM
Hmm,
you mean use only IP for that options wich can use the more than once address, but use the FQDN for options which can contain only one address?
Interesting trick. But i see a little problem. In all service wich work with DN we have the OOS detection. And standart behavior for SIP - mark DN as out-of service and don't use it while oos timer is not counted. Did you already test such config ? And for me the most impotant question - did you ever try it with third-party recorer, Zoom for example?
Title: Re: Resource Manager NLB HA behaviour
Post by: Kubig on July 02, 2013, 03:46:04 AM
Yes, that is what I was thinking. I have real experience with using it (a-a cluster based on IP only) with ZOOM recorder and it works fine. I mean better than active-standby mode. In active-standby mode sometimes the "switchover" takes a long time before the backup node became to primary node..
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on July 02, 2013, 04:44:11 PM
Fanx, Kubig! You words is make the hope grew in my soul =)
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on July 05, 2013, 09:15:02 AM
Looks like it's work =)) Fanx again to all in this topic!
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on July 09, 2013, 01:06:23 PM
>:D
ROFL =)))) Look at this answer from official  Genesys Support:
"From provided configuration I can see that you are running RMs in active-active mode.
Please note method required external load balancer which owns a virtual IP address that is used to forward requests to the active cluster"
Title: Re: Resource Manager NLB HA behaviour
Post by: bublepaw on July 09, 2013, 07:52:11 PM
Ignore it and point them to manual :>. What problem You had?
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on July 10, 2013, 05:38:52 AM
Sip server is not evenly distributes the call on RM.  And the main problem - RM have the "stalled" call which leads to the port leak
Title: Re: Resource Manager NLB HA behaviour
Post by: bublepaw on July 10, 2013, 07:20:36 AM
Try setting option  sip-enable-rfc3263 to true to solve distribution issue. As mentioned previously feature was designed to work with FQDN's and it works much better in such setup
Title: Re: Resource Manager NLB HA behaviour
Post by: Kubig on July 10, 2013, 08:05:50 AM
Additional question : It is necessary or recommended to add RM in active-active cluster to HotStandby mode on application object under CME or GA?
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on July 10, 2013, 09:37:28 AM
[quote author=bublepaw link=topic=7736.msg34476#msg34476 date=1373440836]
Try setting option  sip-enable-rfc3263 to true to solve distribution issue. As mentioned previously feature was designed to work with FQDN's and it works much better in such setup
[/quote]
already enabled
Title: Re: Resource Manager NLB HA behaviour
Post by: bublepaw on July 10, 2013, 12:10:01 PM
[quote author=Kubig link=topic=7736.msg34477#msg34477 date=1373443550]
Additional question : It is necessary or recommended to add RM in active-active cluster to HotStandby mode on application object under CME or GA?
[/quote]

No - RM pair should be configured as two separate applications without any HA settings
Title: Re: Resource Manager NLB HA behaviour
Post by: Kubig on July 30, 2013, 08:47:06 AM
Hi guys,

I have registered an strange issue with a-a RM cluster based on IP addresses only - when I configured routeset on MCPs to use both IP addresses of RM, sometimes happens that MCP sends requests to offline RM instead to alive RM :-( If I deleted both IP from routeset, all is working fine.

Have anyone registered similiar behavior?

thx in advance
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on July 31, 2013, 07:59:53 PM
FYI
GVP 8.1.7 released
WBR Timur
Title: Re: Resource Manager NLB HA behaviour
Post by: Kubig on August 01, 2013, 08:33:10 AM
Yes, I know - already installed on my LAB. I am starting with testing, so I will let you know ;-)
Title: Re: Resource Manager NLB HA behaviour
Post by: Timur Karimov on August 02, 2013, 11:35:14 AM
Look like it more, much more stable and error free. For example, for last day i have not a hundreds of stuck call in system.
WBR Timur