Max ors sessions config - System already at maximum number of active sessions

[prb75]

We recently received the following exception - we are using ORS for both voice routing and to serve up some web service responses to inform other system of hours and routing outcomes.

There is a proxy that sits in front of the calls to ORS web services that only makes a new session when there is a timeout - but somehow we hit a ceiling on the maximum number of  sessions as noted below.

Can someone please give some suggestions about ORS session config and how to prevent an error like below ?  Which settings should I be concerned with and how can I measure the config of ORS with the expected number of new sessions being created and the amount of traffic?

We are only getting about 500 concurrent voice calls for now but the web services could have about 100 concurrent requests.

Really I am looking for some metrics on how many ORS servers we need and what the ors session settings should be based on the rate of new sessions.

One setting we had very high was max-session-age => 60000 .  I think this might have been the killer.  We have dropped this to 500, but I wonder if there are other settings that we might want to tweak to avoid hitting a max ?

Any help would be greatly appreciated.


364862 17:39:16.101 Std 23005 ORS SCXML Session Create failed for session 01HUU0HTOKCPV3IIC5LHK2LAES008UB5 reason: System already at maximum number of active sessions
364863 17:39:16.101 Alr 23015 Orchestration session: 01HUU0HTOKCPV3IIC5LHK2LAES008UB5 create failed for reason: System already at maximum number of active sessions

[eugene]

do you have reservation on standing up additional ORSs to support the infra?  the clustering capability is pretty solid and you can scale horizontally with ease.

ORS historically from my experience is pretty gross in terms of CPU consumption namely due to workflows that has a lot of blocks.

i actually separate out voice ors & eservice clusters with the latter servicing also some of the external ws calls.

you sure you have the correct option there? i dont' remember ever having to touch those default options.  seems to default to a pretty high value to begin with so not sure the idea behind bring it down to 500?  wouldn't this cause calls that sit in queue longer than say 8mins to default?

[prb75]

I wasn't sure what the correct setting is for max-session-age - 8 min. is too short, so i will probably set this to something like 6000 for voice.

How do you stand up e-service clusters so that they do not take voice calls ?

[eugene]

double check the deployment guide as i don't want to miss any steps to give you, but at a high level in your ORS list object you woulddefine the cluster section you wish to target for specific flows. 

for example, in my case i create eservice-cluster and voice-cluster then for the voice interaction you simply define on the RP to target the voice-cluster as the ors cluster to use to process your workflow.  for eservice flows you would define it in your interaction submitter script object and specify the eservice-cluster there.

i'll note that if you're using ORS external interface then this clustering will not apply and that you'll need to  A) rely on http load balancer or B) use GMS to front your ORS workflow service instead.  I like the latter since GMS can manage the load algorithm as well as handle ORS failover situation.

[prb75]

Thanks for the suggestion to have a specific cluster for e-services we are going that route.
The problem i am still having however is that when i make concurrent requests using the same token to the same ors server i get a timeout event and it forces me to make a new token to continue interacting with services.

Do you have any web-service specific settings that would help an ors cluster dedicated to web services be more responsive to concurrent requests ?  It seems if I add some pause in my load test or force the web requests to run from a synchronized thread it doesn't time out - so it seems to be some sort of timing issue ?