Genesys CTI User Forum
Genesys CTI User Forum => Genesys CTI Technical Discussion => Topic started by: cavagnaro on June 11, 2019, 03:50:27 PM
-
Hi guys,
Any of you have faced this issue at SM Platform?
2019-06-11T10:43:06.812 Std 09900 WARN Create AMERICA_ENGLISH index failed. Internal Server Error
I tried reinstalling the .net core 2.0.6 as per eSR recommendation but no luck.
Can't find further logs about this
-
Did you configure SM logging? If not, configure it to see further details about the error.
-
Seems error is from ElasticSearch complaining about not able to connect to port 8080...
Can't find anything useful at internet about it
Will post logs later
Enviado de meu SM-G9650 usando o Tapatalk
-
SM has nothing to do with elastic search. It seems your problem is on IRWS level. Could you post any further information?
-
? It has. Last version needs ES as a prerequisite. Indexes are now created there.
Enviado de meu SM-G9650 usando o Tapatalk
-
I do not think so you are right - where did you get this information? ES cluster is required by IRWS not by SM at all. Of course, the SM requires IRWS also, but there is no direct reference between SM and ES if I recall well.
-
Well last documentation states this very clear. Pain in the arse as no good details are provided. So, hard to troubleshoot unless you are a ES guy already
Enviado de meu SM-G9650 usando o Tapatalk
-
Can you please point me out to the documentation where this is stated? I am not able to find anything similiar and also I have never heart about ES requirements for SM (except IRWS as I mentioned before)
-
What is even worst, SM requires ES version 5.x while IRWS can work with 1.x only. So you have to install separate instances of ES. In case of multi-tenant we need dedicated SM and ES for each tenant.
-
You need 8.5.511 administration guide for ES part.
-
Ahh, you are working on GIA (not GIR as I supposed). Is your ES cluster up?Are you able to check the HTTP API health?
-
indexer logs:
[quote]
2019-06-13 00:02:33.993 [WRN] Failed to disable Elasticsearch automatic index creation.
2019-06-13 00:02:34.029 [INF] Connected to Elasticsearch.
2019-06-13 00:03:23.042 [WRN] Failed to disable Elasticsearch automatic index creation.
2019-06-13 00:03:23.088 [INF] Connected to Elasticsearch.
2019-06-13 00:05:04.034 [INF] Tenant "default": creating index "default_sm_index_english".
2019-06-13 00:05:25.244 [WRN] Tenant "default": failed to create index.
System.Exception: Failed to create index. Invalid NEST response built from a unsuccessful low level call on PUT: /default_sm_index_english
# Audit trail of this API call:
- [1] AllNodesDead: Took: 00:00:00.0000023
- [2] Resurrection: Node: http://172.17.27.19:8080/ Took: 00:00:00.0000007
- [3] PingFailure: Node: http://172.17.27.19:8080/ Exception: PipelineException Took: 00:00:21.0363196
# OriginalException: Elasticsearch.Net.ElasticsearchClientException: One or more errors occurred. (Failed to ping the specified node.) ---> System.AggregateException: One or more errors occurred. (Failed to ping the specified node.) ---> Elasticsearch.Net.PipelineException: Failed to ping the specified node. ---> Elasticsearch.Net.PipelineException: An error occurred trying to write the request data to the specified node.
at Elasticsearch.Net.RequestPipeline.Ping(Node node) in C:\Users\russ\source\elasticsearch-net-5.x\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 279
--- End of inner exception stack trace ---
at Elasticsearch.Net.RequestPipeline.Ping(Node node) in C:\Users\russ\source\elasticsearch-net-5.x\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 286
at Elasticsearch.Net.Transport`1.Ping(IRequestPipeline pipeline, Node node) in C:\Users\russ\source\elasticsearch-net-5.x\src\Elasticsearch.Net\Transport\Transport.cs:line 188
at Elasticsearch.Net.Transport`1.Request[TReturn](HttpMethod method, String path, PostData`1 data, IRequestParameters requestParameters) in C:\Users\russ\source\elasticsearch-net-5.x\src\Elasticsearch.Net\Transport\Transport.cs:line 69
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
# Audit exception in step 3 PingFailure:
Elasticsearch.Net.PipelineException: An error occurred trying to write the request data to the specified node.
at Elasticsearch.Net.RequestPipeline.Ping(Node node) in C:\Users\russ\source\elasticsearch-net-5.x\src\Elasticsearch.Net\Transport\Pipeline\RequestPipeline.cs:line 279
# Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
# Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
at Indexer.ElasticWrapper.IndexCreator.createIndex(IIndexState indexState, String indexName) in c:\jenkins\workspace\indexer-microservice-build-ver-1.2.0\Indexer\ElasticWrapper\IndexCreator.cs:line 106
at Indexer.ElasticWrapper.IndexCreator.createIndex(String indexName, String analyzerName, Language language, Int32 numberOfShards, Int32 numberOfReplicas, String tenant) in c:\jenkins\workspace\indexer-microservice-build-ver-1.2.0\Indexer\ElasticWrapper\IndexCreator.cs:line 98
at Indexer.ElasticWrapper.IndexCreator.createIndex(Int32 numberOfShards, Int32 numberOfReplicas, String indexName, String tenant) in c:\jenkins\workspace\indexer-microservice-build-ver-1.2.0\Indexer\ElasticWrapper\IndexCreator.cs:line 45
at Indexer.Controllers.IndexController.createIndex(CreateIndexRestObject index) in c:\jenkins\workspace\indexer-microservice-build-ver-1.2.0\Indexer\Controllers\IndexController.cs:line 33
2019-06-13 00:05:46.294 [INF] Tenant "default": creating index "default_sm_index_spanish".
2019-06-13 00:06:07.322 [WRN] Tenant "default": failed to create index.
[/quote]
-
What is GIA? I was talking about SpeechMiner, the component of GIR.
-
After looking into your log, I would recommend to check ES logs and Wireshark trace for communication with ES cluster.
You can check ES status with commands like:
curl 'localhost:9205/_cat/health?v'
curl 'localhost:9205/_cat/nodes?v'
curl 'localhost:9205/_cat/indices?v'
-
What is the port of your ES? In log I see something about 8080. Is it really that one?
-
GIA is for speech analysis. GIR requires ES cluster just for IRWS, not even for SM. I also suppose your ES cluster is running on different port then 8080.
-
You are right for SM versions up to 8.5.510. Index files are used for searching in these versions. Starting from 8.5.511 index is moved to external Elasticsearch server.
From Administration Guide:
"Support for Elasticsearch: The solution no longer uses Lucene to store the index for search and retrieval purposes.
The solution now uses Elasticsearch, which is a more scalable search engine based on a distributed storage that
streamlines the backup process and provides robust functionality around data integrity. As a result, all index data must
be migrated from Lucene to Elasticsearch as part of the upgrade. For details, refer to Install, Configure and Run Elasticsearch"
-
Thanks for clarification. Good news about the ES cluster requirement. I have to update my LAB environment to keep knowledge up-to-date :)
-
[quote author=Janis link=topic=11354.msg51781#msg51781 date=1560409147]
After looking into your log, I would recommend to check ES logs and Wireshark trace for communication with ES cluster.
You can check ES status with commands like:
curl 'localhost:9205/_cat/health?v'
curl 'localhost:9205/_cat/nodes?v'
curl 'localhost:9205/_cat/indices?v'
[/quote]
ES logs show almost nothing actually. According to Genesys is not the core where the indexer runs. Indexer is a web app on IIS that consumes ES only.
I also noticed that 8080 port, but no clue about it, never configured such port during installation phase.
-
You have to install ES and check if it is up and running.
Indexer app in IIS is reading Windows Environment Variable "esNodes" to connect to ES cluster. The value is semicolon separated string of all ES nodes "http://ES_Node_1:<ES_port>;http://ES_Node_2:<ES_port>;http://ES_Node_3:<ES_port>". The default port is 9200.
You can check ES in browser as well. Use links:
http://<any_ES_node>:<ES_port>/_cat/health?v
http://<any_ES_node>:<ES_port>/_cat/nodes?v
http://<any_ES_node>:<ES_port>/_cat/indices?v
-
I only have 1 node
Health:
1560453217 14:13:37 elasticsearch green 1 1 0 0 0 0 0 0 - 100.0%
Nodes:
172.117.179.119 13 92 0 md * DOM-RRCT10
Indices:
<blank> what is supposed to come here?
-
Later when indexer will be able to connect to ES, it will create one.
-
Yeah.. That is the issue right now. Totally blind on what is Indexer doing
Enviado de meu SM-G9650 usando o Tapatalk
-
What url you used to check ES status? What is in the esNodes environment variable?
-
Didn't find that environment variable. Documentation doesn't mention it at all.
The URL was the mentioned at docs, http://server:9200/_cluster/health
And it is green
Enviado de meu SM-G9650 usando o Tapatalk
-
Hmm... try to create it in system variables and restart IIS. Support told me about that variable.
-
What should be the value?
Enviado de meu SM-G9650 usando o Tapatalk
-
“http://server:9200”, if you are using default 9200 port
-
Willing to kiss you!! The environment variable was there but with a wrong value and there was that port 8080! Fuck sake!
Now, I am getting this at Web logs
[quote]
2019-06-13T15:16:55.222 Std 09900 WARN SQL Error: SELECT Id FROM partitionTbl WHERE name IN()
Incorrect syntax near ')'.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
at System.Data.SqlClient.SqlDataReader.get_MetaData()
at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.Fill(DataTable[] dataTables, Int32 startRecord, Int32 maxRecords, IDbCommand command, CommandBehavior behavior)
at System.Data.Common.DbDataAdapter.Fill(DataTable dataTable)
at ca.a[a](String A_0, DbParameter[] A_1, Int32 A_2, cb A_3, ICollection`1 A_4, Boolean A_5)
[/quote]
Any idea?
-
Just for your information, I’m a man :) I have Latvian name pronounced as Yanis.
But OK I can accept it, one kiss will not kill me.
-
I know you are a man hahaha don't care [emoji23][emoji23][emoji23] you just deserve, an Italian kiss [emoji1787][emoji1787][emoji1787][emoji1787]
Enviado de meu SM-G9650 usando o Tapatalk
-
Looks like there must be something in the brackets after IN. Will check what is in that partitions table on proper system tommorow. Maybe you are missing some option in SM config? Default workgroup or how it was called there?
-
Nope, can't find any missing info.
Saw this at Wireshark while doing a search
[quote]}[color=rgb(127, 0, 0)][font=verdana][size=78%]GET /speechminer/api/v1/speakerTypes HTTP/1.1[/size][/font][/color] [color=#7f0000]Host: 172.17.79.19[/color]
[color=#7f0000]Accept: application/json, text/plain, */*[/color]
[color=#7f0000]User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36[/color]
[color=#7f0000]Referer: [url=http://172.17.79.19/speechminer/pages/ui/]http://172.17.79.19/speechminer/pages/ui/[/url][/color]
[color=#7f0000]Accept-Encoding: gzip, deflate[/color]
[color=#7f0000]Accept-Language: es-ES,es;q=0.9[/color]
[color=#7f0000]Cookie: AuthenticationType=Genesys; .ASPXAUTH=13C118D0968CCBA5B7C451722E6D6603629A8E23D9BAFBB18330091D562055DED63DECE7FCA566B56B3C7D98F7CE43713B9BB8903A61A37C7CF2D5EC9CB50FF7AC2CF7D72285FB24B24956DAACF2C87BD8AB832AAFF7C07FF1B43B90B8D358B9CCADEA4A58FE219CA986E34F74A01342; ASP.NET_SessionId=kpr1pf3twgfp32ev4sre3hxg[/color]
[color=#7f0000]X-Forwarded-For: 172.22.10.240[/color]
[color=#7f0000]Cache-Control: max-stale=0[/color]
[color=#7f0000]Connection: Keep-Alive[/color]
[color=#7f0000]X-BlueCoat-Via: 615971f7a9aabed0[/color]
[color=#00007f]HTTP/1.1 200 OK[/color]
[color=#00007f]Cache-Control: no-cache[/color]
[color=#00007f]Pragma: no-cache[/color]
[color=#00007f]Content-Type: application/json; charset=utf-8[/color]
[color=#00007f]Expires: -1[/color]
[color=#00007f]Accept-Ranges: bytes[/color]
[color=#00007f]Server: Microsoft-IIS/8.5[/color]
[color=#00007f]X-AspNet-Version: 4.0.30319[/color]
[color=#00007f]X-Powered-By: ASP.NET[/color]
[color=#00007f]X-Frame-Options: SAMEORIGIN[/color]
[color=#00007f]X-XSS-Protection: 1; mode=block[/color]
[color=#00007f]Date: Thu, 13 Jun 2019 21:17:51 GMT[/color]
[color=#00007f]Content-Length: 187[/color]
[color=#00007f]{[/color]
[color=#00007f] "items": [[/color]
[color=#00007f] {[/color]
[color=#00007f] "id": 1,[/color]
[color=#00007f] "name": "agent"[/color]
[color=#00007f] },[/color]
[color=#00007f] {[/color]
[color=#00007f] "id": 2,[/color]
[color=#00007f] "name": "customer"[/color]
[color=#00007f] }[/color]
[color=#00007f] ],[/color]
[color=#00007f] "href": "api/v1/speakertypes",[/color]
[color=#00007f] "statusCode": 0[/color]
[color=#00007f]}[/color][color=#7f0000]GET /speechminer/api/v1/interactions?searchId=11 HTTP/1.1[/color]
[color=#7f0000]Host: 172.17.79.19[/color]
[color=#7f0000]Accept: application/json, text/plain, */*[/color]
[color=#7f0000]User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36[/color]
[color=#7f0000]Referer: [url=http://172.17.79.19/speechminer/pages/ui/]http://172.17.79.19/speechminer/pages/ui/[/url][/color]
[color=#7f0000]Accept-Encoding: gzip, deflate[/color]
[color=#7f0000]Accept-Language: es-ES,es;q=0.9[/color]
[color=#7f0000]Cookie: AuthenticationType=Genesys; .ASPXAUTH=13C118D0968CCBA5B7C451722E6D6603629A8E23D9BAFBB18330091D562055DED63DECE7FCA566B56B3C7D98F7CE43713B9BB8903A61A37C7CF2D5EC9CB50FF7AC2CF7D72285FB24B24956DAACF2C87BD8AB832AAFF7C07FF1B43B90B8D358B9CCADEA4A58FE219CA986E34F74A01342; ASP.NET_SessionId=kpr1pf3twgfp32ev4sre3hxg[/color]
[color=#7f0000]X-Forwarded-For: 172.22.10.240[/color]
[color=#7f0000]Cache-Control: max-stale=0[/color]
[color=#7f0000]Connection: Keep-Alive[/color]
[color=#7f0000]X-BlueCoat-Via: 615971f7a9aabed0[/color]
[color=#00007f]HTTP/1.1 500 Internal Server Error[/color]
[color=#00007f]Cache-Control: no-cache[/color]
[color=#00007f]Pragma: no-cache[/color]
[color=#00007f]Content-Type: application/json; charset=utf-8[/color]
[color=#00007f]Expires: -1[/color]
[color=#00007f]Accept-Ranges: bytes[/color]
[color=#00007f]Server: Microsoft-IIS/8.5[/color]
[color=#00007f]X-AspNet-Version: 4.0.30319[/color]
[color=#00007f]X-Powered-By: ASP.NET[/color]
[color=#00007f]X-Frame-Options: SAMEORIGIN[/color]
[color=#00007f]X-XSS-Protection: 1; mode=block[/color]
[color=#00007f]Date: Thu, 13 Jun 2019 21:17:51 GMT[/color]
[color=#00007f]Content-Length: 123[/color]
[color=#00007f]{[/color]
[color=#00007f] "statusCode": -1,[/color]
[color=#00007f] "errorMessage": "The operation cannot be completed. Please contact your System Administrator."[/color]
[/quote]
-
In the working system I have following request in the Web log:
[code]Running SQL Command: SELECT Id FROM partitionTbl WHERE name IN (N'Users',N'Outbound Users',N'Email Users',N'/ϰa𢠧rupa/janisk')[/code]
Here we have listed all four Access Groups I assigned to user which is signing into SM. Seems your user does not have any.
-
Yes and no.
Was a permissions issue with the Role.
Genesys guy connected yesterday and fixed.
Thanks for the help!
Enviado de meu SM-G9650 usando o Tapatalk