Genesys CTI User Forum

Genesys CTI User Forum => Genesys CTI Technical Discussion => Topic started by: samir67000 on April 04, 2017, 01:34:39 PM

Title: ELK suite Genesys log
Post by: samir67000 on April 04, 2017, 01:34:39 PM
Hello Everyone,

My name is samir and I am a trainee in a company in France. I have a project that aims to establish the ELK suite on the different logs produced by Genesys (URS, Tserver ...).
I wanted to know if you could help me because I have some difficulties.
Has anyone ever done this kind of project ? Please Tell me.

Thaaanks !
Title: Re: ELK suite Genesys log
Post by: Dionysis on April 04, 2017, 01:52:01 PM
Not sure there's many people who can help with "some difficulties", Is there a specific problem you're facing?

Title: Re: ELK suite Genesys log
Post by: Kubig on April 04, 2017, 02:30:48 PM
What kind of difficulties? Try to be more specific on that
Title: Re: ELK suite Genesys log
Post by: samir67000 on April 04, 2017, 02:37:34 PM
Yes !

I explain my problem, This is my logstash configuration file :

https://cdn-enterprise.discourse.org/elastic/uploads/default/original/3X/5/5/554329891050f778def16c5fef4037671bb2f53b.png

And this is the content of the log :

https://cdn-enterprise.discourse.org/elastic/uploads/default/original/3X/5/9/595257a618954c0971d94c5904233b472141b464.png

As you have seen I work in mutiline. (Multiple lines in one document).

My goal is to extract only lines containing "#####".
I tried with the following filter : match => {"message" => "(?(?:^|\n).#####.(?:^\n|$))"}
But logstash sends me the whole document while I want only the line.

For example I want him to return :
https://cdn-enterprise.discourse.org/elastic/uploads/default/original/3X/7/f/7fdc3c4161d4cb1e2f3ba158dbfcaa9ad61aa9f8.png

But he return this :
https://cdn-enterprise.discourse.org/elastic/uploads/default/original/3X/a/8/a8622d1a370b8c69a3a6f1cd3e54a33d02270e5b.png

I hope you understand my problem and that you could help me.

Thanks so much !!!!!
Title: Re: ELK suite Genesys log
Post by: Kubig on April 04, 2017, 03:12:15 PM
The used regex seems to be wrong
Title: Re: ELK suite Genesys log
Post by: samir67000 on April 06, 2017, 07:48:46 AM
Hello EveryOne,

I succeeded. Thank you very much.
I have just a last question, in the URS log, The structure is as follows :

[b]10:00:17.307[/b] Int 22000 ##### EI_COF_RCCAuto_v7 - 00f7029cddc0cf9f - RCCAUTO-0614304044- MAJ CFA suite
[b]10:00:17.307[/b]_I_I_00f7029cddc0cf9f [07:48] func will be continued(0,0000000030c000c6)
[b]10:00:17.307[/b]_M_I_00f7029cddc0cf9f [17:11] VQ 000000000478eca0 first available call: none, reason=(0)strategy
[b]10:00:17.307[/b]_I_I_00f7029cddc0cf9f [09:04] <<<<<<<<<<<<suspend interp(JUMPING), func:CallStrategy timers:00001


For each document I would like to extract the time and put it in a timestamp variable.
Can you help me ?
Title: Re: ELK suite Genesys log
Post by: PeteHoyle on April 06, 2017, 09:26:09 AM
Hi,

Can you explain how you fixed your original issue...

For the timestamp would something like this work?

%{TIME:timestamp}

https://www.youtube.com/watch?v=YIKm6WUgFTY

https://grokdebug.herokuapp.com/
Title: Re: ELK suite Genesys log
Post by: samir67000 on April 06, 2017, 10:02:09 AM
Yes I did this,
This sends me the time in a timestamp variable but I want it to send it into the @timestamp logstash. I did this but it does not work :

filter {
grok {
match => { "message" => "^%{TIME:timestamp}"}
  }
date {
    match => [ "timestamp", "HH:mm:ss.SSS"]
target => "@timestamp"}
Title: Re: ELK suite Genesys log
Post by: samir67000 on April 06, 2017, 10:07:10 AM
For my original issue I changed the grok filter  :

match => {"message" => "(?<diese>(?:^|\n).*#####.*(?:\n|$))"}

to  :

match => {"message" => "(?<diese>(?:^|\n)[^\n]*#####[^\n]*(?:\n|$))"}.

:)
Title: Re: ELK suite Genesys log
Post by: PeteHoyle on April 06, 2017, 05:29:09 PM
This works for me..

[code]input
{
file {
    path => "C:\logs\InteractionWorkspace\TEST.txt"
start_position => "beginning"
    ignore_older => 0 
  }
}


filter { 
    grok {
      match => ["message", "^%{TIME:timestamp}"]
    }
date {
    match => [ "timestamp" , "HH:mm:ss.SSS" ]
    target => "@timestamp"
locale => "en"
timezone => "UTC"
}
}


output
{
  stdout { codec => rubydebug }
  file {
                codec => line {
                        format => "%{[time]} | %{[severity]} : /%{[thread]}/ %{[loggername]} %{[timestamp]} (%{[sourcemethodname]}) %{[timestamp]} - %{[message]} PETE: %{[timestamp]}"
                }
                path => "C:\logs\%{host}\PETE-%{+YYYY-MM-dd}.log"
        }
}[/code]

Test.txt:
[code]10:00:18.307 Int 22000 ##### Did you see this one??
10:00:18.307 Int 22000 ##### Did you see this one???[/code]


Console Output:
[code]{
          "path" => "C:\\logs\\InteractionWorkspace\\TEST.txt",
    "@timestamp" => 2017-01-01T10:00:18.307Z,
      "@version" => "1",
          "host" => "xxxxx",
      "message" => "10:00:18.307 Int 22000 ##### Did you see this one??\r",
    "timestamp" => "10:00:18.307"
}
{
          "path" => "C:\\logs\\InteractionWorkspace\\TEST.txt",
    "@timestamp" => 2017-01-01T10:00:18.307Z,
      "@version" => "1",
          "host" => "xxxxxx",
      "message" => "10:00:18.307 Int 22000 ##### Did you see this one???\r",
    "timestamp" => "10:00:18.307"
}[/code]

The date part of the @timestamp defaults to the first day of the year because there is no date in your log file timestamp.
Title: Re: ELK suite Genesys log
Post by: samir67000 on April 07, 2017, 07:53:46 AM
Thank you very much it works !!

Yes, the date is false. The log date is in the header that looks like this :
[img]https://cdn-enterprise.discourse.org/elastic/uploads/default/original/3X/e/e/ee49b2d285121d5420f87a23abef706202df8b88.png[/img].

You think I can get the date highlighted and put it also in the timestamp to have the right date and the right time ?

Title: Re: ELK suite Genesys log
Post by: PeteHoyle on April 07, 2017, 08:35:41 AM
The short answer is I don't know.

If the log spans multiple days then that date will not be valid anyway.

It doesn't look like anyone is using Logstash with their Genesys logs on this forum, it might be best contacting Logstash to see if they can offer any help, but keep us informed if you find out how to do it.
Title: Re: ELK suite Genesys log
Post by: eugene on April 09, 2017, 03:31:20 PM
Peter, if you happen to work for Genesys please let the engineers know that we would love to see native application logging support to the Elastic stack.  I've been using ORS's capabilities for quite some time now and both for development or operational use has proven to be invaluable.
Title: Re: ELK suite Genesys log
Post by: jarrod on April 12, 2017, 11:41:57 AM
So glad I found this post, we are looking at doing this as well.
Can not see Genesys adding this into the product and time soon.
Title: Re: ELK suite Genesys log
Post by: genesysguru on April 12, 2017, 03:34:57 PM
Jarrod - also see http://www.sggu.com/smf/index.php/topic,10025.msg45471.html#msg45471

Craig
Title: Re: ELK suite Genesys log
Post by: jarrod on April 13, 2017, 12:42:37 AM
Thanks Craig. Its amazing what can be achieved by open communities like this.
Looking forwards to playing with this, will post our findings.
Title: Re: ELK suite Genesys log
Post by: genesysguru on June 14, 2017, 11:07:50 AM
A follow up on this:

http://genesysguru.com/blog/blog/2017/06/14/genesys-integration-on-the-wire-iotw/