How do you manage T-Server file log

[kip1]

Hi all. It is not a secret that when you are debugging T-Server to file in time you will face a hardware limitations. In my case the value "expire" is 80 minutes. So I am thinking of creating a job which will take the debug logs txt`s, transfer them to another dedicated server, and then ZIP them - the files will be approximately 40 times smaller (logs for 53 hours ). But if you have another ideas or I`m missing something, please advice. 10x in advance.

The force will be with you......always

[cmcshane]

I didn't think you could set expire option to minute intervals ? If you have '80' as the option value that means you keep 80 log files of the size specified in the 'segment' scetion. If you have '80 days' that means you keep the files for 80 days. Perhaps I'm missing something here ? We keep our t-server log files to 'all' level and keep 100x30Mb files for troubleshooting. It all depends on how busy your t-server is and how much disk space you've allocated for logging.

[kip1]

Hi man, I did not said it right. In my case [b]expire = minutes[/b], because:
1. Segment which has 4 possible options is set to 10000:
[i]false - No segmentation is allowed.
<number> KB (Sets the maximum segment size, in kilobytes. The minimum segment size is 100 KB.) or <number> MB or <number> hr (Sets the number of hours for the segment to stay open.The minimum number is 1 hour.)[/i]

2. Expire which has also 3 options is set to 80:
[i]false No expiration - all generated segments are stored.
<number> file (Sets the maximum number of log files to store. Specify a number from 1–100.) or <number> day (Sets the maximum number of days before log files are deleted. Specify a number from 1–100.[/i]

Of course T-Server is building 1 log above the option you set - in your case you you should have 101 files in your log folder (excluding snapshots and other folders)

The first file in this moment is [b]T-Server_Prim.20080821_170602_621.log[/b] and last is [b]T-Server_Prim.20080821_154556_887.log[/b]
So as you can see in my case T-Server is building 80 log files for nearly 80 minutes - one log file of 10000 kylobytes  in a minute.

So just for me and maybe not only [b]expire = minutes[/b]

And this 80 minutes are not enough. I want to have log for at least 48 hours And that is why I am wondering what kind of [b]Magic[/b] to do

[René]

Hi,

If my calculation is correct you will have around 30 gigs of log after 48 hours... Does it make any sense to have such amount of log? I don't think so as it would be almost impossible to analyse these...

What logging level is enabled - debug? If the answer is yes I would consider about lowering log level. At least for one reason - handling of such big amount of files has impact on TServers performance.

R.

[astrakid]

We keep our logfiles for at least 30 days. and we have 5 tservers and 6 router in place, but also more than 300gb log-filesystem.
for compressing a job is running, that takes files older than 2 hours and compresses them. so we can do analysing problems afterwards. because some problems are not mentioned directly but some hours or days later...
regards,
andre

[CTIgem]

I would come up with dedicated PC with lots of space, nowdays 1TB of SATA is pretty cheap, and map it, tell Tserver to write logs there.

[astrakid]

Just think of network load you are producing...
In our environment each of our 5 TServers writes about 25MB per minute, now you can calculate the bandwidth...

[kip1]

10X to all. The Decision obviously is a Dedicated Server with a job of transferring the log files and then compress them. I like very much what  [b]astrakid[/b] said [i][b]"We keep our logfiles for at least 30 days"[/b][/i] . Man, if you have a little time, please give a hint how exactly you are doing it and should I look over my shoulder for any bugs, that you have experienced.

The force will be with you....always.

[astra kid]

well, we have a job scheduled by crontab. it starts every 10 minutes and looks for logfiles that are configured in a config-file (so we can easily change some details).
in each configured logfilefolder the system looks for <logfile>*.log which are older than a configured time (e.g. 2 hours) and compresses it and moves it to another location (we take the same filesystem, but another subfolder).

thats it. the script we are using grew from time to time and is very complicated, but in general you can set up a simple script as dscribed above.

kind regards,
astrakid

[kip1]

10X Man.

The force will be with you....always.