ETL Aggregation Issues

[izai]

I'm extracting from Brio
ETL_Runtime & Data Sourcer version 7.0.1

[izai]

question:
can 2 report layouts using the same layout template but with different time profile ?

[cavagnaro]

[quote author=izai link=topic=4047.msg18028#msg18028 date=1239350489]
I'm extracting from Brio
ETL_Runtime & Data Sourcer version 7.0.1
[/quote]

I do remember having a similar issue on a installation with that version were values for the group were twice the sum of the agents alone. I did an upgrade to 7.2 and problems got solved...maybe you are having a similar issue.

[tony]

Someone else needs to verify/confirm this but it looks like your [i]Transform[/i] is set to run every minute...? [b]StartTime=0:00+[u]0:01[/u][/b]

Also - yes, your versions are very much out of date - we've recently start moving towards 7.6...  A lot  of things have been fixed in the meantime...

Tony

[quote author=izai link=topic=4047.msg18026#msg18026 date=1239338238]
guys, tq for your reply, really appreciate it.

ecki - to answer your question, rgrding null data, like I said earlier, I dont see any 'null' value, except '0',
did you mean 'null' or '0' ?

Tony - N_inbound in my posted .xls is original N_inbound, it was not a computed item

For your recommendations under item 1, 1a & 1b, below are our current configuration of our ETL. So I guess, the time for app & transferring should not conflict, am I right ?
FYI, we have 4 ETL_runtime running on the same server. As I've chked, all have the same configuration as on below :


[ETL_Agg_Only]
Command=java -Xmx256m -jar transform.jar -conf etl.properties -aggOnly -
Command0=loglevel=SQL-STATS:3,EVENT:3,COMMAND:3,START_AGG:3,AGG:9,INFO:9,WARNING:8,ERROR:9,_ALL_:0 -
Command1=maxLevelOfAgg=AGG_BY_YEAR -
Command2=updateStatsForLevel0Views -updateStatsForTablesAndIndexes
Priority=medium
StartTime=0:11+1:00

[ETL_Purging]
Command=java -jar purge.jar -conf purge.properties
Priority=medium
StartTime=20:25

[ETL_Tracking]
Command=java -jar conf_report.jar -conf etl.properties
Priority=medium
StartTime=00:23+1:00

[ETL_Trans_Only]
Command=java -Xmx256m -jar transform.jar -conf etl.properties -
Command0=loglevel=WAKE:3,START_WRITE_MON:3,DROP:9,WRITE:3,INFO:9,PURGE:9,WARNING:8,ERROR:9,_ALL_:0 -
Command1=maxNumberOfConnToDist=2
Priority=medium
StartTime=0:00+0:01

I'll start looking the other possibilities as suggested in here.
Thanks
[/quote]

[catanirex]

But DataSourcer and ETL are only available in 7.2 versions.

[izai]

guys;
last friday, I'd de-activated an existing template which pointed to 'AGENT' template - time profile was set 15mins
I'd created new report layout using existing 'AGENT' template and set the time profile to 30mins collection.
Reason : I need to refresh the 30 mins data collection since existing one having a problem (.xls posted)

But somehow, I've a problem to retrieve the data either using brio. or sql tool from v_agent_hour, in t_agent_hour the time key is exist and uptodate - it seems hang.
When I check in oracle alert long - we got 'WARNING: aiowait timed out 1 times'
is this I/O problem or sthing related to my report layout - because the data are now mix 15mins & 30mins ?

Pls help.

[tony]

izai,

We began to gather information around the circumstances of your original problem - we've never really got to the bottom of the problem.  Now you have found a new problem whilst attempting to circumvent your original problem, by using a new Layout based on another Time Profile.  The Layout is still using the core components which may actually be the problem.

If it were me, I would not attempt to build anything new (or as a replacement) until I was sure that the components parts (ETL/DS 7.0.1) were not the root cause.  The first advice was for you to upgrade your components to at least version beyond 7.2.x - is it possible for you to do this..?

Tony

[izai]

We will consider about the upgrade very soon.

At this point of time, i need to resolve this internally first.
As per yesterday, I managed to create new report layout using new layout template, and the data is now looks tally between hourly & 30mins.

I think the problem was might due to report layout conflict - multiple layout defined in DMA, but at least, with your help, I got an accurate data for 30mins report, eventually.

Thanks guys.