DMA : Receive Time > Transfer Time ?

[toumtoum]

Hi all,

We have erase all our Reports Layouts in DMA. So, the OL_CHUNK_LOG table in the ODS DataBase was empty.
But now, when we create and active a new Layout, there is a problem with the "Transfer Time" date.

We don't understand why the Transfer Time date is before the Receive Time date ?
Where can we find the default date of the DMA or of the Data Sourcer ?

Many thanks,

toumtoum

[Tony Tillyer]

I had a similar issue which indicated a corrupted Database Table. Are you using SQL?

Tony

[Wyn Owen]

If you've wiped all your layouts I suggest you export the templates, drop the whole ODS database and start again.

[toumtoum]

You're right Pavel Owen !

I was discovered that when I delete all my layouts & templates, and when I restart, my Transfer Time is OK !

But I've got an other little problem : the Delete Time !

The Receive Time and the Transfer Time is now ok, but there is not a Delete Time. That's meanc the datas are never deleted...?

Does anyone know this problem ? Any idea ?

Thanks,

toumtoum

[W]

Do you have the option dropTransferredTables set for ETL Runtime?
If this is not set the ol_data_xx tables will not be dropped after transformation, and there will not be a date in the delete_time field.
The tables will not be dropped until every row in them has been transferred. The number of rows is configured in the DataSourcer.

[Wyn Owen]

The W isn't me, but he/she is right. Even if this setting is set, if the amount of data being saved is low (low numbebr of objects x only a few templates) then there can be a number of days before the data is dropped from the ODS. So patience is a virtue. There is a setting to delete the data more often but my notes aren't handy.....

[toumtoum]

Hi,

First of all, thanks a lot for your answers.

I have verified the dropTransferredTables option in the etl.properties file, it is correctly set.

I have found that in the ETL Runtime User's guide :
"Also, if the dropTransferredTables ETL Runtime parameter is used, ETL Runtime deletes the corresponding tables from ODS after it determines that data transfer successfully completed."

So, is anybody knows how can I find and configure the parameter which indicates the ETL Runtime that data transfer successfully completed.

Thanks++

toumtoum

[Short a.k.a W]

Hi again,

As I wrote earlier, the ol_data table will not be dropped (delete time filled in the ol_chunk_log table) until every row in the current table is used. The option for this is the DataSourcers option: data ablesize

Controls the size of OL_DATAn tables that store Stat Server statistics. Data Sourcer receives data from Stat Server and writes the data into this table until the number of records exceeds the value you specify. Then, Data Sourcer creates a new OL_DATAn+1 table.
Default Value: 1,000,000
Valid Values: Positive integers greater than or equal to 1,000
Change Takes Effect: Upon restart

If you do not collect a lot of data, a single OL_DATA table can be used for some time = no delet_time filled in until the table is full & dropped!

I hope this helps! If it is a test environment, lower the option above, restart DataSourcer, and see for your self!

Good Luck!

[Vic]

You mean the data is not being deleted?
At all?

There are several reasons for that, and I will start with the most overlooked once first:

1. etl. properties and trans_and_agg.properties which one are you using? I used to mix them up
2. Make sure that the DB security grants DELETE privileges to your user
3. check etl_trans.log for DELETE FROM statements are there any errors?

4. Use DMA to look back several days and see if any data is deleted

Most of the times it is usually just delete taking too long to process all your records. Look back three days or more and if your do not have DeletedStamp, then you should start to worry... Otherwise, don't really worry about it