WfM Corrupt Data Removal?

[tony]

Hello all,

So, we've got WfM 7.2.  We've had it for a while, through various versions and some test data is still in there, somewhere.  Also, there are some entries noteable when building a Forecasts, within the historical data which clearly are corrupt (AHT is null whilst IV is 15, IV is null whilst AHT is 235, both are null, both are zero... etc.)

Aside from clearing the data down prior to a particular date or within a given time frame (which I know can be done with the tools for WfM), has anyone tried the "search and destroy" method of corrupt data removal yet?  I mean defining a set of rules for data removal and actually applying it through SQL...

The advice from Genesys is create Events to exclude corrupt data but that's a very large undertaking when data appears to be missing all over the place...  I want to be able to get rid of it from source, so that it can never be included.  Strange thing is, I have seen some remarks in the guides for WfM which state that low level calculations within the Forecasting algorithm are made based on *all* of the available data (for example, EAE uses a specified time range and then yearly fluctuations are applied, etc.) and, if data is incorrect rather than null or zero, it will have some sort of effect on the resultant Forecast.  But it doesn't tell you how it might affect a Forecast nor how to clean up the data at source...

Has anyone come across this or tried it yet?

Thanks!

Tony

[René]

Hi Tony,

It isn't hard to get rid of corrupted data but it won't help you. Even you delete corrupted intervals from WFM database the system will replace that missing data/intervals with zeros when building the forecast. So your forecast will be affected by that.

If you want to use your historical data for forecasting then it's necessary to repair these. There are several way how to do that:

1/ Replacing wrong data with to correct ones e.g. using the data from a day with similar traffic etc.
2/ Combining historical data and templates when building the forecast
3/ Using Events as recommended by Genesys

Sorry for bad news. I know from my experience it isn't easy to "clean" the WFM data.

René

[tony]

Thanks Rene,

Oh dear! So, even if we could identify the corrupt data and replace it with NULL, WfM still reads it as zero's?  I read somewhere that WfM ignores NULL but does not ignore zero's - is that correct?

Also - an afterthought - how would we apply an Event or Overlay effectively, to enable us to ignore all of the Historical Data...?  I mean, all historical data except for that which you specify in your Forecast.  I have been lead to believe that, even though you may specify 6 weeks for an EAE, the algorthm used in WfM Forecasting also relies on low level frequency filters being applied to all of the available historical volume and I want to make sure we only use the 6 week period specified - and nothing else...

Any pointers would be appreciated.

Thanks!

Tony

[Laura]

Hi there Tony etal

Following a conversation with Rene before his last comment I did some investigation. It is actually misleading the info I gave. WFM will see NULL differently to zero data. however if you could replace the zero data with null you could try:
Export your hist data to CSV. change the data by entering no value on the timesteps. and then importing. This should NULL your data


Regarding the question of EAE and specified data - WFM does not look at any other data other than the dates you have stated.

If you want to use EAE for a large period of time where there is a big gap in the middle for instance then I would use the Event option to 'miss' that data. Thats if you dont want to make the data 'right' by importing or overlapping. Its the 'correct' way.
If its old data you just want to get rid of then try the NULL thing above

Hope that helps

Laura

[Tefleon]

Tony.

Sorry to go back to your first post.  You said that this was to remove test data which was still on the system.  If so the easiest way to fix this IMO would be to get one of the analists to create a .csv file for what the data should have been for that period and then import it through the Import History option.

This will correct the bad data in one action and since the .csv can be big, you could replace several months in one go. 

Let me know if this is something that helped.


Tef

[mark]

That you Mr B?

[tony]

Thanks Laura,

I have a WfM Forecasting White Paper from Genesys which states that, although the User may specify a particular historical time range within the build of an EAE Forecast, this range is applied to an alogrithm which also applies "low level frequency" filters, utilizing any/all available historical data, by default.  I want to ensure that these types of "noise" are not affecting the results of the Forecast, by allowing the User to exclude *everything* except the range they specify.  If what you are stating means that clicking the "do no use historical data" when building the Forecast excludes not only the historical data but also the low level frequency filters used in the Forecasting algorithm, then that is OK with me... but I do need convincing that that would be the end of it, since I am concerend that fliters which the User has no control over may be using out of date historical trends (even at a low level).  Essentially, I want to ensure that there is no historical data outside of the range specified by the User which might affect our Forecasts.

- I hope I explained this OK...?

Tony



[quote author=Laura link=topic=2455.msg9338#msg9338 date=1189548880]
Hi there Tony etal

Following a conversation with Rene before his last comment I did some investigation. It is actually misleading the info I gave. WFM will see NULL differently to zero data. however if you could replace the zero data with null you could try:
Export your hist data to CSV. change the data by entering no value on the timesteps. and then importing. This should NULL your data


Regarding the question of EAE and specified data - WFM does not look at any other data other than the dates you have stated.

If you want to use EAE for a large period of time where there is a big gap in the middle for instance then I would use the Event option to 'miss' that data. Thats if you dont want to make the data 'right' by importing or overlapping. Its the 'correct' way.
If its old data you just want to get rid of then try the NULL thing above

Hope that helps

Laura


[/quote]