Author Topic: DataSourcer components  (Read 2168 times)

Offline JJR

  • Jr. Member
  • **
  • Posts: 56
  • Karma: 0
DataSourcer components
« on: March 18, 2011, 10:42:28 PM »

Can any one help me out on the below.

1.Recently the datasourcer components switchover happen. When i am trying to restart them in the corret mode, they are not starting in the correct mode. Primary datasourcer starting in the back up mode and backup data sourcer starting the primary mode. I stopped both the components, started primary component first and then secondary component. But still the primary components starts in back up mode and secondary component in primary mode. Can some one help me out what could be the reson for this.
2. Data sourcer has not creating the log file though it has log file path set up in the configuration. Help me out here.

Thanks - JJR

Offline Tambo

  • Sr. Member
  • ****
  • Posts: 456
  • Karma: 5
Re: DataSourcer components
« Reply #1 on: March 19, 2011, 06:23:48 PM »

data sourcer takes ages to come back up (as much as 15 - 20 mins) although it shows running in Primary mode in SCI it doesn't mean it is actually up when you see it. So it could be that when you restart them backup is coming up faster than Primary and beating it to be running first. This is what we have found when restarting them.



  • Guest
Re: DataSourcer components
« Reply #2 on: March 19, 2011, 06:53:48 PM »
Ah... Data Sourcers... wonderful enigmas! :)

Tambo is right - Data Sourcers can take ages to "start" - but that depends on the jobs they have to perform.  Their primary task is ETL - and you may find it doesn't have anything to do once it is started, purely because there are no ETL jobs that need to run.

Also, Data Sourcer is Java - it's not the same as the other CCA components.  My advice is that you start only one collection of Data Sourcers - your Primary.  The reason for this is that Data Sourcers in Primary or Back Up perform exactly the same tasks, using the same credentials against the database.  Starting both might seem like the right thing to do but it *will* confuse the database when both try to connect.

Consider that Data Sourcers are reading off raw data chunks that are provided through CCA - and Data Sourcer will "catch up" once it is up and running.  So you don't need to panic if Data Sourcer is not running - it is an ETL tool, not a data provision tool.

In my eyes, the Primary Data Sourcers should be the only ones running - if the Primary Server/Host fails, then you switch over to Back Up...

In the meantime, make sure that your settings are correct for your "emergency files" to be stored on both Primary and Back-Up - Data Sourcer will clear them down once it is up and running again...

Good luck! :)


Offline borkokrz

  • Full Member
  • ***
  • Posts: 154
  • Karma: 6
Re: DataSourcer components
« Reply #3 on: March 22, 2011, 02:34:13 AM »
I'm not sure but I think that primary status is stored in ODS database. That's way i f you start DS in "correct" order, they read database settings on startup and start in opposite mode. I don't have any environment with HA DS under my maintenance, so I can't verify this.

According to documentation:
The operating mode of the Data Sourcer pair in this HA architecture, can be
different from that specified in the Configuration Server. Upon startup, both
primary and backup Data Sourcers compete for control of the ODS database.
Regardless of the initial configuration, the Data Sourcer that acquires controls
of the ODS database, becomes the primary Data Sourcer. The other Data
Sourcer becomes the backup.


  • Guest
Re: DataSourcer components
« Reply #4 on: March 22, 2011, 07:43:02 PM »
thanks borkokrz - that was my point - you could start up both but they "fight" for control.  In my experience, it's better to just run one "set" - and then failover manually if required.