Google
Information Storage and Retrieval: Incremental Data Extraction

Pages

Thursday, January 21, 2010

Incremental Data Extraction

Define a logic to incrementally extract data through an Informatica Mapping. On every session run, the mapping should start loading data from the row where it ended loading in the last run. There is column rowdatetime for every row in the table.

Solution:

Create a user defined mapping variable $$LastUpdateDateTime that saves the timestamp of the last row, the integration service read in the previous session. There is a build-in variable $$SessStartTime for the end time stamp. Use the following filter in source filter of source qualifier to incremetally extract data:

Mytable.rowdatetime > TO_DATE('$$LastUpdateDateTime') and mytable.rowdatetime < TO_DATE('$$SessStartTime')

 

2 comments:

Web Data Extractor said...

Hello,

The incremental technique is a way to solve the issue of added-in data without re-implementing the original algorithm in a dynamic database. Extraction is the operation of extracting data from a source system for further use in a data warehouse environment. Thanks a lot.

James Zicrov said...

Thank you so much for providing Informatica and highlighting its usage and prospects to the extreme level.

Informatica Read JSON