Hi, We are making OAI-PMH requests invoked within Http.
On Tue, Sep 20, 2016 at 9:31 AM, Pierre Villard <[email protected] > wrote: > Hi Selvam, > > Supposing that your source if a SQL-like source, you should have a look at > QueryDatabaseTable [1] processor. It proposes a 'Maximum-value Columns' > that gives you the possibility to specify the column containing an ID and > or timestamp. The processor will keep track of the maximum value for each > column that has been returned since the processor started running. This can > be used to retrieve only those rows that have been added/updated since the > last retrieval. Note that some JDBC types such as bit/boolean are not > conducive to maintaining maximum value, so columns of these types should > not be listed in this property, and will result in error(s) during > processing. If no columns are provided, all rows from the table will be > considered, which could have a performance impact. > > [1] https://nifi.apache.org/docs/nifi-docs/ > > Pierre > > > 2016-09-20 8:50 GMT+02:00 Selvam Raman <[email protected]>: > >> >> > Hi, >> > >> > We have a requirement to pull data periodically from a data source. For >> this to work we would like nifi to keep track of the last id or time stamp >> that was pulled successfully so that the next pull starts from that point. >> > >> > Let me know if nifi supports this? If yes how do we configure? >> > >> > -- >> > Selvam Raman >> > "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து" >> > > -- Selvam Raman "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
