I'm evaluating Apache Nifi as data ingestion tool to load data from an RDBMS into S3. A first test shows odd behavior where the same rows are written to the flowfile over and over again while i expected that only new rows are written.
In fact i was missing configuration options to specify what column could be used to query only for new rows. Taking a look at the processor implementation makes me believe that the only option is to define a query including OFFSET n LIMIT m where "n" is dynamically set based upon previous onTriggers; would this even be possible? Some setup info: nifi: 0.6.0 backend: postgresql driver: postgresql-9.4.1208.jre6.jar query: select * from addresses More in general i don't see a use-case where the current ExecuteSQL processor fits as a processor (without input flowfile). Someone can explain? Paul
