Ok, When it comes to only managing new inserts, it's something you can easily do with QueryDatabaseTable as long as their is a column that is incrementing with your newly inserted lines like an insert timestamp or a key ID.
When it's about any kind of operation on your source database, then it's really a CDC problem and you have to look at more specific solutions based on your input database. As of now, there is a processor for MySQL [1], and there are pull requests for Microsoft SQL [2] and PostgreSQL. You'd have to look into that direction and check if it matches your requirements. [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-cdc-mysql-nar/1.11.2/org.apache.nifi.cdc.mysql.processors.CaptureChangeMySQL/index.html [2] https://github.com/apache/nifi/pull/2231 [3] https://github.com/apache/nifi/pull/4065 Thanks, Pierre Le ven. 21 févr. 2020 à 13:19, KhajaAsmath Mohammed <[email protected]> a écrit : > Hi , > > I am looking for both solutions actually. > > Thanks, > Asmath > > Sent from my iPhone > > On Feb 21, 2020, at 1:47 PM, Pierre Villard <[email protected]> > wrote: > > > Hi, > > When you mean incremental load, do you mean only newly inserted lines in > the source database? or do you also want to capture deletes and updates? > > Thanks, > Pierre > > Le ven. 21 févr. 2020 à 05:16, KhajaAsmath Mohammed < > [email protected]> a écrit : > >> Hi, >> >> I am looking for solution to do incremental load from database once the >> initial load is done. Incremental should happen automatically with the flow >> . Any suggestions >> >> Does capturechangesql help in this case ? >> >> Thanks, >> Asmath >> >> Sent from my iPhone > >
