As an interesting architectural approach we took eons ago, before NiFi, was
to take daily snapshots of a full table. Every row would then be
hashed/digested or in any other way uniquely identified and 2 datasets
would be crossed and compared to find inserts/deletes/updates. It was
involved, but worked.

Andrew

On Sat, Sep 16, 2017, 2:38 AM Uwe Geercken <uwe.geerc...@web.de> wrote:

> Bryan,
>
> yes, the change log would be possible. In my use case I have Oracle 11 as
> the source - and I can not change the source easily (takes long - is
> expensive).
>
> I was expecting this answer but wanted to make sure that I have not missed
> anything. I will try to build my use case around something else then.
>
> Thanks for your response(s).
>
> Rgds,
>
> Uwe
>
> *Gesendet:* Freitag, 15. September 2017 um 16:15 Uhr
> *Von:* "Bryan Bende" <bbe...@gmail.com>
> *An:* users@nifi.apache.org
> *Betreff:* Re: QueryDatabaseTable - Deleted Records
> Uwe,
>
> Typically you need to process the change log of the database in this
> case, which unfortunately usually becomes database specific.
>
> I believe we have a processor CaptureChangeMySQL that can process the
> MySQL change log.
>
> -Bryan
>
>
> On Tue, Sep 12, 2017 at 1:39 PM, Uwe Geercken <uwe.geerc...@web.de> wrote:
> > Hello,
> >
> > apparently the QueryDatabaseTable processor catches changes made to the
> data
> > of the source database - updates and inserts.
> >
> > Has anybody a good idea or strategy how to handle deletes in the source
> > database? Of course one could flag a record as deleted instead of
> phisically
> > deleting it. But this means changing the source system in many cases and
> > that is sometimes not possible. And yes, if you process the change log
> (if
> > available) of the source system that is also a good option.
> >
> > Would be greatful for any tips or a best practive of how you do it.
> >
> > Rgds,
> >
> > Uwe
>

Reply via email to