Hi Leon, I want to try to rule out something. Are there any logs we can search for that would help us find when provenance events are cleared out? I want to rule out the CREATE event getting created and getting immediately deleted due to some race condition.
Some more detail is that the initial processor in our flow is a ListSFTP. Leon, could you confirm whether there were any provenance events for that file? Thanks, Eric On Tue, Dec 8, 2020 at 3:32 PM Leon Yu <leon...@macrohealth.com> wrote: > Hello, > > > > This is the first time I’ve emailed here so please bear with me. > > > > We are using nifi-1.12.1-RC2 and for one of our tests, we are seeing > missing data provenance in NiFi despite the DB showing the flow being > processed. > > A simplified explanation of how the system works is NiFi ingests a text > file, creates a flow file from text file, and some data is written to the > DB. > > The tests I was running ran two files through NiFi about 20 minutes > apart. The DB has shown both files having successfully been processed. No > other process writes to the DB other than NiFi. NiFi’s data provenance > only showed data for the 1st file, there was no trace of the 2nd file > despite success flow. > > > > Thank you, > > > > > > [image: A close up of a sign Description automatically generated] > > * Leon Yu* > > SDET, Macrohealth > > > > [image: A close up of a sign Description automatically generated] > <http://www.macrohealth.com/> [image: A picture containing drawing > Description automatically generated] > <https://www.linkedin.com/company/macrohealth/> > > >
