Leon and I did some more digging and found this in both of our logs when we were experiencing the lack of provenance. I think the missing ".prov" file could be why those events are not visible.
We also grepped for the filename and component ID we were using in the provenance query and both searches came up with nothing. When provenance was working again I tried grepping for filenames and container IDs and I was successful. Here are the logs we got when grepping for "grep -i provenance . -r | grep -v "Provenance Query" | grep -v "NiFi Web Server"" ./nifi-app_2020-12-17_19.0.log:2020-12-17 19:06:11,959 INFO [Timer-Driven > Process Thread-7] o.a.n.p.store.WriteAheadStorePartition Successfully > rolled over Event Writer for Provenance Event Store > Partition[directory=./provenance_repository] due to MAX_TIME_REACHED > ./nifi-app_2020-12-17_19.0.log:2020-12-17 19:16:12,497 INFO [Timer-Driven > Process Thread-5] o.a.n.p.store.WriteAheadStorePartition Successfully > rolled over Event Writer for Provenance Event Store > Partition[directory=./provenance_repository] due to MAX_TIME_REACHED > ./nifi-app_2020-12-17_19.0.log:2020-12-17 19:26:12,919 INFO [Timer-Driven > Process Thread-8] o.a.n.p.store.WriteAheadStorePartition Successfully > rolled over Event Writer for Provenance Event Store > Partition[directory=./provenance_repository] due to MAX_TIME_REACHED > ./nifi-app_2020-12-17_19.0.log:2020-12-17 19:26:12,923 ERROR [Compress > Provenance Logs-1-thread-3] o.a.n.p.s.EventFileCompressor Failed to > compress ./provenance_repository/395.prov on rollover > ./nifi-app_2020-12-17_19.0.log:java.io.FileNotFoundException: > ./provenance_repository/395.prov (No such file or directory) > ./nifi-app_2020-12-17_19.0.log: at > org.apache.nifi.provenance.serialization.EventFileCompressor.compress(EventFileCompressor.java:164) > ./nifi-app_2020-12-17_19.0.log: at > org.apache.nifi.provenance.serialization.EventFileCompressor.run(EventFileCompressor.java:115) > ./nifi-app_2020-12-17_19.0.log:2020-12-17 19:26:12,923 WARN [Compress > Provenance Logs-1-thread-3] o.a.n.p.s.EventFileCompressor Failed to delete > ./provenance_repository/395.prov; this file should be cleaned up manually > ./nifi-app_2020-12-17_19.0.log:2020-12-17 19:36:13,781 INFO [Timer-Driven > Process Thread-2] o.a.n.p.store.WriteAheadStorePartition Successfully > rolled over Event Writer for Provenance Event Store > Partition[directory=./provenance_repository] due to MAX_TIME_REACHED > ./nifi-app_2020-12-17_19.0.log:2020-12-17 19:46:13,876 INFO [Timer-Driven > Process Thread-3] o.a.n.p.store.WriteAheadStorePartition Successfully > rolled over Event Writer for Provenance Event Store > Partition[directory=./provenance_repository] due to MAX_TIME_REACHED > ./nifi-app_2020-12-17_19.0.log:2020-12-17 19:46:13,876 ERROR [Compress > Provenance Logs-1-thread-3] o.a.n.p.s.EventFileCompressor Failed to > compress ./provenance_repository/1331.prov on rollover > ./nifi-app_2020-12-17_19.0.log:java.io.FileNotFoundException: > ./provenance_repository/1331.prov (No such file or directory) > ./nifi-app_2020-12-17_19.0.log: at > org.apache.nifi.provenance.serialization.EventFileCompressor.compress(EventFileCompressor.java:164) > ./nifi-app_2020-12-17_19.0.log: at > org.apache.nifi.provenance.serialization.EventFileCompressor.run(EventFileCompressor.java:115) > ./nifi-app_2020-12-17_19.0.log:2020-12-17 19:46:13,876 WARN [Compress > Provenance Logs-1-thread-3] o.a.n.p.s.EventFileCompressor Failed to delete > ./provenance_repository/1331.prov; this file should be cleaned up manually > ./nifi-app_2020-12-17_19.0.log:2020-12-17 19:56:14,656 INFO [Timer-Driven > Process Thread-5] o.a.n.p.store.WriteAheadStorePartition Successfully > rolled over Event Writer for Provenance Event Store > Partition[directory=./provenance_repository] due to MAX_TIME_REACHED > > Thanks, Eric On Thu, Dec 17, 2020 at 11:38 AM Eric Secules <esecu...@gmail.com> wrote: > I am also trying out a simple example. I connected a Generate Flowfile > processor to a LogAttribute processor and even though I could see flowfiles > moving through I didn't see any events for either of the component ids nor > did I see any events in the overall view of all provenance events. > > I did some grepping around the "provenance_repositoy" directory and I was > able to find the component ids and the filenames I was setting in some of > the files there. But nothing showed up in the UI. > > I am not sure of the conditions needed to reproduce this, but I do know it > happens often enough to be a problem. > > Thanks, > Eric > > On Wed, Dec 9, 2020 at 10:41 AM Eric Secules <esecu...@gmail.com> wrote: > >> Hi Leon, >> >> I want to try to rule out something. Are there any logs we can search for >> that would help us find when provenance events are cleared out? I want to >> rule out the CREATE event getting created and getting immediately deleted >> due to some race condition. >> >> Some more detail is that the initial processor in our flow is a ListSFTP. >> >> Leon, could you confirm whether there were any provenance events for that >> file? >> >> Thanks, >> Eric >> >> >> >> On Tue, Dec 8, 2020 at 3:32 PM Leon Yu <leon...@macrohealth.com> wrote: >> >>> Hello, >>> >>> >>> >>> This is the first time I’ve emailed here so please bear with me. >>> >>> >>> >>> We are using nifi-1.12.1-RC2 and for one of our tests, we are seeing >>> missing data provenance in NiFi despite the DB showing the flow being >>> processed. >>> >>> A simplified explanation of how the system works is NiFi ingests a text >>> file, creates a flow file from text file, and some data is written to the >>> DB. >>> >>> The tests I was running ran two files through NiFi about 20 minutes >>> apart. The DB has shown both files having successfully been processed. No >>> other process writes to the DB other than NiFi. NiFi’s data provenance >>> only showed data for the 1st file, there was no trace of the 2nd file >>> despite success flow. >>> >>> >>> >>> Thank you, >>> >>> >>> >>> >>> >>> [image: A close up of a sign Description automatically generated] >>> >>> * Leon Yu* >>> >>> SDET, Macrohealth >>> >>> >>> >>> [image: A close up of a sign Description automatically generated] >>> <http://www.macrohealth.com/> [image: A picture containing drawing >>> Description automatically generated] >>> <https://www.linkedin.com/company/macrohealth/> >>> >>> >>> >>
