Ah! That fixed my problem! I am running a secure/authenticated configuration. My user had the correct policy permissions when viewed from the hamburger menu, but not on the access-policies for the individual processors/canvas.
Very confusing! One would think if the admin user had the permissions, they would apply everywhere, but apparently not. Anyway, now I can see provenance data. Thanks for the tip! ________________________________ From: Shawn Weeks <swe...@weeksconsulting.us> Sent: Monday, August 10, 2020 5:19 PM To: users@nifi.apache.org <users@nifi.apache.org> Subject: Re: Data Provenance Stops Working That sounds like a permission issue. Are you running in secure mode? If so right click on the main canvas and go to access policies and make sure your current user is allowed provenance access, you’ll also have to go to the policies section in the system menu in the upper right hand corner and do the same. Thanks Shawn From: Darren Govoni <dar...@ontrenet.com> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> Date: Monday, August 10, 2020 at 3:52 PM To: "users@nifi.apache.org" <users@nifi.apache.org> Subject: Re: Data Provenance Stops Working I also use 1.11.4 and out of the box there IS NO provenance data whatsoever. It just doesn't work if you install and run nifi as is. Sent from my Verizon, Samsung Galaxy smartphone Get Outlook for Android<https://aka.ms/ghei36> ________________________________ From: Shawn Weeks <swe...@weeksconsulting.us> Sent: Monday, August 10, 2020 2:23:19 PM To: users@nifi.apache.org <users@nifi.apache.org> Subject: Re: Data Provenance Stops Working It sounds like if I expand the retention time a lot, say 30 days the issue should be less bad? Thanks Shawn From: Mark Payne <marka...@hotmail.com> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> Date: Monday, August 10, 2020 at 12:37 PM To: "users@nifi.apache.org" <users@nifi.apache.org> Subject: Re: Data Provenance Stops Working Shawn / Wyll, I think you’re probably running into NIFI-7346 [1], which basically says there’s a case in which NiFi may “age off” old data even when it’s still the file that’s being actively written to. In Linux/OSX this results in simply deleting the file, and then anything else written to it disappears into the ether. Of course, now the file never exceeds the max size, since it’ll be 0 bytes forever, os it never rolls over again. So when this happens, no more provenance data gets created until NiFi is restarted. It’s also possible that you’re hitting NIFI-7375 [2]. This Jira only affects you if you get to provenance by right-clicking on a Processor and clicking View Provenance (i.e., not if you go to the Hamburger Menu in the top-right corner and go to Provenance from there and search that way). If this is the problem, once you right-click and go to View Provenance, you can actually click the Search icon (magnifying glass) in that empty Provenance Results panel and click Search and then it will actually bring back the results. So that’s obnoxious but it’s a workaround that may help. The good news is that both of these have been addressed for 1.12.0, which sounds like it should be coming out very soon! Thanks -Mark [1] https://issues.apache.org/jira/browse/NIFI-7346 [2] https://issues.apache.org/jira/browse/NIFI-7375 On Aug 10, 2020, at 1:26 PM, Joe Witt <joe.w...@gmail.com<mailto:joe.w...@gmail.com>> wrote: shawn - i believe it is related to our default settings and have phoned a friend to jump in here when able. but default retention and default sharding i *think* can result in this. You can generate a thread dump before and after the locked state to see what it is stuck/sitting on. That will help here Thanks On Mon, Aug 10, 2020 at 10:24 AM Shawn Weeks <swe...@weeksconsulting.us<mailto:swe...@weeksconsulting.us>> wrote: Out of the box even the initial admin user has to be granted permission I think, mine worked fine for several months since 1.11.1 was released and just started having an issues a couple of weeks ago. I’ve increasing the retention time a bit to see if that improves the situation a bit. Thanks Shawn Weeks From: Wyll Ingersoll <wyllys.ingers...@keepertech.com<mailto:wyllys.ingers...@keepertech.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Monday, August 10, 2020 at 12:22 PM To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: Data Provenance Stops Working I run 1.11.4 in a cluster on AWS also and have a similar issue with the provenance data, I can't ever view it. It's probably somehow misconfigured but I haven't been able to figure it out. ________________________________ From: Andy LoPresto <alopre...@apache.org<mailto:alopre...@apache.org>> Sent: Monday, August 10, 2020 1:11 PM To: users@nifi.apache.org<mailto:users@nifi.apache.org> <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: Data Provenance Stops Working Shawn, I don’t know if this is specifically related, but there were a number of critical issues discovered in the 1.11.x release line that have been fixed in 1.11.4. I would not recommend running any prior version on that release line. 1.12.0 should be coming imminently, so if you are going to upgrade anyway, you may want to wait a week or so and get the newest bits with hundreds of new features, but for stability alone, I would strongly recommend 1.11.4. https://cwiki.apache.org/confluence/display/NIFI/Release+Notes Andy LoPresto alopre...@apache.org<mailto:alopre...@apache.org> alopresto.apa...@gmail.com<mailto:alopresto.apa...@gmail.com> He/Him PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Aug 10, 2020, at 10:00 AM, Shawn Weeks <swe...@weeksconsulting.us<mailto:swe...@weeksconsulting.us>> wrote: I’m running a three node NiFi Cluster on AWS EC2s using integrated Zookeeper and SSL Enabled. Version is 1.11.1, OS is RedHat 7.7, Java is 1.8.0_242. For some reason after a period of time Data Provenance goes blank, old records are no longer queryable and new data provenance doesn’t appear to get written. No Lucene or other exceptions are logged and restarting the node causes data provenance to go back to being written however old data provenance does not re-appear. No exceptions appear when querying data provenance. All tests have been run as the initial admin user and roles and permissions appear to be correct. I’ve also checked selinux audit logs to see if something is being blocked but nothing appears. WriteAheadProvenanceRepository, max storage is set to 24 hours, 1GB, 30 seconds for rollover, 100mb rollover size, 2 query threads, 2 index threads, compress on rollover, and don’t sync always. Thanks Shawn Weeks
