I'll give it a try, but I am nowhere near 1G currently. # du -ksh provenance_repository/* 144K provenance_repository/index-1561998104305 4.0K provenance_repository/toc
My flows are batch jobs that run once a day, they generate a few 1000s events per day at most. Different node with 1.8.0 and default settings running the same flows using PersistentProvenanceRepository does not have this problem. Thank you. From: [email protected] At: 07/12/19 11:58:14To: Mikhail Rolshud (BLOOMBERG/ 120 PARK ) Cc: [email protected] Subject: Re: 1.9.2 Does not show provenance events OK, so the problem is probably these 2 lines: nifi.provenance.repository.max.storage.size=1 GB nifi.provenance.repository.index.shard.size=500 MB You allow up to 1 GB of storage space, which involves both the index (used for searching/lookup) and the event files, which contain the actual events. The index can take up to about 500 MB each. There will be 1-2. Which leaves very little space for event files. These are the default settings and are okay for setting up a small flow on your laptop and playing around with it. But if you have any sort of high volumes at all, you'll want to increase the storage space. The issue is likely that the events are just aging off too quickly for you to see them. Possibly even before they are indexed, if it's running at a pretty high rate. For a "tiny" production or integration system I'd say to set the max.storage.size to 10 GB instead of 1 GB. That should allow you to hold a lot more and will likely address the issue. Thanks -Mark On Jul 12, 2019, at 11:29 AM, Mikhail Rolshud (BLOOMBERG/ 120 PARK) <[email protected]> wrote: # Provenance Repository Properties nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository nifi.provenance.repository.debug.frequency=1_000_000 nifi.provenance.repository.encryption.key.provider.implementation= nifi.provenance.repository.encryption.key.provider.location= nifi.provenance.repository.encryption.key.id= nifi.provenance.repository.encryption.key= # Persistent Provenance Repository Properties nifi.provenance.repository.directory.default=./provenance_repository nifi.provenance.repository.max.storage.time=24 hours nifi.provenance.repository.max.storage.size=1 GB nifi.provenance.repository.rollover.time=30 secs nifi.provenance.repository.rollover.size=100 MB nifi.provenance.repository.query.threads=2 nifi.provenance.repository.index.threads=2 nifi.provenance.repository.compress.on.rollover=true nifi.provenance.repository.always.sync=false # Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are: # EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, Relationship, Details nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship # FlowFile Attributes that should be indexed and made searchable. Some examples to consider are filename, uuid, mime.type nifi.provenance.repository.indexed.attributes= # Large values for the shard size will result in more Java heap usage when searching the Provenance Repository # but should provide better performance nifi.provenance.repository.index.shard.size=500 MB # Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from # the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved. nifi.provenance.repository.max.attribute.length=65536 nifi.provenance.repository.concurrent.merge.threads=2 # Volatile Provenance Respository Properties nifi.provenance.repository.buffer.size=100000 From: [email protected] At: 07/12/19 10:53:13 To: Mikhail Rolshud (BLOOMBERG/ 120 PARK ) , [email protected] Subject: Re: 1.9.2 Does not show provenance events What do your provenance-related settings look like in nifi.properties? Thanks -Mark On Jul 12, 2019, at 10:21 AM, Mikhail Rolshud (BLOOMBERG/ 120 PARK) <[email protected]> wrote: WriteAheadProvenanceRepository might be the culprit here. I have another instance running 1.8.0 and it is using PersistentProvenanceRepository. It's been running for months with same flows and has no issues displaying provenance events. In 1.9.2 PersistentProvenanceRepository is deprecated and by default nifi.properties have nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository From: [email protected] At: 07/12/19 00:42:17 To: [email protected] Subject: Re: 1.9.2 Does not show provenance events I have the same behaviour, with the same setup (container, “external” mount to network drive) on 1.9.2 On Thu, 11 Jul 2019, at 21:08, Mikhail Rolshud (BLOOMBERG/ 120 PARK) wrote: This is a clean install of 1.9.2 And I can see the provenance events for a couple of days if I wipe out provenance_repository and restart the node. After awhile nifi seems to keep writing them but not showing anything in ui. From: [email protected] At: 07/11/19 15:01:21 To: Mikhail Rolshud (BLOOMBERG/ 120 PARK ) , [email protected] Subject: Re: 1.9.2 Does not show provenance events Hello I suspect you have to add the new policy. Please see in the migration guide from old version you had until now. Thanks On Thu, Jul 11, 2019, 2:56 PM Mikhail Rolshud (BLOOMBERG/ 120 PARK) <[email protected]> wrote: Hi, We noticed that after some time 1.9.2 stops showing provenance events. Page just comes blank. I checked the provenance_repository/ folder and I can see files there get modified as events are happening, just can't see anything on UI side. We are running this build: 1.9.2 04/03/2019 15:25:53 UTC Tagged nifi-1.9.2-RC2 From ff01ff6 on branch NIFI-6169-RC2 It runs inside a docker container with provenance_repository/ mapped to external drive mounted to a network storage. # ls -ltr provenance_repository/ total 8 drwxrwxr-x. 2 nifi nifi 4096 Jul 7 12:00 toc drwxrwxr-x. 2 nifi nifi 4096 Jul 11 18:52 index-1561998104305 there are files in provenance_repository/index-1561998104305/ that get updated as events are happening. Thanks
