# Provenance Repository Properties nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository nifi.provenance.repository.debug.frequency=1_000_000 nifi.provenance.repository.encryption.key.provider.implementation= nifi.provenance.repository.encryption.key.provider.location= nifi.provenance.repository.encryption.key.id= nifi.provenance.repository.encryption.key=
# Persistent Provenance Repository Properties nifi.provenance.repository.directory.default=./provenance_repository nifi.provenance.repository.max.storage.time=24 hours nifi.provenance.repository.max.storage.size=1 GB nifi.provenance.repository.rollover.time=30 secs nifi.provenance.repository.rollover.size=100 MB nifi.provenance.repository.query.threads=2 nifi.provenance.repository.index.threads=2 nifi.provenance.repository.compress.on.rollover=true nifi.provenance.repository.always.sync=false # Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are: # EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, Relationship, Details nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship # FlowFile Attributes that should be indexed and made searchable. Some examples to consider are filename, uuid, mime.type nifi.provenance.repository.indexed.attributes= # Large values for the shard size will result in more Java heap usage when searching the Provenance Repository # but should provide better performance nifi.provenance.repository.index.shard.size=500 MB # Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from # the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved. nifi.provenance.repository.max.attribute.length=65536 nifi.provenance.repository.concurrent.merge.threads=2 # Volatile Provenance Respository Properties nifi.provenance.repository.buffer.size=100000 From: [email protected] At: 07/12/19 10:53:13To: Mikhail Rolshud (BLOOMBERG/ 120 PARK ) , [email protected] Subject: Re: 1.9.2 Does not show provenance events What do your provenance-related settings look like in nifi.properties? Thanks -Mark On Jul 12, 2019, at 10:21 AM, Mikhail Rolshud (BLOOMBERG/ 120 PARK) <[email protected]> wrote: WriteAheadProvenanceRepository might be the culprit here. I have another instance running 1.8.0 and it is using PersistentProvenanceRepository. It's been running for months with same flows and has no issues displaying provenance events. In 1.9.2 PersistentProvenanceRepository is deprecated and by default nifi.properties have nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository From: [email protected] At: 07/12/19 00:42:17 To: [email protected] Subject: Re: 1.9.2 Does not show provenance events I have the same behaviour, with the same setup (container, “external” mount to network drive) on 1.9.2 On Thu, 11 Jul 2019, at 21:08, Mikhail Rolshud (BLOOMBERG/ 120 PARK) wrote: This is a clean install of 1.9.2 And I can see the provenance events for a couple of days if I wipe out provenance_repository and restart the node. After awhile nifi seems to keep writing them but not showing anything in ui. From: [email protected] At: 07/11/19 15:01:21 To: Mikhail Rolshud (BLOOMBERG/ 120 PARK ) , [email protected] Subject: Re: 1.9.2 Does not show provenance events Hello I suspect you have to add the new policy. Please see in the migration guide from old version you had until now. Thanks On Thu, Jul 11, 2019, 2:56 PM Mikhail Rolshud (BLOOMBERG/ 120 PARK) <[email protected]> wrote: Hi, We noticed that after some time 1.9.2 stops showing provenance events. Page just comes blank. I checked the provenance_repository/ folder and I can see files there get modified as events are happening, just can't see anything on UI side. We are running this build: 1.9.2 04/03/2019 15:25:53 UTC Tagged nifi-1.9.2-RC2 From ff01ff6 on branch NIFI-6169-RC2 It runs inside a docker container with provenance_repository/ mapped to external drive mounted to a network storage. # ls -ltr provenance_repository/ total 8 drwxrwxr-x. 2 nifi nifi 4096 Jul 7 12:00 toc drwxrwxr-x. 2 nifi nifi 4096 Jul 11 18:52 index-1561998104305 there are files in provenance_repository/index-1561998104305/ that get updated as events are happening. Thanks
