No issues here. Sounds like a timezone / system clock / clock drift
issue (in a cluster).
On 4/10/2020 11:59 AM, Joe Witt wrote:
The provenance repo is in large scale use by many many users so
fundamentally it does work. There are conditions that apparently need
improving. In the past couple days these items have been flagged by
folks on this list, JIRAs and PRs raised and merged, all good. If you
can help by creating a build of the latest and confirm it fixes your
case then please do so.
Thanks
On Fri, Apr 10, 2020 at 12:48 PM Darren Govoni <[email protected]
<mailto:[email protected]>> wrote:
It would seem the feature is either broken completely or only
works in specific conditions.
Can the Nifi team put a fix on their road map for this?
Its a rather central feature to Nifi.
Sent from my Verizon, Samsung Galaxy smartphone
------------------------------------------------------------------------
*From:* Wyllys Ingersoll <[email protected]
<mailto:[email protected]>>
*Sent:* Friday, April 10, 2020 11:17:42 AM
*To:* [email protected] <mailto:[email protected]>
<[email protected] <mailto:[email protected]>>
*Subject:* Re: Not Seeing Provenance data
I have a similar problem with viewing provenance. I have a 3-node
cluster in a kubernetes environment, the provenance_repository
directory for each node is on a persistent data store so it is not
deleted or lost between container restarts (which are not very
common). My nifi.provenance.repository.max.storage.time is 24 hours.
Whenever I try to view any provenance, nothing is ever shown. If
I manually inspect the provenance_repository directory, there is a
lucene index and TOC being created.
I see log messages like these:
Submitting query +processorId:882133fe-b684-148b-ad88-7850437ca591
with identifier 64a703fe-0171-1000-0000-000065abd91a against index
directories [./provenance_repository/lucene-8-index-1560864819888]
Returning the following list of index locations because they were
finished being written to before 1586531601311: []
Found no events in the Provenance Repository. In order to perform
maintenace of the indices, will assume that the first event time
is now (1586531601311)
Any suggestions?
-Wyllys Ingersoll
On Thu, Apr 9, 2020 at 11:25 AM Dobbernack, Harald (Key-Work)
<[email protected]
<mailto:[email protected]>> wrote:
Hey Mark,
great news and thank you very much!
Happy Holidays!
Harald
*Von:* Mark Payne <[email protected]
<mailto:[email protected]>>
*Gesendet:* Donnerstag, 9. April 2020 17:18
*An:* [email protected] <mailto:[email protected]>
*Betreff:* Re: Not Seeing Provenance data
Thanks Harald,
I have created a Jira [1] for this. There’s currently a PR up
for it as well.
Thanks
-Mark
[1] https://issues.apache.org/jira/browse/NIFI-7346
On Apr 9, 2020, at 11:14 AM, Dobbernack, Harald (Key-Work)
<[email protected]
<mailto:[email protected]>> wrote:
Hi Mark,
I can confirm after testing that if no provenance event
has been generated in a time greater than the
setnifi.provenance.repository.max.storage.time then as
expected the last recorded provenance events don’t exist
anymore but also from then on any new provenance events
are also not searchable, the provenance Search remains
completely empty regardless of how many flows are active.
As described also *.prov file is then missing in
provenance repository. After restart of Nifi new prov File
will be generated and provenance will work again, but only
showing stuff generated since last NiFi Start.
So yes, I’d say your Idea
‘If so, then I think that would understand why it
deleted the data. It’s trying to age off old data
but unfortunately it doesn’t perform a check to first
determine whether or not the “old file”
that it’s about to delete is also the “active file”.’
fits very nicely to my test.
As a workaround we’re going to set a
greaternifi.provenance.repository.max.storage.time until
this can be resolved.
Thanks again for looking into this.
Harald
*Von:*Dobbernack, Harald (Key-Work)
*Gesendet:*Donnerstag, 9. April 2020 15:22
*An:*[email protected] <mailto:[email protected]>
*Betreff:*AW: Not Seeing Provenance data
Hi Mark,
thank you for looking into this.
The nifi.provenance.repository.max.storage.time setting
might explain why I haven’t been experiencing the effect
so often since changing from the default to 120 hours a
few months ago😉
But I believe provenance stopped working last time
although there was an ‘active’ flows in wait Processor,
expiring every hour, going on to ‘send a message’ before
being rerouted to the same wait processor. I would have
expected this generates provenance entries? As I am not
actually 100% sure if that wait processor was in use when
last provenance got lost I will check with a testing
system to see if I can reproduce provenance breakage when
no active flows are around for a time greater
nifi.provenance.repository.max.storage.timeand I will get
back to you.
Thank you!
Harald
*Von:*Mark Payne <[email protected]
<mailto:[email protected]>>
*Gesendet:*Donnerstag, 9.April 2020 14:41
*An:*[email protected] <mailto:[email protected]>
*Betreff:*Re: Not Seeing Provenance data
Hey Daren, Herald,
Thanks for the note. I have seen this once before but
couldn’t figure out what caused it. Restarting addressed
the issue.
I think I may understand the problem, now, though, after
looking at it again.
In nifi.properties, there are a couple of property named
“nifi.provenance.repository.max.storage.time” that
defaults to “24 hours"
Is it possible that you went 24 hours (or whatever value
is set for that property) without generating any
Provenance events?
If so, then I think that would understand why it deleted
the data. It’s trying to age off old data but
unfortunately it doesn’t perform a check to first
determine whether or not the “old file” that it’s about to
delete is also the “active file”.
Can you confirm whether or not you would expect to see 24
hours pass without any provenance data?
Thanks
-Mark
On Apr 9, 2020, at 4:32 AM, Dobbernack, Harald
(Key-Work) <[email protected]
<mailto:[email protected]>> wrote:
What I noticed is that as long as provenance is
working there will be *.prov files in the directory.
When Provenance isn’t working these files are not to
be seen. Maybe some Cleaning Process deletes those
files prematurely or the process building them doesn’t
work any more?
*Von:*Dobbernack, Harald (Key-Work)
<[email protected]
<mailto:[email protected]>>
*Gesendet:*Donnerstag, 9. April 2020 10:27
*An:*[email protected] <mailto:[email protected]>
*Betreff:*AW: Not Seeing Provenance data
This is something I experience too from time to time.
My quick and dirty workaround is stop nifi, delete
everything in the provenance directory, restart….
Then Provenance is usable again (of course only with
data since the delete) . I’m hoping very much there is
a better way, someone can show us better settings or a
potential bug can be discovered…
*Von:*Darren Govoni <[email protected]
<mailto:[email protected]>>
*Gesendet:*Mittwoch, 8. April 2020 20:31
*An:*[email protected] <mailto:[email protected]>
*Betreff:*Not Seeing Provenance data
Hi,
When I go to "View data provenance" in Nifi, I never
see any logs for my flow. Am I missing some
configuration setting somewhere?
thanks,
Darren
*Harald Dobbernack**
*Key-Work Consulting GmbH | Kriegsstr. 100 | 76133 |
Karlsruhe | Germany
|https://www.key-work.de|Datenschutz
<https://www.key-work.de/de/footer/datenschutz.html>
Fon: +49-721-78203-264 |
E-Mail:[email protected]
<mailto:[email protected]>| Fax:
+49-721-78203-10
Key-Work Consulting GmbH, Karlsruhe, HRB 108695, HRG
Mannheim
Geschäftsführer: Andreas Stappert, Tobin Wotring
