[
https://issues.apache.org/jira/browse/NIFI-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206244#comment-17206244
]
Mengze Li edited comment on NIFI-7856 at 10/2/20, 3:06 PM:
---
Hi [~markap14], that's a great question since that's one major reason that we
upgraded to 1.12.0 since sometimes the data provenance is missing in the
processor view but visible in the global view in 1.11.3 that was fixed in
1.12.0.
So to answer your question, we do both and in 1.12.0, the missing and delaying
data provenance records are consistent from either view.
In global view, we usually get component id and search that way since it is
unique.
We are speculating that some processes is compressing the files before the
scheduled time and those ones didn't make it to the lucene index to be searched.
Will try to dig around the logs more to provide information that I can could
help you debug further.
was (Author: leeyoda):
Hi [~markap14], that's a great question since that's one major reason that we
upgraded to 1.12.0 since sometimes the data provenance is missing in the
processor view but visible in the global view.
So to answer your question, we do both and in 1.12.0, the missing and delaying
data provenance records are consistent from either view.
In global view, we usually get component id and search that way since it is
unique.
We are speculating that some processes is compressing the files before the
scheduled time and those ones didn't make it to the lucene index to be searched.
Will try to dig around the logs more to provide information that I can could
help you debug further.
> Provenance failed to be compressed after nifi upgrade to 1.12
> -
>
> Key: NIFI-7856
> URL: https://issues.apache.org/jira/browse/NIFI-7856
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.0
>Reporter: Mengze Li
>Priority: Major
> Attachments: 1683472.prov, ls.png, screenshot-1.png,
> screenshot-2.png, screenshot-3.png
>
>
> We upgraded our nifi cluster from 1.11.3 to 1.12.0.
> The nodes come up and everything looks to be functional. I can see 1.12.0 is
> running.
> Later on, we discovered that the data provenance is missing. From checking
> our logs, we see tons of errors compressing the logs.
> {code}
> 2020-09-28 03:38:35,205 ERROR [Compress Provenance Logs-1-thread-1]
> o.a.n.p.s.EventFileCompressor Failed to compress
> ./provenance_repository/2752821.prov on rollover
> {code}
> This didn't happen in 1.11.3.
> Is this a known issue? We are considering reverting back if there is no
> solution for this since we can't go prod with no/broken data provenance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
