[jira] [Commented] (NIFI-4077) o.a.n.p.index.lucene.LuceneEventIndex Failed to retrieve Provenance Events from store
[ https://issues.apache.org/jira/browse/NIFI-4077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103177#comment-16103177 ] David A. Wynne commented on NIFI-4077: -- I am seeing the same issue 2017-07-26 12:00:10,753 ERROR [Provenance Query-2] o.a.n.provenance.index.lucene.QueryTask Failed to query events against index /usr/hdp/provenance_repository/index-1500466113184 org.apache.nifi.provenance.index.SearchFailedException: Unable to retrieve events from the Provenance Store at org.apache.nifi.provenance.index.lucene.QueryTask.readDocuments(QueryTask.java:198) at org.apache.nifi.provenance.index.lucene.QueryTask.run(QueryTask.java:149) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.FileNotFoundException: Unable to locate file /usr/hdp/provenance_repository/11495395.prov at org.apache.nifi.provenance.serialization.RecordReaders.newRecordReader(RecordReaders.java:119) at org.apache.nifi.provenance.WriteAheadProvenanceRepository.lambda$initialize$1(WriteAheadProvenanceRepository.java:125) at org.apache.nifi.provenance.store.iterator.SelectiveRecordReaderEventIterator.nextEvent(SelectiveRecordReaderEventIterator.java:137) at org.apache.nifi.provenance.store.iterator.AuthorizingEventIterator.nextEvent(AuthorizingEventIterator.java:47) at org.apache.nifi.provenance.store.PartitionedEventStore.getEvents(PartitionedEventStore.java:192) at org.apache.nifi.provenance.store.PartitionedEventStore.getEvents(PartitionedEventStore.java:167) at org.apache.nifi.provenance.index.lucene.QueryTask.readDocuments(QueryTask.java:196) ... 6 common frames omitted > o.a.n.p.index.lucene.LuceneEventIndex Failed to retrieve Provenance Events > from store > - > > Key: NIFI-4077 > URL: https://issues.apache.org/jira/browse/NIFI-4077 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Joseph Gresock > > I get this error when trying to display Data Provenance on a 4-node cluster: > 2017-06-15 18:28:23,945 ERROR [Provenance Query-2] > o.a.n.p.index.lucene.LuceneEventIndex Failed to retrieve Provenance Events > from store > java.io.FileNotFoundException: Unable to locate file > /data/nifi/provenance_repository/85309321.prov > at > org.apache.nifi.provenance.serialization.RecordReaders.newRecordReader(RecordReaders.java:119) > at > org.apache.nifi.provenance.WriteAheadProvenanceRepository.lambda$initialize$46(WriteAheadProvenanceRepository.java:125) > at > org.apache.nifi.provenance.store.iterator.SelectiveRecordReaderEventIterator.nextEvent(SelectiveRecordReaderEventIterator.java:137) > at > org.apache.nifi.provenance.store.iterator.AuthorizingEventIterator.nextEvent(AuthorizingEventIterator.java:47) > at > org.apache.nifi.provenance.store.PartitionedEventStore.getEvents(PartitionedEventStore.java:192) > at > org.apache.nifi.provenance.store.PartitionedEventStore.getEvents(PartitionedEventStore.java:167) > at > org.apache.nifi.provenance.index.lucene.LuceneEventIndex.lambda$submitQuery$58(LuceneEventIndex.java:442) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > I don't know if this is related, but I also see this error in the same logs: > 2017-06-15 18:28:42,923 ERROR [Compress Provenance Logs-1-thread-1] > o.a.n.p.s.EventFileCompressor Failed to read TOC File > /data/nifi/provenance_repository/toc/87247412.toc > java.io.EOFException: null > at > org.apache.nifi.provenance.toc.StandardTocReader.(StandardTocReader.java:48) > at > org.apache.nifi.provenance.serialization.EventFileCompressor.run(EventFileCompressor.java:93) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748)
[jira] [Created] (NIFI-3751) Add support for deletion of counters
David A. Wynne created NIFI-3751: Summary: Add support for deletion of counters Key: NIFI-3751 URL: https://issues.apache.org/jira/browse/NIFI-3751 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: David A. Wynne Priority: Minor Add support for deletion of counters -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3304) In the Administration Guide, there is an incorrect JAVA 8 parameter for bootstrap.conf file
David A. Wynne created NIFI-3304: Summary: In the Administration Guide, there is an incorrect JAVA 8 parameter for bootstrap.conf file Key: NIFI-3304 URL: https://issues.apache.org/jira/browse/NIFI-3304 Project: Apache NiFi Issue Type: Bug Components: Documentation & Website Reporter: David A. Wynne Priority: Minor In the Administration guide, section Bootstrap properties, there is a paragraph about Java 8 handling of codecache, the listed properties are accurate for Java version 7, but for Java version 8, one has changed This is correct for Java version 7: java.arg.8=-XX:CodeCacheFlushingMinimumFreeSpace=10m But for Java version 8, it should be java.arg.8=-XX:CodeCacheMinimumFreeSpace=10m -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2886) Framework doesn't release thread if processor administratively yields
David A. Wynne created NIFI-2886: Summary: Framework doesn't release thread if processor administratively yields Key: NIFI-2886 URL: https://issues.apache.org/jira/browse/NIFI-2886 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: David A. Wynne Fix For: 1.1.0 If a processor yields due to a exception from onScheduled it doesn't immediately release the thread back to the pool. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2780) Have NiFi automatically clean up the persistent provenance data after switching
David A. Wynne created NIFI-2780: Summary: Have NiFi automatically clean up the persistent provenance data after switching Key: NIFI-2780 URL: https://issues.apache.org/jira/browse/NIFI-2780 Project: Apache NiFi Issue Type: Wish Reporter: David A. Wynne Priority: Trivial Current behavior of NiFi requires the user to manually clean up provenance repository data left on disk after switching from persistent to volatile. This ticket is a request to have NiFi automatically clean up the persistent provenance data left on disk after switching. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2745) Add source destination to FlowFile attribute in ConsumeJMS processor
David A. Wynne created NIFI-2745: Summary: Add source destination to FlowFile attribute in ConsumeJMS processor Key: NIFI-2745 URL: https://issues.apache.org/jira/browse/NIFI-2745 Project: Apache NiFi Issue Type: Improvement Reporter: David A. Wynne Priority: Minor The 'destination_name" header does not always correspond to the source destination when consuming messages. This is managed by JMS provider and depends on a lot of factors. So we essentially need additional attribute (i.e., "source.destination") in FlowFile attributes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
