Carlos Due to some stability and performance concerns a new provenance implementation was written called WriteAheadProvenanceRepository. We have not yet switched to it as the default but we've found far better performance. Stability is harder to judge in comparison since the older repository has been used so successfully for so long.
To switch to it simply change this word 'PersistentProvenanceRepository' to 'WriteAheadProvenanceRepository' in your nifi.properties and restart. If you don't see any related JIRAs to what you highlighted though it is best to create a JIRA with the stacktrace you show and also include a complete nifi thread dump with it as well as showing your nifi.properties settings for at least the provenance repository. Thanks On Wed, Jan 17, 2018 at 10:48 AM, Carlos Manuel Fernandes (DSI) <[email protected]> wrote: > Hi, > > > > I’m on Nifi 1.4 and provenance properties by default. Time to times I Get > the Exception below. After this Exception the CPU is always high , the UI > is slow and some processes get Stucked. After nifi restart all be ok. > > > > 1. I think this is related with NIFI-4023 (resolved in 1.4) with the > difference my problem is in merging journals not indexing. > > > > > > 2018-01-17 06:04:36,375 ERROR [Provenance Repository Rollover Thread-2] > o.a.n.p.PersistentProvenanceRepository Failed to merge Journal Files > [./provenance_repository/journals/249818.journal.0, > ./provenance_repository/journals/249818.journal.1, > ./provenance_repository/journals/249818.journal.2, > ./provenance_repository/journals/249818.journal.3, > ./provenance_repository/journals/249818.journal.4, > ./provenance_repository/journals/249818.journal.5, > ./provenance_repository/journals/249818.journal.6, > ./provenance_repository/journals/249818.journal.7, > ./provenance_repository/journals/249818.journal.8, > ./provenance_repository/journals/249818.journal.9, > ./provenance_repository/journals/249818.journal.10, > ./provenance_repository/journals/249818.journal.11, > ./provenance_repository/journals/249818.journal.12, > ./provenance_repository/journals/249818.journal.13, > ./provenance_repository/journals/249818.journal.14, > ./provenance_repository/journals/249818.journal.15] into a Provenance Log > File due to org.apache.lucene.store.LockObtainFailedException: Lock obtain > timed out: > NativeFSLock@/apps/nifi-1.4.0/provenance_repository/index-1508748590000/write.lock > > 2018-01-17 06:04:36,375 ERROR [Provenance Repository Rollover Thread-2] > o.a.n.p.PersistentProvenanceRepository > > org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: > NativeFSLock@/apps/nifi-1.4.0/provenance_repository/index-1508748590000/write.lock > > at org.apache.lucene.store.Lock.obtain(Lock.java:89) > > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755) > > at > org.apache.nifi.provenance.lucene.SimpleIndexManager.createWriter(SimpleIndexManager.java:198) > > at > org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:227) > > at > org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournals(PersistentProvenanceRepository.java:1712) > > at > org.apache.nifi.provenance.PersistentProvenanceRepository$8.run(PersistentProvenanceRepository.java:1300) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > 2018-01-17 06:04:36,375 WARN [Provenance Repository Rollover Thread-2] > o.a.n.p.PersistentProvenanceRepository Couldn't merge journals. Will try > again. journalsToMerge: [./provenance_repository/journals/249818.journal.0, > ./provenance_repository/journals/249818.journal.1, > ./provenance_repository/journals/249818.journal.2, > ./provenance_repository/journals/249818.journal.3, > ./provenance_repository/journals/249818.journal.4, > ./provenance_repository/journals/249818.journal.5, > ./provenance_repository/journals/249818.journal.6, > ./provenance_repository/journals/249818.journal.7, > ./provenance_repository/journals/249818.journal.8, > ./provenance_repository/journals/249818.journal.9, > ./provenance_repository/journals/249818.journal.10, > ./provenance_repository/journals/249818.journal.11, > ./provenance_repository/journals/249818.journal.12, > ./provenance_repository/journals/249818.journal.13, > ./provenance_repository/journals/249818.journal.14, > ./provenance_repository/journals/249818.journal.15], storageDir: > ./provenance_repository > > > > Thanks > > Carlos > >
