Joe, thanks for your fast answer. I will make the switch and see the results.
Carlos -----Original Message----- From: Joe Witt [mailto:joe.w...@gmail.com] Sent: quarta-feira, 17 de janeiro de 2018 15:58 To: users@nifi.apache.org Subject: Re: PersistentProvenanceRepository Failed to merge Journal Files Carlos Due to some stability and performance concerns a new provenance implementation was written called WriteAheadProvenanceRepository. We have not yet switched to it as the default but we've found far better performance. Stability is harder to judge in comparison since the older repository has been used so successfully for so long. To switch to it simply change this word 'PersistentProvenanceRepository' to 'WriteAheadProvenanceRepository' in your nifi.properties and restart. If you don't see any related JIRAs to what you highlighted though it is best to create a JIRA with the stacktrace you show and also include a complete nifi thread dump with it as well as showing your nifi.properties settings for at least the provenance repository. Thanks On Wed, Jan 17, 2018 at 10:48 AM, Carlos Manuel Fernandes (DSI) <carlos.antonio.fernan...@cgd.pt> wrote: > Hi, > > > > I’m on Nifi 1.4 and provenance properties by default. Time to times I > Get the Exception below. After this Exception the CPU is always high > , the UI is slow and some processes get Stucked. After nifi restart all be > ok. > > > > 1. I think this is related with NIFI-4023 (resolved in 1.4) with the > difference my problem is in merging journals not indexing. > > > > > > 2018-01-17 06:04:36,375 ERROR [Provenance Repository Rollover > Thread-2] o.a.n.p.PersistentProvenanceRepository Failed to merge > Journal Files [./provenance_repository/journals/249818.journal.0, > ./provenance_repository/journals/249818.journal.1, > ./provenance_repository/journals/249818.journal.2, > ./provenance_repository/journals/249818.journal.3, > ./provenance_repository/journals/249818.journal.4, > ./provenance_repository/journals/249818.journal.5, > ./provenance_repository/journals/249818.journal.6, > ./provenance_repository/journals/249818.journal.7, > ./provenance_repository/journals/249818.journal.8, > ./provenance_repository/journals/249818.journal.9, > ./provenance_repository/journals/249818.journal.10, > ./provenance_repository/journals/249818.journal.11, > ./provenance_repository/journals/249818.journal.12, > ./provenance_repository/journals/249818.journal.13, > ./provenance_repository/journals/249818.journal.14, > ./provenance_repository/journals/249818.journal.15] into a Provenance > Log File due to org.apache.lucene.store.LockObtainFailedException: > Lock obtain timed out: > NativeFSLock@/apps/nifi-1.4.0/provenance_repository/index-150874859000 > 0/write.lock > > 2018-01-17 06:04:36,375 ERROR [Provenance Repository Rollover > Thread-2] o.a.n.p.PersistentProvenanceRepository > > org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: > NativeFSLock@/apps/nifi-1.4.0/provenance_repository/index-150874859000 > 0/write.lock > > at org.apache.lucene.store.Lock.obtain(Lock.java:89) > > at > org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755) > > at > org.apache.nifi.provenance.lucene.SimpleIndexManager.createWriter(Simp > leIndexManager.java:198) > > at > org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter > (SimpleIndexManager.java:227) > > at > org.apache.nifi.provenance.PersistentProvenanceRepository.mergeJournal > s(PersistentProvenanceRepository.java:1712) > > at > org.apache.nifi.provenance.PersistentProvenanceRepository$8.run(Persis > tentProvenanceRepository.java:1300) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511 > ) > > at > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.a > ccess$301(ScheduledThreadPoolExecutor.java:180) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.r > un(ScheduledThreadPoolExecutor.java:294) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j > ava:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. > java:617) > > at java.lang.Thread.run(Thread.java:745) > > 2018-01-17 06:04:36,375 WARN [Provenance Repository Rollover Thread-2] > o.a.n.p.PersistentProvenanceRepository Couldn't merge journals. Will > try again. journalsToMerge: > [./provenance_repository/journals/249818.journal.0, > ./provenance_repository/journals/249818.journal.1, > ./provenance_repository/journals/249818.journal.2, > ./provenance_repository/journals/249818.journal.3, > ./provenance_repository/journals/249818.journal.4, > ./provenance_repository/journals/249818.journal.5, > ./provenance_repository/journals/249818.journal.6, > ./provenance_repository/journals/249818.journal.7, > ./provenance_repository/journals/249818.journal.8, > ./provenance_repository/journals/249818.journal.9, > ./provenance_repository/journals/249818.journal.10, > ./provenance_repository/journals/249818.journal.11, > ./provenance_repository/journals/249818.journal.12, > ./provenance_repository/journals/249818.journal.13, > ./provenance_repository/journals/249818.journal.14, > ./provenance_repository/journals/249818.journal.15], storageDir: > ./provenance_repository > > > > Thanks > > Carlos > >
