Michael W Moser created NIFI-12468:
--------------------------------------
Summary: Xodus Flow Configuration History does not recover from
exception
Key: NIFI-12468
URL: https://issues.apache.org/jira/browse/NIFI-12468
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 2.0.0-M1
Reporter: Michael W Moser
I caused Xodus to timeout writing a transation to flow configuration history.
Thereafter, every operation that would update flow configuration history, or
trying to view flow configuration history, continues to throw an exception.
The UI shows "An unexpected error has occurred. Please check the logs for
additional details" based on a 500 response from the REST API
/nifi-api/flow/history
I had to restart NiFi to recover. Hopefully, this code can be improved to
detect exceptions and recover on its own.
I caused the initial problem by setting a breakpoint while a debugger was
attached to NiFi. I dragged a new Process Group onto the graph from a flow
definition file and let it sit at the breakpoint for a while.
{noformat}
2023-12-04 16:32:00,448 ERROR [NiFi Web Server-466]
jetbrains.exodus.env.EnvironmentImpl Failed to flush transaction
java.io.IOException: Stream Closed
at java.base/java.io.RandomAccessFile.length0(Native Method)
at java.base/java.io.RandomAccessFile.length(RandomAccessFile.java:653)
at jetbrains.exodus.io.FileDataWriter.ensureFile(FileDataWriter.kt:172)
at jetbrains.exodus.io.FileDataWriter.write(FileDataWriter.kt:59)
at
jetbrains.exodus.log.BufferedDataWriter.writePage(BufferedDataWriter.java:264)
at
jetbrains.exodus.log.BufferedDataWriter.flush(BufferedDataWriter.java:176)
at jetbrains.exodus.log.Log.flush(Log.kt:631)
at jetbrains.exodus.log.Log.flush$default(Log.kt:630)
at jetbrains.exodus.log.Log.flush(Log.kt)
at
jetbrains.exodus.env.EnvironmentImpl.flushTransaction(EnvironmentImpl.java:731)
at
jetbrains.exodus.env.EnvironmentImpl.commitTransaction(EnvironmentImpl.java:684)
at
jetbrains.exodus.env.ReadWriteTransaction.commit(ReadWriteTransaction.java:96)
at
jetbrains.exodus.entitystore.PersistentStoreTransaction.doCommit(PersistentStoreTransaction.java:192)
at
jetbrains.exodus.entitystore.PersistentStoreTransaction.commit(PersistentStoreTransaction.java:180)
at
jetbrains.exodus.entitystore.PersistentEntityStoreImpl.executeInExclusiveTransaction(PersistentEntityStoreImpl.java:720)
at
org.apache.nifi.admin.service.EntityStoreAuditService.addActions(EntityStoreAuditService.java:116)
at org.apache.nifi.audit.NiFiAuditor.saveActions(NiFiAuditor.java:66)
at org.apache.nifi.audit.NiFiAuditor.saveAction(NiFiAuditor.java:53)
at
org.apache.nifi.audit.ProcessGroupAuditor.saveUpdateAction(ProcessGroupAuditor.java:359)
at
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:305)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
{noformat}
Afterwards, any attempt to read or write from Xodus produces an exception
similar to this:
{noformat}
2023-12-04 16:32:53,111 WARN [NiFi Web Server-371]
o.apache.nifi.audit.ProcessGroupAuditor Unable to record actions:
jetbrains.exodus.ExodusException: Environment is inoperative
at
jetbrains.exodus.ExodusException.toExodusException(ExodusException.java:49)
at
jetbrains.exodus.env.EnvironmentImpl.checkIsOperative(EnvironmentImpl.java:1069)
at
jetbrains.exodus.env.EnvironmentImpl.beginTransaction(EnvironmentImpl.java:613)
at
jetbrains.exodus.env.EnvironmentImpl.beginExclusiveTransaction(EnvironmentImpl.java:264)
at
jetbrains.exodus.entitystore.PersistentStoreTransaction.<init>(PersistentStoreTransaction.java:130)
at
jetbrains.exodus.entitystore.PersistentEntityStoreImpl.beginExclusiveTransaction(PersistentEntityStoreImpl.java:539)
at
jetbrains.exodus.entitystore.PersistentEntityStoreImpl.executeInExclusiveTransaction(PersistentEntityStoreImpl.java:717)
at
org.apache.nifi.admin.service.EntityStoreAuditService.addActions(EntityStoreAuditService.java:116)
at org.apache.nifi.audit.NiFiAuditor.saveActions(NiFiAuditor.java:66)
at org.apache.nifi.audit.NiFiAuditor.saveAction(NiFiAuditor.java:53)
at
org.apache.nifi.audit.ProcessGroupAuditor.saveUpdateAction(ProcessGroupAuditor.java:359)
at
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:305)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
...
...
Caused by: java.io.IOException: Stream Closed
at java.base/java.io.RandomAccessFile.length0(Native Method)
at java.base/java.io.RandomAccessFile.length(RandomAccessFile.java:653)
at jetbrains.exodus.io.FileDataWriter.ensureFile(FileDataWriter.kt:172)
at jetbrains.exodus.io.FileDataWriter.write(FileDataWriter.kt:59)
at
jetbrains.exodus.log.BufferedDataWriter.writePage(BufferedDataWriter.java:264)
at
jetbrains.exodus.log.BufferedDataWriter.flush(BufferedDataWriter.java:176)
at jetbrains.exodus.log.Log.flush(Log.kt:631)
at jetbrains.exodus.log.Log.flush$default(Log.kt:630)
at jetbrains.exodus.log.Log.flush(Log.kt)
at
jetbrains.exodus.env.EnvironmentImpl.flushTransaction(EnvironmentImpl.java:731)
at
jetbrains.exodus.env.EnvironmentImpl.commitTransaction(EnvironmentImpl.java:684)
at
jetbrains.exodus.env.ReadWriteTransaction.commit(ReadWriteTransaction.java:96)
at
jetbrains.exodus.entitystore.PersistentStoreTransaction.doCommit(PersistentStoreTransaction.java:192)
at
jetbrains.exodus.entitystore.PersistentStoreTransaction.commit(PersistentStoreTransaction.java:180)
at
jetbrains.exodus.entitystore.PersistentEntityStoreImpl.executeInExclusiveTransaction(PersistentEntityStoreImpl.java:720)
at
org.apache.nifi.admin.service.EntityStoreAuditService.addActions(EntityStoreAuditService.java:116)
at org.apache.nifi.audit.NiFiAuditor.saveActions(NiFiAuditor.java:66)
at org.apache.nifi.audit.NiFiAuditor.saveAction(NiFiAuditor.java:53)
at
org.apache.nifi.audit.ProcessGroupAuditor.saveUpdateAction(ProcessGroupAuditor.java:359)
at
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:305)
at
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
