[
https://issues.apache.org/jira/browse/NIFI-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15892870#comment-15892870
]
ASF GitHub Bot commented on NIFI-3523:
--------------------------------------
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1542#discussion_r104014802
--- Diff:
nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/serialization/CompressableRecordReader.java
---
@@ -283,7 +283,20 @@ public StandardProvenanceEventRecord nextRecord()
throws IOException {
}
if (isData()) {
- return nextRecord(dis, serializationVersion);
+ while (true) {
+ try {
+ return nextRecord(dis, serializationVersion);
+ } catch (final IOException ioe) {
+ throw ioe;
+ } catch (final Exception e) {
+ // This would only happen if a bug were to exist such
that an 'invalid' event were written
+ // out. For example an Event that has no FlowFile
UUID. While there is in fact an underlying
+ // cause that would need to be sorted out in this
case, the Provenance Repository should be
+ // resilient enough to handle this. Otherwise, we end
up throwing an Exception, which may
+ // prevent iterating over additional events in the
repository.
+ logger.error("Failed to read Provenance Event from " +
filename + "; will skip this event and continue reading subsequent events", e);
+ }
+ }
--- End diff --
@olegz I agree that it is a stylistic preference. I don't really mind
either one. `while(true)` doesn't bother me at all and I believe it's very
clear -- "continue doing the below until told to stop." The proposal that you
lay out also relies "on the hope that something in the loop will eventually set
the value to the assumed stop condition" -- 6 of one, half dozen of the other,
I think. That being said, I would prefer to keep it as-is over changing the
code like that for stylistic changes, given that it means that we'd then have
to re-test to ensure correctness (sure, there are unit tests, but don't I
wouldn't change something like this without going beyond unit testing and
testing manually in the application as well).
> WriteAheadProvenanceRepository fails to initialize properly if it contains an
> invalid Provenance Event
> ------------------------------------------------------------------------------------------------------
>
> Key: NIFI-3523
> URL: https://issues.apache.org/jira/browse/NIFI-3523
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
>
> With the new WriteAheadProvenanceRepository, when trying to initialize
> itself, the repository will throw an Exception preventing NiFi from starting
> if it encounters an invalid Provenance Event. While it can properly handle
> IOExceptions that are thrown, other Exceptions (such as those thrown by
> NIFI-3522) will still cause problems. While such events should not be
> generated, the Provenance Repository should be resilient enough to handle
> them and continue on.
> 2017-02-22 15:47:11,896 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> org.apache.nifi.web.NiFiCoreException: Unable to start Flow Controller.
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:85)
> ~[na:na]
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:837)
> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:533)
> ~[jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:810)
> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:345)
> ~[jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1404)
> ~[jetty-webapp-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1366)
> ~[jetty-webapp-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772)
> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262)
> ~[jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520)
> ~[jetty-webapp-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:231)
> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
> at org.eclipse.jetty.server.Server.start(Server.java:411)
> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106)
> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
> at org.eclipse.jetty.server.Server.doStart(Server.java:378)
> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517]
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517]
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:736)
> ~[nifi-jetty-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at org.apache.nifi.NiFi.<init>(NiFi.java:157)
> [nifi-runtime-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at org.apache.nifi.NiFi.main(NiFi.java:264)
> [nifi-runtime-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'flowService': FactoryBean threw exception on object
> creation; nested exception is
> org.springframework.beans.factory.BeanCreationException: Error creating bean
> with name 'flowController': FactoryBean threw exception on object creation;
> nested exception is java.lang.RuntimeException: Unable to create Provenance
> Repository
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1585)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1060)
> ~[spring-context-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:53)
> ~[na:na]
> ... 28 common frames omitted
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'flowController': FactoryBean threw exception on
> object creation; nested exception is java.lang.RuntimeException: Unable to
> create Provenance Repository
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1585)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1060)
> ~[spring-context-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.spring.StandardFlowServiceFactoryBean.getObject(StandardFlowServiceFactoryBean.java:48)
> ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> ... 34 common frames omitted
> Caused by: java.lang.RuntimeException: Unable to create Provenance Repository
> at
> org.apache.nifi.controller.FlowController.<init>(FlowController.java:462)
> ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.nifi.controller.FlowController.createStandaloneInstance(FlowController.java:378)
> ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.nifi.spring.FlowControllerFactoryBean.getObject(FlowControllerFactoryBean.java:74)
> ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
> ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> ... 41 common frames omitted
> Caused by: java.lang.IllegalStateException: Cannot create Provenance Event
> Record because FlowFile UUID is not set
> at
> org.apache.nifi.provenance.StandardProvenanceEventRecord$Builder.assertSet(StandardProvenanceEventRecord.java:770)
> ~[nifi-data-provenance-utils-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.nifi.provenance.StandardProvenanceEventRecord$Builder.build(StandardProvenanceEventRecord.java:791)
> ~[nifi-data-provenance-utils-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.nifi.provenance.schema.LookupTableEventRecord.getEvent(LookupTableEventRecord.java:331)
> ~[na:na]
> at
> org.apache.nifi.provenance.EventIdFirstSchemaRecordReader.readRecord(EventIdFirstSchemaRecordReader.java:111)
> ~[na:na]
> at
> org.apache.nifi.provenance.EventIdFirstSchemaRecordReader.nextRecord(EventIdFirstSchemaRecordReader.java:100)
> ~[na:na]
> at
> org.apache.nifi.provenance.serialization.CompressableRecordReader.nextRecord(CompressableRecordReader.java:286)
> ~[na:na]
> at
> org.apache.nifi.provenance.serialization.CompressableRecordReader.getMaxEventId(CompressableRecordReader.java:218)
> ~[na:na]
> at
> org.apache.nifi.provenance.store.WriteAheadStorePartition.initialize(WriteAheadStorePartition.java:126)
> ~[na:na]
> at
> org.apache.nifi.provenance.store.PartitionedEventStore.initialize(PartitionedEventStore.java:68)
> ~[na:na]
> at
> org.apache.nifi.provenance.store.PartitionedWriteAheadEventStore.initialize(PartitionedWriteAheadEventStore.java:97)
> ~[na:na]
> at
> org.apache.nifi.provenance.WriteAheadProvenanceRepository.initialize(WriteAheadProvenanceRepository.java:141)
> ~[na:na]
> at
> org.apache.nifi.controller.FlowController.<init>(FlowController.java:460)
> ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> ... 44 common frames omitted
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
