[
https://issues.apache.org/jira/browse/FALCON-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743320#comment-14743320
]
Ajay Yadava commented on FALCON-1401:
-------------------------------------
The "magical" value (IGNORE) is added when outputs don't exist, so what does it
mean that there is an output feed?
Moreover, whatever be the thinking behind it (changing from "NONE" to
"IGNORE"), it is wrong to support two magical values, even worse if you can't
make out one value by reading the code. It is for all present and future
consumers of those arguments and shouldn't be changed.
We shouldn't change magical values conditionally also, if there are use cases
to provide extra signals then we should add a new arg. If a feed exists then
it's name should always be present, it should be left to consumer to decide
what to ignore, as contrary to one consumer other consumers might choose to not
ignore it. Ideally it should have been caught in review of FALCON-1091 but even
now we should revert to a single value of "NONE".
> MetadataMappingService fails to add an edge for a process instance
> ------------------------------------------------------------------
>
> Key: FALCON-1401
> URL: https://issues.apache.org/jira/browse/FALCON-1401
> Project: Falcon
> Issue Type: Bug
> Reporter: Pallavi Rao
> Assignee: Pallavi Rao
> Priority: Minor
> Attachments: FALCON-1401.patch
>
>
> If a process has no output feeds, the metadata mapping service fails to add
> an edge for the process instances. It fails with NPE:
> {code}
> 2015-08-14 15:31:00,950 INFO - [ActiveMQ Session Task:] ~ Adding process
> instance: pig-sleep-process/2015-07-28T07:40Z
> (InstanceRelationshipGraphBuilder:74)
> 2015-08-14 15:31:00,951 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=pig-sleep-process/2015-07-28T07:40Z, type=PROCESS_INSTANCE
> (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:00,957 DEBUG - [ActiveMQ Session Task:] ~ Creating a new
> vertex for: name=pig-sleep-process/2015-07-28T07:40Z, type=PROCESS_INSTANCE
> (RelationshipGraphBuilder:101)
> 2015-08-14 15:31:00,981 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=pig-sleep-process, type=PROCESS_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:00,984 INFO - [ActiveMQ Session Task:] ~ Vertex exists?
> name=pig-sleep-process, type=PROCESS_ENTITY, v=v[16]
> (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:00,988 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=local, type=CLUSTER_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:00,990 INFO - [ActiveMQ Session Task:] ~ Vertex exists?
> name=local, type=CLUSTER_ENTITY, v=v[4] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:00,992 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=pallavi.rao, type=USER (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:00,995 INFO - [ActiveMQ Session Task:] ~ Vertex exists?
> name=pallavi.rao, type=USER, v=v[8] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:00,997 INFO - [ActiveMQ Session Task:] ~ Computing feed
> instance for : name= IGNORE path= IGNORE, in cluster: local
> (InstanceRelationshipGraphBuilder:243)
> 2015-08-14 15:31:01,009 INFO - [ActiveMQ Session Task:] ~ Adding process
> instance: pig-sleep-process/2015-07-28T07:40Z
> (InstanceRelationshipGraphBuilder:74)
> 2015-08-14 15:31:01,009 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=pig-sleep-process/2015-07-28T07:40Z, type=PROCESS_INSTANCE
> (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,012 DEBUG - [ActiveMQ Session Task:] ~ Creating a new
> vertex for: name=pig-sleep-process/2015-07-28T07:40Z, type=PROCESS_INSTANCE
> (RelationshipGraphBuilder:101)
> 2015-08-14 15:31:01,033 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=pig-sleep-process, type=PROCESS_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,035 INFO - [ActiveMQ Session Task:] ~ Vertex exists?
> name=pig-sleep-process, type=PROCESS_ENTITY, v=v[16]
> (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,039 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=local, type=CLUSTER_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,041 INFO - [ActiveMQ Session Task:] ~ Vertex exists?
> name=local, type=CLUSTER_ENTITY, v=v[4] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,043 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=pallavi.rao, type=USER (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,044 INFO - [ActiveMQ Session Task:] ~ Vertex exists?
> name=pallavi.rao, type=USER, v=v[8] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,047 INFO - [ActiveMQ Session Task:] ~ Computing feed
> instance for : name= IGNORE path= IGNORE, in cluster: local
> (InstanceRelationshipGraphBuilder:243)
> 2015-08-14 15:31:01,073 INFO - [ActiveMQ Session Task:] ~ Adding process
> instance: pig-sleep-process/2015-07-28T07:40Z
> (InstanceRelationshipGraphBuilder:74)
> 2015-08-14 15:31:01,073 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=pig-sleep-process/2015-07-28T07:40Z, type=PROCESS_INSTANCE
> (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,076 DEBUG - [ActiveMQ Session Task:] ~ Creating a new
> vertex for: name=pig-sleep-process/2015-07-28T07:40Z, type=PROCESS_INSTANCE
> (RelationshipGraphBuilder:101)
> 2015-08-14 15:31:01,096 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=pig-sleep-process, type=PROCESS_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,098 INFO - [ActiveMQ Session Task:] ~ Vertex exists?
> name=pig-sleep-process, type=PROCESS_ENTITY, v=v[16]
> (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,102 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=local, type=CLUSTER_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,103 INFO - [ActiveMQ Session Task:] ~ Vertex exists?
> name=local, type=CLUSTER_ENTITY, v=v[4] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,105 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex
> for: name=pallavi.rao, type=USER (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,108 INFO - [ActiveMQ Session Task:] ~ Vertex exists?
> name=pallavi.rao, type=USER, v=v[8] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,110 INFO - [ActiveMQ Session Task:] ~ Computing feed
> instance for : name= IGNORE path= IGNORE, in cluster: local
> (InstanceRelationshipGraphBuilder:243)
> 2015-08-14 15:31:01,111 ERROR - [ActiveMQ Session Task:] ~ Error in listener
> org.apache.falcon.metadata.MetadataMappingService
> (WorkflowJobEndNotificationService:226)
> org.apache.falcon.FalconException: java.lang.RuntimeException:
> java.lang.NullPointerException
> at
> org.apache.falcon.metadata.MetadataMappingService.onSuccess(MetadataMappingService.java:274)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:222)
> at
> org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifySuccess(WorkflowJobEndNotificationService.java:105)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:169)
> at
> org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:112)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1229)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:134)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:205)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at
> com.tinkerpop.blueprints.util.TransactionRetryStrategy$AbstractRetryStrategy.execute(TransactionRetryStrategy.java:176)
> at
> com.tinkerpop.blueprints.util.TransactionRetryHelper.use(TransactionRetryHelper.java:115)
> at
> com.tinkerpop.blueprints.util.TransactionRetryHelper.exponentialBackoff(TransactionRetryHelper.java:94)
> at
> org.apache.falcon.metadata.MetadataMappingService.onSuccess(MetadataMappingService.java:263)
> ... 12 more
> Caused by: java.lang.NullPointerException
> at org.apache.falcon.entity.FeedHelper.getCluster(FeedHelper.java:80)
> at
> org.apache.falcon.entity.FeedHelper.getStorageType(FeedHelper.java:235)
> at
> org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.getFeedInstanceName(InstanceRelationshipGraphBuilder.java:280)
> at
> org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.addFeedInstance(InstanceRelationshipGraphBuilder.java:245)
> at
> org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.addOutputFeedInstances(InstanceRelationshipGraphBuilder.java:151)
> at
> org.apache.falcon.metadata.MetadataMappingService.onProcessInstanceExecuted(MetadataMappingService.java:313)
> at
> org.apache.falcon.metadata.MetadataMappingService.onSuccessfulExecution(MetadataMappingService.java:282)
> at
> org.apache.falcon.metadata.MetadataMappingService.access$100(MetadataMappingService.java:53)
> at
> org.apache.falcon.metadata.MetadataMappingService$3.execute(MetadataMappingService.java:267)
> at
> org.apache.falcon.metadata.MetadataMappingService$3.execute(MetadataMappingService.java:264)
> at
> com.tinkerpop.blueprints.util.TransactionRetryStrategy$AbstractRetryStrategy.execute(TransactionRetryStrategy.java:146)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
