Urmi Mustafi created GOBBLIN-1984:
-------------------------------------
Summary: Add Consensus FlowExecutionId to FlowSpec used for
Compilation and Tracking Dag
Key: GOBBLIN-1984
URL: https://issues.apache.org/jira/browse/GOBBLIN-1984
Project: Apache Gobblin
Issue Type: Bug
Components: gobblin-service
Reporter: Urmi Mustafi
Assignee: Abhishek Tiwari
In a previous change, we noticed the compiled flowExecutionId was different
from the one during execution and tried to fix it by updating the flowMetadata
used to send GTE relating to compilation. We succeeded in updating the
compilation event to use the flowExecutionId but did not update the FlowSpec
used by Compiler to create the Dag which is used during the execution phase of
flow management. The compiler retrieves a flowExecutionId if one exists in the
flowSpec (which it does for adhoc flows), otherwise generates a new one
([code|[https://jarvis.corp.linkedin.com/codesearch/result/?name=MultiHopFlowCompiler.java&path=gobblin-elr%2Fgobblin-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice%2Fmodules%2Fflow&reponame=linkedin%2Fgobblin-elr#301]).]
We do need to modify the FlowSpec then with the consensus flowExecutionId
before forwarding it to the DagManager.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)