[ 
https://issues.apache.org/jira/browse/GOBBLIN-1984?focusedWorklogId=899469&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-899469
 ]

ASF GitHub Bot logged work on GOBBLIN-1984:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 12/Jan/24 18:56
            Start Date: 12/Jan/24 18:56
    Worklog Time Spent: 10m 
      Work Description: phet commented on code in PR #3857:
URL: https://github.com/apache/gobblin/pull/3857#discussion_r1450815488


##########
gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/DagActionStoreChangeMonitor.java:
##########
@@ -276,8 +277,12 @@ protected void 
submitFlowToDagManagerHelper(DagActionStore.DagAction dagAction,
     try {
       URI flowUri = FlowSpec.Utils.createFlowSpecUri(flowId);
       spec = (FlowSpec) flowCatalog.getSpecs(flowUri);

Review Comment:
   if you're wondering how Java knows which to choose at runtime, given this is 
still an \@Override, per usual it's chosen dynamically based on the runtime 
type.
   
   the difference here is the more-specific typing in cases where the 
`SpecCatalog` derived type can be guaranteed statically.
   
   to illustrate, these two are legal:
   ```
   URI uri = ...
   SpecCatalog sc = ...
   Spec s = sc.getSpecs(uri);
   ```
   and
   ```
   FlowSpecCatalog fsc = ...
   FlowSpec fs = fsc.getSpecs(uri);
   ```
   but this does NOT type-check:
   ```
   SpecCatalog sc = ...
   FlowSpec fs = sc.getSpecs(uri);
   ```





Issue Time Tracking
-------------------

    Worklog Id:     (was: 899469)
    Time Spent: 1h 50m  (was: 1h 40m)

> Add Consensus FlowExecutionId to FlowSpec used for Compilation and Tracking 
> Dag
> -------------------------------------------------------------------------------
>
>                 Key: GOBBLIN-1984
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1984
>             Project: Apache Gobblin
>          Issue Type: Bug
>          Components: gobblin-service
>            Reporter: Urmi Mustafi
>            Assignee: Abhishek Tiwari
>            Priority: Major
>          Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> In a previous change, we noticed the compiled flowExecutionId was different 
> from the one during execution and tried to fix it by updating the 
> flowMetadata used to send GTE relating to compilation. We succeeded in 
> updating the compilation event to use the flowExecutionId but did not update 
> the FlowSpec used by Compiler to create the Dag which is used during the 
> execution phase of flow management. The compiler retrieves a flowExecutionId 
> if one exists in the flowSpec (which it does for adhoc flows), otherwise 
> generates a new one 
> ([code|[https://jarvis.corp.linkedin.com/codesearch/result/?name=MultiHopFlowCompiler.java&path=gobblin-elr%2Fgobblin-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice%2Fmodules%2Fflow&reponame=linkedin%2Fgobblin-elr#301]).]
>  We do need to modify the FlowSpec then with the consensus flowExecutionId 
> before forwarding it to the DagManager.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to