[ 
https://issues.apache.org/jira/browse/GOBBLIN-1984?focusedWorklogId=899300&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-899300
 ]

ASF GitHub Bot logged work on GOBBLIN-1984:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 12/Jan/24 00:07
            Start Date: 12/Jan/24 00:07
    Worklog Time Spent: 10m 
      Work Description: umustafi opened a new pull request, #3857:
URL: https://github.com/apache/gobblin/pull/3857

   Dear Gobblin maintainers,
   
   Please accept this PR. I understand that it will not be reviewed until I 
have checked off all the steps below!
   
   
   ### JIRA
   - [X] My PR addresses the following [Gobblin 
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references 
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
       - https://issues.apache.org/jira/browse/GOBBLIN-1984
   
   
   ### Description
   - [X] Here are some details about my PR, including screenshots (if 
applicable):
   In a previous change, we noticed the compiled `flowExecutionId` was 
different from the one during execution and tried to fix it by updating the 
`flowMetadata` used to send GTE relating to compilation. See problematic patch 
https://github.com/apache/gobblin/pull/3823 
   We succeeded in updating the compilation event to use the `flowExecutionId` 
but did not update the `FlowSpec` used by `Compiler` to create the `Dag` which 
is used during the execution phase of flow management. The `compiler` retrieves 
a `flowExecutionId` if one exists in the `flowSpec` (which it does for adhoc 
flows), otherwise generates a new one 
([code](https://github.com/apache/gobblin/compare/master...umustafi:gobblin:updateFlowSpecWithId?expand=1#301]).)
 We do need to modify the `FlowSpec` then with the consensus `flowExecutionId` 
before forwarding it to the `DagManager`.
   
   ### Tests
   - [X] My PR adds the following unit tests __OR__ does not need testing for 
this extremely good reason:
   Updates validation helper tests and adds FlowSpec updateProperty test
   
   ### Commits
   - [X] My commits all reference JIRA issues in their subject lines, and I 
have squashed multiple commits if they address the same issue. In addition, my 
commits follow the guidelines from "[How to write a good git commit 
message](http://chris.beams.io/posts/git-commit/)":
       1. Subject is separated from body by a blank line
       2. Subject is limited to 50 characters
       3. Subject does not end with a period
       4. Subject uses the imperative mood ("add", not "adding")
       5. Body wraps at 72 characters
       6. Body explains "what" and "why", not "how"
   
   




Issue Time Tracking
-------------------

            Worklog Id:     (was: 899300)
    Remaining Estimate: 0h
            Time Spent: 10m

> Add Consensus FlowExecutionId to FlowSpec used for Compilation and Tracking 
> Dag
> -------------------------------------------------------------------------------
>
>                 Key: GOBBLIN-1984
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1984
>             Project: Apache Gobblin
>          Issue Type: Bug
>          Components: gobblin-service
>            Reporter: Urmi Mustafi
>            Assignee: Abhishek Tiwari
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> In a previous change, we noticed the compiled flowExecutionId was different 
> from the one during execution and tried to fix it by updating the 
> flowMetadata used to send GTE relating to compilation. We succeeded in 
> updating the compilation event to use the flowExecutionId but did not update 
> the FlowSpec used by Compiler to create the Dag which is used during the 
> execution phase of flow management. The compiler retrieves a flowExecutionId 
> if one exists in the flowSpec (which it does for adhoc flows), otherwise 
> generates a new one 
> ([code|[https://jarvis.corp.linkedin.com/codesearch/result/?name=MultiHopFlowCompiler.java&path=gobblin-elr%2Fgobblin-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice%2Fmodules%2Fflow&reponame=linkedin%2Fgobblin-elr#301]).]
>  We do need to modify the FlowSpec then with the consensus flowExecutionId 
> before forwarding it to the DagManager.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to