[
https://issues.apache.org/jira/browse/GOBBLIN-1984?focusedWorklogId=899300&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-899300
]
ASF GitHub Bot logged work on GOBBLIN-1984:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 12/Jan/24 00:07
Start Date: 12/Jan/24 00:07
Worklog Time Spent: 10m
Work Description: umustafi opened a new pull request, #3857:
URL: https://github.com/apache/gobblin/pull/3857
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [X] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1984
### Description
- [X] Here are some details about my PR, including screenshots (if
applicable):
In a previous change, we noticed the compiled `flowExecutionId` was
different from the one during execution and tried to fix it by updating the
`flowMetadata` used to send GTE relating to compilation. See problematic patch
https://github.com/apache/gobblin/pull/3823
We succeeded in updating the compilation event to use the `flowExecutionId`
but did not update the `FlowSpec` used by `Compiler` to create the `Dag` which
is used during the execution phase of flow management. The `compiler` retrieves
a `flowExecutionId` if one exists in the `flowSpec` (which it does for adhoc
flows), otherwise generates a new one
([code](https://github.com/apache/gobblin/compare/master...umustafi:gobblin:updateFlowSpecWithId?expand=1#301]).)
We do need to modify the `FlowSpec` then with the consensus `flowExecutionId`
before forwarding it to the `DagManager`.
### Tests
- [X] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Updates validation helper tests and adds FlowSpec updateProperty test
### Commits
- [X] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
Issue Time Tracking
-------------------
Worklog Id: (was: 899300)
Remaining Estimate: 0h
Time Spent: 10m
> Add Consensus FlowExecutionId to FlowSpec used for Compilation and Tracking
> Dag
> -------------------------------------------------------------------------------
>
> Key: GOBBLIN-1984
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1984
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In a previous change, we noticed the compiled flowExecutionId was different
> from the one during execution and tried to fix it by updating the
> flowMetadata used to send GTE relating to compilation. We succeeded in
> updating the compilation event to use the flowExecutionId but did not update
> the FlowSpec used by Compiler to create the Dag which is used during the
> execution phase of flow management. The compiler retrieves a flowExecutionId
> if one exists in the flowSpec (which it does for adhoc flows), otherwise
> generates a new one
> ([code|[https://jarvis.corp.linkedin.com/codesearch/result/?name=MultiHopFlowCompiler.java&path=gobblin-elr%2Fgobblin-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice%2Fmodules%2Fflow&reponame=linkedin%2Fgobblin-elr#301]).]
> We do need to modify the FlowSpec then with the consensus flowExecutionId
> before forwarding it to the DagManager.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)