[
https://issues.apache.org/jira/browse/GOBBLIN-1837?focusedWorklogId=862489&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-862489
]
ASF GitHub Bot logged work on GOBBLIN-1837:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 26/May/23 21:06
Start Date: 26/May/23 21:06
Worklog Time Spent: 10m
Work Description: umustafi opened a new pull request, #3700:
URL: https://github.com/apache/gobblin/pull/3700
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I
have checked off all the steps below!
### JIRA
- [X] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1837
### Description
- [X] Here are some details about my PR, including screenshots (if
applicable):
This task will include the implementation of non-blocking, multi-active
scheduler for each host. It will NOT include metric emission or unit tests for
validation. That will be done in a separate follow-up ticket. The work in this
ticket includes
- define a table to do scheduler lease determination for each flow's trigger
event and related methods to execute actions on this table
- update DagActionStore schema and DagActionStoreMonitor to act upon new
"LAUNCH" type events in addition to KILL/RESUME
- update scheduler/orchestrator logic to apply the non-blocking algorithm
when "multi-active scheduler mode" is enabled, otherwise submit events directly
to the DagManager after receiving a scheduler trigger
- implement the non-blocking algorithm, particularly handling reminder
events if another host is in the process of securing the lease for a particular
flow trigger
NOTE: because I'm updating the DagActionStore schema this change will
require manually altering the primary key of the table before deploying these
changes. MySQL only creates/updates the table if the same table name does not
exist.
### Tests
- [X] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Limiting the scope of this PR to the implementation and will focus on
metrics, logging for validation, and unit tests in a separate PR.
### Commits
- [X] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
Issue Time Tracking
-------------------
Worklog Id: (was: 862489)
Remaining Estimate: 0h
Time Spent: 10m
> Implement multi-active, non blocking for leader host
> ----------------------------------------------------
>
> Key: GOBBLIN-1837
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1837
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This task will include the implementation of non-blocking, multi-active
> scheduler for each host. It will NOT include metric emission or unit tests
> for validation. That will be done in a separate follow-up ticket. The work in
> this ticket includes
> * define a table to do scheduler lease determination for each flow's trigger
> event and related methods to execute actions on this table
> * update DagActionStore schema and DagActionStoreMonitor to act upon new
> "LAUNCH" type events in addition to KILL/RESUME
> * update scheduler/orchestrator logic to apply the non-blocking algorithm
> when "multi-active scheduler mode" is enabled, otherwise submit events
> directly to the DagManager after receiving a scheduler trigger
> * implement the non-blocking algorithm, particularly handling reminder
> events if another host is in the process of securing the lease for a
> particular flow trigger
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
