umustafi opened a new pull request, #3700: URL: https://github.com/apache/gobblin/pull/3700
Dear Gobblin maintainers, Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below! ### JIRA - [X] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR" - https://issues.apache.org/jira/browse/GOBBLIN-1837 ### Description - [X] Here are some details about my PR, including screenshots (if applicable): This task will include the implementation of non-blocking, multi-active scheduler for each host. It will NOT include metric emission or unit tests for validation. That will be done in a separate follow-up ticket. The work in this ticket includes - define a table to do scheduler lease determination for each flow's trigger event and related methods to execute actions on this table - update DagActionStore schema and DagActionStoreMonitor to act upon new "LAUNCH" type events in addition to KILL/RESUME - update scheduler/orchestrator logic to apply the non-blocking algorithm when "multi-active scheduler mode" is enabled, otherwise submit events directly to the DagManager after receiving a scheduler trigger - implement the non-blocking algorithm, particularly handling reminder events if another host is in the process of securing the lease for a particular flow trigger NOTE: because I'm updating the DagActionStore schema this change will require manually altering the primary key of the table before deploying these changes. MySQL only creates/updates the table if the same table name does not exist. ### Tests - [X] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason: Limiting the scope of this PR to the implementation and will focus on metrics, logging for validation, and unit tests in a separate PR. ### Commits - [X] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)": 1. Subject is separated from body by a blank line 2. Subject is limited to 50 characters 3. Subject does not end with a period 4. Subject uses the imperative mood ("add", not "adding") 5. Body wraps at 72 characters 6. Body explains "what" and "why", not "how" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
