sv2000 opened a new pull request #3002: URL: https://github.com/apache/incubator-gobblin/pull/3002
…icide when unhealthy Dear Gobblin maintainers, Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below! ### JIRA - [x] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR" - https://issues.apache.org/jira/browse/GOBBLIN-1162 ### Description - [x] Here are some details about my PR, including screenshots (if applicable): In execution environments such as Gobblin-on-Yarn, where Gobblin workers can be re-assigned when the worker dies or is killed, it is useful to add a mode where each Gobblin task running inside a Gobblin worker can perform application-level health checks and report results of the health checks back to the worker hosting the tasks. The worker on receiving a health check failure event can be configured to exit the JVM. In the case of Gobblin-on-Yarn mode, this will result in the worker getting re-assigned to a different node. The proposed behavior is similar in flavor to speculative execution modes provided in other execution frameworks such as MapReduce. This change also provides an example of such an application-level health check that arises in the case of Kafka ingestion. In addition, this PR adds an EventBusFactory as an implementation of SharedResourcesBrokerFactory that allows communication between different components of GobblinTaskRunner via a shared EventBus instance. ### Tests - [x] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason: Added unit tests: 1.EventBusFactoryTest 2. GobblinTaskRunnerTest 3. KafkaIngestionHealthCheckTest ### Commits - [x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)": 1. Subject is separated from body by a blank line 2. Subject is limited to 50 characters 3. Subject does not end with a period 4. Subject uses the imperative mood ("add", not "adding") 5. Body wraps at 72 characters 6. Body explains "what" and "why", not "how" ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
