[
https://issues.apache.org/jira/browse/GOBBLIN-490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuai Yu updated GOBBLIN-490:
----------------------------
Description: This new job launcher will submit original job to Helix
instead of the generated work units. This will allow Helix to re-distribute the
original jobs to different worker nodes. Each worker then process the original
jobs and launch its own GobblinHelixJobLauncher. By doing this, we will relieve
manager node because the task driver logic (mainly generate work units) is now
distributed to worker nodes.
> Allow job to be distributed by Helix
> ------------------------------------
>
> Key: GOBBLIN-490
> URL: https://issues.apache.org/jira/browse/GOBBLIN-490
> Project: Apache Gobblin
> Issue Type: New Feature
> Reporter: Kuai Yu
> Assignee: Kuai Yu
> Priority: Major
>
> This new job launcher will submit original job to Helix instead of the
> generated work units. This will allow Helix to re-distribute the original
> jobs to different worker nodes. Each worker then process the original jobs
> and launch its own GobblinHelixJobLauncher. By doing this, we will relieve
> manager node because the task driver logic (mainly generate work units) is
> now distributed to worker nodes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
