Zihan Li created GOBBLIN-717:
--------------------------------
Summary: Filter Out Empty MultiWorkUnits
Key: GOBBLIN-717
URL: https://issues.apache.org/jira/browse/GOBBLIN-717
Project: Apache Gobblin
Issue Type: Improvement
Reporter: Zihan Li
Now when we run a job, Gobblin use the value of max mappers or the target size
of a mapper to determine the number of mappers. But since one partition cannot
be divided into several WorkUnits, work cannot be evenly distributed, there are
many mappers(MultiWorkUnits) have no work to do. This will waste a lot of
resources. So we need to filter out MultiWorkUnits which contains no WorkUnit
when we determine the number of mappers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
