[
https://issues.apache.org/jira/browse/GOBBLIN-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sudarshan Vasudevan updated GOBBLIN-474:
----------------------------------------
Fix Version/s: (was: 0.16.0)
Affects Version/s: (was: 0.13.0)
> Add end-of-datasets marker to workunit correctly to avoid an additional run
> when end of datasets is reached.
> -------------------------------------------------------------------------------------------------------------
>
> Key: GOBBLIN-474
> URL: https://issues.apache.org/jira/browse/GOBBLIN-474
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-compliance
> Affects Versions: 0.15.0
> Reporter: Sudarshan Vasudevan
> Assignee: Sudarshan Vasudevan
> Priority: Major
> Fix For: 0.15.0
>
>
> When the number of maximum workunits per run setting in
> LoopingDatasetFinderSource equals the number of remaining datasets to process
> before the end of the loop, we do not place the "End-of-Dataset" marker in
> the last workunit. The subsequent run turns out to be essentially a no-op and
> is avoidable by correctly detecting the end-of-datasets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
