[
https://issues.apache.org/jira/browse/BEAM-8872?focusedWorklogId=423707&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-423707
]
ASF GitHub Bot logged work on BEAM-8872:
----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Apr/20 20:24
Start Date: 16/Apr/20 20:24
Worklog Time Spent: 10m
Work Description: boyuanzz commented on pull request #11418: [BEAM-8872]
Support split at fraction for OffsetRangeTracker
URL: https://github.com/apache/beam/pull/11418#discussion_r409826636
##########
File path:
runners/core-java/src/main/java/org/apache/beam/runners/core/OutputAndTimeBoundedSplittableProcessElementInvoker.java
##########
@@ -210,9 +210,10 @@ public FinishBundleContext
finishBundleContext(DoFn<InputT, OutputT> doFn) {
// the call says that not the whole restriction has been processed. So
we need to take
// a checkpoint now: checkpoint() guarantees that the primary
restriction describes exactly
// the work that was done in the current ProcessElement call, and
returns a residual
- // restriction that describes exactly the work that wasn't done in the
current call.
+ // restriction that describes exactly the work that wasn't done in the
current call. The
+ // residual is null when the entire restriction has been processed.
if (processContext.numClaimedBlocks > 0) {
- residual = checkNotNull(processContext.takeCheckpointNow());
+ residual = processContext.takeCheckpointNow();
Review comment:
I guess the original assumption is, checkpoint should happen after at least
one `tryClaim` called. Since we change the assumption, the `numClaimedBlocks `
can also be removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 423707)
Time Spent: 2h 40m (was: 2.5h)
> Add support for splitting at fractions > 0 to
> org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker
> --------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-8872
> URL: https://issues.apache.org/jira/browse/BEAM-8872
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-core
> Reporter: Luke Cwik
> Assignee: Boyuan Zhang
> Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> org.apache.beam.sdk.transforms.splittabledofn.OffsetRangeTracker only
> supports checkpointing.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
