[
https://issues.apache.org/jira/browse/BEAM-5974?focusedWorklogId=163299&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-163299
]
ASF GitHub Bot logged work on BEAM-5974:
----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Nov/18 00:55
Start Date: 07/Nov/18 00:55
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #6949:
[BEAM-5974] Fix ByteKeyRangeTracker to handle tryClaim(ByteKey.EMPTY) instead
of exposing markDone
URL: https://github.com/apache/beam/pull/6949#discussion_r231348892
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/splittabledofn/ByteKeyRangeTracker.java
##########
@@ -64,23 +87,36 @@ public synchronized ByteKeyRange checkpoint() {
/**
* Attempts to claim the given key.
*
- * <p>Must be larger than the last successfully claimed key.
+ * <p>Must be larger than the last attempted key. Note that passing in
{@link ByteKey#EMPTY}
+ * claims all keys to the end of range and can only be claimed once.
*
* @return {@code true} if the key was successfully claimed, {@code false}
if it is outside the
* current {@link ByteKeyRange} of this tracker (in that case this
operation is a no-op).
*/
@Override
protected synchronized boolean tryClaimImpl(ByteKey key) {
+ // Handle claiming the end of range EMPTY key
+ if (key.isEmpty()) {
+ checkArgument(
+ lastAttemptedKey == null || !lastAttemptedKey.isEmpty(),
+ "Trying to claim key %s while last attempted key was %s",
+ key,
+ lastAttemptedKey);
+ lastAttemptedKey = key;
Review comment:
The range is always a semi-bounded open interval and hence "" being the end
key can never be claimed.
So updating lastAttemptedKey and returning false is the correct thing to do.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 163299)
Time Spent: 1h 20m (was: 1h 10m)
> Migrate ByteKeyRangeTracker to use tryClaim(ByteKey.EMPTY) as end of range
> claim instead of markDone
> ----------------------------------------------------------------------------------------------------
>
> Key: BEAM-5974
> URL: https://issues.apache.org/jira/browse/BEAM-5974
> Project: Beam
> Issue Type: Bug
> Components: io-java-hbase, sdk-java-core
> Affects Versions: 2.8.0
> Reporter: Luke Cwik
> Assignee: Luke Cwik
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> org.apache.beam.sdk.transforms.splittabledofn.ByteKeyRangeTracker doesn't
> handle tryClaim(ByteKey.EMPTY) and the related doneness check or
> checkpointing since doneness checking can't handle the empty interval,
> checkpointing returns invalid checkpoints or errors out since it is using the
> lastClaimedKey vs the lastAttemptedKey for doneness checking
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
