[
https://issues.apache.org/jira/browse/BEAM-8871?focusedWorklogId=427123&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427123
]
ASF GitHub Bot logged work on BEAM-8871:
----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Apr/20 23:07
Start Date: 24/Apr/20 23:07
Worklog Time Spent: 10m
Work Description: lukecwik edited a comment on pull request #11454:
URL: https://github.com/apache/beam/pull/11454#issuecomment-619274251
> > Testing that output is only produced after the DoFn has done the claim
call makes a lot of sense and ensuring that only the part that has been claimed
was produced as output would cover correctness.
>
> Thanks for the answer (again). It does not seem trivial to be able to test
this from the outside of the DoFn without having to play with DoFnRunners and
friends. Or is there something that would help to do this?
Users should rely on PAssert/TestStream for correctness testing. Runners
should perform dynamic work rebalancing on these pipelines which via test
flakes should show correctness issues.
As a simpler test, if the user doesn't use
StartBundleContext/FinishBundleContext/ProcessContext, it should be easy for
the user to instantiate their DoFn and call the methods on it directly if needs
be.
To check splitting efficiency, runners that support dynamic work rebalancing
can tell users that their SDFs aren't behaving well by seeing that the SDF is
not splitting or making a lot of unbalanced splits.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 427123)
Time Spent: 5h (was: 4h 50m)
> Add support for splitting at fractions > 0 to
> org.apache.beam.sdk.transforms.splittabledofn.ByteKeyRangeTracker
> ---------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-8871
> URL: https://issues.apache.org/jira/browse/BEAM-8871
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-core
> Reporter: Luke Cwik
> Assignee: Boyuan Zhang
> Priority: Major
> Fix For: 2.22.0
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> org.apache.beam.sdk.transforms.splittabledofn.ByteKeyRangeTracker only
> supports checkpointing
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
