[
https://issues.apache.org/jira/browse/BEAM-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16744473#comment-16744473
]
Scott Wegner commented on BEAM-6352:
------------------------------------
Status update:
I made an attempt to extract the extract APIs off of {{GrowthTracker}} into a
separate class local to the ProcessElement invocation. But, it was not as easy
to isolate as I thought; there's some shared state that's needed for
checkpointing. Initial attempt here: https://github.com/apache/beam/pull/7520
In parallel, I tried rolling back PR#7520 so see how easy that would be, and it
seems that everything is passing: https://github.com/apache/beam/pull/7540. So
we have rollback as an option.
I'm going to spend a little more time grokking the Watch.Growth implementation
and see if I can come up with a better way of untangling it. If I don't come to
a solution today, I propose we rollback to unblock the release.
Note: we could also choose to rollback only on the release branch. But, that
keeps master in a broken state and if we don't find a solution today, it's not
clear to me how long it will take. My preference is not to keep things broken
for too long.
> Watch PTransform is broken
> --------------------------
>
> Key: BEAM-6352
> URL: https://issues.apache.org/jira/browse/BEAM-6352
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: 2.9.0
> Reporter: Gleb Kanterov
> Assignee: Scott Wegner
> Priority: Blocker
> Fix For: 2.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> List of affected tests:
> org.apache.beam.sdk.transforms.WatchTest >
> testSinglePollMultipleInputsWithSideInput FAILED
> org.apache.beam.sdk.transforms.WatchTest > testMultiplePollsWithKeyExtractor
> FAILED
> org.apache.beam.sdk.transforms.WatchTest > testSinglePollMultipleInputs FAILED
> org.apache.beam.sdk.transforms.WatchTest >
> testMultiplePollsWithTerminationDueToTerminationCondition FAILED
> org.apache.beam.sdk.transforms.WatchTest > testMultiplePollsWithManyResults
> FAILED
> org.apache.beam.sdk.transforms.WatchTest > testSinglePollWithManyResults
> FAILED
> org.apache.beam.sdk.transforms.WatchTest >
> testMultiplePollsStopAfterTimeSinceNewOutput
> org.apache.beam.sdk.transforms.WatchTest >
> testMultiplePollsWithTerminationBecauseOutputIsFinal FAILED
> org.apache.beam.sdk.io.AvroIOTest$NeedsRunnerTests >
> testContinuouslyWriteAndReadMultipleFilepatterns[0: true] FAILED
> org.apache.beam.sdk.io.AvroIOTest$NeedsRunnerTests >
> testContinuouslyWriteAndReadMultipleFilepatterns[1: false] FAILED
> org.apache.beam.sdk.io.FileIOTest > testMatchWatchForNewFiles FAILED
> org.apache.beam.sdk.io.TextIOReadTest$BasicIOTest > testReadWatchForNewFiles
> FAILED
> {code}
> java.lang.IllegalArgumentException:
> org.apache.beam.sdk.transforms.Watch$WatchGrowthFn, @ProcessElement
> process(ProcessContext, GrowthTracker): Has tracker type
> Watch.GrowthTracker<OutputT, KeyT, TerminationStateT>, but the DoFn's tracker
> type must be of type RestrictionTracker.
> {code}
> Relevant pull requests:
> - https://github.com/apache/beam/pull/6467
> - https://github.com/apache/beam/pull/7374
> Now tests are marked with @Ignore referencing this JIRA issue
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
