[
https://issues.apache.org/jira/browse/BEAM-10670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334972#comment-17334972
]
Kenneth Knowles commented on BEAM-10670:
----------------------------------------
So the idea is this:
- Every BoundedSource-based Read transform should still have Read in its
expansion. I don't know if this was changed. The SDK should be able to build
one graph and have it work optimally on runners based on their preferences.
- Inside that expansion there may be SDF, but any runner can ignore it, or
have a flag to enable it. Again, I don't think the SDK should change its
expansion.
Can you itemize where this does not work? Was the SparkRunner changed? Were the
IOs made incompatible?
> Make non-portable Splittable DoFn the only option when executing Java "Read"
> transforms
> ---------------------------------------------------------------------------------------
>
> Key: BEAM-10670
> URL: https://issues.apache.org/jira/browse/BEAM-10670
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Luke Cwik
> Assignee: Kenneth Knowles
> Priority: P3
> Labels: Clarified
> Fix For: 2.30.0
>
> Time Spent: 37h 50m
> Remaining Estimate: 0h
>
> All runners seem to be capable of migrating to splittable DoFn for
> non-portable execution except for Dataflow runner v1 which will internalize
> the current primitive read implementation that is shared across runner
> implementations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
