[
https://issues.apache.org/jira/browse/BEAM-10670?focusedWorklogId=495467&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-495467
]
ASF GitHub Bot logged work on BEAM-10670:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Oct/20 17:31
Start Date: 05/Oct/20 17:31
Worklog Time Spent: 10m
Work Description: lukecwik edited a comment on pull request #12603:
URL: https://github.com/apache/beam/pull/12603#issuecomment-703777328
> > @iemejia Since streaming is effectively broken due to lack of support
for watermark holds. What do you think about enabling SDF for Spark and it only
working in batch?
>
> Can you be more explicit on what you mean on 'it only working in batch'?
Isn't it the current case for Bounded SDF?
> Or what you mean is to have now Unbounded Reads translated using SDF still
with the same level of functionality than the old translation (no holds)?
BoundedSource converted to an SDF will work just as well as the current
BoundedSource implementation since they don't rely on watermark holds. The
current implementation of UnboundedSource and the new implementation using
UnboundedSource as an SDF both set the watermark but triggers don't honor it
since there is lack of support for watermark holds.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 495467)
Time Spent: 25h (was: 24h 50m)
> Make non-portable Splittable DoFn the only option when executing Java "Read"
> transforms
> ---------------------------------------------------------------------------------------
>
> Key: BEAM-10670
> URL: https://issues.apache.org/jira/browse/BEAM-10670
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Luke Cwik
> Assignee: Luke Cwik
> Priority: P2
> Time Spent: 25h
> Remaining Estimate: 0h
>
> All runners seem to be capable of migrating to splittable DoFn for
> non-portable execution except for Dataflow runner v1 which will internalize
> the current primitive read implementation that is shared across runner
> implementations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
