[
https://issues.apache.org/jira/browse/BEAM-11105?focusedWorklogId=757079&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-757079
]
ASF GitHub Bot logged work on BEAM-11105:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Apr/22 16:46
Start Date: 14/Apr/22 16:46
Worklog Time Spent: 10m
Work Description: damccorm commented on PR #17267:
URL: https://github.com/apache/beam/pull/17267#issuecomment-1099392945
> I think the difficulty here is that we can't meaningfully test the
effectiveness of the watermark estimation until we have ProcessContinuations
in, but I see (other than the last two suggested changes) no reason not to get
it in at the moment.
I would argue that we can to some degree with splitting and a little hacking
to set the mode to streaming (I've run this with dataflow and seen it correctly
update the watermarks on splits as described in my description). That isn't a
complete demonstration of functionality, but the implementation isn't complete
either since it doesn't try to handle processContinuations yet (that will need
to be a future pr). Either way, I'm on board with your overall assessment that
we shouldn't block this change on that, if for no other reason than it blocks
future watermark work.
Thanks for the linting callouts, I committed both of them
Issue Time Tracking
-------------------
Worklog Id: (was: 757079)
Time Spent: 3.5h (was: 3h 20m)
> [Go SDK] Watermark Estimation
> -----------------------------
>
> Key: BEAM-11105
> URL: https://issues.apache.org/jira/browse/BEAM-11105
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Danny McCormick
> Priority: P3
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Allow self checkpointing SplittableDoFns to specify a watermark estimator.
> (To be updated once [https://github.com/apache/beam/pull/13160] is merged and
> the programming guide updated with SDF content.)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
