[
https://issues.apache.org/jira/browse/BEAM-3061?focusedWorklogId=258924&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-258924
]
ASF GitHub Bot logged work on BEAM-3061:
----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Jun/19 17:32
Start Date: 12/Jun/19 17:32
Worklog Time Spent: 10m
Work Description: steveniemitz commented on issue #7805: [BEAM-3061] Done
notifications for BigtableIO.Write
URL: https://github.com/apache/beam/pull/7805#issuecomment-501377866
hah I had forgotten I even had this PR open still...answers are inline.
> Sorry about the delay.
>
> So IIUC, for each bundle (not window) a null value will be emitted as the
output ?
Both really, for every bundle that arrives at the BigtableIO, when all
writes in the bundle complete, it will emit a single Void (in this case it is
null, but the value isn't important) for every window in the bundle.
> Since this signals that part of a data has been written, not all data, how
do you think users will use this notification ?
I included an example of how it would be used above in my initial comment,
was that unclear? The typical use-case would be to do something after a window
closes and all writes have completed.
> Also, please update documentation and add unit tests. Also make sure to
test this with a real BT cluster and/or add an integration test (optional).
I'll add unit tests and rebase this soon. We've been using this in
production for months now, so it's gotten battle tested pretty well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 258924)
Time Spent: 2h (was: 1h 50m)
> BigtableIO should support emitting a sentinel "done" value when a bundle
> completes
> ----------------------------------------------------------------------------------
>
> Key: BEAM-3061
> URL: https://issues.apache.org/jira/browse/BEAM-3061
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Reporter: Steve Niemitz
> Assignee: Steve Niemitz
> Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> There was some discussion of this on the dev@ mailing list [1]. This
> approach was taken based on discussion there.
> [1]
> https://lists.apache.org/thread.html/949b33782f722a9000c9bf9e37042739c6fd0927589b99752b78d7bd@%3Cdev.beam.apache.org%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
