MiguelAnzoWizeline commented on a change in pull request #14811:
URL: https://github.com/apache/beam/pull/14811#discussion_r636404119
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/BatchSpannerRead.java
##########
@@ -104,15 +98,31 @@ public void teardown() throws Exception {
}
@ProcessElement
- public void processElement(ProcessContext c) throws Exception {
+ public void processElement(ProcessContext c,
RestrictionTracker<OffsetRange, Long> tracker)
Review comment:
Sure, my design idea was that the element should be the ReadOperation
which is going to be splitted into partitions, and the restriction is an
OffsetRange that represents the partitions left to be processed, and since
initially we don’t know how many partitions are going to be created, the
initial restriction was set to OffsetRange(0, Long.MAX_RANGE).
However I'm still unsure if my approach was correct or if im getting
something wrong about how Splittable DoFn are handled?
Thanks for your responses
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
