[
https://issues.apache.org/jira/browse/BEAM-12164?focusedWorklogId=750742&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-750742
]
ASF GitHub Bot logged work on BEAM-12164:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Mar/22 04:47
Start Date: 31/Mar/22 04:47
Worklog Time Spent: 10m
Work Description: thiagotnunes commented on a change in pull request
#17200:
URL: https://github.com/apache/beam/pull/17200#discussion_r839164220
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/dofn/ReadChangeStreamPartitionDoFn.java
##########
@@ -146,6 +156,17 @@ public TimestampRange initialRestriction(@Element
PartitionMetadata partition) {
return TimestampRange.of(startTimestamp, endTimestamp);
}
+ @GetSize
+ public double getSize(@Element PartitionMetadata partition, @Restriction
TimestampRange range)
+ throws Exception {
+ final BigDecimal timeGapInSeconds =
+ BigDecimal.valueOf(newTracker(partition,
range).getProgress().getWorkRemaining());
Review comment:
Ah ok. Do you think we could extract the progress tracking into its own
class that is used both here and in the restriction tracker? You would be able
to create a single instance in the setup method and re-use it? I feel it might
be overkill to create a new restriction tracker inside this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 750742)
Time Spent: 65h (was: 64h 50m)
> SpannerIO Change Stream Connector
> ---------------------------------
>
> Key: BEAM-12164
> URL: https://issues.apache.org/jira/browse/BEAM-12164
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Thiago Nunes
> Assignee: Thiago Nunes
> Priority: P2
> Fix For: 2.37.0
>
> Time Spent: 65h
> Remaining Estimate: 0h
>
> We would like to augment the existing Google Cloud SpannerIO connector
> ([https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java)]
> with the support for Spanner Change Streams (CDC). CDC support is just being
> implemented in Spanner and it will be exposed through a gRPC API. We will use
> such API to create a new SpannerIO.readChangeStream(...) implementation.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
