prodriguezdefino commented on code in PR #32529:
URL: https://github.com/apache/beam/pull/32529#discussion_r1799964018
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/providers/BigQueryStorageWriteApiSchemaTransformProvider.java:
##########
@@ -498,5 +563,52 @@ BigQueryIO.Write<Row>
createStorageWriteApiTransform(Schema schema) {
return write;
}
+
+ BigQueryIO.Write<Row> validateAndIncludeCDCInformation(
+ BigQueryIO.Write<Row> write, Schema schema) {
+ checkArgument(
+
schema.getFieldNames().containsAll(Arrays.asList(ROW_PROPERTY_MUTATION_INFO,
"record")),
+ "When writing using CDC functionality, we expect Row Schema with a "
+ + "\""
+ + ROW_PROPERTY_MUTATION_INFO
+ + "\" Row field and a \"record\" Row field.");
+ checkArgument(
+ schema
+ .getField(ROW_PROPERTY_MUTATION_INFO)
+ .getType()
+ .getRowSchema()
+ .equals(ROW_SCHEMA_MUTATION_INFO),
+ "When writing using CDC functionality, we expect a \""
+ + ROW_PROPERTY_MUTATION_INFO
+ + "\" field of Row type with fields \""
+ + ROW_PROPERTY_MUTATION_TYPE
+ + "\" and \""
+ + ROW_PROPERTY_MUTATION_SQN
+ + "\" both of type string.");
+
+ String tableDestination = null;
+
+ if (configuration.getTable().equals(DYNAMIC_DESTINATIONS)) {
+ validateDynamicDestinationsExpectedSchema(schema);
+ } else {
+ tableDestination = configuration.getTable();
+ }
+
+ return write
+ .to(
+ new CdcWritesDynamicDestination(
+ schema.getField("record").getType().getRowSchema(),
+ tableDestination,
+ configuration.getPrimaryKey()))
Review Comment:
Changed this to use `RowDynamicDestinations` after modifications, but the
`.to(...)` set is needed because of previous comment's response.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
