Abacn commented on code in PR #31823:
URL: https://github.com/apache/beam/pull/31823#discussion_r1672411824
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java:
##########
@@ -413,8 +413,15 @@ public static String toTableSpec(TableReference ref) {
}
public static String dataCatalogName(TableReference ref) {
- return String.format(
- "bigquery:%s.%s.%s", ref.getProjectId(), ref.getDatasetId(),
ref.getTableId());
+ String tableIdBase;
+ int ix = ref.getTableId().indexOf('$');
+ if (ix == -1) {
+ tableIdBase = ref.getTableId();
+ } else {
+ tableIdBase = ref.getTableId().substring(0, ix);
+ }
+ System.out.println("XXX " + ref.getTableId() + " " + ix + tableIdBase);
Review Comment:
is this `System.out.println` a leftover, or one should use slf4j logger?
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWriteUnshardedRecords.java:
##########
@@ -1089,6 +1098,8 @@ public void process(
initializedDatasetService,
initializedWriteStreamService,
pipelineOptions.as(BigQueryOptions.class)));
+ Lineage.getSinks()
Review Comment:
TableUrn has format "projects/%s/datasets/%s/tables/%s". We should be able
to recover the table reference from urn so no need to change DestinationState?
Otherwise could the change on DestinationState breaking upgrade compatibility?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
