ahmedabu98 commented on code in PR #28189:
URL: https://github.com/apache/beam/pull/28189#discussion_r1307928791
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java:
##########
@@ -215,6 +216,7 @@ public void processElement(
firstPaneCreateDisposition,
dynamicDestinations,
destination);
+ LOG.debug("Use TableSchema for {}: {}", destination, tableSchema);
Review Comment:
I like "Fetching tableschema..." because it makes it clear we are getting it
from BQ table, not from an input schema.
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWriteUnshardedRecords.java:
##########
@@ -430,8 +430,13 @@ SchemaAndDescriptor getCurrentTableSchema(String stream,
@Nullable TableSchema u
WriteStream writeStream =
Preconditions.checkStateNotNull(maybeDatasetService).getWriteStream(streamName);
if (writeStream != null && writeStream.hasTableSchema()) {
- currentSchema.set(writeStream.getTableSchema());
+ TableSchema updatedFromStream = writeStream.getTableSchema();
+ currentSchema.set(updatedFromStream);
updated.set(true);
+ LOG.debug(
+ "Updated currentSchema according to stream {}: {}",
+ streamName,
+ updatedFromStream);
Review Comment:
"Fetched updated schema for table {}: \n\t{}" ?
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWriteUnshardedRecords.java:
##########
@@ -430,8 +430,13 @@ SchemaAndDescriptor getCurrentTableSchema(String stream,
@Nullable TableSchema u
WriteStream writeStream =
Preconditions.checkStateNotNull(maybeDatasetService).getWriteStream(streamName);
if (writeStream != null && writeStream.hasTableSchema()) {
- currentSchema.set(writeStream.getTableSchema());
+ TableSchema updatedFromStream = writeStream.getTableSchema();
+ currentSchema.set(updatedFromStream);
updated.set(true);
+ LOG.debug(
+ "Updated currentSchema according to stream {}: {}",
+ streamName,
+ updatedFromStream);
Review Comment:
Also this is in the context of a `DestinationState`, so we can display the
table (similar to what you do in StorageApiWritesShardedRecords) using
`tableUrn`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
