ahmedabu98 commented on code in PR #35737:
URL: https://github.com/apache/beam/pull/35737#discussion_r2285918594
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/providers/BigQueryWriteConfiguration.java:
##########
@@ -58,6 +59,41 @@ public abstract static class Builder {
}
}
+ @AutoValue
+ public abstract static class TimePartitioningConfig implements Serializable {
+ @SchemaFieldDescription("The time partitioning type.")
+ public abstract @Nullable String getType();
+
+ @SchemaFieldDescription(
+ "If not set, the table is partitioned by pseudo column
'_PARTITIONTIME'; if set, the table is partitioned by this field.")
+ public abstract @Nullable String getField();
Review Comment:
Can we do some validation that this field exists in the input PCollection
Schema? Maybe in the constructor of `PortableBigQueryDestinations`
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/providers/PortableBigQueryDestinations.java:
##########
@@ -83,9 +87,40 @@ public String getDestination(@Nullable
ValueInSingleWindow<Row> element) {
@Override
public TableDestination getTable(String destination) {
+ TimePartitioning timePartitioning = null;
+
+ if (timePartitioningConfig != null) {
+ String type = timePartitioningConfig.getType();
+ String field = timePartitioningConfig.getField();
+ Long expirationMs = timePartitioningConfig.getExpirationMs();
+ Boolean requirePartitionFilter =
timePartitioningConfig.getRequirePartitionFilter();
+
+ if (type == null) {
+ throw new IllegalArgumentException(
+ "TimePartitioning 'type' must be specified (DAY, HOUR, MONTH, or
YEAR).");
+ }
Review Comment:
Just remove @Nullable from getType() in the configuration above and you
won't need this.
But please add extra logic to make sure the value of "type" is one of {DAY,
HOUR, MONTH, YEAR}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
