TheNeuralBit commented on a change in pull request #15485:
URL: https://github.com/apache/beam/pull/15485#discussion_r801149024
##########
File path:
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOWriteTest.java
##########
@@ -2320,6 +2322,46 @@ public void testWriteToTableDecorator() throws Exception
{
p.run();
}
+ @Test
+ public void testWriteWithAllowTruncatedTimestamps() throws IOException,
InterruptedException {
+ if (useStorageApi) {
+ // TODO: to support storage API, changes have to be made to
+ // org.apache.beam.sdk.io.gcp.bigquery.BeamRowToStorageApiProto
+ return;
Review comment:
Could this test that we fail gracefully in this case? It would also be
good to file a jira for the follow up work and reference that, e.g.
TODO(BEAM-XXX):
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java
##########
@@ -594,10 +604,32 @@ public static TableRow toTableRow(Row row) {
java.time.format.DateTimeFormatter localDateTimeFormatter =
(0 == localDateTime.getNano()) ? ISO_LOCAL_DATE_TIME :
BIGQUERY_DATETIME_FORMATTER;
return localDateTimeFormatter.format(localDateTime);
- } else if ("Enum".equals(identifier)) {
+ } else if (EnumerationType.IDENTIFIER.equals(identifier)) {
Review comment:
Thanks for the cleanup :)
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
##########
@@ -2372,6 +2377,10 @@ static String getExtractDestinationUri(String
extractDestinationDir) {
return toBuilder().setExtendedErrorInfo(true).build();
}
+ public Write<T> withAllowTruncatedTimestamps() {
Review comment:
Please add a docstring for this
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TestBigQuery.java
##########
@@ -211,7 +211,7 @@ public TableReference tableReference() {
public TableDataInsertAllResponse insertRows(Schema rowSchema, Row... rows)
throws IOException {
List<Rows> bqRows =
Arrays.stream(rows)
- .map(row -> new Rows().setJson(BigQueryUtils.toTableRow(row)))
Review comment:
Could you also add an overload, `BigQueryUtils.toTableRow(row)`, that
preserves the existing API like you did with `BigQueryUtils.toTableRow()`? Then
you wouldn't have to change calls like this one, and we preserve the existing
API which is technically public (although practically no one should be relying
on this).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
