Ben Birt created BEAM-10021:
-------------------------------
Summary: BigQueryUtils DATETIME conversion requires unix millis
Key: BEAM-10021
URL: https://issues.apache.org/jira/browse/BEAM-10021
Project: Beam
Issue Type: Bug
Components: beam-model
Reporter: Ben Birt
I'm trying to run a very simple GCP Dataflow SQL job (copy all columns from
table A to B), and it's failing with the following exception:
{code:java}
Caused by: java.lang.ClassCastException: org.apache.avro.util.Utf8 cannot be
cast to java.lang.Longat
org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.truncateToMillis
(BigQueryUtils.java:643)at
org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.convertAvroFormat
(BigQueryUtils.java:597)at
org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.lambda$toBeamRow$5
(BigQueryUtils.java:389)
{code}
It looks to me like this is because BigQueryUtils.convertAvroFormat expects
that any value whose (beam) type is "DATETIME" must be expressed in unix
millis. However, these values come out of BigQuery as a string, (AFAICT) in ISO
8601 format.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
