Pascal GILLET created BEAM-12479:
------------------------------------
Summary: UnsupportedOperationException when reading from BigQuery
tables and converting TableRows to Beam Rows
Key: BEAM-12479
URL: https://issues.apache.org/jira/browse/BEAM-12479
Project: Beam
Issue Type: Bug
Components: io-java-gcp
Affects Versions: 2.29.0
Environment: Beam Java SDK
Reporter: Pascal GILLET
Assignee: Pascal GILLET
Fix For: 2.32.0
{{UnsupportedOperationException}}s are thrown in
{{org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils#toBeamValue(FieldType,
Object)}}
when reading from BigQuery tables with
{{org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO#readTableRowsWithSchema()}}
and converting the returned {{TableRow}}s to Beam {{Row}}s
Example:
{code:java}
PCollection<Row> rows =
pipeline
.apply(
"Read from BigQuery table",
BigQueryIO.readTableRowsWithSchema().from(String.format("%s:%s.%s", project,
dataset, table)))
.apply(Convert.toRows());{code}
{{UnsupportedOperationException}} messages are of the type:
{{Converting BigQuery type "java.lang.Boolean" to "BOOLEAN" is not supported}}
{{Converting BigQuery type "java.lang.Double" to "DOUBLE" is not supported}}
And so on for numeric types...
Indeed, the method {{BigQueryUtils#toBeamValue(FieldType, Object)}} expects
only {{String}} objects or {{Collection}}s of {{String}}s.
I propose a pull request to also support {{Number}} and {{Boolean}} objects.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
