[
https://issues.apache.org/jira/browse/BEAM-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pascal GILLET updated BEAM-12479:
---------------------------------
Description:
{{UnsupportedOperationExceptions}} are thrown in
{{org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils#toBeamValue(FieldType,
Object)}}
when reading from BigQuery tables with
{{org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO#readTableRowsWithSchema()}}
and converting the returned {{TableRows}} to Beam {{Rows}}
Example:
{code:java}
PCollection<Row> rows =
pipeline
.apply(
"Read from BigQuery table",
BigQueryIO.readTableRowsWithSchema().from(String.format("%s:%s.%s", project,
dataset, table)))
.apply(Convert.toRows());{code}
{{UnsupportedOperationException}} messages are of the type:
{{Converting BigQuery type "java.lang.Boolean" to "BOOLEAN" is not supported}}
{{Converting BigQuery type "java.lang.Double" to "DOUBLE" is not supported}}
And so on for other numeric types...while the conversion of these Java types
should be straightforward.
Indeed, the method {{BigQueryUtils#toBeamValue(FieldType, Object)}} expects
only {{String}} objects or {{Collections}} of {{Strings}}.
I propose a pull request to also support {{Number}} and {{Boolean}} objects.
was:
{{UnsupportedOperationExceptions}} are thrown in
{{org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils#toBeamValue(FieldType,
Object)}}
when reading from BigQuery tables with
{{org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO#readTableRowsWithSchema()}}
and converting the returned {{TableRows}} to Beam {{Rows}}
Example:
{code:java}
PCollection<Row> rows =
pipeline
.apply(
"Read from BigQuery table",
BigQueryIO.readTableRowsWithSchema().from(String.format("%s:%s.%s", project,
dataset, table)))
.apply(Convert.toRows());{code}
{{UnsupportedOperationException}} messages are of the type:
{{Converting BigQuery type "java.lang.Boolean" to "BOOLEAN" is not supported}}
{{Converting BigQuery type "java.lang.Double" to "DOUBLE" is not supported}}
And so on for numeric types...
Indeed, the method {{BigQueryUtils#toBeamValue(FieldType, Object)}} expects
only {{String}} objects or {{Collections}} of {{Strings}}.
I propose a pull request to also support {{Number}} and {{Boolean}} objects.
> UnsupportedOperationException when reading from BigQuery tables and
> converting TableRows to Beam Rows
> -----------------------------------------------------------------------------------------------------
>
> Key: BEAM-12479
> URL: https://issues.apache.org/jira/browse/BEAM-12479
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.29.0
> Environment: Beam Java SDK
> Reporter: Pascal GILLET
> Assignee: Pascal GILLET
> Priority: P2
> Labels: pull-request-available
> Fix For: 2.32.0
>
>
> {{UnsupportedOperationExceptions}} are thrown in
> {{org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils#toBeamValue(FieldType,
> Object)}}
> when reading from BigQuery tables with
> {{org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO#readTableRowsWithSchema()}}
> and converting the returned {{TableRows}} to Beam {{Rows}}
> Example:
> {code:java}
> PCollection<Row> rows =
> pipeline
> .apply(
> "Read from BigQuery table",
> BigQueryIO.readTableRowsWithSchema().from(String.format("%s:%s.%s", project,
> dataset, table)))
> .apply(Convert.toRows());{code}
>
> {{UnsupportedOperationException}} messages are of the type:
> {{Converting BigQuery type "java.lang.Boolean" to "BOOLEAN" is not supported}}
> {{Converting BigQuery type "java.lang.Double" to "DOUBLE" is not supported}}
> And so on for other numeric types...while the conversion of these Java types
> should be straightforward.
> Indeed, the method {{BigQueryUtils#toBeamValue(FieldType, Object)}} expects
> only {{String}} objects or {{Collections}} of {{Strings}}.
> I propose a pull request to also support {{Number}} and {{Boolean}} objects.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
