[
https://issues.apache.org/jira/browse/BEAM-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091719#comment-17091719
]
Reuven Lax commented on BEAM-9613:
----------------------------------
TableRow is a json object, so fundamentally its values are interpreted as
strings.
It looks like Beam does BQ exports in Avro files, and then converts the Avro to
TableRow objects. If you look at BigQueryUtils.convertAvroPrimitiveType, it
converts the values into primitive objects (Byte, Long, Double, etc.), not into
String objects. I'm assuming this was done to save on memory usage in the
worker.
> BigQuery IO not support convert double type for beam row
> --------------------------------------------------------
>
> Key: BEAM-9613
> URL: https://issues.apache.org/jira/browse/BEAM-9613
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Reporter: MAKSIM TSYGAN
> Priority: Major
>
> If execute query with double column via BigQueryIO.readFrom(), I get
> exception:
> Caused by: java.lang.UnsupportedOperationException: Converting BigQuery type
> 'class java.lang.Double' to 'FieldType\{typeName=DOUBLE, nullable=true,
> logicalType=null, collectionElementType=null, mapKeyType=null,
> mapValueType=null, rowSchema=null, metadata={}}' is not supported
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.toBeamValue(BigQueryUtils.java:532)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.toBeamRowFieldValue(BigQueryUtils.java:483)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.lambda$toBeamRow$6(BigQueryUtils.java:469)
> at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtils.toBeamRow(BigQueryUtils.java:470)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
