[
https://issues.apache.org/jira/browse/BEAM-11047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonas Grabber updated BEAM-11047:
---------------------------------
Description:
For some reason, convertRequiredField used in
BigQueryAvroUtils.convertGenericRecordToTableRow [casts values with the Avro
INTEGER type to Long, but then converts them to String objects via
toString|https://github.com/apache/beam/blob/v2.23.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryAvroUtils.java#L326].
I'm quite unsure where convertGenericRecordToTableRow is used elsewhere, but we
use it to utilize BigQuery's Storage API reads.
I'm fairly certain this is not expected behaviour because other types are
converted properly and due to the cast to Long it's ensured that any values
will fit into Long objects anyways.
was:
For some reason, `convertRequiredField` used in
`BigQueryAvroUtils.convertGenericRecordToTableRow` casts values with the Avro
`INTEGER` type to `Long`, but then converts them to `String` objects via
`toString`.
I'm quite unsure where `convertGenericRecordToTableRow` is used elsewhere, but
we use it to utilize BigQuery's Storage API reads.
I'm fairly certain this is not expected behaviour because other types are
converted properly and due to the cast to `Long` it's ensured that any values
will fit into `Long` objects anyways.
> BigQuery IO: Avro INTEGER values get converted to String objects
> ----------------------------------------------------------------
>
> Key: BEAM-11047
> URL: https://issues.apache.org/jira/browse/BEAM-11047
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.22.0, 2.23.0, 2.24.0
> Reporter: Jonas Grabber
> Priority: P2
> Labels: bigquery, java
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For some reason, convertRequiredField used in
> BigQueryAvroUtils.convertGenericRecordToTableRow [casts values with the Avro
> INTEGER type to Long, but then converts them to String objects via
> toString|https://github.com/apache/beam/blob/v2.23.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryAvroUtils.java#L326].
> I'm quite unsure where convertGenericRecordToTableRow is used elsewhere, but
> we use it to utilize BigQuery's Storage API reads.
> I'm fairly certain this is not expected behaviour because other types are
> converted properly and due to the cast to Long it's ensured that any values
> will fit into Long objects anyways.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
