[
https://issues.apache.org/jira/browse/BEAM-9321?focusedWorklogId=396290&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396290
]
ASF GitHub Bot logged work on BEAM-9321:
----------------------------------------
Author: ASF GitHub Bot
Created on: 02/Mar/20 18:37
Start Date: 02/Mar/20 18:37
Worklog Time Spent: 10m
Work Description: regadas commented on issue #10869: [BEAM-9321] Add
BigQuery Avro logical type support on write
URL: https://github.com/apache/beam/pull/10869#issuecomment-593551872
Hi, I think we need to consider a few things:
1) in BigQuery this is not the default behavior; Users need to explicitly
enable `useAvroLogicalTypes` to have their types converted to the data types
mentioned in the doc as opposed to the underlying avro raw type. This means
that tables will have different schemas!
2) The previous Beam version was already released with this; Logical Types
are not respected and created Table will have schemas where types match the
avro raw types.
It makes sense that logical types should be respected by default (it option
to disable), but because of the previous points I don't think we can. or even
if it's a good idea.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 396290)
Time Spent: 1.5h (was: 1h 20m)
> BigQuery avro write logical type support
> ----------------------------------------
>
> Key: BEAM-9321
> URL: https://issues.apache.org/jira/browse/BEAM-9321
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.19.0
> Reporter: Filipe Regadas
> Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> With 2.18.0 we are able to write GenericRecords to BigQuery. However, writing
> does not respect Avro <-> BigQuery data type conversion
> ([docs|https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#logical_types])
> we need to set the useAvroLogicalTypes option.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
