[
https://issues.apache.org/jira/browse/BEAM-9051?focusedWorklogId=371734&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-371734
]
ASF GitHub Bot logged work on BEAM-9051:
----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Jan/20 17:45
Start Date: 14/Jan/20 17:45
Worklog Time Spent: 10m
Work Description: RyanBerti commented on issue #10540: [BEAM-9051]
BigQueryUtils toBeamRow support the Avro timestamp-millis logical type
URL: https://github.com/apache/beam/pull/10540#issuecomment-574292337
@TheNeuralBit looks like AvroUtils.toBeamRowStrict can replace
BigQueryUtils.toBeamRow in our use case; and the implementation of logical
types in Beam schemas makes sense. I'm going to close this PR and the
associated tickets which add features to BigQueryUtils.toBeamRow. Sounds like
we should get some more info from @kennknowles about whether the avro support
in BigQueryUtils should be deprecated? Having the two implementations (with the
BigQueryUtils implementation not supporting as many Avro types as AvroUtils)
can be misleading.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 371734)
Time Spent: 1h 50m (was: 1h 40m)
> BigQueryUtils toBeamRow to support GenericRecords with millisecond precision
> timestamps
> ---------------------------------------------------------------------------------------
>
> Key: BEAM-9051
> URL: https://issues.apache.org/jira/browse/BEAM-9051
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Affects Versions: 2.16.0
> Reporter: Ryan Berti
> Assignee: Ryan Berti
> Priority: Minor
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently, BigQueryUtils assumes all Avro GenericRecords which utilize
> timestamp fields include timestamp data in microseconds when converting from
> GenericRecords to Beam Rows
> ([https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java#L568]).
> The implementation should also support timestamp data in milliseconds; this
> should be feasible by referencing the LogicalType associated with the avro
> field to determine if the underlying value is represented in millis or micros.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
