Re: BigQueryUtils improvements for Avro Bytes / Timestamp (millis)
Hello, I've got a PR that affects the java gcp component, specifically BigQueryUtils. Can anyone help me with a review? I've tagged the owner of the component on the PR but haven't heard anything for a week, so I figured I'd send an e-mail to this list. https://issues.apache.org/jira/browse/BEAM-9051 https://github.com/apache/beam/pull/10540 Thanks! Ryan On Thu, Jan 2, 2020 at 6:15 PM Ryan Berti wrote: > Hello, > > I just wanted to send a quick note to the mailing list letting you know I > opened two tickets: > > https://issues.apache.org/jira/browse/BEAM-9051 > https://issues.apache.org/jira/browse/BEAM-9052 > > In both cases, I encountered the lack of functionality after using an > external library to generate Avro GenericRecords from Scala case classes. > These records could be written via ParquetIO without any issues, but when > converting them to Beam Rows, and then BigQuery TableRows (via AvroUtils > and BigQueryUtils), I found that the records were rejected by BigQuery. > I'll setup some unit tests and open PRs in the next week or so to add this > functionality. > > Thanks! > Ryan >
BigQueryUtils improvements for Avro Bytes / Timestamp (millis)
Hello, I just wanted to send a quick note to the mailing list letting you know I opened two tickets: https://issues.apache.org/jira/browse/BEAM-9051 https://issues.apache.org/jira/browse/BEAM-9052 In both cases, I encountered the lack of functionality after using an external library to generate Avro GenericRecords from Scala case classes. These records could be written via ParquetIO without any issues, but when converting them to Beam Rows, and then BigQuery TableRows (via AvroUtils and BigQueryUtils), I found that the records were rejected by BigQuery. I'll setup some unit tests and open PRs in the next week or so to add this functionality. Thanks! Ryan
Contributor Permission for BEAM-8953
Hello, My name is Ryan Berti, working at Quibi. We're using Beam via SCIO, and I've run into a situation where I'd like to contribute some minor improvements to the java sdk, specifically to ParquetIO. I've created BEAM-8953 and would like some feedback on the improvement. My JIRA user name is 'Ryan Berti'. Thanks! Ryan
