[
https://issues.apache.org/jira/browse/BEAM-8933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17311819#comment-17311819
]
Brian Hulette commented on BEAM-8933:
-------------------------------------
[~isidro.martinez] note there is also
[10384|https://github.com/apache/beam/pull/10384], which is included in 10369.
So we have:
- 10384: Arrow to Beam Row conversion code
- 10369: Adds support for reading Arrow format to BigQueryIO
- 10572: Measure performance of Arrow reads
10572 is a nice to have, but it doesn't need to be part of this work. The
important thing is to revive 10384 and then 10369. This could be done as two
separate PRs as it is now, or you could go ahead and put up one PR for both. I
think two separate PRs is preferable, but however you want to do it is fine.
> BigQuery IO should support reading Arrow format over Storage API
> ----------------------------------------------------------------
>
> Key: BEAM-8933
> URL: https://issues.apache.org/jira/browse/BEAM-8933
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Reporter: Kirill Kozlov
> Priority: P3
> Time Spent: 13h
> Remaining Estimate: 0h
>
> As of right now BigQuery uses Avro format for reading and writing.
> We should add a config to BigQueryIO to specify which format to use: Arrow or
> Avro (with Avro as default).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
