[ https://issues.apache.org/jira/browse/BEAM-2879?focusedWorklogId=339673&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339673 ]
ASF GitHub Bot logged work on BEAM-2879: ---------------------------------------- Author: ASF GitHub Bot Created on: 07/Nov/19 01:22 Start Date: 07/Nov/19 01:22 Worklog Time Spent: 10m Work Description: steveniemitz commented on issue #9665: [BEAM-2879] Support writing data to BigQuery via avro URL: https://github.com/apache/beam/pull/9665#issuecomment-550576486 > Thanks. Looks great to me. > > To make sure I understood correctly, this will not be enabled for existing users by default and to enable this users have to specify withAvroFormatFunction(), correct ? Correct, with schemas I think we could make this enabled transparently, but for now its opt-in only. > Also, can we add a version of BigQueryIOIT so that we can continue to monitor both Avro and JSON based BQ write transforms ? Yeah I can add that in there. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 339673) Time Spent: 2h 50m (was: 2h 40m) > Implement and use an Avro coder rather than the JSON one for intermediary > files to be loaded in BigQuery > -------------------------------------------------------------------------------------------------------- > > Key: BEAM-2879 > URL: https://issues.apache.org/jira/browse/BEAM-2879 > Project: Beam > Issue Type: Improvement > Components: io-java-gcp > Reporter: Black Phoenix > Assignee: Steve Niemitz > Priority: Minor > Labels: starter > Time Spent: 2h 50m > Remaining Estimate: 0h > > Before being loaded in BigQuery, temporary files are created and encoded in > JSON. Which is a costly solution compared to an Avro alternative -- This message was sent by Atlassian Jira (v8.3.4#803005)