[
https://issues.apache.org/jira/browse/BEAM-6611?focusedWorklogId=287056&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-287056
]
ASF GitHub Bot logged work on BEAM-6611:
----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Aug/19 21:34
Start Date: 01/Aug/19 21:34
Worklog Time Spent: 10m
Work Description: ttanay commented on issue #9216: [BEAM-6611] Move edge
case comment to TriggerCopyJobs
URL: https://github.com/apache/beam/pull/9216#issuecomment-517466582
Run Portable_Python PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 287056)
Time Spent: 10.5h (was: 10h 20m)
> A Python Sink for BigQuery with File Loads in Streaming
> -------------------------------------------------------
>
> Key: BEAM-6611
> URL: https://issues.apache.org/jira/browse/BEAM-6611
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Reporter: Pablo Estrada
> Assignee: Tanay Tummalapalli
> Priority: Major
> Labels: gsoc, gsoc2019, mentor
> Time Spent: 10.5h
> Remaining Estimate: 0h
>
> The Java SDK supports a bunch of methods for writing data into BigQuery,
> while the Python SDK supports the following:
> - Streaming inserts for streaming pipelines [As seen in [bigquery.py and
> BigQueryWriteFn|https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigquery.py#L649-L813]]
> - File loads for batch pipelines [As implemented in [PR
> 7655|https://github.com/apache/beam/pull/7655]]
> Qucik and dirty early design doc: https://s.apache.org/beam-bqfl-py-streaming
> The Java SDK also supports File Loads for Streaming pipelines [see BatchLoads
> application|https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L1709-L1776].
> File loads have the advantage of being much cheaper than streaming inserts
> (although they also are slower for the records to show up in the table).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
