[
https://issues.apache.org/jira/browse/BEAM-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Kirpichov closed BEAM-3683.
----------------------------------
Resolution: Fixed
Fix Version/s: 2.4.0
> Support BigQuery column-based time partitioning
> -----------------------------------------------
>
> Key: BEAM-3683
> URL: https://issues.apache.org/jira/browse/BEAM-3683
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Reporter: Eugene Kirpichov
> Assignee: Eugene Kirpichov
> Priority: Major
> Fix For: 2.4.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> BigQuery now supports tables partitioned by a DATE or TIMESTAMP column. This
> is very useful for backfilling, because now it doesn't require 1 load job per
> partition (1 load job for the whole table is fine now), and in case of
> BigQueryIO.write(), doesn't require using DynamicDestinations - one only
> needs to specify which field to partition on.
> It is specified via TimePartitioning.field:
> [https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load]
> (configuration.load.timePartitioning.field).
> Seems that the only thing that's needed is to update the BigQuery client -
> then users can use BigQueryIO.write().withTimePartitioning() in some cases
> where they previously needed to use write().to(DynamicDestinations).
> Plus publicity (e.g. a StackOverflow answer)
> CC: [~reuvenlax] [~chamikara]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
