[
https://issues.apache.org/jira/browse/BEAM-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16032409#comment-16032409
]
Eric Johston commented on BEAM-2390:
------------------------------------
My initial commit had some errors in that
a) TimePartitioning is not serializable, and
b) CreateTables#possibleCreate fails when TableDestination includes a partition
with $
I've made changes to these by storing TimePartitioning in Json format similar
to how the schemas are propagated. I've also modified the table creation such
that when creating a table Beam only looks at the part before $. It seems to be
working now (running this from my own fork in production)
> allow user to use .setTimePartitioning in BigQueryIO.write
> ----------------------------------------------------------
>
> Key: BEAM-2390
> URL: https://issues.apache.org/jira/browse/BEAM-2390
> Project: Beam
> Issue Type: Improvement
> Components: beam-model-runner-api
> Affects Versions: 2.0.0
> Reporter: Eric Johston
> Assignee: Kenneth Knowles
> Labels: easyfix, features, newbie
> Fix For: 2.0.0
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Currently when writing to a table with BigQueryIO sink, there is no way to
> create a new table that is date partitioned. This would be very useful, since
> currently the only way to do this is by manually creating a table ahead of
> time. We should be able to leverage the automatic table creation
> functionality for date partitioned tables.
> The best way to do this would be to have a withTimePartitioning method in the
> BigQueryIO class.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
