[
https://issues.apache.org/jira/browse/BEAM-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333413#comment-16333413
]
Jean-Baptiste Onofré commented on BEAM-3506:
--------------------------------------------
I already have a branch with {{withBatchSize()}} option on the Write
PTransform. However, I like your idea: the user can have different batch size
depending of the collections. The same can apply for other IOs as well. I will
do the change as well. Thanks !
> JdbcIO: Support writing iterables (i.e. collections) of rows instead of only
> single rows
> ----------------------------------------------------------------------------------------
>
> Key: BEAM-3506
> URL: https://issues.apache.org/jira/browse/BEAM-3506
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-extensions
> Affects Versions: 2.3.0
> Reporter: Knut Olav Loite
> Assignee: Jean-Baptiste Onofré
> Priority: Minor
> Labels: JdbcIO, jdbc
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> The current JdbcIO write interface expects a PCollection<T> where T is the
> row to be written. Each instance of T is then added to a batch and written to
> the database. The user has little control over how many rows will be added to
> one batch. If JdbcIO would also support writing a PCollection<Iterable<T>>
> the user would have more control over the number of rows in one batch.
> Especially when writing to cloud databases, such as Google Cloud Spanner, the
> batching of multiple rows together is important for performance.
> I already have a solution locally and I will submit a pull request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
