[
https://issues.apache.org/jira/browse/BEAM-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Knut Olav Loite updated BEAM-3506:
----------------------------------
Description:
The current JdbcIO write interface expects a PCollection<T> where T is the row
to be written. Each instance of T is then added to a batch and written to the
database. The user has little control over how many rows will be added to one
batch. If JdbcIO would also support writing a PCollection<Iterable<T>> the user
would have more control over the number of rows in one batch. Especially when
writing to cloud databases, such as Google Cloud Spanner, the batching of
multiple rows together is important for performance.
I already have a solution locally and I will submit a pull request.
was:The current JdbcIO write interface expects a PCollection<T> where T is
the row to be written. Each instance of T is then added to a batch and written
to the database. The user has little control over how many rows will be added
to one batch. If JdbcIO would also support writing a PCollection<Iterable<T>>
the user would have more control over the number of rows in one batch.
Especially when writing to cloud databases, such as Google Cloud Spanner, the
batching of multiple rows together is important for performance.
> JdbcIO: Support writing iterables (i.e. collections) of rows instead of only
> single rows
> ----------------------------------------------------------------------------------------
>
> Key: BEAM-3506
> URL: https://issues.apache.org/jira/browse/BEAM-3506
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-extensions
> Affects Versions: 2.3.0
> Reporter: Knut Olav Loite
> Assignee: Reuven Lax
> Priority: Minor
> Labels: JdbcIO, jdbc
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> The current JdbcIO write interface expects a PCollection<T> where T is the
> row to be written. Each instance of T is then added to a batch and written to
> the database. The user has little control over how many rows will be added to
> one batch. If JdbcIO would also support writing a PCollection<Iterable<T>>
> the user would have more control over the number of rows in one batch.
> Especially when writing to cloud databases, such as Google Cloud Spanner, the
> batching of multiple rows together is important for performance.
> I already have a solution locally and I will submit a pull request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
