Pablo Estrada created BEAM-14161:
------------------------------------
Summary: Add dynamic splitting to JdbcIO.readWithPartitions
Key: BEAM-14161
URL: https://issues.apache.org/jira/browse/BEAM-14161
Project: Beam
Issue Type: Improvement
Components: io-java-jdbc
Reporter: Pablo Estrada
Assignee: Jean-Baptiste Onofré
Fix For: Not applicable
Now, the JDBC IO is basically a {{DoFn}} executed with a {{ParDo}}. So, it
means that parallelism is "limited" and executed on one executor.
We can imagine to create several JDBC {{BoundedSource}}s splitting the SQL
query in subset (for instance using row id paging or any "splitting/limit" we
can figure based on the original SQL query) (something similar to what Sqoop is
doing).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
