GitHub user jkff opened a pull request:
https://github.com/apache/beam/pull/3800
[BEAM-2803, BEAM-2706] JdbcIO improvements
This solves two issues:
* https://issues.apache.org/jira/browse/BEAM-2803 by changing the strategy
of breaking fusion: instead of a regular reshuffle, materializes via side input
first.
* https://issues.apache.org/jira/browse/BEAM-2706 by introducing
JdbcIO.readAll().
Also brings the validation messages in accordance with the updated guidance
in PTransform Style Guide.
R: @jbonofre
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jkff/incubator-beam jdbcio-improvements
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3800.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3800
----
commit be7b47e286bc441c3edb5f9417a12364e809396b
Author: Eugene Kirpichov <[email protected]>
Date: 2017-08-31T00:22:45Z
[BEAM-2803] Improves performance of JdbcIO for a very large number of query
results
commit e5daf608122345c36eafd010b00c4d940adbff7d
Author: Eugene Kirpichov <[email protected]>
Date: 2017-08-31T00:40:16Z
Introduces JdbcIO.readAll()
Also cleans up validation messages.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
