GitHub user ilganeli opened a pull request:
https://github.com/apache/spark/pull/16685
[SPARK-19935] Introduce insert, update, and upsert commands to the
JdbcUtils class
## What changes were proposed in this pull request?
Adds the ability to perform an insert, update, or update command to the
JdbcUtils class which supports writing DataFrames to databases via JDBC
This functionality has not existed heretofore within Spark and doing an
Upsert efficiently is generally difficult. The method presented here strikes a
reasonable balance between simplicity and performance and has shown reasonably
efficient scaling. The insert operation, while already existing, is implemented
slightly differently in this approach to be consistent with how update is
implemented.
## How was this patch tested?
This functionality has been tested through extensive manual testing and
tuning while developing this patch. If the committers believe that this is a
valuable addition, I will be happy to add additional unit tests around this
feature.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ilganeli/spark SPARK-19935
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16685.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16685
----
commit 8d499fe908c29f3b84236315a65e9221ae08cb14
Author: Ilya Ganelin <[email protected]>
Date: 2017-01-24T00:28:16Z
Introduce insert, update, and upsert commands to the JdbcUtils class
commit 89cef373077283627cc896dce4ab95c9d5aa41de
Author: Ilya Ganelin <[email protected]>
Date: 2017-01-24T00:32:49Z
Extra line
commit e1fc6f6697a00567015c47d13173ec4976e7cbb3
Author: Ilya Ganelin <[email protected]>
Date: 2017-01-24T00:37:12Z
Fixed merge conflicts
commit a64719b2c0b687cbe0b854d4a0c5e6e02f75a0bc
Author: Ilya Ganelin <[email protected]>
Date: 2017-01-24T00:39:13Z
Reverted changes to df writer
commit ca494ebdf9110b67c96fc1c3df8463a4d63a56da
Author: Ilya Ganelin <[email protected]>
Date: 2017-01-24T00:39:46Z
Reverted changes to savemode
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
