GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/13919
[SPARK-16222] [SQL] JDBC Sources - Handling illegal input values for
`fetchsize` and `batchsize`
#### What changes were proposed in this pull request?
For JDBC data sources, users can specify `batchsize` for multi-row inserts
and `fetchsize` for multi-row fetch. A few issues exist:
- The property keys are case sensitive. Thus, the existing test cases for
`fetchsize` use incorrect names. Basically, the test cases are broken.
- No test cases exist for `batchsize`.
- We do not detect the illegal input values for `fetchsize` and
`batchsize`.
For example, when `batchsize` is zero, we got the following exception:
```
Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most
recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost):
java.lang.ArithmeticException: / by zero
```
when `fetchsize` is less than zero, we got the exception from the
underlying JDBC driver:
```
Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most
recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost):
org.h2.jdbc.JdbcSQLException: Invalid value "-1" for parameter "rows"
[90008-183]
```
This PR fixes all the above issues, and issue the appropriate exceptions
when detecting the illegal inputs for `fetchsize` and `batchsize`. Also update
the function descriptions.
#### How was this patch tested?
Test cases are fixed and added.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark jdbcProperties
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13919.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13919
----
commit 852c0e9bd7ecebf1bce1906ad641d8f714b9fe45
Author: gatorsmile <[email protected]>
Date: 2016-06-27T01:44:25Z
batchsize and fetchsize
commit cf2ce283407c62f3fd4d64cf380ddfcf262d7ecc
Author: gatorsmile <[email protected]>
Date: 2016-06-27T02:18:36Z
fix
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
