GitHub user mchalek opened a pull request:
https://github.com/apache/spark/pull/14420
[SPARK-14204] [SQL] register driverClass rather than user-specified class
This is a pull request that was originally merged against branch-1.6 as
#12000, now being merged into master as well. @srowen @zzcclp @JoshRosen
This pull request fixes an issue in which cluster-mode executors fail to
properly register a JDBC driver when the driver is provided in a jar by the
user, but the driver class name is derived from a JDBC URL (rather than
specified by the user). The consequence of this is that all JDBC accesses under
the described circumstances fail with an IllegalStateException. I reported the
issue here: https://issues.apache.org/jira/browse/SPARK-14204
My proposed solution is to have the executors register the JDBC driver
class under all circumstances, not only when the driver is specified by the
user.
This patch was tested manually. I built an assembly jar, deployed it to a
cluster, and confirmed that the problem was fixed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mchalek/spark mchalek-jdbc_driver_registration
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14420.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14420
----
commit c0bd08ef36b8b4ed073fcdc2f88927389593e47f
Author: Kevin McHale <[email protected]>
Date: 2016-03-23T17:11:54Z
register driverClass rather than user class
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
