GitHub user sureshthalamati opened a pull request:
https://github.com/apache/spark/pull/8676
[SPARK-9078] [SQL] Allow jdbc dialects to override the query used to check
the table.
Current implementation uses query with a LIMIT clause to find if table
already exists. This syntax works only in some database systems. This patch
changes the default query to the one that is likely to work on most databases,
and adds a new method to the JdbcDialect abstract class to allow dialects to
override the default query.
I looked at using the JDBC meta data calls, it turns out there is no common
way to find the current schema, catalog..etc. There is a new method
Connection.getSchema() , but that is available only starting jdk1.7 , and
existing jdbc drivers may not have implemented it. Other option was to use
jdbc escape syntax clause for LIMIT, not sure on how well this supported in all
the databases also. After looking at all the jdbc metadata options my
conclusion was most common way is to use the simple select query with 'where 1
=0' , and allow dialects to customize as needed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sureshthalamati/spark table_exists_spark-9078
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8676.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8676
----
commit d4787548cc0ec9408c36deaa64443554d19e7f5f
Author: sureshthalamati <[email protected]>
Date: 2015-09-10T01:35:24Z
Modifying query to check table exists to be more generic, and allow dialect
implementations to specify the query.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
