[ https://issues.apache.org/jira/browse/SPARK-9078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yin Huai resolved SPARK-9078. ----------------------------- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8676 [https://github.com/apache/spark/pull/8676] > Use of non-standard LIMIT keyword in JDBC tableExists code > ---------------------------------------------------------- > > Key: SPARK-9078 > URL: https://issues.apache.org/jira/browse/SPARK-9078 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.3.1, 1.4.0 > Reporter: Robert Beauchemin > Priority: Minor > Fix For: 1.6.0 > > > tableExists in > spark/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcUtils.scala uses > non-standard SQL (specifically, the LIMIT keyword) to determine whether a > table exists in a JDBC data source. This will cause an exception in many/most > JDBC databases that doesn't support LIMIT keyword. See > http://stackoverflow.com/questions/1528604/how-universal-is-the-limit-statement-in-sql > To check for table existence or an exception, it could be recrafted around > "select 1 from $table where 0 = 1" which isn't the same (it returns an empty > resultset rather than the value '1'), but would support more data sources and > also support empty tables. Arguably ugly and possibly queries every row on > sources that don't support constant folding, but better than failing on JDBC > sources that don't support LIMIT. > Perhaps "supports LIMIT" could be a field in the JdbcDialect class for > databases that support keyword this to override. The ANSI standard is (OFFSET > and) FETCH. > The standard way to check for table existence would be to use > information_schema.tables which is a SQL standard but may not work for other > JDBC data sources that support SQL, but not the information_schema. The JDBC > DatabaseMetaData interface provides getSchemas() that allows checking for > the information_schema in drivers that support it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org