Github user rick-ibm commented on the pull request:
https://github.com/apache/spark/pull/9202#issuecomment-152638711
Thanks for that feedback, Michael. To answer your question:
MA> I haven't looked closely at the implementation, but one high level
question is whether this is breaking the use case where a users gives a
subquery instead of a table name (i.e. dbtable = (SELECT ...).) This is an
important part of the API that we can't break.
Reynold posed the same question at
https://issues.apache.org/jira/browse/SPARK-10857?focusedCommentId=14967827&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14967827
and I replied at
https://issues.apache.org/jira/browse/SPARK-10857?focusedCommentId=14967930&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14967930
Can you give me an example of the syntax you have in mind which might trip
this problem? I tried the following experiment...
val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:derby:/Users/rhillegas/derby/databases/derby1",
"dbtable" -> "select * from app.t")).load()
...but that statement doesn't tickle JdbcDialect.getTableExistsQuery().
Instead, the statement fails because the following ungrammatical query is sent
to the database:
SELECT * FROM select * from app.t WHERE 1=0
That, in turn, is because JDBCRDD.resolveTable() has its own hard-coded
query which probes for the existence of a table. I have logged
https://issues.apache.org/jira/browse/SPARK-11426 to track this issue. The
header comment on JDBCRDD.resolveTable() indicates that the method expects a
table name and not an arbitrary query.
Thanks,
-Rick
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
