[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[ https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105439#comment-16105439 ] Paul Wu commented on SPARK-17614: - Oh, sorry. I thought I could use a query hereas I do with other rdbms. Things become complicated for this Cassandra case after I think more on thisI'll accept your comment. > sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra > does not support > - > > Key: SPARK-17614 > URL: https://issues.apache.org/jira/browse/SPARK-17614 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 > Environment: Any Spark Runtime >Reporter: Paul Wu >Assignee: Sean Owen >Priority: Minor > Labels: cassandra-jdbc, sql > Fix For: 2.1.0 > > > I have the code like the following with Cassandra JDBC > (https://github.com/adejanovski/cassandra-jdbc-wrapper): > final String dbTable= "sql_demo"; > Dataset jdbcDF > = sparkSession.read() > .jdbc(CASSANDRA_CONNECTION_URL, dbTable, > connectionProperties); > List rows = jdbcDF.collectAsList(); > It threw the error: > Exception in thread "main" java.sql.SQLTransientException: > com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable > alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...) > at > com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48) > The reason is that the Spark jdbc code uses the sql syntax "where 1=0" > somewhere (to get the schema?), but Cassandra does not support this syntax. > Not sure how this issue can be resolved...this is because CQL is not standard > sql. > The following log shows more information: > 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s; > Rack: %s > 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM > sql_demo WHERE 1=0 > 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322] > com.datastax.driver.core.Statement$1@41ccb3b9 > 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[ https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105431#comment-16105431 ] Sean Owen commented on SPARK-17614: --- Well, it's unrelated to this issue, so this isn't the place. And you seem to be reporting syntax that Cassandra doesn't support, which isn't a Spark issue. > sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra > does not support > - > > Key: SPARK-17614 > URL: https://issues.apache.org/jira/browse/SPARK-17614 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 > Environment: Any Spark Runtime >Reporter: Paul Wu >Assignee: Sean Owen >Priority: Minor > Labels: cassandra-jdbc, sql > Fix For: 2.1.0 > > > I have the code like the following with Cassandra JDBC > (https://github.com/adejanovski/cassandra-jdbc-wrapper): > final String dbTable= "sql_demo"; > Dataset jdbcDF > = sparkSession.read() > .jdbc(CASSANDRA_CONNECTION_URL, dbTable, > connectionProperties); > List rows = jdbcDF.collectAsList(); > It threw the error: > Exception in thread "main" java.sql.SQLTransientException: > com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable > alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...) > at > com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48) > The reason is that the Spark jdbc code uses the sql syntax "where 1=0" > somewhere (to get the schema?), but Cassandra does not support this syntax. > Not sure how this issue can be resolved...this is because CQL is not standard > sql. > The following log shows more information: > 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s; > Rack: %s > 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM > sql_demo WHERE 1=0 > 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322] > com.datastax.driver.core.Statement$1@41ccb3b9 > 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[ https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105425#comment-16105425 ] Paul Wu commented on SPARK-17614: - So create a new issue? Or this is not an issue to you? > sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra > does not support > - > > Key: SPARK-17614 > URL: https://issues.apache.org/jira/browse/SPARK-17614 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 > Environment: Any Spark Runtime >Reporter: Paul Wu >Assignee: Sean Owen >Priority: Minor > Labels: cassandra-jdbc, sql > Fix For: 2.1.0 > > > I have the code like the following with Cassandra JDBC > (https://github.com/adejanovski/cassandra-jdbc-wrapper): > final String dbTable= "sql_demo"; > Dataset jdbcDF > = sparkSession.read() > .jdbc(CASSANDRA_CONNECTION_URL, dbTable, > connectionProperties); > List rows = jdbcDF.collectAsList(); > It threw the error: > Exception in thread "main" java.sql.SQLTransientException: > com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable > alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...) > at > com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48) > The reason is that the Spark jdbc code uses the sql syntax "where 1=0" > somewhere (to get the schema?), but Cassandra does not support this syntax. > Not sure how this issue can be resolved...this is because CQL is not standard > sql. > The following log shows more information: > 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s; > Rack: %s > 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM > sql_demo WHERE 1=0 > 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322] > com.datastax.driver.core.Statement$1@41ccb3b9 > 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[ https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512743#comment-15512743 ] Apache Spark commented on SPARK-17614: -- User 'srowen' has created a pull request for this issue: https://github.com/apache/spark/pull/15196 > sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra > does not support > - > > Key: SPARK-17614 > URL: https://issues.apache.org/jira/browse/SPARK-17614 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 > Environment: Any Spark Runtime >Reporter: Paul Wu > Labels: cassandra-jdbc, sql > > I have the code like the following with Cassandra JDBC > (https://github.com/adejanovski/cassandra-jdbc-wrapper): > final String dbTable= "sql_demo"; > Dataset jdbcDF > = sparkSession.read() > .jdbc(CASSANDRA_CONNECTION_URL, dbTable, > connectionProperties); > List rows = jdbcDF.collectAsList(); > It threw the error: > Exception in thread "main" java.sql.SQLTransientException: > com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable > alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...) > at > com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48) > The reason is that the Spark jdbc code uses the sql syntax "where 1=0" > somewhere (to get the schema?), but Cassandra does not support this syntax. > Not sure how this issue can be resolved...this is because CQL is not standard > sql. > The following log shows more information: > 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s; > Rack: %s > 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM > sql_demo WHERE 1=0 > 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322] > com.datastax.driver.core.Statement$1@41ccb3b9 > 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[ https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510707#comment-15510707 ] Sean Owen commented on SPARK-17614: --- Yup, that much is clearly a bug. Go for a fix, anyone who wants to - or I'll fix that to try to unblock further experimentation. > sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra > does not support > - > > Key: SPARK-17614 > URL: https://issues.apache.org/jira/browse/SPARK-17614 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 > Environment: Any Spark Runtime >Reporter: Paul Wu > Labels: cassandra-jdbc, sql > > I have the code like the following with Cassandra JDBC > (https://github.com/adejanovski/cassandra-jdbc-wrapper): > final String dbTable= "sql_demo"; > Dataset jdbcDF > = sparkSession.read() > .jdbc(CASSANDRA_CONNECTION_URL, dbTable, > connectionProperties); > List rows = jdbcDF.collectAsList(); > It threw the error: > Exception in thread "main" java.sql.SQLTransientException: > com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable > alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...) > at > com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48) > The reason is that the Spark jdbc code uses the sql syntax "where 1=0" > somewhere (to get the schema?), but Cassandra does not support this syntax. > Not sure how this issue can be resolved...this is because CQL is not standard > sql. > The following log shows more information: > 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s; > Rack: %s > 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM > sql_demo WHERE 1=0 > 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322] > com.datastax.driver.core.Statement$1@41ccb3b9 > 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[ https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510709#comment-15510709 ] Paul Wu commented on SPARK-17614: - Create pull request: https://github.com/apache/spark/pull/15183 > sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra > does not support > - > > Key: SPARK-17614 > URL: https://issues.apache.org/jira/browse/SPARK-17614 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 > Environment: Any Spark Runtime >Reporter: Paul Wu > Labels: cassandra-jdbc, sql > > I have the code like the following with Cassandra JDBC > (https://github.com/adejanovski/cassandra-jdbc-wrapper): > final String dbTable= "sql_demo"; > Dataset jdbcDF > = sparkSession.read() > .jdbc(CASSANDRA_CONNECTION_URL, dbTable, > connectionProperties); > List rows = jdbcDF.collectAsList(); > It threw the error: > Exception in thread "main" java.sql.SQLTransientException: > com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable > alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...) > at > com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48) > The reason is that the Spark jdbc code uses the sql syntax "where 1=0" > somewhere (to get the schema?), but Cassandra does not support this syntax. > Not sure how this issue can be resolved...this is because CQL is not standard > sql. > The following log shows more information: > 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s; > Rack: %s > 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM > sql_demo WHERE 1=0 > 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322] > com.datastax.driver.core.Statement$1@41ccb3b9 > 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[ https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510704#comment-15510704 ] Apache Spark commented on SPARK-17614: -- User 'paulzwu' has created a pull request for this issue: https://github.com/apache/spark/pull/15183 > sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra > does not support > - > > Key: SPARK-17614 > URL: https://issues.apache.org/jira/browse/SPARK-17614 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 > Environment: Any Spark Runtime >Reporter: Paul Wu > Labels: cassandra-jdbc, sql > > I have the code like the following with Cassandra JDBC > (https://github.com/adejanovski/cassandra-jdbc-wrapper): > final String dbTable= "sql_demo"; > Dataset jdbcDF > = sparkSession.read() > .jdbc(CASSANDRA_CONNECTION_URL, dbTable, > connectionProperties); > List rows = jdbcDF.collectAsList(); > It threw the error: > Exception in thread "main" java.sql.SQLTransientException: > com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable > alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...) > at > com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48) > The reason is that the Spark jdbc code uses the sql syntax "where 1=0" > somewhere (to get the schema?), but Cassandra does not support this syntax. > Not sure how this issue can be resolved...this is because CQL is not standard > sql. > The following log shows more information: > 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s; > Rack: %s > 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM > sql_demo WHERE 1=0 > 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322] > com.datastax.driver.core.Statement$1@41ccb3b9 > 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[ https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510626#comment-15510626 ] Paul Wu commented on SPARK-17614: - No, Custom JdbcDialect won't resolve the problem since DataFrameReader uses JDBCRDD and the later has a hard code line val statement = conn.prepareStatement(s"SELECT * FROM $table WHERE 1=0") for getting the table existence. See line 61 at https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala > sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra > does not support > - > > Key: SPARK-17614 > URL: https://issues.apache.org/jira/browse/SPARK-17614 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 2.0.0 > Environment: Any Spark Runtime >Reporter: Paul Wu >Priority: Minor > Labels: cassandra-jdbc, sql > > I have the code like the following with Cassandra JDBC > (https://github.com/adejanovski/cassandra-jdbc-wrapper): > final String dbTable= "sql_demo"; > Dataset jdbcDF > = sparkSession.read() > .jdbc(CASSANDRA_CONNECTION_URL, dbTable, > connectionProperties); > List rows = jdbcDF.collectAsList(); > It threw the error: > Exception in thread "main" java.sql.SQLTransientException: > com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable > alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...) > at > com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48) > The reason is that the Spark jdbc code uses the sql syntax "where 1=0" > somewhere (to get the schema?), but Cassandra does not support this syntax. > Not sure how this issue can be resolved...this is because CQL is not standard > sql. > The following log shows more information: > 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s; > Rack: %s > 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM > sql_demo WHERE 1=0 > 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322] > com.datastax.driver.core.Statement$1@41ccb3b9 > 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[
https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510525#comment-15510525
]
Paul Wu commented on SPARK-17614:
-
Thanks. I tried to register my custom dialect as following, but it does not
reach the getTableExistsQuery() method. Could anyone help?
import org.apache.spark.sql.jdbc.JdbcDialect;
public class NRSCassandraDialect extends JdbcDialect {
@Override
public boolean canHandle(String url) {
System.out.println("came here.."+ url.startsWith("jdbc:cassandra"));
return url.startsWith("jdbc:cassandra");
}
@Override
public String getTableExistsQuery (String table) {
System.out.println("query?");
return "SELECT * from " + table + " LIMIT 1";
}
}
--
public class CassJDBC implements Serializable {
private static final org.apache.log4j.Logger LOGGER =
org.apache.log4j.Logger.getLogger(CassJDBC.class);
private static final String _CONNECTION_URL =
"jdbc:cassandra://ulpd326..com/test?loadbalancing=DCAwareRoundRobinPolicy(%22datacenter1%22)";
private static final String _USERNAME = "";
private static final String _PWD = "";
private static final SparkSession sparkSession
= SparkSession.builder() .config("spark.sql.warehouse.dir",
"file:///home/zw251y/tmp").master("local[*]").appName("Spark2JdbcDs").getOrCreate();
public static void main(String[] args) {
JdbcDialects.registerDialect(new NRSCassandraDialect());
final Properties connectionProperties = new Properties();
final String dbTable= "sql_demo";
Dataset jdbcDF
= sparkSession.read()
.jdbc(_CONNECTION_URL, dbTable, connectionProperties);
jdbcDF.show();
}
}
Error message:
came here..true
parameters = "datacenter1"
Exception in thread "main" java.sql.SQLTransientException:
com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable
alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...)
at
com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108)
at
com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371)
at
com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348)
at
com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48)
> sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra
> does not support
> -
>
> Key: SPARK-17614
> URL: https://issues.apache.org/jira/browse/SPARK-17614
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 2.0.0
> Environment: Any Spark Runtime
>Reporter: Paul Wu
>Priority: Minor
> Labels: cassandra-jdbc, sql
>
> I have the code like the following with Cassandra JDBC
> (https://github.com/adejanovski/cassandra-jdbc-wrapper):
> final String dbTable= "sql_demo";
> Dataset jdbcDF
> = sparkSession.read()
> .jdbc(CASSANDRA_CONNECTION_URL, dbTable,
> connectionProperties);
> List rows = jdbcDF.collectAsList();
> It threw the error:
> Exception in thread "main" java.sql.SQLTransientException:
> com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable
> alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...)
> at
> com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108)
> at
> com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371)
> at
> com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348)
> at
> com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48)
> The reason is that the Spark jdbc code uses the sql syntax "where 1=0"
> somewhere (to get the schema?), but Cassandra does not support this syntax.
> Not sure how this issue can be resolved...this is because CQL is not standard
> sql.
> The following log shows more information:
> 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s;
> Rack: %s
> 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM
> sql_demo WHERE 1=0
> 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322]
> com.datastax.driver.core.Statement$1@41ccb3b9
> 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[ https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508059#comment-15508059 ] Josh Rosen commented on SPARK-17614: You should be able to customize the "table existence" query by defining a custom JdbcDialect and registering it with Spark; see https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala > sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra > does not support > - > > Key: SPARK-17614 > URL: https://issues.apache.org/jira/browse/SPARK-17614 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.0.0 > Environment: Any Spark Runtime >Reporter: Paul Wu > Labels: cassandra-jdbc, sql > > I have the code like the following with Cassandra JDBC > (https://github.com/adejanovski/cassandra-jdbc-wrapper): > final String dbTable= "sql_demo"; > Dataset jdbcDF > = sparkSession.read() > .jdbc(CASSANDRA_CONNECTION_URL, dbTable, > connectionProperties); > List rows = jdbcDF.collectAsList(); > It threw the error: > Exception in thread "main" java.sql.SQLTransientException: > com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable > alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...) > at > com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348) > at > com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48) > The reason is that the Spark jdbc code uses the sql syntax "where 1=0" > somewhere (to get the schema?), but Cassandra does not support this syntax. > Not sure how this issue can be resolved...this is because CQL is not standard > sql. > The following log shows more information: > 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s; > Rack: %s > 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM > sql_demo WHERE 1=0 > 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322] > com.datastax.driver.core.Statement$1@41ccb3b9 > 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17614) sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra does not support
[
https://issues.apache.org/jira/browse/SPARK-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15507950#comment-15507950
]
Paul Wu commented on SPARK-17614:
-
Work around: Rebuild the Cassandra JDBC wrapper by modifying
CassandraPreparedStatement.java at
(https://github.com/adejanovski/cassandra-jdbc-wrapper/blob/master/src/main/java/com/github/adejanovski/cassandra/jdbc/CassandraPreparedStatement.java,
pulled 09/20/2016) . Add the following 2 lines before line 87:
this.cql = cql.replace("WHERE 1=0", "limit 1");
cql = this.cql;
> sparkSession.read() .jdbc(***) use the sql syntax "where 1=0" that Cassandra
> does not support
> -
>
> Key: SPARK-17614
> URL: https://issues.apache.org/jira/browse/SPARK-17614
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 2.0.0
> Environment: Any Spark Runtime
>Reporter: Paul Wu
> Labels: cassandra-jdbc, sql
>
> I have the code like the following with Cassandra JDBC
> (https://github.com/adejanovski/cassandra-jdbc-wrapper):
> final String dbTable= "sql_demo";
> Dataset jdbcDF
> = sparkSession.read()
> .jdbc(CASSANDRA_CONNECTION_URL, dbTable,
> connectionProperties);
> List rows = jdbcDF.collectAsList();
> It threw the error:
> Exception in thread "main" java.sql.SQLTransientException:
> com.datastax.driver.core.exceptions.SyntaxError: line 1:29 no viable
> alternative at input '1' (SELECT * FROM sql_demo WHERE [1]...)
> at
> com.github.adejanovski.cassandra.jdbc.CassandraPreparedStatement.(CassandraPreparedStatement.java:108)
> at
> com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:371)
> at
> com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:348)
> at
> com.github.adejanovski.cassandra.jdbc.CassandraConnection.prepareStatement(CassandraConnection.java:48)
> The reason is that the Spark jdbc code uses the sql syntax "where 1=0"
> somewhere (to get the schema?), but Cassandra does not support this syntax.
> Not sure how this issue can be resolved...this is because CQL is not standard
> sql.
> The following log shows more information:
> 16/09/20 13:16:35 INFO CassandraConnection 138: Datacenter: %s; Host: %s;
> Rack: %s
> 16/09/20 13:16:35 TRACE CassandraPreparedStatement 98: CQL: SELECT * FROM
> sql_demo WHERE 1=0
> 16/09/20 13:16:35 TRACE RequestHandler 71: [19400322]
> com.datastax.driver.core.Statement$1@41ccb3b9
> 16/09/20 13:16:35 TRACE RequestHandler 272: [19400322-1] Starting
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
