[
https://issues.apache.org/jira/browse/SPARK-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262442#comment-14262442
]
Alex Liu commented on SPARK-4943:
---------------------------------
The seq can be like
{code}
tableName, databaseName, clusterName, catalog
{code}
First element is tableName, the next one is databaseName, then clusterName and
catalog(option).
SQL query looks like
{code}
SELECT test_table.column1 FROM catalog.cluster.database.table AS test_table
SELECT test_table.column1 FROM cluster.database.table AS test_table
SELECT test_table.column1 FROM database.table AS test_table
SELECT table.column1 FROM table
{code}
Potentially we should not use AS clause here.
Parser only allow maximum four levels in full table name.
[catalog].[cluster].[database].[table]
> Parsing error for query with table name having dot
> --------------------------------------------------
>
> Key: SPARK-4943
> URL: https://issues.apache.org/jira/browse/SPARK-4943
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.2.0
> Reporter: Alex Liu
>
> When integrating Spark 1.2.0 with Cassandra SQL, the following query is
> broken. It was working for Spark 1.1.0 version. Basically we use the table
> name having dot to include database name
> {code}
> [info] java.lang.RuntimeException: [1.29] failure: ``UNION'' expected but
> `.' found
> [info]
> [info] SELECT test1.a FROM sql_test.test1 AS test1 UNION DISTINCT SELECT
> test2.a FROM sql_test.test2 AS test2
> [info] ^
> [info] at scala.sys.package$.error(package.scala:27)
> [info] at
> org.apache.spark.sql.catalyst.AbstractSparkSQLParser.apply(SparkSQLParser.scala:33)
> [info] at
> org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.scala:79)
> [info] at
> org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.scala:79)
> [info] at
> org.apache.spark.sql.catalyst.SparkSQLParser$$anonfun$org$apache$spark$sql$catalyst$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:174)
> [info] at
> org.apache.spark.sql.catalyst.SparkSQLParser$$anonfun$org$apache$spark$sql$catalyst$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:173)
> [info] at
> scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:136)
> [info] at
> scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:135)
> [info] at
> scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
> [info] at
> scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
> [info] at
> scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
> [info] at
> scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254)
> [info] at
> scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254)
> [info] at
> scala.util.parsing.combinator.Parsers$Failure.append(Parsers.scala:202)
> [info] at
> scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)
> [info] at
> scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)
> [info] at
> scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
> [info] at
> scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
> [info] at
> scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
> [info] at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
> [info] at
> scala.util.parsing.combinator.Parsers$$anon$2.apply(Parsers.scala:890)
> [info] at
> scala.util.parsing.combinator.PackratParsers$$anon$1.apply(PackratParsers.scala:110)
> [info] at
> org.apache.spark.sql.catalyst.AbstractSparkSQLParser.apply(SparkSQLParser.scala:31)
> [info] at
> org.apache.spark.sql.SQLContext$$anonfun$parseSql$1.apply(SQLContext.scala:83)
> [info] at
> org.apache.spark.sql.SQLContext$$anonfun$parseSql$1.apply(SQLContext.scala:83)
> [info] at scala.Option.getOrElse(Option.scala:120)
> [info] at org.apache.spark.sql.SQLContext.parseSql(SQLContext.scala:83)
> [info] at
> org.apache.spark.sql.cassandra.CassandraSQLContext.cassandraSql(CassandraSQLContext.scala:53)
> [info] at
> org.apache.spark.sql.cassandra.CassandraSQLContext.sql(CassandraSQLContext.scala:56)
> [info] at
> com.datastax.spark.connector.sql.CassandraSQLSpec$$anonfun$20.apply$mcV$sp(CassandraSQLSpec.scala:169)
> [info] at
> com.datastax.spark.connector.sql.CassandraSQLSpec$$anonfun$20.apply(CassandraSQLSpec.scala:168)
> [info] at
> com.datastax.spark.connector.sql.CassandraSQLSpec$$anonfun$20.apply(CassandraSQLSpec.scala:168)
> [info] at
> org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
> [info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
> [info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> [info] at org.scalatest.Transformer.apply(Transformer.scala:22)
> [info] at org.scalatest.Transformer.apply(Transformer.scala:20)
> [info] at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1647)
> [info] at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
> [info] at org.scalatest.FlatSpec.withFixture(FlatSpec.scala:1683)
> [info] at
> org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1644)
> [info] at
> org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
> [info] at
> org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
> [info] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
> [info] at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1656)
> [info] at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683)
> [info] at
> org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
> [info] at
> org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
> [info] at
> org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
> [info] at
> org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
> [info] at scala.collection.immutable.List.foreach(List.scala:318)
> [info] at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
> [info] at
> org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
> [info] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
> [info] at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1714)
> [info] at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683)
> [info] at org.scalatest.Suite$class.run(Suite.scala:1424)
> [info] at
> org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683)
> [info] at
> org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
> [info] at
> org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
> [info] at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
> [info] at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1760)
> [info] at org.scalatest.FlatSpec.run(FlatSpec.scala:1683)
> [info] at
> org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:466)
> [info] at
> org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:677)
> [info] at sbt.ForkMain$Run$2.call(ForkMain.java:294)
> [info] at sbt.ForkMain$Run$2.call(ForkMain.java:284)
> [info] at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [info] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [info] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [info] at java.lang.Thread.run(Thread.java:745)
> [info] - should allow to select rows with union distinct clause *** FAILED
> *** (46 milliseconds)
> [info] java.lang.RuntimeException: [1.29] failure: ``UNION'' expected but
> `.' found
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
