[ https://issues.apache.org/jira/browse/SPARK-21486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095689#comment-16095689 ]
Liang-Chi Hsieh edited comment on SPARK-21486 at 7/21/17 2:34 AM: ------------------------------------------------------------------ Since 2.2.0, it is not allowed to use the qualifier inside a subquery. Please see SPARK-21335. It is considered a wrong SQL query. So, this query works: {code} scala> sql("SELECT ta FROM (SELECT a as ta FROM test a21)").show() +---+ | ta| +---+ | 1| | 2| | 3| +---+ {code} But this query can't work: {code} scala> sql("SELECT a21.ta FROM (SELECT a as ta FROM test a21)").show() org.apache.spark.sql.AnalysisException: cannot resolve '`a21.ta`' given input columns: [__auto_generated_subquery_n ame.ta]; line 1 pos 7; 'Project ['a21.ta] +- SubqueryAlias __auto_generated_subquery_name +- Project [a#7 AS ta#31] +- SubqueryAlias a21 +- SubqueryAlias test +- Project [_1#3 AS a#7, _2#4 AS b#8, _3#5 AS c#9] +- LocalRelation [_1#3, _2#4, _3#5] {code} was (Author: viirya): Since 2.2.0, it is not allowed to use the qualifier inside a subquery. Please see SPARK-21335. So, this query works: {code} scala> sql("SELECT ta FROM (SELECT a as ta FROM test a21)").show() +---+ | ta| +---+ | 1| | 2| | 3| +---+ {code} But this query can't work: {code} scala> sql("SELECT a21.ta FROM (SELECT a as ta FROM test a21)").show() org.apache.spark.sql.AnalysisException: cannot resolve '`a21.ta`' given input columns: [__auto_generated_subquery_n ame.ta]; line 1 pos 7; 'Project ['a21.ta] +- SubqueryAlias __auto_generated_subquery_name +- Project [a#7 AS ta#31] +- SubqueryAlias a21 +- SubqueryAlias test +- Project [_1#3 AS a#7, _2#4 AS b#8, _3#5 AS c#9] +- LocalRelation [_1#3, _2#4, _3#5] {code} > Fail when using aliased column of a aliased table from a subquery > ----------------------------------------------------------------- > > Key: SPARK-21486 > URL: https://issues.apache.org/jira/browse/SPARK-21486 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.1.0, 2.1.1, 2.2.0 > Reporter: Miguel Angel Fernandez Diaz > > SparkSQL cannot resolve a qualified column when using the column alias and > the table alias from a subquery. > Example: > {code:sql} > SELECT a21.ta FROM (SELECT tip_amount as ta FROM yellow a21) > {code} > Error: > {code} > org.apache.spark.sql.AnalysisException: cannot resolve '`a21.ta`' given input > columns: [ta]; line 1 pos 7; > 'Project ['a21.ta] > +- Project [tip_amount#505 AS ta#489] > +- SubqueryAlias a21 > +- SubqueryAlias yellow > +- > Relation[VendorID#490,tpep_pickup_datetime#491,tpep_dropoff_datetime#492,passenger_count#493,trip_distance#494,pickup_longitude#495,pickup_latitude#496,RatecodeID#497,store_and_fwd_flag#498,dropoff_longitude#499,dropoff_latitude#500,payment_type#501,fare_amount#502,extra#503,mta_tax#504,tip_amount#505,tolls_amount#506,improvement_surcharge#507,total_amount#508] > csv > at > org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42) > at > org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$2.applyOrElse(CheckAnalysis.scala:88) > at > org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$2.applyOrElse(CheckAnalysis.scala:85) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformUp$1.apply(TreeNode.scala:289) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformUp$1.apply(TreeNode.scala:289) > at > org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:70) > at > org.apache.spark.sql.catalyst.trees.TreeNode.transformUp(TreeNode.scala:288) > at > org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$transformExpressionsUp$1.apply(QueryPlan.scala:268) > at > org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$transformExpressionsUp$1.apply(QueryPlan.scala:268) > at > org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpression$1(QueryPlan.scala:279) > at > org.apache.spark.sql.catalyst.plans.QueryPlan.org$apache$spark$sql$catalyst$plans$QueryPlan$$recursiveTransform$1(QueryPlan.scala:289) > at > org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$org$apache$spark$sql$catalyst$plans$QueryPlan$$recursiveTransform$1$1.apply(QueryPlan.scala:293) > at > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) > at > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) > at scala.collection.immutable.List.foreach(List.scala:381) > at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) > at scala.collection.immutable.List.map(List.scala:285) > at > org.apache.spark.sql.catalyst.plans.QueryPlan.org$apache$spark$sql$catalyst$plans$QueryPlan$$recursiveTransform$1(QueryPlan.scala:293) > at > org.apache.spark.sql.catalyst.plans.QueryPlan$$anonfun$6.apply(QueryPlan.scala:298) > at > org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:187) > at > org.apache.spark.sql.catalyst.plans.QueryPlan.mapExpressions(QueryPlan.scala:298) > at > org.apache.spark.sql.catalyst.plans.QueryPlan.transformExpressionsUp(QueryPlan.scala:268) > at > org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:85) > at > org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:78) > at > org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:127) > at > org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:78) > at > org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:91) > at > org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:52) > at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:66) > at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:623) > ... 48 elided > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org