[
https://issues.apache.org/jira/browse/SPARK-49674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicholas Rushton updated SPARK-49674:
-------------------------------------
Priority: Major (was: Blocker)
> Unable to use alias with LATERAL VIEW in Spark SQL
> --------------------------------------------------
>
> Key: SPARK-49674
> URL: https://issues.apache.org/jira/browse/SPARK-49674
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.5.2
> Reporter: Nicholas Rushton
> Priority: Major
>
> When using Spark SQL with a lateral view as a source and an alias with a
> projection fails to parse:
> {code:java}
> scala> spark.sql("SELECT ELEMENT_AT(arr,1) as c FROM LATERAL VIEW OUTER
> INLINE(ARRAYS_ZIP(ARRAY(ARRAY(1,2),ARRAY(3,4)))) `t` as `arr`").show
> org.apache.spark.sql.catalyst.parser.ParseException:
> [PARSE_SYNTAX_ERROR] Syntax error at or near 'INLINE'.(line 1, pos 54)== SQL
> ==
> SELECT ELEMENT_AT(arr,1) as c FROM LATERAL VIEW OUTER
> INLINE(ARRAYS_ZIP(ARRAY(ARRAY(1,2),ARRAY(3,4)))) `t` as `arr`
> ------------------------------------------------------^^^ at
> org.apache.spark.sql.catalyst.parser.ParseException.withCommand(parsers.scala:257)
> at
> org.apache.spark.sql.catalyst.parser.AbstractParser.parse(parsers.scala:98)
> at
> org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:54)
> at
> org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(AbstractSqlParser.scala:68)
> at org.apache.spark.sql.SparkSession.$anonfun$sql$5(SparkSession.scala:684)
> at
> org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:138)
> at org.apache.spark.sql.SparkSession.$anonfun$sql$4(SparkSession.scala:683)
> at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:682)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:713)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:744)
> ... 47 elided {code}
> without the alias, the query works fine:
> {code:java}
> scala> spark.sql("SELECT ELEMENT_AT(arr,1) FROM LATERAL VIEW OUTER
> INLINE(ARRAYS_ZIP(ARRAY(ARRAY(1,2),ARRAY(3,4)))) `t` as `arr`").show
> +----+
> |FROM|
> +----+
> | 1|
> | 3|
> +----+ {code}
> side note: the assigned alias in the results is FROM, which doesn't seem
> right.
> It seems the SQL is not being parsed correctly in this case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
