Nicholas Rushton created SPARK-49674:
----------------------------------------
Summary: Unable to use alias with LATERAL VIEW
Key: SPARK-49674
URL: https://issues.apache.org/jira/browse/SPARK-49674
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.5.2
Reporter: Nicholas Rushton
When using Spark SQL with a lateral view as a source and an alias with a
projection fails to parse:
{code:java}
scala> spark.sql("SELECT ELEMENT_AT(arr,1) as c FROM LATERAL VIEW OUTER
INLINE(ARRAYS_ZIP(ARRAY(ARRAY(1,2),ARRAY(3,4)))) `t` as `arr`").show
org.apache.spark.sql.catalyst.parser.ParseException:
[PARSE_SYNTAX_ERROR] Syntax error at or near 'INLINE'.(line 1, pos 54)== SQL ==
SELECT ELEMENT_AT(arr,1) as c FROM LATERAL VIEW OUTER
INLINE(ARRAYS_ZIP(ARRAY(ARRAY(1,2),ARRAY(3,4)))) `t` as `arr`
------------------------------------------------------^^^ at
org.apache.spark.sql.catalyst.parser.ParseException.withCommand(parsers.scala:257)
at org.apache.spark.sql.catalyst.parser.AbstractParser.parse(parsers.scala:98)
at
org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:54)
at
org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(AbstractSqlParser.scala:68)
at org.apache.spark.sql.SparkSession.$anonfun$sql$5(SparkSession.scala:684)
at
org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:138)
at org.apache.spark.sql.SparkSession.$anonfun$sql$4(SparkSession.scala:683)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:682)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:713)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:744)
... 47 elided {code}
without the alias, the query works fine:
{code:java}
scala> spark.sql("SELECT ELEMENT_AT(arr,1) FROM LATERAL VIEW OUTER
INLINE(ARRAYS_ZIP(ARRAY(ARRAY(1,2),ARRAY(3,4)))) `t` as `arr`").show
+----+
|FROM|
+----+
| 1|
| 3|
+----+ {code}
note: the assigned alias in the results is FROM, which doesn't seem right.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
