[
https://issues.apache.org/jira/browse/SPARK-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin updated SPARK-8930:
-------------------------------
Target Version/s: 1.5.0
> Throw a AnalysisException with meaningful messages if DataFrame#explode takes
> a star in expressions
> ---------------------------------------------------------------------------------------------------
>
> Key: SPARK-8930
> URL: https://issues.apache.org/jira/browse/SPARK-8930
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.4.0
> Reporter: Takeshi Yamamuro
>
> The current implementation throws an exception with meaningless messages if
> DataFrame#explode contain a star '*' (ISTM that explode cannot take a star in
> expressions) like codes blow;
> {code}
> val df = Seq(("1", "1,2"), ("2", "4"), ("3", "7,8,9")).toDF("prefix",
> "csv")
> df.explode($"*") { case Row(prefix: String, csv: String) =>
> csv.split(",").map(v => Tuple1(prefix + ":" + v))
> }
> {code}
> {code}
> [info] - explode takes UnresolvedStar *** FAILED *** (14 milliseconds)
> [info] org.apache.spark.sql.AnalysisException: cannot resolve '_1' given
> input columns prefix, csv;
> [info] at
> org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
> [info] at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$2.applyOrElse(CheckAnalysis.scala:55)
> [info] at
> org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$2.applyOrElse(CheckAnalysis.scala:52)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
