[ https://issues.apache.org/jira/browse/SPARK-11619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15021509#comment-15021509 ]
Wenchen Fan commented on SPARK-11619: ------------------------------------- Actually this line: https://github.com/apache/spark/blob/branch-1.5/sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala#L689 When we use `selectExpr`, we pass in `UnresolvedFunction` to `DataFrame.select` and fall in the last case. A workaround is to do special handling for UDTF like we did for `explode`(and `json_tuple` in 1.6), wrap it with `MultiAlias`. Another workaround is using `expr`, for example, `df.select(expr("explode(a)").as(Nil))`, I think `selectExpr` is no longer needed after we have the `expr` function.... > cannot use UDTF in DataFrame.selectExpr > --------------------------------------- > > Key: SPARK-11619 > URL: https://issues.apache.org/jira/browse/SPARK-11619 > Project: Spark > Issue Type: Bug > Components: SQL > Reporter: Wenchen Fan > Priority: Minor > > Currently if use UDTF like `explode`, `json_tuple` in `DataFrame.selectExpr`, > it will be parsed into `UnresolvedFunction` first, and then alias it with > `expr.prettyString`. However, UDTF may need MultiAlias so we will get error > if we run: > {code} > val df = Seq((Map("1" -> 1), 1)).toDF("a", "b") > df.selectExpr("explode(a)").show() > {code} > [info] org.apache.spark.sql.AnalysisException: Expect multiple names given > for org.apache.spark.sql.catalyst.expressions.Explode, > [info] but only single name ''explode(a)' specified; -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org