[ https://issues.apache.org/jira/browse/SPARK-7067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan updated SPARK-7067: ------------------------------- Description: In order to avoid breaking existing HiveQL queries, the current way we resolve column in ORDER BY is: first resolve based on what comes from the select clause and then fall back on its child only when this fails. However, this case will fail: {code} test("orderby queries") { jsonRDD(sparkContext.makeRDD( """{"a": {"b": [{"c": 1}]}, "b": [{"d": 1}]}""" :: Nil)).registerTempTable("t") sql("SELECT a.b FROM t ORDER BY b[0].d").queryExecution.analyzed } {code} As hive doesn't support resolve ORDER BY attribute not exist in select clause, so this problem is spark sql only. was: In order to avoid breaking existing HiveQL queries, the current way we resolve column in ORDER BY is: first resolve based on what comes from the select clause and then fall back on its child only when this fails. However, this case will fail: {code} test("orderby queries") { jsonRDD(sparkContext.makeRDD( """{"a": {"b": [{"c": 1}]}, "b": [{"d": 1}]}""" :: Nil)).registerTempTable("t") sql("SELECT a.b FROM t ORDER BY b[0].d").queryExecution.analyzed } {code} As hive doesn't support resolve order by attribute not exist in select clause, so this problem is spark sql only. > Can't resolve nested column in ORDER BY > --------------------------------------- > > Key: SPARK-7067 > URL: https://issues.apache.org/jira/browse/SPARK-7067 > Project: Spark > Issue Type: Bug > Components: SQL > Reporter: Wenchen Fan > > In order to avoid breaking existing HiveQL queries, the current way we > resolve column in ORDER BY is: first resolve based on what comes from the > select clause and then fall back on its child only when this fails. > However, this case will fail: > {code} > test("orderby queries") { > jsonRDD(sparkContext.makeRDD( > """{"a": {"b": [{"c": 1}]}, "b": [{"d": 1}]}""" :: > Nil)).registerTempTable("t") > sql("SELECT a.b FROM t ORDER BY b[0].d").queryExecution.analyzed > } > {code} > As hive doesn't support resolve ORDER BY attribute not exist in select > clause, so this problem is spark sql only. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org