Github user peter-toth commented on a diff in the pull request:
https://github.com/apache/spark/pull/22817#discussion_r228737835
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -2578,4 +2578,12 @@ class DataFrameSuite extends QueryTest with
SharedSQLContext {
Row ("abc", 1))
}
}
+
+ test("SPARK-25816 ResolveReferences works with nested extractors") {
+ val df0 = Seq((1, Map(1 -> "a")), (2, Map(2 -> "b"))).toDF("1", "2")
+ val df1 = df0.select($"1".as("2"), $"2".as("1"))
+ val df2 = df1.filter($"1"(map_keys($"1")(0)) > "a")
--- End diff --
@cloud-fan unfortunately, that would be a bit too much of simplification.
In your example `ResolveMissingReferences` after `ResolveReferences` is
able to resolve the `extraction` correctly even without this PR. The issue
comes up when we have another level of projection before the extractor.
I simplified the example a bit though.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
