sigmod commented on code in PR #39479: URL: https://github.com/apache/spark/pull/39479#discussion_r1065456673
########## sql/core/src/test/resources/sql-tests/results/join-lateral.sql.out: ########## @@ -837,6 +837,132 @@ struct<c1:int,c2:array<int>,c3:int> NULL [4] NULL +-- !query +SELECT * FROM t1, LATERAL RANGE(3) +-- !query schema +struct<c1:int,c2:int,id:bigint> +-- !query output +0 1 0 +0 1 1 +0 1 2 +1 2 0 +1 2 1 +1 2 2 + + +-- !query +SELECT * FROM t1, LATERAL EXPLODE(ARRAY(c1, c2)) t2(c3) +-- !query schema +struct<c1:int,c2:int,c3:int> +-- !query output +0 1 0 +0 1 1 +1 2 1 +1 2 2 + + +-- !query +SELECT * FROM t3, LATERAL EXPLODE(c2) t2(v) +-- !query schema +struct<c1:int,c2:array<int>,v:int> +-- !query output +0 [0,1] 0 +0 [0,1] 1 +1 [2] 2 +NULL [4] 4 + + +-- !query +SELECT * FROM t3, LATERAL EXPLODE_OUTER(c2) t2(v) +-- !query schema +struct<c1:int,c2:array<int>,v:int> +-- !query output +0 [0,1] 0 +0 [0,1] 1 +1 [2] 2 +2 [] NULL +NULL [4] 4 + + +-- !query +SELECT * FROM EXPLODE(ARRAY(1, 2)) t(v), LATERAL (SELECT v + 1) +-- !query schema +struct<v:int,(outer(t.v) + 1):int> +-- !query output +1 2 +2 3 + + +-- !query +SELECT * FROM t1 JOIN LATERAL EXPLODE(ARRAY(c1, c2)) t(c3) ON t1.c1 = c3 +-- !query schema +struct<c1:int,c2:int,c3:int> +-- !query output +0 1 0 +1 2 1 + + +-- !query +SELECT * FROM t3 JOIN LATERAL EXPLODE(c2) t(c3) ON t3.c1 = c3 +-- !query schema +struct<c1:int,c2:array<int>,c3:int> +-- !query output +0 [0,1] 0 + + +-- !query +SELECT * FROM t3 LEFT JOIN LATERAL EXPLODE(c2) t(c3) ON t3.c1 = c3 +-- !query schema +struct<c1:int,c2:array<int>,c3:int> +-- !query output +0 [0,1] 0 +1 [2] NULL +2 [] NULL +NULL [4] NULL + + +-- !query +SELECT * FROM t1, LATERAL (SELECT * FROM EXPLODE(ARRAY(c1, c2))) +-- !query schema +struct<c1:int,c2:int,col:int> +-- !query output +0 1 0 +0 1 1 +1 2 1 +1 2 2 + + +-- !query +SELECT * FROM t1, LATERAL (SELECT t1.c1 + c3 FROM EXPLODE(ARRAY(c1, c2)) t(c3)) +-- !query schema +struct<c1:int,c2:int,(outer(spark_catalog.default.t1.c1) + c3):int> +-- !query output +0 1 0 +0 1 1 +1 2 2 +1 2 3 + + +-- !query +SELECT * FROM t1, LATERAL (SELECT t1.c1 + c3 FROM EXPLODE(ARRAY(c1, c2)) t(c3) WHERE t1.c2 > 1) +-- !query schema +struct<c1:int,c2:int,(outer(spark_catalog.default.t1.c1) + c3):int> +-- !query output +1 2 2 +1 2 3 + + +-- !query +SELECT * FROM t1, LATERAL (SELECT * FROM EXPLODE(ARRAY(c1, c2)) l(x) JOIN EXPLODE(ARRAY(c2, c1)) r(y) ON x = y) Review Comment: Does it make sense to add a SQL query into SchemaPruningSuite for "... from lateral explode .. " or "... from explode ..." to make sure schema pruning behavior is the same: https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/SchemaPruningSuite.scala Or alternatively, add a plan unit test to assert "... from Lateral explode(...)" and "... from explode(...)" is always compiled to a plan with `Generate`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
