rui-mo commented on issue #9965:
URL:
https://github.com/apache/incubator-gluten/issues/9965#issuecomment-2996687305
@wenfang6 I find there's a plan difference. In the plan you provide, no
filter is pushed down to the TableScan, and there's one Filter operator to
handle the filtering. While in my plan, filter pushdown works well and no
Filter operator. Would you please take a look which Scan transformer was used
in your testing? We have the `HiveTableScanExecTransformer.scala`,
`BatchScanExecTransformer.scala` and `FileSourceScanExecTransformer.scala`.
The error plan:
```
-- Project[4][expressions: (n4_3:INTEGER, hash_with_seed(42,"n2_4")),
(n4_4:INTEGER, "n2_2"), (n4_5:VARCHAR, "n2_3"), (n4_6:INTEGER, "n2_4")] ->
n4_3:INTEGER, n4_4:INTEGER, n4_5:VARCHAR, n4_6:INTEGER
-- TopNRowNumber[3][partition by (n2_4) order by (n2_3 DESC NULLS LAST)
limit 1] -> n2_2:INTEGER, n2_3:VARCHAR, n2_4:INTEGER
-- Project[2][expressions: (n2_2:INTEGER, "n0_0"), (n2_3:VARCHAR,
"n0_1"), (n2_4:INTEGER, "n0_0")] -> n2_2:INTEGER, n2_3:VARCHAR, n2_4:INTEGER
-- Filter[1][expression:
and(isnotnull("n0_1"),equalto("n0_1","wukong"))] -> n0_0:INTEGER, n0_1:VARCHAR
-- TableScan[0][table: hive_table] -> n0_0:INTEGER, n0_1:VARCHAR
```
Plan in my testing:
```
-- Project[3][expressions: (n3_3:INTEGER, hash_with_seed(42,"n1_4")),
(n3_4:INTEGER, "n1_2"), (n3_5:VARCHAR, "n1_3"), (n3_6:INTEGER, "n1_4")] ->
n3_3:INTEGER, n3_4:INTEGER, n3_5:VARCHAR, n3_6:INTEGER
-- TopNRowNumber[2][partition by (n1_4) order by (n1_3 DESC NULLS LAST)
limit 1] -> n1_2:INTEGER, n1_3:VARCHAR, n1_4:INTEGER
-- Project[1][expressions: (n1_2:INTEGER, "n0_0"), (n1_3:VARCHAR,
"n0_1"), (n1_4:INTEGER, "n0_0")] -> n1_2:INTEGER, n1_3:VARCHAR, n1_4:INTEGER
-- TableScan[0][table: hive_table, remaining filter:
(and(isnotnull("name"),equalto("name","wukong"))), data columns:
ROW<id:INTEGER,name:VARCHAR>] -> n0_0:INTEGER, n0_1:VARCHAR
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
