Ted-Jiang opened a new issue, #2725:
URL: https://github.com/apache/arrow-datafusion/issues/2725
**Describe the bug**
Filter push down not consider alias columns
**To Reproduce**
Logical plan
```
Received plan for execution: Limit: 50000
Projection: #LINEORDER.LO_SHIPMODE, #TEST1
Projection: #LO_SHIPMODE AS LINEORDER.LO_SHIPMODE,
#testBITMAPCOUNTDISTINCT(_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_) AS TEST1
Aggregate: groupBy=[[#LO_SHIPMODE]],
aggr=[[testBITMAPCOUNTDISTINCT(#_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_)]]
Projection: #LO_SHIPMODE, #dummy_LINEORDER_LO_SUPPKEY,
#_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_
Filter: #LO_SHIPMODE IN ([CAST(Int32(1) AS Utf8), CAST(Int32(2) AS
Utf8), CAST(Int32(3) AS Utf8), CAST(Int32(4) AS Utf8), CAST(Int32(5) AS Utf8)])
Projection: #LO_ORDERKEY, #LO_LINENUMBER, #LO_CUSTKEY,
#LO_PARTKEY, #dummy_LINEORDER_LO_SUPPKEY, #LO_ORDERDATE, #LO_ORDERPRIORITY,
#LO_SHIPPRIORITY, #LO_QUANTITY, #LO_EXTENDEDPRICE, #LO_ORDTOTALPRICE,
#LO_DISCOUNT, #LO_TAX, #LO_COMMITDATE, #LO_SHIPMODE, #_KY_COUNT__,
#_KY_COUNT_DISTINCT_LINEORDER_LO_ORDERKEY_,
#_KY_COUNT_DISTINCT_LINEORDER_LO_SHIPPRIORITY_LINEORDER_LO_SUPPKEY_,
#_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_,
#_KY_PERCENTILE_APPROX_LINEORDER_LO_EXTENDEDPRICE_,
#_KY_APPROX_TOPN_SUM_LINEORDER_LO_EXTENDEDPRICE_,
#_KY_APPROX_BITOPN_SUM_LINEORDER_LO_EXTENDEDPRICE_,
#_KY_APPROX_SUM_TOPN_LINEORDER_LO_EXTENDEDPRICE_,
#_KY_APPROX_SUM_BITOPN_LINEORDER_LO_EXTENDEDPRICE_,
#dummy_LINEORDER__KY_SUM_LINEORDER_LO_EXTENDEDPRICE_,
#dummy_LINEORDER__KY_MAX_LINEORDER_LO_EXTENDEDPRICE_
Projection: #ssb@[email protected] AS
LO_ORDERKEY, #ssb@[email protected] AS LO_LINENUMBER,
#ssb@[email protected] AS LO_CUSTKEY,
#ssb@[email protected] AS LO_PARTKEY, Utf8("1") AS
dummy_LINEORDER_LO_SUPPKEY, #ssb@[email protected] AS
LO_ORDERDATE, #ssb@[email protected] AS LO_ORDERPRIORITY,
#ssb@[email protected] AS LO_SHIPPRIORITY,
#ssb@[email protected] AS LO_QUANTITY,
#ssb@[email protected] AS LO_EXTENDEDPRICE,
#ssb@[email protected] AS LO_ORDTOTALPRICE,
#ssb@[email protected] AS LO_DISCOUNT,
#ssb@[email protected] AS LO_TAX,
#ssb@[email protected] AS LO_COMMITDATE,
#ssb@[email protected] AS LO_SHIPMODE,
#ssb@[email protected] AS _KY_COUNT__,
#ssb@[email protected]
AS _KY_COUNT_DISTINCT_LINEORDER_LO_ORDERKEY_,
#ssb@[email protected] AS
_KY_COUNT_DISTINCT_LINEORDER_LO_SHIPPRIORITY_LINEORDER_LO_SUPPKEY_,
#ssb@[email protected] AS
_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_,
#ssb@[email protected] AS
_KY_PERCENTILE_APPROX_LINEORDER_LO_EXTENDEDPRICE_,
#ssb@[email protected] AS
_KY_APPROX_TOPN_SUM_LINEORDER_LO_EXTENDEDPRICE_,
#ssb@[email protected] AS
_KY_APPROX_BITOPN_SUM_LINEORDER_LO_EXTENDEDPRICE_,
#ssb@[email protected] AS
_KY_APPROX_SUM_TOPN_LINEORDER_LO_EXTENDEDPRICE_,
#ssb@[email protected] AS
_KY_APPROX_SUM_BITOPN_LINEORDER_LO_EXTENDEDPRICE_, Utf8("1") AS
dummy_LINEORDER__KY_SUM_LINEORDER_LO_EXTENDEDPRICE_, Utf8("1") AS
dummy_LINEORDER__KY_MAX_LINEORDER_LO_EXTENDEDPRICE_
TableScan: ssb@test_udaf_cube_update@17179869183
projection=Some([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37,
38, 39, 40, 41, 42, 43, 44])
```
logical plan optimize
```
Calculated optimized plan: Limit: 50000
Projection: #LINEORDER.LO_SHIPMODE, #TEST1
Projection: #LO_SHIPMODE AS LINEORDER.LO_SHIPMODE,
#testBITMAPCOUNTDISTINCT(_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_) AS TEST1
Aggregate: groupBy=[[#LO_SHIPMODE]],
aggr=[[testBITMAPCOUNTDISTINCT(#_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_)]]
Projection: #LO_SHIPMODE, #_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_
Projection: #LO_SHIPMODE, #_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_
Projection: #ssb@[email protected] AS
LO_SHIPMODE, #ssb@[email protected] AS
_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_
Filter: #LO_SHIPMODE IN ([Utf8("1"), Utf8("2"), Utf8("3"),
Utf8("4"), Utf8("5")])
TableScan: ssb@test_udaf_cube_update@17179869183
projection=Some([12, 37]), partial_filters=[#LO_SHIPMODE IN ([Utf8("1"),
Utf8("2"), Utf8("3"), Utf8("4"), Utf8("5")])]
```
physical plan optimize
```
create_physical_plan optimized plan: Limit: 50000
Projection: #LINEORDER.LO_SHIPMODE, #TEST1
Projection: #LO_SHIPMODE AS LINEORDER.LO_SHIPMODE,
#testBITMAPCOUNTDISTINCT(_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_) AS TEST1
Aggregate: groupBy=[[#LO_SHIPMODE]],
aggr=[[testBITMAPCOUNTDISTINCT(#_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_)]]
Projection: #LO_SHIPMODE, #_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_
Projection: #LO_SHIPMODE, #_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_
Projection: #ssb@[email protected] AS
LO_SHIPMODE, #ssb@[email protected] AS
_KY_COUNT_DISTINCT_LINEORDER_LO_SUPPKEY_
Filter: #LO_SHIPMODE IN ([Utf8("1"), Utf8("2"), Utf8("3"),
Utf8("4"), Utf8("5")])
TableScan: ssb@test_udaf_cube_update@17179869183
projection=Some([12, 37]), partial_filters=[#LO_SHIPMODE IN ([Utf8("1"),
Utf8("2"), Utf8("3"), Utf8("4"), Utf8("5")])]
```
in logical plan `Filter: #LO_SHIPMODE IN ` is after alias `Projection:
#LO_ORDERKEY`
After optimize `Filter` is before `Projection`.
cause :
```
ERROR Could not create physical plan: Error during planning: No field named
'<unqualified>.LO_SHIPMODE'. Valid fields are
'ssb@[email protected]',
'ssb@[email protected]'.
```
**Expected behavior**
could find alias col
**Additional context**
Add any other context about the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
