HeartSaVioR commented on pull request #31355:
URL: https://github.com/apache/spark/pull/31355#issuecomment-806332643
I took the diff on printed plans (expected, actual) in stack trace, and they
seem to be identical.
```
TakeOrderedAndProject
[i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov]
WholeStageCodegen (9)
HashAggregate
[i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
[count(ss_quantity),avg(ss_quantity),stddev_samp(cast(ss_quantity as
double)),count(sr_return_quantity),avg(sr_return_quantity),stddev_samp(cast(sr_return_quantity
as double)),count(cs_quantity),avg(cs_quantity),stddev_samp(cast(cs_quantity
as
double)),store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
InputAdapter
Exchange [i_item_id,i_item_desc,s_state] #1
WholeStageCodegen (8)
HashAggregate
[i_item_id,i_item_desc,s_state,ss_quantity,sr_return_quantity,cs_quantity]
[count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
Project
[ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project
[ss_item_sk,ss_quantity,sr_return_quantity,cs_quantity,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project
[ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project
[ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,cs_quantity,cs_sold_date_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project
[ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,sr_returned_date_sk,cs_quantity,cs_sold_date_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project
[ss_item_sk,ss_store_sk,ss_quantity,ss_sold_date_sk,sr_return_quantity,sr_returned_date_sk,cs_quantity,cs_sold_date_sk]
BroadcastHashJoin
[sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
Project
[ss_item_sk,ss_store_sk,ss_quantity,ss_sold_date_sk,sr_item_sk,sr_customer_sk,sr_return_quantity,sr_returned_date_sk]
BroadcastHashJoin
[ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
Filter
[ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet
default.store_sales
[ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast
[d_date_sk] #1
ReusedExchange
[d_date_sk] #2
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Filter
[sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet
default.store_returns
[sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
SubqueryBroadcast
[d_date_sk] #2
ReusedExchange
[d_date_sk] #4
InputAdapter
BroadcastExchange #5
WholeStageCodegen (2)
Filter
[cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet
default.catalog_sales
[cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
SubqueryBroadcast
[d_date_sk] #3
ReusedExchange
[d_date_sk] #4
InputAdapter
BroadcastExchange #2
WholeStageCodegen (3)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet
default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (4)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet default.date_dim
[d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet default.store
[s_store_sk,s_state]
InputAdapter
BroadcastExchange #7
WholeStageCodegen (7)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet default.item
[i_item_sk,i_item_id,i_item_desc]
```
```
TakeOrderedAndProject
[i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov]
WholeStageCodegen (9)
HashAggregate
[i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
[count(ss_quantity),avg(ss_quantity),stddev_samp(cast(ss_quantity as
double)),count(sr_return_quantity),avg(sr_return_quantity),stddev_samp(cast(sr_return_quantity
as double)),count(cs_quantity),avg(cs_quantity),stddev_samp(cast(cs_quantity
as
double)),store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
InputAdapter
Exchange [i_item_id,i_item_desc,s_state] #1
WholeStageCodegen (8)
HashAggregate
[i_item_id,i_item_desc,s_state,ss_quantity,sr_return_quantity,cs_quantity]
[count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
Project
[ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project
[ss_item_sk,ss_quantity,sr_return_quantity,cs_quantity,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project
[ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project
[ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,cs_quantity,cs_sold_date_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project
[ss_item_sk,ss_store_sk,ss_quantity,sr_return_quantity,sr_returned_date_sk,cs_quantity,cs_sold_date_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project
[ss_item_sk,ss_store_sk,ss_quantity,ss_sold_date_sk,sr_return_quantity,sr_returned_date_sk,cs_quantity,cs_sold_date_sk]
BroadcastHashJoin
[sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
Project
[ss_item_sk,ss_store_sk,ss_quantity,ss_sold_date_sk,sr_item_sk,sr_customer_sk,sr_return_quantity,sr_returned_date_sk]
BroadcastHashJoin
[ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
Filter
[ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet
default.store_sales
[ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast
[d_date_sk] #1
ReusedExchange
[d_date_sk] #2
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Filter
[sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet
default.store_returns
[sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
SubqueryBroadcast
[d_date_sk] #2
ReusedExchange
[d_date_sk] #4
InputAdapter
BroadcastExchange #5
WholeStageCodegen (2)
Filter
[cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet
default.catalog_sales
[cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
SubqueryBroadcast
[d_date_sk] #3
ReusedExchange
[d_date_sk] #4
InputAdapter
BroadcastExchange #2
WholeStageCodegen (3)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet
default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (4)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet default.date_dim
[d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet default.store
[s_store_sk,s_state]
InputAdapter
BroadcastExchange #7
WholeStageCodegen (7)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet default.item
[i_item_sk,i_item_id,i_item_desc]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
