puchengy commented on issue #4997:
URL: https://github.com/apache/iceberg/issues/4997#issuecomment-1150594476
@kbendick thanks for the response, here is the query plan you suggest me to
try, however, they both seems do not help much.
> Can you set spark.sql.autoBroadcastJoinThreshold = -1 in your spark config
and then try running the explain again?
```
spark-sql> set spark.sql.autoBroadcastJoinThreshold=-1;
spark-sql> explain select * FROM db.tbl WHERE dt BETWEEN
date_sub('2022-06-06', 1) and '2022-06-06';
...
plan
== Physical Plan ==
*(1) Project [col1#36, col2#37L, ..., dt#59, bool_partition_col#60]
+- *(1) Filter (((cast(dt#59 as date) >= 2022-06-05) AND isnotnull(dt#59))
AND (dt#59 <= 2022-06-06))
+- BatchScan[col1#36, col2#37L, ..., dt#59, bool_partition_col#60]
spark_catalog.db.tbl [filters=dt IS NOT NULL, dt <= '2022-06-06']
RuntimeFilters: []
```
> but what plan gets generated if you use INTERVAL?
```
spark-sql> explain select * FROM db.tbl WHERE dt BETWEEN cast('2022-06-06'
as date) - interval 1 day and '2022-06-06';
...
plan
== Physical Plan ==
*(1) Project [col1#96, col2#97L, app#98, ..., dt#119, bool_partition_col#120]
+- *(1) Filter (((cast(dt#119 as date) >= 2022-06-05) AND isnotnull(dt#119))
AND (dt#119 <= 2022-06-06))
+- BatchScan[col1#96, col2#97L, ..., dt#119, bool_partition_col#120]
spark_catalog.db.tbl [filters=dt IS NOT NULL, dt <= '2022-06-06']
RuntimeFilters: []
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
