monkeyboy123 commented on PR #44431:
URL: https://github.com/apache/spark/pull/44431#issuecomment-1865359053
if we execute SQL:
```
select * from
tableA
where dt =20231212
limit 100;
```
Before this pr:
```
=== Applying Rule
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations ===
+- 'Filter (('dt = 20231212) AND ('hour = 10)) +-
Filter ((cast(dt#520 as int) = 20231212) AND (cast(hour#521 as int) = 10))
! +- 'UnresolvedRelation [default, tableA], [], false +-
SubqueryAlias spark_catalog.default.tableA
!
+- RelationV2[] spark_catalog.default.tableA tableA
=== Applying Rule
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$CombinedTypeCoercionRule
===
! +- 'Filter ((dt#520 = 20231212) AND (hour#521 = 10)) +-
Filter ((cast(dt#520 as int) = 20231212) AND (cast(hour#521 as int) = 10))
+- SubqueryAlias spark_catalog.default.tableA
+- SubqueryAlias spark_catalog.default.tableA
+- RelationV2[xxx] spark_catalog.default.tableA tableA
+- RelationV2[xxx] spark_catalog.default.tableA tableA
INFO
(org.apache.spark.sql.execution.datasources.v2.V2ScanRelationPushDown:60) -
Pushing operators to tableA
Pushed Filters: IsNotNull(dt), IsNotNull(hour)
Post-Scan Filters: (cast(dt#520 as int) = 20231212),(cast(hour#521 as int) =
10)
```
After this pr:
```
=== Applying Rule
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations ===
+- 'Filter (('dt = 20231212) AND ('hour = 10)) +-
'Filter (('dt = 20231212) AND ('hour = 10))
! +- 'UnresolvedRelation [default, tableA], [], false +-
SubqueryAlias spark_catalog.default.tableA
!
+- RelationV2[xxx] spark_catalog.default.tableA tableA
=== Applying Rule
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$CombinedTypeCoercionRule
===
! +- 'Filter ((dt#520 = 20231212) AND (hour#521 = 10))
+- Filter ((dt#520 = cast(20231212 as string)) AND (hour#521 = cast(10 as
string)))
+- SubqueryAlias spark_catalog.default.tableA
+- SubqueryAlias spark_catalog.default.tableA
+- RelationV2[xxx] spark_catalog.default.tableA tableA
+- RelationV2[xxx] spark_catalog.default.tableA tableA
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.ConstantFolding ===
! +- Filter ((dt#520 = cast(20231212 as string)) AND (hour#521 =
cast(10 as string))) +- Filter ((dt#520 = 20231212) AND (hour#521
= 10))
+- RelationV2[xxx] spark_catalog.default.tableA tableA
+- RelationV2[xxx]
spark_catalog.default.tableA tableA
Pushing operators to tableA
Pushed Filters: IsNotNull(dt), IsNotNull(hour), EqualTo(dt,20231212),
EqualTo(hour,10)
Post-Scan Filters:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
