[
https://issues.apache.org/jira/browse/SPARK-48265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
angerszhu updated SPARK-48265:
------------------------------
Description:
{code:java}
24/05/13 17:39:25 ERROR [main] PlanChangeLogger:
=== Result of Batch LocalRelation ===
GlobalLimit 21
GlobalLimit 21
+- LocalLimit 21
+- LocalLimit 21
! +- Union false, false
+- LocalLimit 21
! :- LocalLimit 21
+- Project
[item_id#647L]
! : +- Project [item_id#647L]
+- Filter
(((isnotnull(tz_type#734) AND (tz_type#734 = local)) AND (grass_region#735 =
BR)) AND isnotnull(grass_region#735))
! : +- Filter (((isnotnull(tz_type#734) AND (tz_type#734 = local)) AND
(grass_region#735 = BR)) AND isnotnull(grass_region#735)) +-
Relation db.table[,... 91 more fields] parquet
! : +- Relation db.table[,... 91 more fields] parquet
! +- LocalLimit 21
! +- Project [item_id#738L]
! +- LocalRelation <empty>, [, ... 91 more fields]
24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch Check Cartesian Products
has no effect.
24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch RewriteSubquery has no
effect.
24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch NormalizeFloatingNumbers
has no effect.
24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch
ReplaceUpdateFieldsExpression has no effect.
24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch Optimize Metadata Only
Query has no effect.
24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch PartitionPruning has no
effect.
24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch InjectRuntimeFilter has
no effect.
24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch Pushdown Filters from
PartitionPruning has no effect.
24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch Cleanup filters that
cannot be pushed down has no effect.
24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch Extract Python UDFs has
no effect.
24/05/13 17:39:25 ERROR [main] PlanChangeLogger:
=== Applying Rule org.apache.spark.sql.catalyst.optimizer.EliminateLimits ===
GlobalLimit 21
GlobalLimit 21
!+- LocalLimit 21
+- LocalLimit least(,
... 2 more fields)
! +- LocalLimit 21
+- Project
[item_id#647L]
! +- Project [item_id#647L]
+- Filter
(((isnotnull(tz_type#734) AND (tz_type#734 = local)) AND (grass_region#735 =
BR)) AND isnotnull(grass_region#735))
! +- Filter (((isnotnull(tz_type#734) AND (tz_type#734 = local)) AND
(grass_region#735 = BR)) AND isnotnull(grass_region#735)) +-
Relation db.table[,... 91 more fields] parquet
! +- Relation db.table[,... 91 more fields] parquet
{code}
> Infer window group limit batch should do constant folding
> ---------------------------------------------------------
>
> Key: SPARK-48265
> URL: https://issues.apache.org/jira/browse/SPARK-48265
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.0.0, 3.5.1
> Reporter: angerszhu
> Priority: Major
>
> {code:java}
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger:
> === Result of Batch LocalRelation ===
> GlobalLimit 21
> GlobalLimit 21
> +- LocalLimit 21
> +- LocalLimit 21
> ! +- Union false, false
> +-
> LocalLimit 21
> ! :- LocalLimit 21
> +-
> Project [item_id#647L]
> ! : +- Project [item_id#647L]
> +-
> Filter (((isnotnull(tz_type#734) AND (tz_type#734 = local)) AND
> (grass_region#735 = BR)) AND isnotnull(grass_region#735))
> ! : +- Filter (((isnotnull(tz_type#734) AND (tz_type#734 = local))
> AND (grass_region#735 = BR)) AND isnotnull(grass_region#735))
> +- Relation db.table[,... 91 more fields] parquet
> ! : +- Relation db.table[,... 91 more fields] parquet
> ! +- LocalLimit 21
> ! +- Project [item_id#738L]
> ! +- LocalRelation <empty>, [, ... 91 more fields]
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch Check Cartesian
> Products has no effect.
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch RewriteSubquery has no
> effect.
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch
> NormalizeFloatingNumbers has no effect.
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch
> ReplaceUpdateFieldsExpression has no effect.
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch Optimize Metadata Only
> Query has no effect.
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch PartitionPruning has
> no effect.
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch InjectRuntimeFilter
> has no effect.
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch Pushdown Filters from
> PartitionPruning has no effect.
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch Cleanup filters that
> cannot be pushed down has no effect.
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger: Batch Extract Python UDFs
> has no effect.
> 24/05/13 17:39:25 ERROR [main] PlanChangeLogger:
> === Applying Rule org.apache.spark.sql.catalyst.optimizer.EliminateLimits ===
> GlobalLimit 21
> GlobalLimit 21
> !+- LocalLimit 21
> +- LocalLimit
> least(, ... 2 more fields)
> ! +- LocalLimit 21
> +- Project
> [item_id#647L]
> ! +- Project [item_id#647L]
> +- Filter
> (((isnotnull(tz_type#734) AND (tz_type#734 = local)) AND (grass_region#735 =
> BR)) AND isnotnull(grass_region#735))
> ! +- Filter (((isnotnull(tz_type#734) AND (tz_type#734 = local)) AND
> (grass_region#735 = BR)) AND isnotnull(grass_region#735)) +-
> Relation db.table[,... 91 more fields] parquet
> ! +- Relation db.table[,... 91 more fields] parquet
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
