2010YOUY01 commented on code in PR #26757: URL: https://github.com/apache/datafusion/pull/26757#discussion_r4431276937
########## datafusion/session/src/physical_analyzer.rs: ########## @@ -0,0 +1,114 @@ +// SPDX-License-Identifier: Apache-2.0 +// SPDX-FileCopyrightText: See the NOTICE file distributed with this work for additional information regarding copyright ownership +// SPDX-FileContributor: Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements + +//! Physical analyzer interfaces. + +use std::fmt::Debug; +use std::sync::Arc; + +use datafusion_common::Result; +use datafusion_common::config::ConfigOptions; +use datafusion_physical_plan::ExecutionPlan; + +use crate::physical_optimizer::PhysicalOptimizerContext; + +/// A `PhysicalAnalyzerRule` runs before requirements are enforced. The +/// analyzer phase runs the rules that must see the plan while each +/// operator's [`ExecutionPlan::required_input_distribution`] and +/// [`ExecutionPlan::required_input_ordering`] are only declared, then +/// `EnsureRequirements`, which repartitions and sorts the inputs to meet them. +/// Every [`PhysicalOptimizerRule`] after it receives a plan that meets its +/// requirements and must return one. +/// +/// A rule belongs in the analyzer when `EnsureRequirements` reads what the +/// rule changes (a requirement, an ordering, or statistics), as with the join +/// mode `join_selection` picks, or when the rule needs the plan before +/// enforcement adds repartitions and sorts, as with `WindowTopN`. A rule that +/// reads what enforcement produced, such as `LimitPushdown`, is a +/// [`PhysicalOptimizerRule`]. Both phases hold +/// performance rules; what differs is the plan each one receives. A rule is +/// one or the other: moving it to the other phase means implementing the +/// other trait. +/// +/// An analyzer rule must: +/// +/// - Keep the query's results unchanged. It may leave a distribution or +/// ordering requirement unmet; `EnsureRequirements` at the end of the phase +/// meets it. +/// - Stay correct when the plan already contains repartitions, sorts, or +/// operators it does not know, as happens when a plan is optimized again. +/// It may skip its rewrite there, and decides what is safe around an +/// operator only from what the operator declares. +/// +/// # Where analyzer rules run +/// +/// [`DefaultPhysicalPlanner::optimize_physical_plan`] runs every analyzer rule, +/// then every optimizer rule. A rule appended to the analyzer list runs after +/// `EnsureRequirements`, so it must leave the plan valid. +/// +/// ```text +/// ExecutionPlan (every operator declares its distribution and ordering +/// requirements; nothing satisfies them yet) +/// │ +/// ▼ +/// ┌─ PhysicalAnalyzer phase ── before enforcement ───────────────┐ +/// │ rules that must run before EnsureRequirements │ +/// │ EnsureRequirements: enforce distribution, ordering │ +/// └──────────────────────────────┬───────────────────────────────┘ +/// │ every requirement is met +/// ▼ +/// ┌─ PhysicalOptimizer phase ── after enforcement ───────────────┐ +/// │ every rule: valid plan in, valid plan out │ +/// └──────────────────────────────┬───────────────────────────────┘ +/// ▼ +/// valid, optimized ExecutionPlan +/// ``` +/// +/// [`DefaultPhysicalPlanner::optimize_physical_plan`]: https://docs.rs/datafusion/latest/datafusion/physical_planner/struct.DefaultPhysicalPlanner.html#method.optimize_physical_plan +/// +/// [`PhysicalOptimizerRule`]: crate::physical_optimizer::PhysicalOptimizerRule +/// [`ExecutionPlan::required_input_distribution`]: datafusion_physical_plan::ExecutionPlan::required_input_distribution +/// [`ExecutionPlan::required_input_ordering`]: datafusion_physical_plan::ExecutionPlan::required_input_ordering +pub trait PhysicalAnalyzerRule: Debug + std::any::Any { Review Comment: WDYT about extending the existing optimizer trait with `fn phase()` to distinguish between analyzer and optimizer rules? They're very similar in essence, and this would reduce the number of APIs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
