NGA-TRAN commented on code in PR #17843:
URL: https://github.com/apache/datafusion/pull/17843#discussion_r2529891968
##########
datafusion/core/src/execution/context/mod.rs:
##########
@@ -1365,6 +1367,18 @@ impl SessionContext {
self.state.write().register_udwf(Arc::new(f)).ok();
}
+ #[cfg(feature = "sql")]
+ /// Registers a [`RelationPlanner`] to customize SQL table-factor planning.
+ ///
+ /// Planners are invoked in reverse registration order, allowing newer
+ /// planners to take precedence over existing ones.
Review Comment:
This comment is important
##########
datafusion/sql/src/relation/mod.rs:
##########
@@ -24,19 +24,119 @@ use datafusion_common::{
not_impl_err, plan_err, DFSchema, Diagnostic, Result, Span, Spans,
TableReference,
};
use datafusion_expr::builder::subquery_alias;
+use datafusion_expr::planner::{
+ PlannedRelation, RelationPlannerContext, RelationPlanning,
+};
use datafusion_expr::{expr::Unnest, Expr, LogicalPlan, LogicalPlanBuilder};
use datafusion_expr::{Subquery, SubqueryAlias};
use sqlparser::ast::{FunctionArg, FunctionArgExpr, Spanned, TableFactor};
mod join;
+struct SqlToRelRelationContext<'a, 'b, S: ContextProvider> {
Review Comment:
I had to review the tests and this a few times to really understand why
things work that way. You’ve built a very powerful framework/extension with
such a simple design—impressive work.
A few thoughts:
- If users know what they’re doing and register the relation planner in the
correct reverse order, will everything consistently work as expected?
- How do we test this assumption? Your example is mind‑blowing, but how can
we be confident it will hold for any order of standard SQL?
I believe it will work, but I keep asking myself: how do we know it will
always work?
```rust
let ctx = SessionContext::new();
// Register custom planners for SQL syntax extensions
ctx.register_relation_planner(Arc::new(TableSamplePlanner))?;
ctx.register_relation_planner(Arc::new(MatchRecognizePlanner))?;
ctx.register_relation_planner(Arc::new(PivotUnpivotPlanner))?;
// Use multiple custom table modifiers together - even in nested JOINs or
CTEs
let df = ctx.sql(r#"
WITH sampled_data AS (
SELECT * FROM stock_prices
TABLESAMPLE BERNOULLI(10 PERCENT) REPEATABLE(42)
)
SELECT symbol, quarter, price
FROM sampled_data
MATCH_RECOGNIZE (
PARTITION BY symbol ORDER BY time
MEASURES LAST(price) AS price, quarter
PATTERN (UP+ DOWN+)
DEFINE
UP AS price > PREV(price),
DOWN AS price < PREV(price)
) AS patterns
PIVOT (
AVG(price) FOR quarter IN ('Q1', 'Q2', 'Q3', 'Q4')
) AS pivoted
"#).await?;
df.show().await?;
```
##########
datafusion/core/src/execution/session_state.rs:
##########
@@ -581,6 +586,24 @@ impl SessionState {
&self.expr_planners
}
+ #[cfg(feature = "sql")]
+ /// Returns the registered relation planners in priority order.
+ pub fn relation_planners(&self) -> &[Arc<dyn RelationPlanner>] {
+ &self.relation_planners
+ }
+
+ #[cfg(feature = "sql")]
+ /// Registers a [`RelationPlanner`] to customize SQL relation planning.
+ ///
+ /// Newly registered planners are given higher priority than existing ones.
Review Comment:
👍 This is consistent with the comment above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
