jayzhan211 commented on code in PR #27470:
URL: https://github.com/apache/datafusion/pull/27470#discussion_r4599748500
##########
datafusion/physical-plan/src/render_tree.rs:
##########
@@ -195,6 +196,16 @@ fn create_tree_recursive(
}
}
+ if show_equivalences {
+ let facts = plan.equivalence_properties().facts();
+ if facts.is_empty() {
+ extra_info.insert("facts".to_string(), "none".to_string());
+ }
+ for (key, value) in facts {
+ extra_info.insert(key.to_string(), value);
+ }
+ }
Review Comment:
In the tree format, the facts are inserted into the same map as the node's
own entries, so they overwrite any entry with the same key:
`PartitionedTopKExec` loses its `order: [b@1 DESC]` and shows the fact ordering
under that label instead, and a projection column aliased `equal` disappears.
My prototype in the issue had the same problem. Putting all facts under one
`facts` entry keeps both, matches the indent format's `facts=[...]`, and drops
the `none` special case, since `Display` already prints `none`:
```suggestion
if show_equivalences {
extra_info.insert(
"facts".to_string(),
plan.equivalence_properties().to_string(),
);
}
```
<details><summary>Repro on this branch (tree format, option on vs
off)</summary>
```sql
CREATE TABLE t (a INT, b INT, c INT) AS VALUES (1,2,3),(3,4,5),(NULL,1,1);
set datafusion.explain.format = tree;
set datafusion.explain.show_equivalences = true;
EXPLAIN SELECT * FROM (SELECT a, b, ROW_NUMBER() OVER (PARTITION BY a ORDER
BY b DESC) as rn FROM t) WHERE rn <= 1;
EXPLAIN SELECT a, b + 1 AS "equal", a AS a2 FROM t;
```
With the option off, the `PartitionedTopKExec` box has `order: [b@1 DESC]`
and the `ProjectionExec` box has `equal: CAST(b AS Int64) + 1`. With it on,
both are replaced by `order: [a@0 ASC NULLS FIRST, b@1 DESC NULLS FIRST] |
[...]` and `equal: {a@0, a2@2}`. With the suggestion applied, both boxes keep
their own entry and add a `facts:` entry. The `explain_tree.slt` case needs
regenerating with `--complete`; the unit tests and clippy pass.
</details>
##########
datafusion/physical-expr/src/equivalence/ordering.rs:
##########
@@ -282,15 +282,42 @@ impl IntoIterator for OrderingEquivalenceClass {
}
}
+/// Renders an ordering with `NULLS FIRST`/`NULLS LAST` spelled out. The
+/// `PhysicalSortExpr` display leaves `NULLS FIRST` implicit for `ASC`, which
is
+/// ambiguous when reading equivalence properties.
+pub(crate) struct ExplicitNullsOrdering<'a>(pub(crate) &'a LexOrdering);
+
+impl Display for ExplicitNullsOrdering<'_> {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ for (idx, sort_expr) in self.0.iter().enumerate() {
+ if idx > 0 {
+ write!(f, ", ")?;
+ }
+ let direction = if sort_expr.options.descending {
+ "DESC"
+ } else {
+ "ASC"
+ };
+ let nulls = if sort_expr.options.nulls_first {
+ "FIRST"
+ } else {
+ "LAST"
+ };
+ write!(f, "{} {direction} NULLS {nulls}", sort_expr.expr)?;
Review Comment:
Optional cleanup: arrow's `SortOptions` `Display` already prints `ASC NULLS
FIRST` / `DESC NULLS LAST` with both parts spelled out, so this can reuse it:
```suggestion
write!(f, "{} {}", sort_expr.expr, sort_expr.options)?;
```
##########
datafusion/sqllogictest/test_files/explain.slt:
##########
@@ -264,6 +264,23 @@ physical_plan DataSourceExec: file_groups={1 group:
[[WORKSPACE_ROOT/datafusion/
statement ok
set datafusion.explain.show_schema = false;
+### tests for EXPLAIN with display equivalences enabled
+statement ok
+set datafusion.explain.show_equivalences = true;
+
+query TT
+EXPLAIN SELECT a, b, c FROM simple_explain_test ORDER BY a;
+----
+logical_plan
+01)Sort: simple_explain_test.a ASC NULLS LAST
+02)--TableScan: simple_explain_test projection=[a, b, c]
+physical_plan
+01)SortExec: expr=[a@0 ASC NULLS LAST], preserve_partitioning=[false],
facts=[order: [a@0 ASC NULLS LAST]]
+02)--DataSourceExec: file_groups={1 group:
[[WORKSPACE_ROOT/datafusion/core/tests/data/example.csv]]}, projection=[a, b,
c], file_type=csv, has_header=true, facts=[none]
Review Comment:
The `explain.slt` cases only show `order` and `none`, so nothing covers
`equal`, a constant with a value, or skipping literals: removing the
`!is_literal` filter on equal classes makes `WHERE a = 5` print `equal: {a@0,
5}` and all of this PR's tests still pass. Could you add this case?
```suggestion
02)--DataSourceExec: file_groups={1 group:
[[WORKSPACE_ROOT/datafusion/core/tests/data/example.csv]]}, projection=[a, b,
c], file_type=csv, has_header=true, facts=[none]
query TT
EXPLAIN SELECT a, b, c FROM simple_explain_test WHERE a = b AND c = 5 ORDER
BY b;
----
logical_plan
01)Sort: simple_explain_test.b ASC NULLS LAST
02)--Filter: simple_explain_test.b = simple_explain_test.a AND
simple_explain_test.c = Int32(5)
03)----TableScan: simple_explain_test projection=[a, b, c],
partial_filters=[simple_explain_test.b = simple_explain_test.a,
simple_explain_test.c = Int32(5)]
physical_plan
01)SortPreservingMergeExec: [b@1 ASC NULLS LAST], facts=[order: [b@1 ASC
NULLS LAST]; equal: {b@1, a@0}; constant: c@2 = 5]
02)--SortExec: expr=[b@1 ASC NULLS LAST], preserve_partitioning=[true],
facts=[order: [b@1 ASC NULLS LAST]; equal: {b@1, a@0}; constant: c@2 = 5]
03)----FilterExec: b@1 = a@0 AND c@2 = 5, facts=[equal: {b@1, a@0};
constant: c@2 = 5]
04)------RepartitionExec: partitioning=RoundRobinBatch(4),
input_partitions=1, facts=[none]
05)--------DataSourceExec: file_groups={1 group:
[[WORKSPACE_ROOT/datafusion/core/tests/data/example.csv]]}, projection=[a, b,
c], file_type=csv, has_header=true, facts=[none]
```
##########
datafusion/physical-expr/src/equivalence/properties/mod.rs:
##########
@@ -1895,34 +1896,84 @@ impl From<EquivalenceProperties> for
OrderingEquivalenceClass {
}
}
-/// More readable display version of the `EquivalenceProperties`.
+impl EquivalenceProperties {
+ /// The facts this object states about its output, as `(key, value)` pairs.
+ ///
+ /// Orderings always spell out NULL placement. Literal-only constants and
+ /// equivalence classes are skipped, and empty sections are omitted.
+ pub fn facts(&self) -> Vec<(&'static str, String)> {
Review Comment:
Optional cleanup: if the tree format prints `Display` under one `facts`
entry, `facts()` has no caller outside this module, and keeping it private
means its `(key, value)` shape can change later without a breaking change.
```suggestion
fn facts(&self) -> Vec<(&'static str, String)> {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]