This is an automated email from the ASF dual-hosted git repository.
dbtsai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-connect-rust.git
The following commit(s) were added to refs/heads/master by this push:
new 56b4e72 [SPARK-60695][CONNECT] Align Rust schema printing with
reference output
56b4e72 is described below
commit 56b4e72ba24643c8f146d969af23d6fa7e6841ce
Author: DB Tsai <[email protected]>
AuthorDate: Thu Nov 5 15:32:11 2026 -0800
[SPARK-60695][CONNECT] Align Rust schema printing with reference output
### What changes were proposed in this pull request?
Address the two follow-up nits from [LC's review of
#131](https://github.com/apache/spark-connect-rust/pull/131#pullrequestreview-5793501399):
use `println!` in Rust `DataFrame::print_schema()` so its output includes the
final blank line, and explain why `StructType.treeString` uses a Python integer
default so explicit `None` raises `TypeError` like PySpark.
### Why are the changes needed?
Follow-up to
[SPARK-60695](https://issues.apache.org/jira/browse/SPARK-60695). Rust schema
printing should match Scala's `println(schema.treeString)` and Python's
`print(tree)`, both of which append a newline to the already newline-terminated
tree.
### Does this PR introduce _any_ user-facing change?
Rust `DataFrame::print_schema()` now ends its output with a blank line,
matching Scala and Python. Python behavior is unchanged.
### How was this patch tested?
- Exact Rust stdout check using a cached nested schema, including the final
blank line.
- 416 Rust library tests passed; all client test targets compiled with and
without `wasm-udf`.
- 540 offline Python tests passed, with 8 skips.
- Official Spark 4.3 type-parity gate: 70 passed, 21 skipped, with no
unexpected failures against an isolated local Connect server.
- Exact schema depth/default/error/escaping checks matched reference
PySpark 4.3; the 12-class API comparison found only the documented transport
exceptions.
- Release wheel build, Rust formatting, and whitespace checks passed.
- Advisory workspace Clippy reports existing binding errors. All reported
diagnostic source locations contain unchanged base text; this was a source
audit, not a separate baseline Clippy run.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: OpenAI Codex (GPT-6).
This pull request and its description were written by Isaac.
Closes #169 from dbtsai/codex/pr131-lc-followup.
Authored-by: DB Tsai <[email protected]>
Signed-off-by: DB Tsai <[email protected]>
---
crates/pyspark-rs/src/types.rs | 1 +
crates/spark-connect/src/dataframe.rs | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/crates/pyspark-rs/src/types.rs b/crates/pyspark-rs/src/types.rs
index 24e147c..7e03242 100644
--- a/crates/pyspark-rs/src/types.rs
+++ b/crates/pyspark-rs/src/types.rs
@@ -1775,6 +1775,7 @@ impl PyStructType {
}
/// Tree string. Mirrors `StructType.treeString(maxDepth)`.
+ // Use an integer default so explicit None raises TypeError, matching
PySpark.
#[allow(non_snake_case)]
#[pyo3(
name = "treeString",
diff --git a/crates/spark-connect/src/dataframe.rs
b/crates/spark-connect/src/dataframe.rs
index 5edae1a..7e64cf2 100644
--- a/crates/spark-connect/src/dataframe.rs
+++ b/crates/spark-connect/src/dataframe.rs
@@ -1491,7 +1491,7 @@ impl DataFrame {
/// Print the schema of this DataFrame.
pub fn print_schema(&self) -> Result<()> {
let schema = self.schema()?;
- print!("{}", schema.tree_string()?);
+ println!("{}", schema.tree_string()?);
Ok(())
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]