This is an automated email from the ASF dual-hosted git repository.

dbtsai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-connect-rust.git


The following commit(s) were added to refs/heads/master by this push:
     new 9feb32b  [SPARK-60695][CONNECT][TESTS] Assert native schema-printing 
stdout
9feb32b is described below

commit 9feb32b3b0a4545f4e7217f8b4b3f23a9b7cdf52
Author: DB Tsai <[email protected]>
AuthorDate: Fri Nov 6 22:05:12 2026 -0800

    [SPARK-60695][CONNECT][TESTS] Assert native schema-printing stdout
    
    ### What changes were proposed in this pull request?
    
    PR #169 aligned Rust `DataFrame::print_schema()` with the reference 
trailing blank line, but its exact Rust stdout check was not committed. Add an 
offline regression that calls the real method in a child process and asserts 
exact output for nested and empty cached schemas. A 30-second child watchdog 
bounds cache/RPC regressions. Document the intentional final blank line; 
production behavior and public APIs are unchanged.
    
    ### Why are the changes needed?
    
    Keep the output contract from 
[SPARK-60695](https://issues.apache.org/jira/browse/SPARK-60695) under durable 
regression coverage. Python stdout tests exercise a separate print path. The 
new test fails when the native `println!` is temporarily changed back to 
`print!`.
    
    ### How was this patch tested?
    
    - 462 Rust release library tests passed, including the actual-stdout 
regression; the negative newline mutation failed as expected and was restored.
    - All client integration targets compiled with and without `wasm-udf`; 
formatting and whitespace checks passed.
    - Fresh native Python extension release build passed; 14 focused Python 
schema tests passed.
    - Official Spark 4.3 type-parity gate: 70 passed, 21 skipped on an isolated 
UTC server with `spark.sql.timeType.enabled=true`. The initial TIME-disabled 
failure reproduced with official PySpark before correcting the server 
configuration.
    - Twelve-class attribute-name comparison found zero unexpected missing 
public names; the documented `Column.to_plan` transport gap and inherited 
private/dunder differences remain visible. This is not a claim of complete 
behavioral parity.
    - Genuine local Claude Code reviewed the original landed patch twice and 
the complete follow-up patch, assessed all findings/nits, and approved this 
final source with no defects.
    
    Draft pending broader repository validation: workspace Clippy exits 101 
with 14 existing approximate-constant errors in files byte-identical to the 
base (source audit, not a separate baseline Clippy run). The complete official 
Connect suite and CI have not been certified by these scoped checks.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: OpenAI Codex (GPT-6), with independent local Claude Code 
review.
    
    This pull request and its description were written by Isaac.
    
    Closes #285 from dbtsai/codex/postmerge-pr-169-schema-stdout-test.
    
    Authored-by: DB Tsai <[email protected]>
    Signed-off-by: DB Tsai <[email protected]>
    Co-authored-by: Isaac <[email protected]>
---
 crates/spark-connect/src/dataframe.rs | 74 +++++++++++++++++++++++++++++++++++
 1 file changed, 74 insertions(+)

diff --git a/crates/spark-connect/src/dataframe.rs 
b/crates/spark-connect/src/dataframe.rs
index 5666c67..0265375 100644
--- a/crates/spark-connect/src/dataframe.rs
+++ b/crates/spark-connect/src/dataframe.rs
@@ -1780,6 +1780,8 @@ impl DataFrame {
     }
 
     /// Print the schema of this DataFrame.
+    ///
+    /// The output ends with a blank line, matching Scala and Python.
     pub fn print_schema(&self) -> Result<()> {
         let schema = self.schema()?;
         println!("{}", schema.tree_string()?);
@@ -3369,6 +3371,78 @@ mod cache_tests {
     use super::*;
     use prost::Message;
 
+    #[test]
+    fn print_schema_includes_final_blank_line() {
+        const CHILD: &str = "SPARK_CONNECT_PRINT_SCHEMA_STDOUT_CHILD";
+        const BEGIN: &str = "__PRINT_SCHEMA_BEGIN__\n";
+        const END: &str = "__PRINT_SCHEMA_END__";
+        if std::env::var_os(CHILD).is_some() {
+            // A cache regression must fail promptly instead of waiting on an 
RPC.
+            let _watchdog = std::thread::spawn(|| {
+                std::thread::sleep(std::time::Duration::from_secs(30));
+                eprintln!("print_schema stdout child timed out");
+                std::process::exit(1);
+            });
+            let spark = SparkSession::builder()
+                .remote("sc://localhost:1")
+                .get_or_create()
+                .unwrap();
+            let nested = DataType::from_json(&serde_json::json!({
+                "type": "struct",
+                "fields": [{"name": "outer", "type": {
+                    "type": "struct",
+                    "fields": [{"name": "inner", "type": "integer", 
"nullable": false, "metadata": {}}]
+                }, "nullable": true, "metadata": {}}]
+            }))
+            .unwrap();
+            print!("{BEGIN}");
+            for schema in [nested, DataType::Struct { fields: vec![] }] {
+                let df = spark.range(0).unwrap();
+                *df.schema_cache.lock().unwrap() = Some(Arc::new(schema));
+                df.print_schema().unwrap();
+            }
+            println!("{END}");
+            return;
+        }
+
+        // A separate process exposes real stdout without changing global 
capture
+        // or sharing it with the other parallel tests in this process.
+        let output = 
std::process::Command::new(std::env::current_exe().unwrap())
+            .args([
+                "--exact",
+                
"dataframe::cache_tests::print_schema_includes_final_blank_line",
+                "--nocapture",
+            ])
+            .env(CHILD, "1")
+            .output()
+            .unwrap();
+        assert!(
+            output.status.success(),
+            "stdout: {}\nstderr: {}",
+            String::from_utf8_lossy(&output.stdout),
+            String::from_utf8_lossy(&output.stderr),
+        );
+        let stdout = String::from_utf8(output.stdout).unwrap();
+        let printed = stdout
+            .split_once(BEGIN)
+            .expect("child must run the schema print test")
+            .1
+            .split_once(END)
+            .expect("child must finish printing schemas")
+            .0;
+        assert_eq!(
+            printed,
+            concat!(
+                "root\n",
+                " |-- outer: struct (nullable = true)\n",
+                " |    |-- inner: integer (nullable = false)\n",
+                "\n",
+                "root\n",
+                "\n",
+            ),
+        );
+    }
+
     #[test]
     fn schema_cache_shares_exact_clones_but_not_changed_plans_or_new_sources() 
{
         let spark = SparkSession::builder()


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to