dbtsai opened a new pull request, #127: URL: https://github.com/apache/spark-connect-rust/pull/127
### What changes were proposed in this pull request? Fast-path exact built-in Python `str` values in `py_to_value` before bool, Decimal and numeric probing. Keep all numeric code unchanged. String subclasses and failed UTF-8 extraction retain the existing generic fallback, including subclass conversion protocols and surrogate errors. This is based directly on master and is independent of PR #111's numeric optimization and PR #114's Arrow encoder work. ### Why are the changes needed? Local string input currently pays generic scalar detection for every value. Five fresh-process samples per client/case with the unchanged frozen benchmark on Apple M3 Max (arm64 Python 3.12.14, release build) give these medians for 10,000 × 3 string inputs: | Source | Master | Candidate | Speedup | |---|---:|---:|---:| | tuples | 20.493 ms | 3.930 ms | 5.21× | | pandas | 22.310 ms | 5.313 ms | 4.20× | The Python reference measures 2.843 ms and 1.784 ms respectively, so a gap remains. All 195 samples across 13 cases and three clients pass the unchanged validators. Numeric, Decimal, date, timestamp, array, nested-result and RPC guards show no material regression; small differences remain noisy with five samples. All CPU-heavy work was serialized. ### Does this PR introduce _any_ user-facing change? Faster construction for exact built-in string values. Value conversion and errors are preserved by baseline/candidate checks; subclasses retain their current numeric conversion behavior. ### How was this patch tested? - Release locked maturin wheel; full default Rust member test compilation; 392 Rust library tests; `cargo fmt --all -- --check`; `git diff --check`. - Baseline and candidate each pass 22 new tests: exact strings, unusual Unicode/NUL, surrogate errors, subclass `__index__`/`__float__`/`__str__` protocol order, nested nulls, inferred/named/explicit schemas. A separate-process localhost mock decodes the actual outbound Arrow IPC and compares every value, including 10,000 × 3 tuple/pandas inputs and nested arrays/maps/structs. - Baseline and candidate each pass all 274 existing offline tests with `pytest --rootdir=python --import-mode=importlib python/tests/test_dropin_offline.py -q`. New multiprocessing protocol tests use normal pytest imports. - Frozen benchmark harness `1bbdec9f3665fcd2ec2ee5129bbcdb95ee4b00c4`, baseline master `d884e347452c6446c7bc54af151755c62801d8cd`, Python reference `ca1514a2b4e72f9529871f34d09881cd4921df82`. Harness files and validators unchanged; isolated candidate environment; five fresh samples each. Construction benchmarks validate DataFrame objects; the separate IPC tests validate the encoded contents. Client-only scripted localhost fixtures; no JVM or live Spark server validation. The initial protocol invocation failed because pytest importlib mode made the spawned test module unimportable; the corrected invocation passes on both clients, and the failed evidence is retained locally along with raw samples, logs, environment freezes, wheel and harness hashes. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
