dbtsai opened a new pull request, #125: URL: https://github.com/apache/spark-connect-rust/pull/125
## What changes were proposed in this pull request? Support integer positional column access such as `df[0]` and `df[-1]`. The binding currently accepts only strings, so these calls raise `TypeError`. Resolve integer positions through the DataFrame's column names and use Python list indexing to preserve negative positions, bool/int subclasses, and `IndexError` for out-of-range or arbitrarily large integers. Keep named lookup unchanged and release the GIL during schema lookup. This is independent of schema caching PR #120. Positional access performs a schema RPC on current master. List projection and Column-filter indexing are outside this change. ## Why are the changes needed? [SPARK-60680](https://issues.apache.org/jira/browse/SPARK-60680): restore the Python Connect client's positional column indexing behavior. ## How was this patch tested? - 19 focused tests using a separate-process mock gRPC server, passing against both the candidate and official Python client. Covers positive/negative positions, bounds and huge integers, booleans, subclasses, empty schemas, invalid types, schema errors, special column names, and named-access RPC guards. - 274 existing offline tests pass on both baseline and candidate with `--rootdir=python --import-mode=importlib`. Supplemental wire comparison confirms ten identical selected column names; candidate performs six schema RPCs versus reference Python's cached one. - 392 native client library tests and 95 core transport library tests passed. Full native tests compiled with default and `wasm-udf` features. Release wheel built with `maturin build --release --locked`; formatting passed. - Unchanged frozen client benchmark: five fresh subprocess samples per case/client across four positional cases and five guards. Candidate and official Python: 45/45 pass each. Master baseline: 25/45 pass, with exactly the 20 expected positional `TypeError` failures. Named-column access and all unrelated guards pass for all clients. Each candidate positional sample performs one schema RPC; named access performs none. Compatibility fix, no speedup claim. Validation uses a scripted localhost gRPC service, with no Spark server or JVM. A workspace-wide compile attempt was blocked by the unrelated WASM example's missing `wasm32-unknown-unknown` target; native tests compile independently. Initial sandbox TLS failure passed with network permissions. Failed setup and baseline evidence are retained in the campaign artifacts. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
