dbtsai commented on PR #108: URL: https://github.com/apache/spark-connect-rust/pull/108#issuecomment-6502194724
Follow-up to the remaining CI-wiring item: pushed e0eb002, which adds `test_arrow_e2e.py` to the Connect parity job in `.github/workflows/build_python_connect.yml`. The step uses the already-built native extension and existing live server, with `SPARK_REMOTE=sc://localhost:15002` and `PYTHONPATH` pointing at this repository's `python` directory. It runs after the official suite and before the local-cluster restart, preserving the pristine-server start for official parity. These three tests now directly exercise the native `toArrow()` path in CI instead of relying on the official-PySpark transport gate. Validated workflow YAML parsing and ran the exact pytest command locally against Spark Connect: **3 passed**. The architecture docs and PR description now reflect the CI wiring. This addresses the previously open wiring request; the new CI run still needs to finish. PyArrow 10 remains covered by the documented local runs, rather than a pinned CI job. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
