agriyakhetarpal opened a new pull request, #55588: URL: https://github.com/apache/arrow/pull/55588
### Rationale for this change Closes #55586. This PR removes the `NodeDriver` part of the Emscripten/Pyodide testing infrastructure to replace it with a native replacement that we offer, called `pyodide venv`. Here is our documentation: https://pyodide-build.readthedocs.io/en/stable/how-to/testing.html Since the Pyodide Python interpreter included in this virtual environment can access the file system, the timezone database directory can now be read, which enables the tests for the ORC reader. ### What changes are included in this PR? This PR includes the following changes: - Removes the `NodeDriver` testing class and `runtime=node` option from `scripts/run_emscripten_tests.py`, and replaces them with a `pyodide venv` invocation, where testing dependencies (either pure Python or WASM wheels) can be pip-installed, akin to a conventional Python virtual environment. The documentation has been updated with a short section to reflect this. - Additional dependencies have been enabled for testing: `cffi`, `cloudpickle`, `fsspec`, and `scipy`, which allow for 380 or so extra tests on Node, and 346 more in Chrome, to be run and pass. The remaining skips are due to features not built for the Emscripten platform, such as S3, Flight, and threading. - The ORC tests have been enabled, since `/usr/share/zoneinfo` can now be accessed. - Two tests out of those, `test_timezone_database_absent` and `test_timezone_absent`, rely on subprocesses and are skipped accordingly. - The Substrait expression serialisation test that was noted to crash Pyodide at some point no longer fails on my machine, so I've unskipped it. :) ### Are these changes tested? Yes, I have built a WASM build for Arrow, and a pyemscripten wheel for PyArrow, and tested the changes locally in a Pyodide virtual environment and with Chrome installed on my macOS machine/ ### Are there any user-facing changes? No, there are no user-facing changes as part of this PR. ### Was AI used for this PR? In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR. **PR code and description written by:** - [x] Human - [ ] AI **Reviewed before submission by:** - [x] Human - [ ] AI - [ ] Not reviewed I have not used any AI tools for this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
