Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20465
I agree that pandas and pyarrow should not be a hard requirement for users,
and this is what it is today: PySpark only throws exception when users try to
use pandas related functions without pandas/pyarrow installed.
My proposal is, pandas and pyarrow should be a hard requirement for our
jenkins, to make sure the features are well tested.
If there is a way to prove that py3 tests run well, and the environment
issue is hard to fix, then we maybe we can deal with it later, after 2.3.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
