shaneknapp edited a comment on issue #24379: [SPARK-25079][python][branch-2.4] update python3 executable to 3.6.x URL: https://github.com/apache/spark/pull/24379#issuecomment-483854051 > I think the 2.3/2.4 branches will have to stay with pyarrow 0.8.0 and pandas 0.19.2 for the tests to pass. There were a few fixes that were done in master to make later versions of pyarrow work. These could be applied to the branches, but would have to be done manually. The risk is that we could end up breaking support for pyarrow 0.8.0, which would be pretty bad. > edit: i guess py36 does support pyarrow 0.8.0! https://conda.anaconda.org/conda-forge/linux-64/pyarrow-0.8.0-py36_0.tar.bz2 > @shaneknapp do you think you can try pyspark.sql.tests + py36 + pandas 0.19.2 + pyarrow 0.8.0 for the 2.4 branch? I think that should work. Sorry, I should have explained that better before.. sure. give me a couple of hours to make the env/run tests.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
