Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
I think we should test PyArrow and Pandas with PyPy and Python 2.x too ...
why not?
@shaneknapp, how about upgrading the current PyArrow (python 3.4.5: pyarrow
0.4.1 -> python 3.4.5: pyarrow 0.8.0), for now and then separately proceed
others later if it is relatively easy to you (assuming you are currently ready
only for upgrading PyArrow for now)?
Looks this upgrade blocks #18754 so upgrading python 3.4.5: pyarrow 0.4.1
-> python 3.4.5: pyarrow 0.8.0 alone should relatively be safe and not break
the tests.
I have seen few tests failed in Python 2 and PyPy with Pandas and PyArrow
in my local if i am not mistaken but haven't got really enough time to check if
it is an actual issue and file a JIRA. But, I am pretty sure some tests will
fail after the upgrade.
(^ Please let me know if anyone tried this thing before as well ..)
If the below works to you @shaneknapp, I would like to suggest:
1. Upgrade python 3.4.5: pyarrow 0.4.1 -> python 3.4.5: pyarrow 0.8.0
alone, and unblock #18754.
2. Investigate and check if the tests with PyPy and Python 2 actually pass
2.1. If not, file a JIRA and fix at our best first.
3. Upgrade others.
If you'd prefer doing it in one-go, I (and probably some guys here) will
try to investigate if the tests pass with Pandas 0.19.2 and PyArrow with PyPy
and Python 2 first quickly and will let you know.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
