[jira] [Commented] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds
[ https://issues.apache.org/jira/browse/SPARK-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566949#comment-16566949 ] shane knapp commented on SPARK-24980: - looking pretty good (you have to go to the very bottom of the 16M build log to get to the output) : [https://amplab.cs.berkeley.edu/jenkins/view/RISELab%20Infra/job/spark-master-test-sbt-hadoop-2.6-ubuntu-test/896/consoleFull] summary: * py27 and py34 pandas and pyarrow tests run * pypy doesn't have any additional libs installed, plus the version we're running is pretty old – 2.5.1. the following tests were skipped: pandas, pyarrow, numpy, scipy * i will need to bump the unittest py2.7 lib to >= 3.3 to support mocks * we should really think about bumping python3 from 3.4 to 3.5 at some point in the not-too-distant-future > add support for pandas/arrow etc for python2.7 and pypy builds > -- > > Key: SPARK-24980 > URL: https://issues.apache.org/jira/browse/SPARK-24980 > Project: Spark > Issue Type: Improvement > Components: Build, PySpark >Affects Versions: 2.3.1 >Reporter: shane knapp >Assignee: shane knapp >Priority: Major > > since we have full support for python3.4 via anaconda, it's time to create > similar environments for 2.7 and pypy 2.5.1. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds
[ https://issues.apache.org/jira/browse/SPARK-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565780#comment-16565780 ] shane knapp commented on SPARK-24980: - alright, pandas 0.19.2 and pyarrow 0.8.0 are installed for py27 on the ubuntu workers. this doesn't provide full coverage for tests, but here's the list of builds pinned to these workers: [https://amplab.cs.berkeley.edu/jenkins/label/ubuntu/] one of my test builds is currently running, which will show if these tests are being run against pandas and pyarrow w/py27: [https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.6-ubuntu-test/] i'll deal w/the centos updates later. > add support for pandas/arrow etc for python2.7 and pypy builds > -- > > Key: SPARK-24980 > URL: https://issues.apache.org/jira/browse/SPARK-24980 > Project: Spark > Issue Type: Improvement > Components: Build, PySpark >Affects Versions: 2.3.1 >Reporter: shane knapp >Assignee: shane knapp >Priority: Major > > since we have full support for python3.4 via anaconda, it's time to create > similar environments for 2.7 and pypy 2.5.1. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds
[ https://issues.apache.org/jira/browse/SPARK-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564813#comment-16564813 ] Hyukjin Kwon commented on SPARK-24980: -- Oh BTW, [~bryanc], do you remember if PyArrow is distributed and available in PyPy too? I remember it wasn't there so I could install PyArrow in PyPy before. Pandas is there for PyPy. > add support for pandas/arrow etc for python2.7 and pypy builds > -- > > Key: SPARK-24980 > URL: https://issues.apache.org/jira/browse/SPARK-24980 > Project: Spark > Issue Type: Improvement > Components: Build, PySpark >Affects Versions: 2.3.1 >Reporter: shane knapp >Assignee: shane knapp >Priority: Major > > since we have full support for python3.4 via anaconda, it's time to create > similar environments for 2.7 and pypy 2.5.1. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds
[ https://issues.apache.org/jira/browse/SPARK-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564808#comment-16564808 ] Hyukjin Kwon commented on SPARK-24980: -- Thank you for cc'ing me Shane! > add support for pandas/arrow etc for python2.7 and pypy builds > -- > > Key: SPARK-24980 > URL: https://issues.apache.org/jira/browse/SPARK-24980 > Project: Spark > Issue Type: Improvement > Components: Build, PySpark >Affects Versions: 2.3.1 >Reporter: shane knapp >Assignee: shane knapp >Priority: Major > > since we have full support for python3.4 via anaconda, it's time to create > similar environments for 2.7 and pypy 2.5.1. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24980) add support for pandas/arrow etc for python2.7 and pypy builds
[ https://issues.apache.org/jira/browse/SPARK-24980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564154#comment-16564154 ] shane knapp commented on SPARK-24980: - [~hyukjin.kwon] thought you might be interested in this! ;) > add support for pandas/arrow etc for python2.7 and pypy builds > -- > > Key: SPARK-24980 > URL: https://issues.apache.org/jira/browse/SPARK-24980 > Project: Spark > Issue Type: Improvement > Components: Build, PySpark >Affects Versions: 2.3.1 >Reporter: shane knapp >Assignee: shane knapp >Priority: Major > > since we have full support for python3.4 via anaconda, it's time to create > similar environments for 2.7 and pypy 2.5.1. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
