Github user staslos commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-123036437
Does this work on executors in YARN cluster?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-55478165
This looks good to me (Davies and I walked through the code offline). I'm
going to merge this into `master`. Thanks!
---
If your project is set up for it, you can
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2144
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-55297241
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20161/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-55305217
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20159/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-55306714
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20161/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-55224000
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/64/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-55295841
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20159/consoleFull)
for PR 2144 at commit
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-55220021
Thanks to @shaneknapp we now have `pypy-2.0.2-1.el6.x86_64` on the Jenkins
workers, so I'm going to try retesting this.
---
If your project is set up for it, you can
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-55220156
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/64/consoleFull)
for PR 2144 at commit
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54731796
I'm waiting to figure out the right procedure for installing `pypy` on our
Jenkins boxes; once I have this figured out, I'll loop back to finish
reviewing. From what
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54654606
@mateiz @JoshRosen @mattf run-tests will try to run tests for spark core
and sql with PyPy.
One known issue is that serialization of array in PyPy is similar to
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54654638
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54669121
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54695060
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19855/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54698587
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19855/consoleFull)
for PR 2144 at commit
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54513664
PyPy is fully compatible with CPython for pure Python code, so it's not
necessary to test against every commit with PyPy.
Maybe we could have nightly tests (for
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54554607
PyPy does not fully support NumPy right now, so MLlib can not run with PyPy.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54564941
So you guys should figure out a way to run this so that it doesn't get
stale. For example it's fine to add some code to the script that runs all the
tests except the
Github user mattf commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54566230
So you guys should figure out a way to run this so that it doesn't get
stale. For example it's fine to add some code to the script that runs all the
tests except the
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54567184
Let's just have the PyPy tests run by default on Jenkins. If this causes
build speed problems later down the road, we can revisit the issue of
selectively running
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54398757
One concern with adding tests for `pypy` is that it might significantly
increase the runtime of the Jenkins tests. We should test regularly with
`pypy` to make sure
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-54399419
How long do the Python tests run now?
Anyway, we could do PyPy only if Python code changed (but I'd still do
Python all the time).
---
If your project is set up
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53979487
Yes, I will do that next week.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53970492
@davies just curious, do all the unit tests run if you do `run-tests` with
`pypy`? We should make sure they do, and add a command in there to test this in
Jenkins (ask
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53938391
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19504/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53941281
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19504/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53783568
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19419/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53804387
**Tests timed out** after a configured wait of `120m`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53807775
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19430/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53817298
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19430/consoleFull)
for PR 2144 at commit
GitHub user davies opened a pull request:
https://github.com/apache/spark/pull/2144
[SPARK-3094] [PySpark] compatitable with PyPy
After this patch, we can run PySpark in PyPy (testing with PyPy 2.3.1 in
Mac 10.9), for example:
```
PYSPARK_PYTHON=pypy ./bin/spark-submit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53475635
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19230/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53495819
**Tests timed out** after a configured wait of `120m`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53498333
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53499028
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19242/consoleFull)
for PR 2144 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53509963
**Tests timed out** after a configured wait of `120m`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/2144#issuecomment-53512874
This looks like it will be tricky to maintain without automated testing.
Can you update dev/run-tests to also run the PySpark tests with PyPy maybe? You
might need help
38 matches
Mail list logo
