Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/19884
@wesm I was able to install pyarrow 0.8.0 to my local environment via
conda. Thanks!
---
-
To unsubscribe, e-mail:
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
I just uploaded the pip packages for Windows and Linux so they are
available. There is an error building the Mac packages, so those will come
later after that is resolved.
---
Github user wesm commented on the issue:
https://github.com/apache/spark/pull/19884
If you want to install pyarrow 0.8.0 via conda it's available now from the
`-c conda-forge` channel (https://anaconda.org/conda-forge/pyarrow). I am not
sure where we are at on PyPI / pip packages --
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
Just for a refreshing reminder about Jenkins, I happened to check what we
have in Jenkins roughly a month ago (just simply by printing out the versions
within PySpark tests) in a specific
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/19884
@wesm Yes, I'd like to use it asap to verify this patch and to confirm the
behavior of my PR #18754 for `DecimalType` support. Thanks.
---
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/19884
the install is done w/pip in a conda environment.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user wesm commented on the issue:
https://github.com/apache/spark/pull/19884
Just as a matter of prioritization -- do you need pip or conda packages to
be able to proceed with finishing/verifying this patch? Getting pip packages up
on PyPI shouldn't take too long after the
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84743 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84743/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84743/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84743 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84743/testReport)**
for PR 19884 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
>yeah, i can do the upgrade next week. i'll be working remotely from the
east coast, but unavailable at all on monday due to travel.
Great, thanks @shaneknapp ! I'll ping you when I
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84740 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84740/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84740/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84740 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84740/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84738 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84738/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84738/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84738 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84738/testReport)**
for PR 19884 at commit
Github user wesm commented on the issue:
https://github.com/apache/spark/pull/19884
The Arrow 0.8.0 release vote just started today. Assuming it passes, the
earliest you could see packages pushed to PyPI or conda-forge would be sometime
on Thursday evening or Friday.
---
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/19884
yeah, i can do the upgrade next week. i'll be working remotely from the
east coast, but unavailable at all on monday due to travel.
On Mon, Dec 11, 2017 at 1:59 PM, Bryan Cutler
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/19884
@BryanCutler could you just pull my changes into this PR since we need to
both changes to pass Jenkins? Thanks!
---
-
To
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
>Jenkins cannot support to install multiple versions of PyArrow?
@zsxwing that's right, we will have to coordinate to make sure the Jenkins
pyarrow is upgraded to version 0.8 as well.
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
>When I tried to run tests locally, I got OutOfMemoryException
@ueshin , you got that error because the latest Arrow has upgraded Netty to
4.1.17 but Spark has an older version on the
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/19884
I saw #18974 tried to upgrade Arrow but got closed due to some Jenkins
issue. @ueshin do you have any idea what may block this PR? Jenkins cannot
support to install multiple versions of PyArrow?
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84663 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84663/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84663/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84663 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84663/testReport)**
for PR 19884 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84616 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84616/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84616/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84616 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84616/testReport)**
for PR 19884 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
@zsxwing, fyi after applying your Netty upgrade patch to Arrow, and then
your other patch for Spark, all of the Spark Scala/Java tests pass
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
cc @zsxwing as well, I saw you opened a JIRA about this - SPARK-22656
---
-
To unsubscribe, e-mail:
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
> Great, @BryanCutler . Could you put the highlight in the PR description,
too?
Sure, thanks @dongjoon-hyun ! Will do, just want to go back and check the
release notes first
---
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
This is a WIP to start updating Spark to use Arrow 0.8.0 which will be
released soon.
TODO:
- [ ] Update to reflect Java API changes
- [ ] Update to reflect Python API
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/19884
Great, @BryanCutler . Could you put the highlight in the PR description,
too?
---
-
To unsubscribe, e-mail:
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
The highlights that pertain to Spark for the update from Arrow versoin
0.4.1 to 0.8.0 include:
* Java refactoring for more simple API
* Type support for DecimalType, ArrayType
*
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84447 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84447/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84447/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #84447 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84447/testReport)**
for PR 19884 at commit
44 matches
Mail list logo
