Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
@HyukjinKwon [ARROW-1949](https://issues.apache.org/jira/browse/ARROW-1949)
was created to add an option to allow truncation when data will be lost. Once
that is in Arrow, we can remove the
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
@BryanCutler, did we resolve
https://github.com/apache/spark/pull/19884#issuecomment-353276931? If not,
shall we file a JIRA?
---
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
> @BryanCutler can you give me a minimal repro for the timestamp issue you
cited above?
Sure @wesm, I'll ping you with a repro
---
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
Thanks all for reviewing and getting the Netty upgrade in also!
---
-
To unsubscribe, e-mail:
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/19884
@HyukjinKwon yeah, I closed the ticket.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user wesm commented on the issue:
https://github.com/apache/spark/pull/19884
@BryanCutler can you give me a minimal repro for the timestamp issue you
cited above?
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
Hi @zsxwing is it okay to resolve SPARK-19552?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
LGTM
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85246/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85246 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85246/testReport)**
for PR 19884 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85246 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85246/testReport)**
for PR 19884 at commit
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/19884
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85244 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85244/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85242/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85244/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85242 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85242/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85244 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85244/testReport)**
for PR 19884 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
I used a workaround for timestamp casts that allows the tests to pass for
me locally, and left a note to look into the root cause later. Hopefully this
should pass now and we will be good to
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85242 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85242/testReport)**
for PR 19884 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19884
LGTM, I'm also fine to ignore some tests if they are hard to fix, to
unblock other PRs sooner.
---
-
To unsubscribe, e-mail:
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
> Quick comment - Hey @BryanCutler, to me I am fine to skip some tests for
now if they take a while to fix.
The last failure was the same one I was seeing locally from
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
Fix comment - Hey @BryanCutler, to me I am fine to skip some tests for now
if they take a while to fix.
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85220/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85220 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85220/testReport)**
for PR 19884 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85220 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85220/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85205/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85205 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85205/testReport)**
for PR 19884 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85205 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85205/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85203 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85203/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85203/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85203 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85203/testReport)**
for PR 19884 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
Ok I did some local testing with these changes and pyarrow 0.8.0 with
different combinations of Python and Pandas:
**python 3.6.3, pandas 0.19.2**
ERROR:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85195/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85195 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85195/testReport)**
for PR 19884 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
Thanks for doing the update @shaneknapp and thanks for looking into the
details @HyukjinKwon !
I'll look into the test issues with python2.7, it looks to be related to
timestamps..
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
I need to go sleep now and I guess @ueshin should be sleeping too. Let me
leave my signoff here - LGTM if the tests pass. I guess now other builds in
other PRs would be broken without this PR.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85195 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85195/testReport)**
for PR 19884 at commit
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/19884
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/19884
alright https://github.com/apache/spark/pull/18754 should now be unblocked.
let me know if there's anything else that needs to happen.
---
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/19884
we should be good to go:
```$ pssh -h jenkins_workers.txt -t 0 "export
PATH=/home/anaconda/envs/py3k/bin$PATH; pip install pyarrow==0.8.0"
[1] 05:55:00 [SUCCESS]
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/19884
@HyukjinKwon @wesm @BryanCutler
alright. here's my plan for right now:
* python 3.4.5 -- upgrade pyarrow --> 0.8.0 (confirmed working on my
staging environment)
what i'm
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
For https://github.com/apache/spark/pull/19884#issuecomment-352993779,
> I have seen few tests failed in Python 2 and PyPy with Pandas and PyArrow
in my local if i am not mistaken and
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85165/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85165 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85165/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85160/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85160 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85160/testReport)**
for PR 19884 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85165 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85165/testReport)**
for PR 19884 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
> We are supporting pyarrow on Python 2.7 except for Windows
Hi @wesm, mind if I ask the details about Windows? I think we should add
few asserts in the version checks separately later
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
I think we should test PyArrow and Pandas with PyPy and Python 2.x too ...
why not?
@shaneknapp, how about upgrading the current PyArrow (python 3.4.5: pyarrow
0.4.1 -> python 3.4.5:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85160 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85160/testReport)**
for PR 19884 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
@shaneknapp afaik Spark is supporting pyarrow with python 2.7 and we should
be testing these also, but I'm not sure about pypy. Maybe @ueshin or
@HyukjinKwon can confirm before we start
Github user wesm commented on the issue:
https://github.com/apache/spark/pull/19884
We need to upgrade to 0.8.0 now because we changed the binary format
---
-
To unsubscribe, e-mail:
Github user wesm commented on the issue:
https://github.com/apache/spark/pull/19884
We are supporting pyarrow on Python 2.7 except for Windows
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/19884
pypy is 2.5.1, no pandas or pyarrow (/usr/bin/pypy -- hand-rolled dist i
put together ~3 years ago)
python 3.4.5: pyarrow 0.4.1, pandas 0.19.2 (managed by anaconda)
python
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85100/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85100 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85100/testReport)**
for PR 19884 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19884
Hmm.. @shaneknapp but if I remember and understood correctly, wouldn't this
fail to build when the PATH is dropped? I ended up with checking both Pandas
and PyArrow at that time because I
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/19884
@BryanCutler @HyukjinKwon
pandas and pyarrow are most definitely installed on all of the jenkins
workers. the 'missing' packages happened after we had a power outage at the
colo, and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85100 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85100/testReport)**
for PR 19884 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19884
Hi @shaneknapp , I think we are all ready here to try updating to pyarrow
0.8.0. The build here should pass once this version is available, if you want
to just try updating a single worker
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85099/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85099 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85099/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85099 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85099/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85048 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85048/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85048/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85048 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85048/testReport)**
for PR 19884 at commit
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/19884
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85044/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85044 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85044/testReport)**
for PR 19884 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85044 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85044/testReport)**
for PR 19884 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19884
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85043/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85043 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85043/testReport)**
for PR 19884 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19884
**[Test build #85043 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85043/testReport)**
for PR 19884 at commit
88 matches
Mail list logo
