Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
For now, you can ruin
https://github.com/apache/spark/blob/master/python/run-tests-with-coverage
script to check the coverage.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
@gatorsmile, I'm working on test coverage report and already almost
finished. Only left job is to setup a Jenkins job as I talked. cc @shaneknapp
---
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/21546
Thanks @HyukjinKwon and others who reviewed!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95397/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95397 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95397/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95390/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95390 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95390/testReport)**
for PR 21546 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/21546
@HyukjinKwon I redid the benchmarks for `toPandas` with the current code
and updated the description. It's not a huge speedup now, but definitely does
improve some. I'll also followup with
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95397 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95397/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/21546
Yes, that is the worst case. If there is some bug with types/schema then
there is an automatic fallback to the non-arrow code path too
On Tue, Aug 28, 2018, 7:16 PM Xiao Li wrote:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
Yup, since `spark.sql.execution.arrow.enabled` is an experimental feature,
we could just turn this off if there are critical bugs found later after the
release.
---
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21546
@BryanCutler The worst case is to turn off
`spark.sql.execution.arrow.enabled`, if the new code path has a bug, right?
---
-
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/21546
@gatorsmile , this is just the format for Arrow IPC between the JVM and
Python process and although it used the Arrow File format, there is nothing
persisted. There is no real reason to keep
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
It's not a bug. Here's what's going on now:
He proposed a way to write out regardless of partition order, I suggested
to revert that part back (see
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95390 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95390/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95363/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95363 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95363/testReport)**
for PR 21546 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95363 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95363/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95340/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95340 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95340/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95340 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95340/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95325/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95325 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95325/testReport)**
for PR 21546 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95325 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95325/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95310/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95310 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95310/testReport)**
for PR 21546 at commit
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/21546
Hey @HyukjinKwon , after going through the previous benchmarks, it seems
out-of-order batches had more of an effect on performance that I thought with
`toPandas`. The current revision of this
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95310 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95310/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/21546
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95296/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95296 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95296/testReport)**
for PR 21546 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95296 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95296/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
Eh @BryanCutler, mind resolving conflicts please?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95145/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95145 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95145/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95145 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95145/testReport)**
for PR 21546 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95137/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95137 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95137/testReport)**
for PR 21546 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95137 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95137/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
LGTM otherwise.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95093 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95093/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95093/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95093 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95093/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/21546
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95083/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95083 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95083/testReport)**
for PR 21546 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95083 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95083/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95055 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95055/testReport)**
for PR 21546 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95055/
Test FAILed.
---
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/21546
@HyukjinKwon @felixcheung , I reverted the out-of-order batching for now
(we can look at that in another PR) and cleaned up things after the Arrow
0.10.0 upgrade. If this looks good to you
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #95055 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95055/testReport)**
for PR 21546 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
Yea.. I hope so. only one comment left.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21546
so... can we still target this for 2.4? sounds like very good to have and
complimentary to hydrogen
---
-
To unsubscribe,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93465/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21546
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21546
**[Test build #93465 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93465/testReport)**
for PR 21546 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21546
SGTM except the discussion going on in
https://github.com/apache/spark/pull/21546#discussion_r204324646
---
-
To
1 - 100 of 117 matches
Mail list logo
