Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20567
I mean I got that a nicer error message is useful of course but wouldn't it
be better to match the behaviour between `toPandas` and `createDataFrame`
before 2.3.0 if its complexity looks
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20567
Yup, I also agree with adding a configuration to control this. I will work
on it for master only later.
For https://github.com/apache/spark/pull/20567#issuecomment-364994740, yup.
I
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/20567
A quick bit: fallback is a single word.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20567
Regarding the error message, this is a good example to show how to provide
a user-friendly message. To the external end users, they might not be aware of
the internal implementation. They might
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20567
This is kind of like what we did for whole-stage codegen. We have a conf
like `spark.sql.codegen.fallback` to decide whether we should fail fast or go
back to the slow path. I would suggest to
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/20567
Sorry I am late to the party.
https://github.com/apache/spark/pull/20567#issuecomment-364639922 Does look
like a bug to me. However, I am a bit concerned that such magic behavior would
be not
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87332/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20567
**[Test build #87332 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87332/testReport)**
for PR 20567 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/809/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20567
**[Test build #87332 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87332/testReport)**
for PR 20567 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20567
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/20567
LGTM. I'd leave it to @HyukjinKwon and @gatorsmile whether we should merge
this into branch-2.3 or not.
---
-
To unsubscribe,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87327/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20567
**[Test build #87327 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87327/testReport)**
for PR 20567 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20567
**[Test build #87327 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87327/testReport)**
for PR 20567 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/805/
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20567
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87324/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20567
**[Test build #87324 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87324/testReport)**
for PR 20567 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20567
**[Test build #87324 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87324/testReport)**
for PR 20567 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/803/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20567
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20567
@ueshin, yup, I initially thought so but realised that it might collect
twice (`_collectAsArrow`, `collect`) and trigger two jobs due to one failure in
execution time. Also, seems it could
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20567
https://github.com/apache/spark/pull/20567#issuecomment-364639922 case is
actually closer to a bug as both output from one without Arrow and with Arrow
are different and inconsistent. The
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20567
Since this PR is not a bug fix, we will not merge it to 2.3. How about
submitting another PR to throw a better error message in to-be-released 2.3?
---
30 matches
Mail list logo
