Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20894
LGTM
Thanks! Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91399/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #91399 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91399/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #91399 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91399/testReport)**
for PR 20894 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
@gatorsmile @cloud-fan @HyukjinKwon @gengliangwang Could you look at the PR
one more time.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91155/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #91155 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91155/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #91155 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91155/testReport)**
for PR 20894 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90993/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90993 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90993/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90993 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90993/testReport)**
for PR 20894 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
@gatorsmile @cloud-fan @HyukjinKwon @gengliangwang May I ask you to look at
the PR again.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90809/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90809 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90809/testReport)**
for PR 20894 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90808/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90808 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90808/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90809 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90809/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90808 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90808/testReport)**
for PR 20894 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90756/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90756 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90756/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90756 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90756/testReport)**
for PR 20894 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90738/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90738 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90738/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90738 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90738/testReport)**
for PR 20894 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20894
I understand the default value of `enforceSchema` must be true to keep
backward compatibility, but shall we suggest users to set it to false to avoid
incorrect result in the document?
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20894
Right, will proceed reviewing this.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90682/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90682 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90682/testReport)**
for PR 20894 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20894
In this PR, when documenting the option `enforceSchema`, we need to
emphasize the schemas are checked by positions instead of by names.
@ssimeonov `UNION` is a good example. Many Spark
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90682 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90682/testReport)**
for PR 20894 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20894
cc @cloud-fan, I would like to listen what you think too.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20894
Can we check if there's any committer positive on this change before going
ahead?
---
-
To unsubscribe, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20894
ping @gengliangwang
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
@gengliangwang @gatorsmile May I ask you to look at this PR one more time.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90598/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90598 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90598/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90598 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90598/testReport)**
for PR 20894 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20894
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
jenkins, retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90589/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90589 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90589/testReport)**
for PR 20894 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20894
Documentation does solve the problem in a way because we are going to claim
what Spark supports and does. How about adding a script or something to find
out by reading ahead or another way?
Github user ssimeonov commented on the issue:
https://github.com/apache/spark/pull/20894
@HyukjinKwon we are one of the Spark users experiencing this problem in the
real world: dealing with dirty data produced by a variety of third party
systems. Documentation doesn't solve anything
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90589 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90589/testReport)**
for PR 20894 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90581 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90581/testReport)**
for PR 20894 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90581/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90581 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90581/testReport)**
for PR 20894 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20894
I don't have a particularly better solution for now. Documentation _might
be_ one option in such case given past discussions and suggestions from other
committers and reviewers.
---
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
> For example, there are so many options that can be potentially added
(other univocity parser options).
You are right, so many things can be added but in this particular case we
are
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20894
I understood the rationale, problem and the approach and I don't feel
strongly - I am less sure since 1. this option is specific to CSV about
columns. It sounds adding complexity. 2. however, I
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
@HyukjinKwon @gengliangwang @gatorsmile Please, have a look at it again.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90215/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90215 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90215/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90215 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90215/testReport)**
for PR 20894 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
jenkins, retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90049/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90049 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90049/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90049 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90049/testReport)**
for PR 20894 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
jenkins, retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
> The case exists in all data source format, right?
Not in all, for example, JSON datasource is more tolerant to field order in
json records. Let's say if you have the schema:
```
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/20894
Some questions here:
1. The case exists in all data source format, right?
2. If user didn't provide schema, should we check the header among CSV
files?
3. Users should be
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90011/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90011 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90011/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #90011 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90011/testReport)**
for PR 20894 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20894
cc @gengliangwang Could you review this CSV PR?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89940/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #89940 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89940/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #89940 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89940/testReport)**
for PR 20894 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
@gatorsmile @HyukjinKwon Could you look at the PR again, please.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89501/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #89501 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89501/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #89501 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89501/testReport)**
for PR 20894 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89497/
Test FAILed.
---
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
jenkins, retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #89497 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89497/testReport)**
for PR 20894 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #89497 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89497/testReport)**
for PR 20894 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #89496 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89496/testReport)**
for PR 20894 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89496/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20894
**[Test build #89496 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89496/testReport)**
for PR 20894 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/20894
jenkins, retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89365/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20894
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
1 - 100 of 136 matches
Mail list logo
