Github user jose-torres commented on the issue:
https://github.com/apache/spark/pull/20397
In general I have very weak opinions on what classes are named :)
I agree that readers and writers are very different in the DataSourceV2
API, and they're even more so in
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20397
what do streaming guys think? cc @tdas @jose-torres
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20397
@cloud-fan, can we revert this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20397
I am careful to say this out .. but let me leave my +0 for
https://github.com/apache/spark/pull/20397#issuecomment-361345426. One option
might be a similar name with `DataWriterFactory` but
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20397
I'm kind of convinced but maybe it's because I wrote this `ReadTask`. Let's
get feedback from more people, cc @RussellSpitzer @VincentPoncet @HyukjinKwon
@wzhfy @dongjoon-hyun @j-baker
---
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20397
This is more confusing, not less. Look at @jiangxb1987's comment above: "We
shall create only one DataReaderFactory, and have that create multiple data
readers." It is not clear why the API requires
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20397
My point is, `ReadTask` is more precise, but `DataReaderFactor` also
works(Spark serialize and send it to executors, and ask it to create data
reader, it's reasonable to call it a factory). If we
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20397
> I think the renaming is worth to remove future confusions.
What future confusion?
I understand that the difference isn't obvious, but making the names less
accurate isn't a good
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20397
About your last point, it's mostly my fault that I didn't schedule the work
well and missed this one. Since the last RC failed and next RC is not started
yet, I think this is a good window to get
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20397
About the renaming, a lot of people complained to me about why the namings
are not consistent, including @rxin . I named it `ReadTask` at the beginning
because it really works like a task. But I
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20397
One last point: should significant changes to public APIs like this go in
just before or just after a release? 2.3.0 candidates have used ReadTask up to
now.
---
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20397
@cloud-fan, thanks for pinging me on this.
-1: I don't think there's a compelling benefit to justify this change, and
I think it makes the API more confusing. I think we should revert this.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20397
The previous commit passed all test, and the last commit just changed some
comment and has nothing to do with the failed test, I'm merging it to
master/2.3, thanks!
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86770/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86770 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86770/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86772/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86772 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86772/testReport)**
for PR 20397 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86772 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86772/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/339/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/336/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86770 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86770/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86763/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86763 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86763/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86763 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86763/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/331/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86754/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86754 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86754/testReport)**
for PR 20397 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20397
LGTM except a few comments. also cc @rdblue
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86754 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86754/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/323/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86721 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86721/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/298/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20397
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86708 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86708/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86708/
Test FAILed.
---
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/20397
cc @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86708 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86708/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/288/
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/20397
I thought the idea behind having a different name was to differentiate from
the writer factory due to the difference in semantics - ReadTask will
create only 1 DataReader in a task, whereas a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86678/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86678 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86678/testReport)**
for PR 20397 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86678 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86678/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/262/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86646/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86646 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86646/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86644/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86644 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86644/testReport)**
for PR 20397 at commit
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20397
This is replacing the original name so LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20397
cc @tdas This PR might resolve your original comment about the
inconsistency between v2.reader and v2.writer
---
-
To
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86646 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86646/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/237/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20397
**[Test build #86644 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86644/testReport)**
for PR 20397 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/235/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20397
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
70 matches
Mail list logo
