[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-11-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22547 I was stuck with some personal business recently, I'll send a PR for batch source after the weekend. --- - To unsubscribe,

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-11-16 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/22547 I agree that there is consensus for the proposal in the design doc and I don't think there are any blockers. If there's something I can do to help, please let me know. Otherwise ping me to review!

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-11-16 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/22547 @cloud-fan @rdblue I believe we've converged on an appropriate API as per our last sync. Do we have a plan to move this forward, with the separated smaller patches? ---

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22547 Let's move the high-level discussion to https://docs.google.com/document/d/1uUmKCpWLdh9vHxP7AWJ9EgbwB_U6T3EJYNjhISGmiQg/edit?usp=sharing ---

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-19 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/22547 @jose-torres, I don't mean that the primary purpose of the v2 API is for catalog integration, I mean that the primary use of v2 is with tables that are stored in some catalog. So we should make sure

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-19 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/22547 I agree that we need a shared understanding of the relationship between this work and the new catalog API. I was not under the impression that the primary purpose of v2 is to integrate catalog

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-19 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/22547 After looking at the changes, I want to reiterate that request for a design doc. I think that code is a great way to prototype a design, but that we need to step back and make sure that the design

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-19 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/22547 @cloud-fan, is there a design doc that outlines these changes and the new API structure? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97540/ Test PASSed. ---

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #97540 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97540/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97538/ Test PASSed. ---

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #97538 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97538/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #97540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97540/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22547 A major part of this PR is to update existing streaming sources, which is just moving code around. There are 3 things we need to pay attention to during review: 1. the naming and

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #97538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97538/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97520/ Test FAILed. ---

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #97520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97520/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #97520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97520/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22547 Hi @rdblue welcome back! I just rebased it so it's ready for review :) --- - To unsubscribe, e-mail:

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-17 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/22547 @cloud-fan, sorry to look at this so late, I was out on vacation for a little while. Is this about ready for review? --- - To

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97206/ Test PASSed. ---

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #97206 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97206/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #97206 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97206/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-10-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96570/ Test PASSed. ---

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #96570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96570/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #96570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96570/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22547 cc @rxin @jose-torres @rdblue --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96567/ Test FAILed. ---

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #96567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96567/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96566/ Test FAILed. ---

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #96566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96566/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #96567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96567/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #96566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96566/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96563/ Test FAILed. ---

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #96563 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96563/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22547 **[Test build #96563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96563/testReport)** for PR 22547 at commit

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22547: [SPARK-25528][SQL] data source V2 read side API refactor...

2018-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22547 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional