[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15397 Thanks! Merging to master and 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-13 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15397 LGTM, thanks for talking it through --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15397 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15397 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66908/ Test PASSed. ---

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15397 **[Test build #66908 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66908/consoleFull)** for PR 15397 at commit

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15397 **[Test build #66908 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66908/consoleFull)** for PR 15397 at commit

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15397 > My main point is that whoever implements SPARK-17812 is going to have to deal with the issue shown in SPARK-17782, which means much of this patch is going to need to be changed anyway.

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-12 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15397 My main point is that whoever implements SPARK-17812 is going to have to deal with the issue shown in SPARK-17782, which means much of this patch is going to need to be changed anyway.

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15397 **[Test build #66849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66849/consoleFull)** for PR 15397 at commit

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15397 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15397 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66849/ Test PASSed. ---

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-12 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15397 @koeninger sorry for the delay. Right now KafkaSource doesn't support external group id, so we don't need to concern about how to fetching committed offsets. Any other cases that I'm missing?

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15397 **[Test build #66849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66849/consoleFull)** for PR 15397 at commit

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-10 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15397 Look at the poll/seek implementation in the DStream's subscribe and subscribe pattern when user offsets are provided, i.e. the problem that triggered this ticket to begin with. You're going

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-09 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15397 > How is this going to work with assign? It seems like it's just avoiding the problem, not fixing it. We can seek to the offsets provided by the user. --- If your project is set up for

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-07 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15397 How is this going to work with assign? It seems like it's just avoiding the problem, not fixing it. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15397 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15397 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66543/ Test PASSed. ---

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15397 **[Test build #66543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66543/consoleFull)** for PR 15397 at commit

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15397 **[Test build #66543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66543/consoleFull)** for PR 15397 at commit

[GitHub] spark issue #15397: [SPARK-17834][SQL]Fetch the earliest offsets manually in...

2016-10-07 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15397 /cc @tdas @koeninger --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes