[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao closed the pull request at: https://github.com/apache/spark/pull/4805 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-120210548 OK, get it, that makes sense. I'm closing it now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-120210226 @jerryshao We have now exposed offsets in both Java/Scala/Python API, the user can post the offsets in their own data stores. As @koeninger has pointed out in the JIRA, usi

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user zzcclp commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-100752171 Hi @koeninger , of cource, I can read kafka offset from ZK when recovering by myself, I just want to know that writing to ZK is built-in support or not. --- If your pro

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-100741752 Is there anything you need to do that couldn't be accomplished by reading from / writing to ZK yourself? Is this just a question of convenient api for kafka

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user zzcclp commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-100741128 Hi @jerryshao , current spark version is not built-in support for upgrade and accumulators when recovering from checkpoint, so I want to add kafka offset to zk, and read

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-100103582 Hi @zzcclp , would you please describe your requirement and scenario to use this feature. We're still not sure about the necessity of this feature to put offset into Z

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user zzcclp commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-100087287 is there any conclusion? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user zzcclp commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-100077362 is there any conclusion? @jerryshao , could it add a config option to decide whether adding zk offest posting? just my bit suggest:) --- If your project is set up fo

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99101507 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/318

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99101501 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99101459 [Test build #31877 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31877/consoleFull) for PR 4805 at commit [`3d7e0c8`](https://gith

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99081192 I just want to re-iterate from the jira discussion that I am thumbs-down to the idea of overloading the meaning of group.id, and thumbs-down to adding options that conf

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99073760 [Test build #31877 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31877/consoleFull) for PR 4805 at commit [`3d7e0c8`](https://githu

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99073321 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99073301 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99069573 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99069575 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/318

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99069570 [Test build #31876 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31876/consoleFull) for PR 4805 at commit [`98ec19f`](https://gith

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99069238 [Test build #31876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31876/consoleFull) for PR 4805 at commit [`98ec19f`](https://githu

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99068861 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99068961 Just rebase the code, I will change the implementation according to feedbacks. --- If your project is set up for it, you can reply to this email and have your reply ap

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99068895 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99013270 Can you update with master. Will take a look tomorrow and decide. On May 5, 2015 2:29 AM, "Saisai Shao" wrote: > Hi @tdas , it's quite

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99008013 Hi @tdas , it's quite a old PR, since @koeninger has several comments on this, I'm not sure which way is the good way :). --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-99006515 This had fallen off my radar. @jerryshao Lets try to get this PR in for Spark 1.4.0 --- If your project is set up for it, you can reply to this email and have your reply ap

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-78121376 @koeninger I agree the purpose of this PR was just to update ZK for tools to work. I am thinking beyond on the bigger goal to make this new API a superset in features of the

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26150162 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -118,6 +123,7 @@ class DirectKafkaInputDStrea

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-78053440 I am now confused about what the purpose of this PR is. The jira seemed to indicate that the problem was "several third-party offset monitoring tools fail to monitor t

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26120607 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -84,6 +83,11 @@ class DirectKafkaInputDStream

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26120488 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -118,6 +123,7 @@ class DirectKafkaInputDStrea

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77988079 If checkpointing is not enabled, then we cannot restart on failure any ways. So the system cannot provide any guarantee. Thats not the point of this update. The point of thi

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77985091 Yes, that's my original purpose of PR, user can monitor and reuse these offsets, also we can offer user the similar functionality as old Kafka stream to get data from p

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26094221 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -118,6 +123,7 @@ class DirectKafkaInputDStream[

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77982638 This got me thinking. To make this direct stream functionally similar to the older stream, we should also pickup the offset position from the ZK if group.id is specified. In

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26093934 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -84,6 +83,11 @@ class DirectKafkaInputDStream[

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26093882 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +165,23 @@ class DirectKafkaInputDStream[

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26093688 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -118,6 +123,7 @@ class DirectKafkaInputDStrea

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26048624 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -118,6 +123,7 @@ class DirectKafkaInputDStrea

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26048829 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -84,6 +83,11 @@ class DirectKafkaInputDStream

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77882744 As it stands now, no offsets are stored by spark unless you're checkpointing. Does it really make sense to have an option to automatically store offsets in Kafka

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77805729 [Test build #28380 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28380/consoleFull) for PR 4805 at commit [`e02e2d2`](https://gith

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77805742 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77805422 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77805418 [Test build #28379 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28379/consoleFull) for PR 4805 at commit [`610979a`](https://gith

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77801344 Hi @koeninger , would you please review this again? Thanks a lot and appreciate your time. Here I still keep using the HashMap for Time -> offset relation mappi

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77800707 [Test build #28380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28380/consoleFull) for PR 4805 at commit [`e02e2d2`](https://githu

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77800424 [Test build #28379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28379/consoleFull) for PR 4805 at commit [`610979a`](https://githu

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26013229 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -82,8 +83,12 @@ class DirectKafkaInputDStream

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26012257 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class DirectKafkaInputDStre

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26012160 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class DirectKafkaInputDStre

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26011997 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class DirectKafkaInputDStre

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r26011962 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/SparkKafkaUtils.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77787450 Hi @koeninger , thanks a lot for your review. I will the fix the all the comments you addressed. The reason why I put updating ZK in `StreamingListener` rather

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25991083 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class DirectKafkaInputDStre

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990950 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class DirectKafkaInputDStre

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990850 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/SparkKafkaUtils.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990774 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/ReliableKafkaReceiver.scala --- @@ -239,21 +239,7 @@ class ReliableKafkaReceiver[

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990749 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class DirectKafkaInputDStre

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990695 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class DirectKafkaInputDStre

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990683 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -158,4 +166,37 @@ class DirectKafkaInputDStre

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25990568 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala --- @@ -82,8 +83,12 @@ class DirectKafkaInputDStream

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4805#discussion_r25987964 --- Diff: project/MimaExcludes.scala --- @@ -153,6 +153,11 @@ object MimaExcludes { ProblemFilters.exclude[MissingClassProblem]("org.apache.spar

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user tdas commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77656954 @koeninger Would be good to have you in the conversation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77107444 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77107436 [Test build #28252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28252/consoleFull) for PR 4805 at commit [`eb273e1`](https://gith

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-77101224 [Test build #28252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28252/consoleFull) for PR 4805 at commit [`eb273e1`](https://githu

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-76458198 [Test build #28078 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28078/consoleFull) for PR 4805 at commit [`526e343`](https://gith

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-76458212 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-76444874 [Test build #28078 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28078/consoleFull) for PR 4805 at commit [`526e343`](https://githu

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-76349481 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-76349478 [Test build #28049 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28049/consoleFull) for PR 4805 at commit [`0ac450a`](https://gith

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4805#issuecomment-76342336 [Test build #28049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28049/consoleFull) for PR 4805 at commit [`0ac450a`](https://githu

[GitHub] spark pull request: [SPARK-6051][Streaming] Add ZooKeeper offest p...

GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/4805 [SPARK-6051][Streaming] Add ZooKeeper offest posting for DirectKafkaInputDStream Currently in DirectKafkaInputDStream, offset is managed by Spark Streaming itself without ZK or Kafka involved, wh