[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155211465 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155211188 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155204978 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155204883 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155211487 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155213853 **[Test build #45423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45423/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155254492 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155255206 Thank you very much for all the feedback @tdas @zsxwing and @harishreedharan . This is definitely my most commented on PR. Even the most commented on per line even :S

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155254346 **[Test build #45445 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45445/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155254635 All right! merging this to master and 1.6. This was a complicated PR and the reviewing was really long. Thank you very much for all the effort @brkyvz and @zsxwing ---

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155254494 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155147279 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155129633 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155129611 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44310193 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -599,6 +621,10 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155147059 **[Test build #45381 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45381/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44310128 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +493,24 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155150877 **[Test build #45387 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45387/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155132675 **[Test build #45381 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45381/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155148740 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155148707 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155129707 Added the test for abrupt close, and added the batching to manual writer --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155172521 **[Test build #45387 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45387/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155172748 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44324712 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +493,24 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9143 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155209961 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155226351 LGTM, will merge when tests pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155227163 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155228476 **[Test build #45434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45434/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155246265 > Test build #45434 has finished for PR 9143 at commit 1fe2e7b. > > This patch fails Spark unit tests. > This patch merges cleanly. > This patch adds the

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155239389 **[Test build #45434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45434/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155239425 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155240395 Failed to a flaky test. Restarting --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155240408 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155241026 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155240999 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155241173 **[Test build #45445 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45445/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155226674 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155226564 **[Test build #45423 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45423/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155226632 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-155226728 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154861970 **[Test build #45317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45317/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154871950 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154871903 **[Test build #45317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45317/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44229377 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +491,12 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154861688 @tdas @zsxwing @harishreedharan I addressed all the comments, added a shutdown check to the ReceiverTracker RPC Endpoint, and removed the possibly flaky test. Could

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154861718 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154861724 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44236326 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,221 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44238297 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogUtilsSuite.scala --- @@ -0,0 +1,286 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44234779 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +493,28 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44236121 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +493,28 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44236366 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,221 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44234476 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +493,28 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44236068 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +493,28 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44239702 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,283 +281,150 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44234566 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +493,28 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44234591 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -437,7 +437,12 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44239889 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,283 +281,150 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44239938 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,283 +281,150 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44239947 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,283 +281,150 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154930525 **[Test build #45341 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45341/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154934173 **[Test build #45345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45345/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44243764 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,223 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154956060 **[Test build #45345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45345/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44239951 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,221 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44240691 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogUtilsSuite.scala --- @@ -0,0 +1,286 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44240881 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,283 +281,150 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44241288 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +493,28 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154931549 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154931541 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154956128 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44240746 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,283 +281,150 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44243570 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -58,49 +71,127 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44244198 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,149 +284,165 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44244797 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,283 +281,150 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154962132 Other than the test, things look LGTM. @zsxwing could you take another pass as well. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154930295 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154930279 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44240131 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,283 +281,150 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44240234 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -58,49 +71,127 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44240396 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -58,49 +71,127 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44240091 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,283 +281,150 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44240066 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/util/WriteAheadLogSuite.scala --- @@ -190,283 +281,150 @@ class WriteAheadLogSuite extends

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154930726 **[Test build #45341 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45341/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154930734 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154342369 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154342398 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154351044 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154350950 **[Test build #45205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45205/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154351047 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154352789 **[Test build #45208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45208/consoleFull)** for PR 9143 at commit

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154353164 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9143#issuecomment-154353160 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44173319 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +491,12 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44173447 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala --- @@ -488,7 +491,12 @@ class ReceiverTracker(ssc:

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44176718 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44174052 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11141][STREAMING] Batch ReceivedBlockTr...

2015-11-06 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9143#discussion_r44174435 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache

  1   2   3   4   >