[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9805 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157903703 Thanks @zsxwing , merging this to master and 1.6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157863731 **[Test build #46238 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46238/consoleFull)** for PR 9805 at commit [`d87a5e6`](https://git

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157863927 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157863926 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157847716 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157847087 **[Test build #46238 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46238/consoleFull)** for PR 9805 at commit [`d87a5e6`](https://gith

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157846597 That's funny! How come the leak did not cause downstream failures? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9805#discussion_r45252508 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/TrackStateByKeySuite.scala --- @@ -436,6 +437,41 @@ class TrackStateByKeySuite extends SparkFun

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/9805#discussion_r45249165 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/TrackStateByKeySuite.scala --- @@ -436,6 +437,41 @@ class TrackStateByKeySuite extends SparkFunSui

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9805#discussion_r45236372 --- Diff: streaming/src/test/scala/org/apache/spark/streaming/TrackStateByKeySuite.scala --- @@ -436,6 +437,41 @@ class TrackStateByKeySuite extends SparkFun

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157805199 I just noticed `TrackStateByKeySuite` leaks a SparkContext. Could you fix it? --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157683585 @zsxwing Please take a look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157682855 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157682856 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157682747 **[Test build #46209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46209/consoleFull)** for PR 9805 at commit [`77f8ee1`](https://git

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157681942 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157681943 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157681828 **[Test build #46208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46208/consoleFull)** for PR 9805 at commit [`8d0a494`](https://git

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157671290 **[Test build #46209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46209/consoleFull)** for PR 9805 at commit [`77f8ee1`](https://gith

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9805#issuecomment-157669825 **[Test build #46208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46208/consoleFull)** for PR 9805 at commit [`8d0a494`](https://gith

[GitHub] spark pull request: [SPARK-11814][Streaming]Add better default che...

2015-11-18 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/9805 [SPARK-11814][Streaming]Add better default checkpoint duration DStream checkpoint interval is by default set at max(10 second, batch interval). That's bad for large batch intervals where the checkpoint