[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10453 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-169862067 LGTM. Merging this to master and 1.6. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-169837010 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-169837009 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-169836865 **[Test build #48980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48980/consoleFull)** for PR 10453 at commit [`28a750d`](https://g

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-16984 **[Test build #48980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48980/consoleFull)** for PR 10453 at commit [`28a750d`](https://gi

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-169831376 just one more comment. then LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r49137355 --- Diff: docs/streaming-programming-guide.md --- @@ -1985,7 +1985,11 @@ To run a Spark Streaming applications, you need to have the following. to increa

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-169818271 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-169818273 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-169817911 **[Test build #48971 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48971/consoleFull)** for PR 10453 at commit [`4d55b03`](https://g

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-169813237 **[Test build #48971 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48971/consoleFull)** for PR 10453 at commit [`4d55b03`](https://gi

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r49127420 --- Diff: docs/configuration.md --- @@ -1600,6 +1600,34 @@ Apart from these, the following properties are also available, and may be useful How many ba

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r49127139 --- Diff: docs/streaming-programming-guide.md --- @@ -2029,6 +2029,11 @@ If the data is being received by the receivers faster than what can be processed y

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r49125847 --- Diff: docs/streaming-programming-guide.md --- @@ -2029,6 +2029,11 @@ If the data is being received by the receivers faster than what can be processed y

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r49125736 --- Diff: docs/configuration.md --- @@ -1600,6 +1600,34 @@ Apart from these, the following properties are also available, and may be useful How many ba

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2016-01-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r49125309 --- Diff: docs/configuration.md --- @@ -1600,6 +1600,34 @@ Apart from these, the following properties are also available, and may be useful How many ba

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-31 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-168194624 @zsxwing Thanks! LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-168094522 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-168094523 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-168094430 **[Test build #48516 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48516/consoleFull)** for PR 10453 at commit [`7d9b038`](https://g

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-168092271 **[Test build #48516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48516/consoleFull)** for PR 10453 at commit [`7d9b038`](https://gi

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-30 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-168091405 > Maybe we can also include that allowBatching is not just helpful when closeFileAfterWrite is enabled, but is also very helpful to scale to a large number of receivers

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-30 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-167973342 Maybe we can also include that `allowBatching` is not just helpful when `closeFileAfterWrite` is enabled, but is also very helpful to scale to a large number of receiver

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-30 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-167973270 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-167903860 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-167903857 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-167903798 **[Test build #48436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48436/consoleFull)** for PR 10453 at commit [`bce7a29`](https://g

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-167902755 **[Test build #48436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48436/consoleFull)** for PR 10453 at commit [`bce7a29`](https://gi

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-167902303 @BenFradet Addressed. Thanks for your reviewing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r48559444 --- Diff: docs/configuration.md --- @@ -1600,6 +1600,33 @@ Apart from these, the following properties are also available, and may be useful How many

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread BenFradet
Github user BenFradet commented on the pull request: https://github.com/apache/spark/pull/10453#issuecomment-167848381 I have a few comments on phrasing but otherwise it lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread BenFradet
Github user BenFradet commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r48559339 --- Diff: docs/streaming-programming-guide.md --- @@ -2029,6 +2029,11 @@ If the data is being received by the receivers faster than what can be processed

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread BenFradet
Github user BenFradet commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r48559275 --- Diff: docs/configuration.md --- @@ -1600,6 +1600,33 @@ Apart from these, the following properties are also available, and may be useful How ma

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread BenFradet
Github user BenFradet commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r48559209 --- Diff: docs/configuration.md --- @@ -1600,6 +1600,33 @@ Apart from these, the following properties are also available, and may be useful How ma

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread BenFradet
Github user BenFradet commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r48559086 --- Diff: docs/configuration.md --- @@ -1600,6 +1600,33 @@ Apart from these, the following properties are also available, and may be useful How ma

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread BenFradet
Github user BenFradet commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r48559015 --- Diff: docs/configuration.md --- @@ -1600,6 +1600,33 @@ Apart from these, the following properties are also available, and may be useful How ma

[GitHub] spark pull request: [SPARK-12507][Streaming][Document]Expose close...

2015-12-29 Thread BenFradet
Github user BenFradet commented on a diff in the pull request: https://github.com/apache/spark/pull/10453#discussion_r48558980 --- Diff: docs/configuration.md --- @@ -1600,6 +1600,33 @@ Apart from these, the following properties are also available, and may be useful How ma