[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56777200 Let me see if I can add a method in the sink that can be used to query if the acks have come. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56777581 That would be great. If you _really_ need to rely on sleeps then it would be good to at least sleep in several smaller intervals rather than 1 second and check multiple

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56778924 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20780/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56778929 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56878711 This is more deterministic. This counts down when a batch is completed. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56878740 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56878879 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20820/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56879646 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20821/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56888031 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56888019 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20820/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56888979 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20821/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56888986 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56891657 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-5689 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20823/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56899263 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56899254 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20823/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56910171 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56910409 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20837/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56914095 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20837/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56914100 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56915169 @pwendell This seems to be stable at this point. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56922316 Thanks Hari - I'll pull it in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2531 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-24 Thread harishreedharan
GitHub user harishreedharan opened a pull request: https://github.com/apache/spark/pull/2531 [SPARK-3686][STREAMING] Wait for sink to commit the channel before check... ...ing for the channel size. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56770669 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20778/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56773737 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20778/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56773739 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-24 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56773776 Jenkins, test this please Thanks, Hari On Wed, Sep 24, 2014 at 9:57 PM, UCB AMPLab notificati...@github.com wrote:

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-24 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56774960 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56775115 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20780/consoleFull) for PR 2531 at commit

[GitHub] spark pull request: [SPARK-3686][STREAMING] Wait for sink to commi...

2014-09-24 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2531#issuecomment-56776994 Hey @harishreedharan - because the Spark tests take a long time, we don't usually allow sleep statements. Is there a way to use some better synchronization to make this