Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/9593
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-185003282
@junhaoMg Thanks a lot. Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-184962897
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-184962894
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-184962774
**[Test build #51396 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51396/consoleFull)**
for PR 9593 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-184939531
**[Test build #51396 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51396/consoleFull)**
for PR 9593 at commit
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-184934114
Will pull in once tests pass
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-184933638
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-166381528
LGTM. Ping @tdas to take a final look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-166178142
**[Test build #48091 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48091/consoleFull)**
for PR 9593 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-166185197
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user junhaoMg commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r48112803
--- Diff: docs/configuration.md ---
@@ -1523,6 +1523,15 @@ Apart from these, the following properties are also
available, and may be useful
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-166185198
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-166185146
**[Test build #48091 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48091/consoleFull)**
for PR 9593 at commit
Github user BenFradet commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r47878665
--- Diff: docs/configuration.md ---
@@ -1523,6 +1523,15 @@ Apart from these, the following properties are also
available, and may be useful
Github user BenFradet commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r47878960
--- Diff: docs/configuration.md ---
@@ -1523,6 +1523,15 @@ Apart from these, the following properties are also
available, and may be useful
Github user BenFradet commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r47879107
--- Diff: docs/configuration.md ---
@@ -1523,6 +1523,15 @@ Apart from these, the following properties are also
available, and may be useful
Github user BenFradet commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-165380452
A few remarks regarding the doc.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-165381302
**[Test build #47907 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47907/consoleFull)**
for PR 9593 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-165381420
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-165381421
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-165399619
**[Test build #47900 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47900/consoleFull)**
for PR 9593 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-165399693
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-165399695
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r47820904
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/receiver/RateLimiter.scala
---
@@ -18,6 +18,7 @@
package
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r47820788
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/receiver/RateLimiter.scala
---
@@ -61,4 +62,21 @@ private[receiver] abstract class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r47820758
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/receiver/RateLimiter.scala
---
@@ -61,4 +62,21 @@ private[receiver] abstract class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r47820721
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/receiver/RateLimiter.scala
---
@@ -61,4 +62,21 @@ private[receiver] abstract class
Github user junhaoMg commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r47876028
--- Diff: docs/configuration.md ---
@@ -1523,6 +1523,15 @@ Apart from these, the following properties are also
available, and may be useful
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r47870905
--- Diff: docs/configuration.md ---
@@ -1523,6 +1523,15 @@ Apart from these, the following properties are also
available, and may be useful
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-165341987
**[Test build #47900 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47900/consoleFull)**
for PR 9593 at commit
Github user junhaoMg commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r47870756
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/receiver/RateLimiter.scala
---
@@ -18,6 +18,7 @@
package
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-165368252
**[Test build #47907 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47907/consoleFull)**
for PR 9593 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-161950061
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-161950062
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-161938126
**[Test build #47195 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47195/consoleFull)**
for PR 9593 at commit
Github user junhaoMg commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-161936369
I hava add the configuration to the docs.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user BenFradet commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159921616
If you introduce a new configuration property, don't forget to add it to
the docs.
---
If your project is set up for it, you can reply to this email and have your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159795582
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159795580
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159793849
**[Test build #46740 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46740/consoleFull)**
for PR 9593 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159791092
**[Test build #46733 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46733/consoleFull)**
for PR 9593 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159791889
**[Test build #46736 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46736/consoleFull)**
for PR 9593 at commit
Github user junhaoMg commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159794454
I have changed 'spark.streaming.receiver.initialRate' to
'spark.streaming.backpressure.initialRate'.
---
If your project is set up for it, you can reply to this email
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159799136
**[Test build #46736 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46736/consoleFull)**
for PR 9593 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159799410
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159799412
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159801849
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159801848
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159833997
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159833994
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159833901
**[Test build #46733 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46733/consoleFull)**
for PR 9593 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-159801774
**[Test build #46740 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46740/consoleFull)**
for PR 9593 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-158229319
**[Test build #46357 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46357/consoleFull)**
for PR 9593 at commit
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-158210110
Jenkins, this is ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-158211547
**[Test build #46357 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46357/consoleFull)**
for PR 9593 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-158229452
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-158229456
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/9593#discussion_r45007951
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/receiver/RateLimiter.scala
---
@@ -61,4 +62,21 @@ private[receiver] abstract class
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-155374156
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user junhaoMg opened a pull request:
https://github.com/apache/spark/pull/9593
[SPARK-11627] Add initial input rate limit for Spark Streaming backpressure
mechanism.
https://issues.apache.org/jira/browse/SPARK-11627
Spark Streaming backpressure mechanism has no
Github user junhaoMg commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-155629739
Setting the rate limit using SparkConf spark.streaming.receiver.maxRate
will cap the receiver rateï¼but select an appropriate value is difficult, if
the value is too
Github user junhaoMg commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-155631922
Given an appropriate initial value can make sure the job success in the
first batch, then the backpressure mechanism can work and adjust data receive
rate according
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/9593#issuecomment-155655693
test this again
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
64 matches
Mail list logo
