[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[rezasafi]

Github user rezasafi commented on the issue:

https://github.com/apache/spark/pull/3
  
Thank you very much @koeninger 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[koeninger]

Github user koeninger commented on the issue:

https://github.com/apache/spark/pull/3
  
Thanks, merging to master


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/3
  
**[Test build #4303 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4303/testReport)**
 for PR 3 at commit 
[`6f8d15a`](https://github.com/apache/spark/commit/6f8d15a86926eb3f34bdf9c58a1dc1d44749abb8).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/3
  
**[Test build #4303 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4303/testReport)**
 for PR 3 at commit 
[`6f8d15a`](https://github.com/apache/spark/commit/6f8d15a86926eb3f34bdf9c58a1dc1d44749abb8).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[koeninger]

Github user koeninger commented on the issue:

https://github.com/apache/spark/pull/3
  
I think since the default behavior is still 1, it's probably ok to let
someone do what they want here

On Wed, Aug 29, 2018 at 3:51 PM, rezasafi  wrote:

> *@rezasafi* commented on this pull request.
> --
>
> In external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/
> DirectKafkaInputDStream.scala
> :
>
> > @@ -154,7 +153,8 @@ private[spark] class DirectKafkaInputDStream[K, V](
>  if (effectiveRateLimitPerPartition.values.sum > 0) {
>val secsPerBatch = 
context.graph.batchDuration.milliseconds.toDouble / 1000
>Some(effectiveRateLimitPerPartition.map {
> -case (tp, limit) => tp -> Math.max((secsPerBatch * 
limit).toLong, 1L)
> +case (tp, limit) => tp -> Math.max((secsPerBatch * limit).toLong,
> +  Math.max(ppc.minRatePerPartition(tp), 1L))
>
> I just didn't want to break the reasoning behind SPARK-18371 to have at
> least 1 always. I didn't have any other reason for this. I can change it 
to
> give the user the freedom.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/3
  
**[Test build #4296 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4296/testReport)**
 for PR 3 at commit 
[`85ece1c`](https://github.com/apache/spark/commit/85ece1c0866164a3f5a260b6e226b01c1fd1dd81).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/3
  
**[Test build #4296 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4296/testReport)**
 for PR 3 at commit 
[`85ece1c`](https://github.com/apache/spark/commit/85ece1c0866164a3f5a260b6e226b01c1fd1dd81).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/3
  
**[Test build #4295 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4295/testReport)**
 for PR 3 at commit 
[`2b0b1ce`](https://github.com/apache/spark/commit/2b0b1ce3876e2f55807156a98f75068280e03054).
 * This patch **fails MiMa tests**.
 * This patch **does not merge cleanly**.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/3
  
**[Test build #4295 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4295/testReport)**
 for PR 3 at commit 
[`2b0b1ce`](https://github.com/apache/spark/commit/2b0b1ce3876e2f55807156a98f75068280e03054).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[rezasafi]

Github user rezasafi commented on the issue:

https://github.com/apache/spark/pull/3
  
Jenkins, retest this please


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[rezasafi]

Github user rezasafi commented on the issue:

https://github.com/apache/spark/pull/3
  
Thank you very much @koeninger. I appreciate it.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/3
  
**[Test build #4286 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4286/testReport)**
 for PR 3 at commit 
[`f37b7e8`](https://github.com/apache/spark/commit/f37b7e81bd3b59afe859df3bec597ff9cafe1bbf).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

[SparkQA]

Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/3
  
**[Test build #4286 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4286/testReport)**
 for PR 3 at commit 
[`f37b7e8`](https://github.com/apache/spark/commit/f37b7e81bd3b59afe859df3bec597ff9cafe1bbf).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org