Hi Aravindh,

spark.streaming.receiver.maxRate  is per receiver. You should multiply with
number of receivers with max rate.


On Mon, Sep 19, 2016 at 9:31 AM, Aravindh [via Apache Spark User List] <
ml-node+s1001560n27754...@n3.nabble.com> wrote:

> I am trying to throttle my spark cluster reading events from kinesis by
> setting spark.streaming.receiver.maxRate to 100. The batch interval is 1
> second. But spark seems to read thousands of events for every second. Am I
> missing anything here?
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-spark-user-list.1001560.n3.nabble.com/Kinesis-
> Receiver-not-respecting-spark-streaming-receiver-maxRate-tp27754.html
> To start a new topic under Apache Spark User List, email
> ml-node+s1001560n1...@n3.nabble.com
> To unsubscribe from Apache Spark User List, click here
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dG9zYWlnYW5lc2hAZ21haWwuY29tfDF8LTExOTE1Nzk1MTU=>
> .
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>

Sai Ganesh
View this message in context: 
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Reply via email to