[
https://issues.apache.org/jira/browse/SPARK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shixiong Zhu resolved SPARK-17510.
----------------------------------
Resolution: Fixed
Assignee: Cody Koeninger
Fix Version/s: 2.1.0
> Set Streaming MaxRate Independently For Multiple Streams
> --------------------------------------------------------
>
> Key: SPARK-17510
> URL: https://issues.apache.org/jira/browse/SPARK-17510
> Project: Spark
> Issue Type: Improvement
> Components: DStreams
> Affects Versions: 2.0.0
> Reporter: Jeff Nadler
> Assignee: Cody Koeninger
> Fix For: 2.1.0
>
>
> We use multiple DStreams coming from different Kafka topics in a Streaming
> application.
> Some settings like maxrate and backpressure enabled/disabled would be better
> passed as config to KafkaUtils.createStream and
> KafkaUtils.createDirectStream, instead of setting them in SparkConf.
> Being able to set a different maxrate for different streams is an important
> requirement for us; we currently work-around the problem by using one
> receiver-based stream and one direct stream.
> We would like to be able to turn on backpressure for only one of the streams
> as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
