[
https://issues.apache.org/jira/browse/SPARK-12209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-12209.
----------------------------------
Resolution: Incomplete
> spark.streaming.concurrentJobs and spark.streaming.backpressure.enabled
> -----------------------------------------------------------------------
>
> Key: SPARK-12209
> URL: https://issues.apache.org/jira/browse/SPARK-12209
> Project: Spark
> Issue Type: Bug
> Components: Scheduler
> Affects Versions: 1.5.2
> Reporter: Andrej Kazakov
> Priority: Major
> Labels: bulk-closed
>
> spark.streaming.backpressure.enabled doesn't take into account
> spark.streaming.concurrentJobs.
> The backpressure reduces the input rate until only one job is being processed
> at a time. Having concurrent jobs is interesting when dealing with
> high-latency backends, where the ingestion time is large enough, yet there is
> enough throughput capacity to have multiple jobs complete in parallel.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
