[
https://issues.apache.org/jira/browse/SPARK-33418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dingbei updated SPARK-33418:
----------------------------
Description:
It begins with the needs to start a lot of spark streaming receivers . *The
launch time gets super long when it comes to more than 300 receivers.* I will
show tests data I did and how I improved this.
*Tests preparation*
There are two cores exists in every executors.(one for receiver and the other
one to process every batch of datas). I observed launch time of all receivers
through spark web UI (duration between the first receiver started to the last
one started).
*Tests and data*
At first, we set the number of executors to 200 which means to start 200
receivers and everything goes well. launch time is around 50s.processing time
for every batch is around 10s.(picture 1)
Then we set the number of executors to 500 which means to start 500 receivers.
The launch time became around 5 mins. processing time for every batch is around
2mins.(picture 2)
Then I start to look for the reason in the source code. I use Thread dump to
check which methods takes relatively long time.
was:
It begins with the needs to start a lot of spark streaming receivers . The
launch time gets super long when it comes to more than 300 receivers. I will
show Tests data I did and how did I improve this. There are two cores in every
executors.(one for receiver and the other one to process bacth jobs)
There is two main metrics i will mention below.
receiver launch time :From the first receiver started to the last one.(observed
through spark web UI)
At first, we set the number of executors to 200 which means to start 200
receivers and everything goes well. launch time is around 50s.processing time
for every batch is around 10s.(picture 1)
Then we set the number of executors to 500 which means to start 500 receivers.
The launch time became around 5 mins. processing time for every batch is around
2mins.(picture 2)
Then I start to look for the reason in the source code. I use Thread dump to
check which methods takes relatively long time.
> TaskSchedulerImpl: Check pending tasks in advance when resource offers
> ----------------------------------------------------------------------
>
> Key: SPARK-33418
> URL: https://issues.apache.org/jira/browse/SPARK-33418
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.0.1
> Reporter: dingbei
> Priority: Major
>
> It begins with the needs to start a lot of spark streaming receivers . *The
> launch time gets super long when it comes to more than 300 receivers.* I will
> show tests data I did and how I improved this.
> *Tests preparation*
> There are two cores exists in every executors.(one for receiver and the other
> one to process every batch of datas). I observed launch time of all receivers
> through spark web UI (duration between the first receiver started to the
> last one started).
> *Tests and data*
> At first, we set the number of executors to 200 which means to start 200
> receivers and everything goes well. launch time is around 50s.processing time
> for every batch is around 10s.(picture 1)
> Then we set the number of executors to 500 which means to start 500
> receivers. The launch time became around 5 mins. processing time for every
> batch is around 2mins.(picture 2)
>
> Then I start to look for the reason in the source code. I use Thread dump to
> check which methods takes relatively long time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
