[
https://issues.apache.org/jira/browse/SPARK-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931806#comment-15931806
]
Apache Spark commented on SPARK-19486:
--------------------------------------
User 'witgo' has created a pull request for this issue:
https://github.com/apache/spark/pull/17139
> Investigate using multiple threads for task serialization
> ---------------------------------------------------------
>
> Key: SPARK-19486
> URL: https://issues.apache.org/jira/browse/SPARK-19486
> Project: Spark
> Issue Type: Improvement
> Components: Scheduler
> Affects Versions: 2.1.0
> Reporter: Shivaram Venkataraman
>
> This is related to SPARK-18890, where all the serialization logic is moved
> into the Scheduler backend thread. As a follow on to this we can investigate
> using a thread pool to serialize a number of tasks together instead of using
> a single thread to serialize all of them.
> Note that this may not yield sufficient benefits unless the driver has enough
> cores and we don't run into contention across threads. We can first
> investigate potential benefits and if there are sufficient benefits we can
> create a PR for this.
> cc [~kayousterhout]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
