[
https://issues.apache.org/jira/browse/SPARK-21171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060348#comment-16060348
]
wangminfeng commented on SPARK-21171:
-------------------------------------
We have modified some code for this feature, i will add a benchmark soon. It
is the first time i contribute to community, give me some time for learn the
rule. Thank you.
> Speculate task scheduling block dirve handle normal task when a job task
> number more than one hundred thousand
> --------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-21171
> URL: https://issues.apache.org/jira/browse/SPARK-21171
> Project: Spark
> Issue Type: Improvement
> Components: Block Manager
> Affects Versions: 2.0.0
> Environment: We have more than two hundred high-performance machine
> to handle more than 2T data by one query
> Reporter: wangminfeng
>
> If a job have more then one hundred thousand tasks and spark.speculation is
> true, when speculable tasks start, choosing a speculable will waste lots of
> time and block other tasks. We do a ad-hoc query for data analyse, we can't
> tolerate one job wasting time even it is a large job
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
