Ye Xianjin created SPARK-2557:
---------------------------------
Summary: createTaskScheduler should be consistent between local
and local-n-failures
Key: SPARK-2557
URL: https://issues.apache.org/jira/browse/SPARK-2557
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 1.0.0
Reporter: Ye Xianjin
Priority: Minor
In SparkContext.createTaskScheduler, we can use {code}local[*]{code} to
estimates the number of cores on the machine. I think we should also be able to
use * in the local-n-failures mode.
And according to the code in the LOCAL_N_REGEX pattern matching code, I believe
the regular expression of LOCAL_N_REGEX is wrong. LOCAL_N_REFEX should be
{code}
"""local\[([0-9]+|\*)\]""".r
{code}
rather than
{code}
"""local\[([0-9\*]+)\]""".r
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
