[
https://issues.apache.org/jira/browse/SPARK-13007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Braithwaite updated SPARK-13007:
-------------------------------------
Description:
While spark is well documented for the most part, often times I have trouble
determining where a configuration applies.
For example, when setting spark.dynamicAllocation.enabled , does it
always apply to the entire cluster manager, or is it possible to configure it
on a per-job level?
Different levels I can think of:
Application
Driver
Executor
Worker
Cluster
And I'm sure there are more. This could be just another column in the
configuration page.
was:
While spark is well documented for the most part, often times I have trouble
determining where a configuration applies.
For example, when setting spark.dynamicAllocation.enabled , does it
always apply to the entire cluster manager, or is it possible to configure it
on a per-job level?
Different levels I can think of:
Driver
Executor
Worker
Cluster
And I'm sure there are more. This could be just another column in the
configuration page.
> Document where configuration / properties are read and applied
> --------------------------------------------------------------
>
> Key: SPARK-13007
> URL: https://issues.apache.org/jira/browse/SPARK-13007
> Project: Spark
> Issue Type: Improvement
> Components: Documentation
> Affects Versions: 1.6.0
> Reporter: Alan Braithwaite
> Priority: Critical
>
> While spark is well documented for the most part, often times I have trouble
> determining where a configuration applies.
> For example, when setting spark.dynamicAllocation.enabled , does it
> always apply to the entire cluster manager, or is it possible to configure it
> on a per-job level?
> Different levels I can think of:
> Application
> Driver
> Executor
> Worker
> Cluster
> And I'm sure there are more. This could be just another column in the
> configuration page.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
