Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3731#discussion_r22139448
--- Diff: docs/job-scheduling.md ---
@@ -56,6 +56,112 @@ the same RDDs. For example, the
[Shark](http://shark.cs.berkeley.edu) JDBC serve
queries. In future releases, in-memory storage systems such as
[Tachyon](http://tachyon-project.org) will
provide another approach to share RDDs.
+## Dynamic Resource Allocation
+
+Spark 1.2 introduces the ability to dynamically scale the set of cluster
resources allocated to
+your application up and down based on the workload. This means that your
application may give
+resources back to the cluster if they are no longer used and request them
again later when there
+is demand. This feature is particularly useful if multiple applications
share resources in your
+Spark cluster. If a subset of the resources allocated to an application
becomes idle, it can be
+returned to the cluster's pool of resources and acquired by other
applications. In Spark, dynamic
+resource allocation is performed on the granularity of the executor and
can be enabled through
+`spark.dynamicAllocation.enabled`.
+
+This feature is currently disabled by default and available only on
[YARN](running-on-yarn.html).
+A future release will extend this to [standalone
mode](spark-standalone.html) and
+[Mesos coarse-grained mode](running-on-mesos.html#mesos-run-modes). Note
that although Spark on
+Mesos already has a similar notion of dynamic resource sharing in
fine-grained mode, enabling
+dynamic allocation allows your Mesos application to take advantage of
coarse-grained low-latency
+scheduling while sharing cluster resources efficiently.
+
+Lastly, it is worth noting that Spark's dynamic resource allocation
mechanism is cooperative.
--- End diff --
fair enough
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
