Repository: spark Updated Branches: refs/heads/branch-2.3 8d7723f2e -> 7102aeeb2
[SPARK-25583][DOC][BRANCH-2.3] Add history-server related configuration in the documentation. ## What changes were proposed in this pull request? This is a follow up PR for the PR, https://github.com/apache/spark/pull/22601. Add history-server related configuration in the documentation for spark2.3 Some of the history server related configurations were missing in the documentation.Like, 'spark.history.store.maxDiskUsage', 'spark.ui.liveUpdate.period' etc. ## How was this patch tested?   Closes #22613 from shahidki31/SPARK-25583. Authored-by: Shahid <shahidk...@gmail.com> Signed-off-by: Dongjoon Hyun <dongj...@apache.org> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7102aeeb Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/7102aeeb Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/7102aeeb Branch: refs/heads/branch-2.3 Commit: 7102aeeb2a01bd3e4cfb5f9e0bb87e1654339721 Parents: 8d7723f Author: Shahid <shahidk...@gmail.com> Authored: Wed Oct 3 04:10:59 2018 -0700 Committer: Dongjoon Hyun <dongj...@apache.org> Committed: Wed Oct 3 04:10:59 2018 -0700 ---------------------------------------------------------------------- docs/configuration.md | 16 ++++++++++++++++ docs/monitoring.md | 8 ++++++++ 2 files changed, 24 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/7102aeeb/docs/configuration.md ---------------------------------------------------------------------- diff --git a/docs/configuration.md b/docs/configuration.md index ec4f5d4..c85c04f 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -791,6 +791,13 @@ Apart from these, the following properties are also available, and may be useful </td> </tr> <tr> + <td><code>spark.ui.dagGraph.retainedRootRDDs</code></td> + <td>Int.MaxValue</td> + <td> + How many DAG graph nodes the Spark UI and status APIs remember before garbage collecting. + </td> +</tr> +<tr> <td><code>spark.ui.enabled</code></td> <td>true</td> <td> @@ -805,6 +812,15 @@ Apart from these, the following properties are also available, and may be useful </td> </tr> <tr> + <td><code>spark.ui.liveUpdate.period</code></td> + <td>100ms</td> + <td> + How often to update live entities. -1 means "never update" when replaying applications, + meaning only the last write will happen. For live applications, this avoids a few + operations that we can live without when rapidly processing incoming task events. + </td> +</tr> +<tr> <td><code>spark.ui.port</code></td> <td>4040</td> <td> http://git-wip-us.apache.org/repos/asf/spark/blob/7102aeeb/docs/monitoring.md ---------------------------------------------------------------------- diff --git a/docs/monitoring.md b/docs/monitoring.md index 6f6cfc1..bfdaea6 100644 --- a/docs/monitoring.md +++ b/docs/monitoring.md @@ -225,6 +225,14 @@ The history server can be configured as follows: </td> </tr> <tr> + <td>spark.history.store.maxDiskUsage</td> + <td>10g</td> + <td> + Maximum disk usage for the local directory where the cache application history information + are stored. + </td> + </tr> + <tr> <td>spark.history.store.path</td> <td>(none)</td> <td> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
