[16/50] [abbrv] hadoop git commit: YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. (cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/0c39681b Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/0c39681b Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/0c39681b Branch: refs/heads/branch-2-jhung-test Commit: 0c39681b2cc24723171c34676760a86ffefeda07 Parents: 03ff0f1 Author: Sunil G Authored: Mon Oct 22 14:15:51 2018 +0530 Committer: Jonathan Hung Committed: Fri Dec 14 16:03:55 2018 -0800 -- .../hadoop-yarn-site/src/site/markdown/NodeManager.md | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/0c39681b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md index 12201b9..18aeff4 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md @@ -77,13 +77,19 @@ Step 2. Configure a path to the local file-system directory where the NodeManag |: |: | | `yarn.nodemanager.recovery.dir` | The local filesystem directory in which the node manager will store state when recovery is enabled. The default value is set to `$hadoop.tmp.dir/yarn-nm-recovery`. | -Step 3. Configure a valid RPC address for the NodeManager. +Step 3: Enable NM supervision under recovery to prevent running containers from getting cleaned up when NM exits. + +| Property | Description | +|: |: | +| `yarn.nodemanager.recovery.supervised` | If enabled, NodeManager running will not try to cleanup containers as it exits with the assumption it will be immediately be restarted and recover containers The default value is set to 'false'. | + +Step 4. Configure a valid RPC address for the NodeManager. | Property | Description | |: |: | | `yarn.nodemanager.address` | Ephemeral ports (port 0, which is default) cannot be used for the NodeManager's RPC server specified via yarn.nodemanager.address as it can make NM use different ports before and after a restart. This will break any previously running clients that were communicating with the NM before restart. Explicitly setting yarn.nodemanager.address to an address with specific port number (for e.g 0.0.0.0:45454) is a precondition for enabling NM restart. | -Step 4. Auxiliary services. +Step 5. Auxiliary services. * NodeManagers in a YARN cluster can be configured to run auxiliary services. For a completely functional NM restart, YARN relies on any auxiliary service configured to also support recovery. This usually includes (1) avoiding usage of ephemeral ports so that previously running clients (in this case, usually containers) are not disrupted after restart and (2) having the auxiliary service itself support recoverability by reloading any previous state when NodeManager restarts and reinitializes the auxiliary service. - To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
[26/50] [abbrv] hadoop git commit: YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/e5033451 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/e5033451 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/e5033451 Branch: refs/heads/HDFS-12943 Commit: e50334513c781a10368e7353bec6674f89eb0a7b Parents: 7db3bb3 Author: Sunil G Authored: Mon Oct 22 14:15:51 2018 +0530 Committer: Sunil G Committed: Mon Oct 22 14:15:51 2018 +0530 -- .../hadoop-yarn-site/src/site/markdown/NodeManager.md | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/e5033451/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md index 12201b9..18aeff4 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md @@ -77,13 +77,19 @@ Step 2. Configure a path to the local file-system directory where the NodeManag |: |: | | `yarn.nodemanager.recovery.dir` | The local filesystem directory in which the node manager will store state when recovery is enabled. The default value is set to `$hadoop.tmp.dir/yarn-nm-recovery`. | -Step 3. Configure a valid RPC address for the NodeManager. +Step 3: Enable NM supervision under recovery to prevent running containers from getting cleaned up when NM exits. + +| Property | Description | +|: |: | +| `yarn.nodemanager.recovery.supervised` | If enabled, NodeManager running will not try to cleanup containers as it exits with the assumption it will be immediately be restarted and recover containers The default value is set to 'false'. | + +Step 4. Configure a valid RPC address for the NodeManager. | Property | Description | |: |: | | `yarn.nodemanager.address` | Ephemeral ports (port 0, which is default) cannot be used for the NodeManager's RPC server specified via yarn.nodemanager.address as it can make NM use different ports before and after a restart. This will break any previously running clients that were communicating with the NM before restart. Explicitly setting yarn.nodemanager.address to an address with specific port number (for e.g 0.0.0.0:45454) is a precondition for enabling NM restart. | -Step 4. Auxiliary services. +Step 5. Auxiliary services. * NodeManagers in a YARN cluster can be configured to run auxiliary services. For a completely functional NM restart, YARN relies on any auxiliary service configured to also support recovery. This usually includes (1) avoiding usage of ephemeral ports so that previously running clients (in this case, usually containers) are not disrupted after restart and (2) having the auxiliary service itself support recoverability by reloading any previous state when NodeManager restarts and reinitializes the auxiliary service. - To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
hadoop git commit: YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
Repository: hadoop Updated Branches: refs/heads/branch-2.8 f697f3c4f -> a6d86c611 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. (cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/a6d86c61 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/a6d86c61 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/a6d86c61 Branch: refs/heads/branch-2.8 Commit: a6d86c611957a8f6d08004c3814cf3f6583a44ea Parents: f697f3c Author: Sunil G Authored: Mon Oct 22 14:15:51 2018 +0530 Committer: Sunil G Committed: Mon Oct 22 14:20:42 2018 +0530 -- .../hadoop-yarn-site/src/site/markdown/NodeManager.md | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/a6d86c61/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md index b3a92dc..fe202ae 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md @@ -83,13 +83,19 @@ Step 2. Configure a path to the local file-system directory where the NodeManag |: |: | | `yarn.nodemanager.recovery.dir` | The local filesystem directory in which the node manager will store state when recovery is enabled. The default value is set to `$hadoop.tmp.dir/yarn-nm-recovery`. | -Step 3. Configure a valid RPC address for the NodeManager. +Step 3: Enable NM supervision under recovery to prevent running containers from getting cleaned up when NM exits. + +| Property | Description | +|: |: | +| `yarn.nodemanager.recovery.supervised` | If enabled, NodeManager running will not try to cleanup containers as it exits with the assumption it will be immediately be restarted and recover containers The default value is set to 'false'. | + +Step 4. Configure a valid RPC address for the NodeManager. | Property | Description | |: |: | | `yarn.nodemanager.address` | Ephemeral ports (port 0, which is default) cannot be used for the NodeManager's RPC server specified via yarn.nodemanager.address as it can make NM use different ports before and after a restart. This will break any previously running clients that were communicating with the NM before restart. Explicitly setting yarn.nodemanager.address to an address with specific port number (for e.g 0.0.0.0:45454) is a precondition for enabling NM restart. | -Step 4. Auxiliary services. +Step 5. Auxiliary services. * NodeManagers in a YARN cluster can be configured to run auxiliary services. For a completely functional NM restart, YARN relies on any auxiliary service configured to also support recovery. This usually includes (1) avoiding usage of ephemeral ports so that previously running clients (in this case, usually containers) are not disrupted after restart and (2) having the auxiliary service itself support recoverability by reloading any previous state when NodeManager restarts and reinitializes the auxiliary service. - To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
hadoop git commit: YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
Repository: hadoop Updated Branches: refs/heads/branch-2.9 622919d8d -> 4a481e1dc YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. (cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/4a481e1d Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/4a481e1d Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/4a481e1d Branch: refs/heads/branch-2.9 Commit: 4a481e1dcc390c4209ea7f07f725fd6e591fbacb Parents: 622919d Author: Sunil G Authored: Mon Oct 22 14:15:51 2018 +0530 Committer: Sunil G Committed: Mon Oct 22 14:19:41 2018 +0530 -- .../hadoop-yarn-site/src/site/markdown/NodeManager.md | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/4a481e1d/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md index 3261cd7..b8dea3d 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md @@ -77,13 +77,19 @@ Step 2. Configure a path to the local file-system directory where the NodeManag |: |: | | `yarn.nodemanager.recovery.dir` | The local filesystem directory in which the node manager will store state when recovery is enabled. The default value is set to `$hadoop.tmp.dir/yarn-nm-recovery`. | -Step 3. Configure a valid RPC address for the NodeManager. +Step 3: Enable NM supervision under recovery to prevent running containers from getting cleaned up when NM exits. + +| Property | Description | +|: |: | +| `yarn.nodemanager.recovery.supervised` | If enabled, NodeManager running will not try to cleanup containers as it exits with the assumption it will be immediately be restarted and recover containers The default value is set to 'false'. | + +Step 4. Configure a valid RPC address for the NodeManager. | Property | Description | |: |: | | `yarn.nodemanager.address` | Ephemeral ports (port 0, which is default) cannot be used for the NodeManager's RPC server specified via yarn.nodemanager.address as it can make NM use different ports before and after a restart. This will break any previously running clients that were communicating with the NM before restart. Explicitly setting yarn.nodemanager.address to an address with specific port number (for e.g 0.0.0.0:45454) is a precondition for enabling NM restart. | -Step 4. Auxiliary services. +Step 5. Auxiliary services. * NodeManagers in a YARN cluster can be configured to run auxiliary services. For a completely functional NM restart, YARN relies on any auxiliary service configured to also support recovery. This usually includes (1) avoiding usage of ephemeral ports so that previously running clients (in this case, usually containers) are not disrupted after restart and (2) having the auxiliary service itself support recoverability by reloading any previous state when NodeManager restarts and reinitializes the auxiliary service. - To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
hadoop git commit: YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
Repository: hadoop Updated Branches: refs/heads/branch-2 e412d8f6c -> 56a27284a YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. (cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/56a27284 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/56a27284 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/56a27284 Branch: refs/heads/branch-2 Commit: 56a27284a77a98f2327b992fdb917de17e02621c Parents: e412d8f Author: Sunil G Authored: Mon Oct 22 14:15:51 2018 +0530 Committer: Sunil G Committed: Mon Oct 22 14:18:37 2018 +0530 -- .../hadoop-yarn-site/src/site/markdown/NodeManager.md | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/56a27284/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md index 12201b9..18aeff4 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md @@ -77,13 +77,19 @@ Step 2. Configure a path to the local file-system directory where the NodeManag |: |: | | `yarn.nodemanager.recovery.dir` | The local filesystem directory in which the node manager will store state when recovery is enabled. The default value is set to `$hadoop.tmp.dir/yarn-nm-recovery`. | -Step 3. Configure a valid RPC address for the NodeManager. +Step 3: Enable NM supervision under recovery to prevent running containers from getting cleaned up when NM exits. + +| Property | Description | +|: |: | +| `yarn.nodemanager.recovery.supervised` | If enabled, NodeManager running will not try to cleanup containers as it exits with the assumption it will be immediately be restarted and recover containers The default value is set to 'false'. | + +Step 4. Configure a valid RPC address for the NodeManager. | Property | Description | |: |: | | `yarn.nodemanager.address` | Ephemeral ports (port 0, which is default) cannot be used for the NodeManager's RPC server specified via yarn.nodemanager.address as it can make NM use different ports before and after a restart. This will break any previously running clients that were communicating with the NM before restart. Explicitly setting yarn.nodemanager.address to an address with specific port number (for e.g 0.0.0.0:45454) is a precondition for enabling NM restart. | -Step 4. Auxiliary services. +Step 5. Auxiliary services. * NodeManagers in a YARN cluster can be configured to run auxiliary services. For a completely functional NM restart, YARN relies on any auxiliary service configured to also support recovery. This usually includes (1) avoiding usage of ephemeral ports so that previously running clients (in this case, usually containers) are not disrupted after restart and (2) having the auxiliary service itself support recoverability by reloading any previous state when NodeManager restarts and reinitializes the auxiliary service. - To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
hadoop git commit: YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
Repository: hadoop Updated Branches: refs/heads/branch-3.0 8360d15ab -> 78cf8c809 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. (cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/78cf8c80 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/78cf8c80 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/78cf8c80 Branch: refs/heads/branch-3.0 Commit: 78cf8c809996b356e26fe7592ad84cdd2e3c2f0f Parents: 8360d15 Author: Sunil G Authored: Mon Oct 22 14:15:51 2018 +0530 Committer: Sunil G Committed: Mon Oct 22 14:17:29 2018 +0530 -- .../hadoop-yarn-site/src/site/markdown/NodeManager.md | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/78cf8c80/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md index 12201b9..18aeff4 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md @@ -77,13 +77,19 @@ Step 2. Configure a path to the local file-system directory where the NodeManag |: |: | | `yarn.nodemanager.recovery.dir` | The local filesystem directory in which the node manager will store state when recovery is enabled. The default value is set to `$hadoop.tmp.dir/yarn-nm-recovery`. | -Step 3. Configure a valid RPC address for the NodeManager. +Step 3: Enable NM supervision under recovery to prevent running containers from getting cleaned up when NM exits. + +| Property | Description | +|: |: | +| `yarn.nodemanager.recovery.supervised` | If enabled, NodeManager running will not try to cleanup containers as it exits with the assumption it will be immediately be restarted and recover containers The default value is set to 'false'. | + +Step 4. Configure a valid RPC address for the NodeManager. | Property | Description | |: |: | | `yarn.nodemanager.address` | Ephemeral ports (port 0, which is default) cannot be used for the NodeManager's RPC server specified via yarn.nodemanager.address as it can make NM use different ports before and after a restart. This will break any previously running clients that were communicating with the NM before restart. Explicitly setting yarn.nodemanager.address to an address with specific port number (for e.g 0.0.0.0:45454) is a precondition for enabling NM restart. | -Step 4. Auxiliary services. +Step 5. Auxiliary services. * NodeManagers in a YARN cluster can be configured to run auxiliary services. For a completely functional NM restart, YARN relies on any auxiliary service configured to also support recovery. This usually includes (1) avoiding usage of ephemeral ports so that previously running clients (in this case, usually containers) are not disrupted after restart and (2) having the auxiliary service itself support recoverability by reloading any previous state when NodeManager restarts and reinitializes the auxiliary service. - To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
hadoop git commit: YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
Repository: hadoop Updated Branches: refs/heads/branch-3.1 84726679f -> 450f231a5 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. (cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/450f231a Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/450f231a Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/450f231a Branch: refs/heads/branch-3.1 Commit: 450f231a535f59b15113b3381bed5ba9a050b4be Parents: 8472667 Author: Sunil G Authored: Mon Oct 22 14:15:51 2018 +0530 Committer: Sunil G Committed: Mon Oct 22 14:16:54 2018 +0530 -- .../hadoop-yarn-site/src/site/markdown/NodeManager.md | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/450f231a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md index 12201b9..18aeff4 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md @@ -77,13 +77,19 @@ Step 2. Configure a path to the local file-system directory where the NodeManag |: |: | | `yarn.nodemanager.recovery.dir` | The local filesystem directory in which the node manager will store state when recovery is enabled. The default value is set to `$hadoop.tmp.dir/yarn-nm-recovery`. | -Step 3. Configure a valid RPC address for the NodeManager. +Step 3: Enable NM supervision under recovery to prevent running containers from getting cleaned up when NM exits. + +| Property | Description | +|: |: | +| `yarn.nodemanager.recovery.supervised` | If enabled, NodeManager running will not try to cleanup containers as it exits with the assumption it will be immediately be restarted and recover containers The default value is set to 'false'. | + +Step 4. Configure a valid RPC address for the NodeManager. | Property | Description | |: |: | | `yarn.nodemanager.address` | Ephemeral ports (port 0, which is default) cannot be used for the NodeManager's RPC server specified via yarn.nodemanager.address as it can make NM use different ports before and after a restart. This will break any previously running clients that were communicating with the NM before restart. Explicitly setting yarn.nodemanager.address to an address with specific port number (for e.g 0.0.0.0:45454) is a precondition for enabling NM restart. | -Step 4. Auxiliary services. +Step 5. Auxiliary services. * NodeManagers in a YARN cluster can be configured to run auxiliary services. For a completely functional NM restart, YARN relies on any auxiliary service configured to also support recovery. This usually includes (1) avoiding usage of ephemeral ports so that previously running clients (in this case, usually containers) are not disrupted after restart and (2) having the auxiliary service itself support recoverability by reloading any previous state when NodeManager restarts and reinitializes the auxiliary service. - To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
hadoop git commit: YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
Repository: hadoop Updated Branches: refs/heads/branch-3.2 a7dd244a4 -> 8724c2900 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. (cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/8724c290 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/8724c290 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/8724c290 Branch: refs/heads/branch-3.2 Commit: 8724c2900fb07462864a1ab3140630d5c185bf1f Parents: a7dd244 Author: Sunil G Authored: Mon Oct 22 14:15:51 2018 +0530 Committer: Sunil G Committed: Mon Oct 22 14:16:24 2018 +0530 -- .../hadoop-yarn-site/src/site/markdown/NodeManager.md | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/8724c290/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md index 12201b9..18aeff4 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md @@ -77,13 +77,19 @@ Step 2. Configure a path to the local file-system directory where the NodeManag |: |: | | `yarn.nodemanager.recovery.dir` | The local filesystem directory in which the node manager will store state when recovery is enabled. The default value is set to `$hadoop.tmp.dir/yarn-nm-recovery`. | -Step 3. Configure a valid RPC address for the NodeManager. +Step 3: Enable NM supervision under recovery to prevent running containers from getting cleaned up when NM exits. + +| Property | Description | +|: |: | +| `yarn.nodemanager.recovery.supervised` | If enabled, NodeManager running will not try to cleanup containers as it exits with the assumption it will be immediately be restarted and recover containers The default value is set to 'false'. | + +Step 4. Configure a valid RPC address for the NodeManager. | Property | Description | |: |: | | `yarn.nodemanager.address` | Ephemeral ports (port 0, which is default) cannot be used for the NodeManager's RPC server specified via yarn.nodemanager.address as it can make NM use different ports before and after a restart. This will break any previously running clients that were communicating with the NM before restart. Explicitly setting yarn.nodemanager.address to an address with specific port number (for e.g 0.0.0.0:45454) is a precondition for enabling NM restart. | -Step 4. Auxiliary services. +Step 5. Auxiliary services. * NodeManagers in a YARN cluster can be configured to run auxiliary services. For a completely functional NM restart, YARN relies on any auxiliary service configured to also support recovery. This usually includes (1) avoiding usage of ephemeral ports so that previously running clients (in this case, usually containers) are not disrupted after restart and (2) having the auxiliary service itself support recoverability by reloading any previous state when NodeManager restarts and reinitializes the auxiliary service. - To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
hadoop git commit: YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
Repository: hadoop Updated Branches: refs/heads/trunk 7db3bb3ac -> e50334513 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/e5033451 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/e5033451 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/e5033451 Branch: refs/heads/trunk Commit: e50334513c781a10368e7353bec6674f89eb0a7b Parents: 7db3bb3 Author: Sunil G Authored: Mon Oct 22 14:15:51 2018 +0530 Committer: Sunil G Committed: Mon Oct 22 14:15:51 2018 +0530 -- .../hadoop-yarn-site/src/site/markdown/NodeManager.md | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/e5033451/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md index 12201b9..18aeff4 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManager.md @@ -77,13 +77,19 @@ Step 2. Configure a path to the local file-system directory where the NodeManag |: |: | | `yarn.nodemanager.recovery.dir` | The local filesystem directory in which the node manager will store state when recovery is enabled. The default value is set to `$hadoop.tmp.dir/yarn-nm-recovery`. | -Step 3. Configure a valid RPC address for the NodeManager. +Step 3: Enable NM supervision under recovery to prevent running containers from getting cleaned up when NM exits. + +| Property | Description | +|: |: | +| `yarn.nodemanager.recovery.supervised` | If enabled, NodeManager running will not try to cleanup containers as it exits with the assumption it will be immediately be restarted and recover containers The default value is set to 'false'. | + +Step 4. Configure a valid RPC address for the NodeManager. | Property | Description | |: |: | | `yarn.nodemanager.address` | Ephemeral ports (port 0, which is default) cannot be used for the NodeManager's RPC server specified via yarn.nodemanager.address as it can make NM use different ports before and after a restart. This will break any previously running clients that were communicating with the NM before restart. Explicitly setting yarn.nodemanager.address to an address with specific port number (for e.g 0.0.0.0:45454) is a precondition for enabling NM restart. | -Step 4. Auxiliary services. +Step 5. Auxiliary services. * NodeManagers in a YARN cluster can be configured to run auxiliary services. For a completely functional NM restart, YARN relies on any auxiliary service configured to also support recovery. This usually includes (1) avoiding usage of ephemeral ports so that previously running clients (in this case, usually containers) are not disrupted after restart and (2) having the auxiliary service itself support recoverability by reloading any previous state when NodeManager restarts and reinitializes the auxiliary service. - To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
