Repository: samza Updated Branches: refs/heads/master 20a1403f0 -> 5d50c0830
SAMZA-829: add doc to configure YARN Node Manager to allow clean shutdown of SamzaContainer w/ large state stores Project: http://git-wip-us.apache.org/repos/asf/samza/repo Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/5d50c083 Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/5d50c083 Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/5d50c083 Branch: refs/heads/master Commit: 5d50c08308b1626be397a11521e3494ce086fb9f Parents: 20a1403 Author: Yi Pan <[email protected]> Authored: Thu Dec 3 15:28:44 2015 -0800 Committer: Yi Pan (Data Infrastructure) <[email protected]> Committed: Thu Dec 3 15:28:44 2015 -0800 ---------------------------------------------------------------------- .../learn/documentation/versioned/yarn/yarn-host-affinity.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/samza/blob/5d50c083/docs/learn/documentation/versioned/yarn/yarn-host-affinity.md ---------------------------------------------------------------------- diff --git a/docs/learn/documentation/versioned/yarn/yarn-host-affinity.md b/docs/learn/documentation/versioned/yarn/yarn-host-affinity.md index 108dfbc..1d9c29e 100644 --- a/docs/learn/documentation/versioned/yarn/yarn-host-affinity.md +++ b/docs/learn/documentation/versioned/yarn/yarn-host-affinity.md @@ -96,7 +96,13 @@ export LOGGED_STORE_BASE_DIR=<path-for-state-stores> <value>1000*</value> <!-- Should be tuned per requirement --> </property> {% endhighlight %} - +3. Configure Yarn Node Manager SIGTERM to SIGKILL timeout to be reasonable time s.t. Node Manager will give Samza Container enough time to perform a clean shutdown in yarn-site.xml {% highlight xml %} +<property> + <name>yarn.nodemanager.sleep-delay-before-sigkill.ms</name> + <description>No. of ms to wait between sending a SIGTERM and SIGKILL to a container</description> + <value>600000</value> <!-- Set it to 10min to allow enough time for clean shutdown of containers --> +</property> +{% endhighlight %} ## Configuring a Samza job to use Host Affinity Any stateful Samza job can leverage this feature to reduce the Mean Time To Restore (MTTR) of it's state stores by setting <code>yarn.samza.host-affinity</code> to true.
