Ori.livneh has uploaded a new change for review. https://gerrit.wikimedia.org/r/227881
Change subject: nutcracker: prevent servers from being marked as dead indefinitely ...................................................................... nutcracker: prevent servers from being marked as dead indefinitely * Set a 30s server-retry-timeout. When this configuration option is unset, a transient connection failure is liable to make Nutcracker mark a server as dead forever (or at least until it is restarted).[1] * Reduce the number of server connections from 1 to 2. Having more than one server connections makes twemproxy reads and writes not repeatable, and I am not persuaded that we correctly account for that everywhere.[2] [1]: https://github.com/twitter/twemproxy/blob/594105/notes/recommendation.md#liveness [2]: https://github.com/twitter/twemproxy/blob/594105/notes/recommendation.md#server_connections--1 Change-Id: I546c1606cd33339ad91eb5c9a1160ed6599d3129 --- M manifests/role/mediawiki.pp 1 file changed, 6 insertions(+), 3 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/81/227881/1 diff --git a/manifests/role/mediawiki.pp b/manifests/role/mediawiki.pp index 84e6d8a..81df319 100644 --- a/manifests/role/mediawiki.pp +++ b/manifests/role/mediawiki.pp @@ -25,8 +25,9 @@ hash => 'md5', listen => '127.0.0.1:11212', preconnect => true, - server_connections => 2, + server_connections => 1, server_failure_limit => 3, + server_retry_timeout => to_milliseconds('30s'), timeout => 250, servers => hiera('mediawiki_memcached_servers'), }, @@ -36,8 +37,9 @@ hash => 'md5', listen => '/var/run/nutcracker/nutcracker.sock 0666', preconnect => true, - server_connections => 2, + server_connections => 1, server_failure_limit => 3, + server_retry_timeout => to_milliseconds('30s'), timeout => 250, servers => hiera('mediawiki_memcached_servers'), }, @@ -49,8 +51,9 @@ hash => 'md5', listen => '127.0.0.1:6380', preconnect => true, - server_connections => 2, + server_connections => 1, server_failure_limit => 3, + server_retry_timeout => to_milliseconds('30s'), timeout => 1000, servers => hiera('mediawiki_session_redis_servers'), }, -- To view, visit https://gerrit.wikimedia.org/r/227881 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I546c1606cd33339ad91eb5c9a1160ed6599d3129 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Ori.livneh <o...@wikimedia.org> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
