EBernhardson has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/370834 )

Change subject: Switch elastic1017-1031 to niofs
......................................................................

Switch elastic1017-1031 to niofs

By default elasticsearch uses an mmap file store implementation
for 64 bit linux hosts. This works on most of our servers, but after
the upgrade from elasticsearch 2 to elasticsearch 5 these servers
have seen drastically increased IO load, and the linux kernel reports
only about half of the available disk caching memory is actually being
mmap'd into the process. On a single test machine a switch to niofs
dramatically decreased IO and appears to have let the servers access
the full disk caching memory.

Bug: T169498
Change-Id: I8ea4689356f738932567ee27b5f4e7f5b4759491
---
M modules/elasticsearch/manifests/init.pp
M modules/elasticsearch/templates/elasticsearch_5.yml.erb
M modules/profile/manifests/elasticsearch.pp
3 files changed, 10 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/34/370834/1

diff --git a/modules/elasticsearch/manifests/init.pp 
b/modules/elasticsearch/manifests/init.pp
index bc7571e..337f3a4 100644
--- a/modules/elasticsearch/manifests/init.pp
+++ b/modules/elasticsearch/manifests/init.pp
@@ -78,6 +78,10 @@
 #        and port are used. Defaults to undef, which means no remote reindex 
can occur.
 # - $script_max_compilations_per_minute: integer, max number of script
 #        compilations per minute, defaults to undef (see T171579).
+# - $index_store: Sets the storage implementation elasticsearch will use to
+#        access index data. Defaults to undef which lets elasticsearch decide.
+#        This should almost always be undef unless there is a specific issue to
+#        be solved.
 #
 # == Sample usage:
 #
@@ -117,6 +121,7 @@
     $curator_uses_unicast_hosts = true,
     $reindex_remote_whitelist = undef,
     $script_max_compilations_per_minute = undef,
+    $index_store = undef,
 ) {
 
     # Check arguments
diff --git a/modules/elasticsearch/templates/elasticsearch_5.yml.erb 
b/modules/elasticsearch/templates/elasticsearch_5.yml.erb
index 971cdd7..2f4e40a 100644
--- a/modules/elasticsearch/templates/elasticsearch_5.yml.erb
+++ b/modules/elasticsearch/templates/elasticsearch_5.yml.erb
@@ -135,6 +135,9 @@
 # Use the Index Status API (<http://localhost:9200/A/_status>) to inspect
 # the index status.
 
+<%if @index_store %>
+index.store.type: <%= @index_store %>
+<% end %>
 
 #################################### Paths ####################################
 
diff --git a/modules/profile/manifests/elasticsearch.pp 
b/modules/profile/manifests/elasticsearch.pp
index 54ae82f..af4e818 100644
--- a/modules/profile/manifests/elasticsearch.pp
+++ b/modules/profile/manifests/elasticsearch.pp
@@ -27,6 +27,7 @@
     $recover_after_time = hiera('profile::elasticsearch::recover_after_time', 
'1s'),
     $recover_after_nodes = 
hiera('profile::elasticsearch::recover_after_nodes', 1),
     $reindex_remote_whitelist = 
hiera('profile::elasticsearch::reindex_remote_whitelist'),
+    $index_store = hiera('profile::elasticsearch::index_store'),
 ) {
     $master_eligible = $::fqdn in $unicast_hosts
 
@@ -96,6 +97,7 @@
         version                            => 5,
         reindex_remote_whitelist           => $reindex_remote_whitelist,
         script_max_compilations_per_minute => 10000,
+        index_store                        => $index_store,
     }
 
     class { '::elasticsearch::https':

-- 
To view, visit https://gerrit.wikimedia.org/r/370834
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8ea4689356f738932567ee27b5f4e7f5b4759491
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: EBernhardson <ebernhard...@wikimedia.org>

_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

Reply via email to