BBlack has submitted this change and it was merged.
Change subject: Set bnx2x num_queues to match physical cores on lvs nodes
......................................................................
Set bnx2x num_queues to match physical cores on lvs nodes
bnx2x actually defaults to num_queues = num_procs, but doesn't
account for hyperthreading and caps itself at an internal
max queue count. In the case of lvs[34]00x, the simple num_procs
is 24, the physical core count is 12, and bnx2x ends up defaulting
to its internal max of 15. We'd rather have 12 queues to balance
over the 12 real cores with RPS than 15/12 with three oddballs
that can't be balanced evenly.
Getting this to take effect requires a reboot or a reload of
the bnx2x module, which it probably isn't wise to puppet-automate.
The RPS-balancing script will work regardless, this just makes
things more-optimal post-reboot/reload.
Change-Id: If88abac93e52147d915e34dc5649cdf8cadf0df3
---
A modules/lvs/lib/facter/physicalcorecount.rb
M modules/lvs/manifests/balancer.pp
A modules/lvs/templates/lvs.conf.erb
3 files changed, 16 insertions(+), 2 deletions(-)
Approvals:
Mark Bergsma: Looks good to me, but someone else must approve
Faidon Liambotis: Looks good to me, but someone else must approve
BBlack: Verified; Looks good to me, approved
jenkins-bot: Verified
diff --git a/modules/lvs/lib/facter/physicalcorecount.rb
b/modules/lvs/lib/facter/physicalcorecount.rb
new file mode 100644
index 0000000..2e6456e
--- /dev/null
+++ b/modules/lvs/lib/facter/physicalcorecount.rb
@@ -0,0 +1,12 @@
+# physicalcorecount isn't fooled by hyperthread siblings
+
+require 'facter'
+require 'pathname'
+
+Facter.add("physicalcorecount") do
+ setcode do
+
Pathname::glob('/sys/devices/system/cpu/cpu[0-9]*/topology/thread_siblings_list').map{|x|
+ File.open(x,'r').read().split(',')[0]
+ }.sort.uniq.count
+ end
+end
diff --git a/modules/lvs/manifests/balancer.pp
b/modules/lvs/manifests/balancer.pp
index f179f29..ade6961 100644
--- a/modules/lvs/manifests/balancer.pp
+++ b/modules/lvs/manifests/balancer.pp
@@ -28,9 +28,8 @@
site => $site
}
- # Tune the ip_vs conn_tab_bits parameter
file { "/etc/modprobe.d/lvs.conf":
- content => "# This file is managed by Puppet!\noptions ip_vs
conn_tab_bits=20\n";
+ content => template("${module_name}/lvs.conf.erb");
}
# Bind balancer IPs to the loopback interface
diff --git a/modules/lvs/templates/lvs.conf.erb
b/modules/lvs/templates/lvs.conf.erb
new file mode 100644
index 0000000..8988348
--- /dev/null
+++ b/modules/lvs/templates/lvs.conf.erb
@@ -0,0 +1,3 @@
+# This file is managed by Puppet!
+options ip_vs conn_tab_bits=20
+options bnx2x num_queues=<%= physicalcorecount %>
--
To view, visit https://gerrit.wikimedia.org/r/127827
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: If88abac93e52147d915e34dc5649cdf8cadf0df3
Gerrit-PatchSet: 5
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: BBlack <bbl...@wikimedia.org>
Gerrit-Reviewer: BBlack <bbl...@wikimedia.org>
Gerrit-Reviewer: Faidon Liambotis <fai...@wikimedia.org>
Gerrit-Reviewer: Mark Bergsma <m...@wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
