Alexandros Kosiaris has submitted this change and it was merged.
Change subject: Don't include a node in its own seeds
......................................................................
Don't include a node in its own seeds
In multi-node clusters we want to make sure that a new node bootstraps (loads
data from other nodes) fully before accepting requests. While not directly
leading to incorrect query results when using LOCALQUORUM reads, an empty node
joining the cluster before a repair will lead to incorrect query results for
queries with consistency level of ONE.
Bootstrapping is disabled if the node is listed among its own seeds. Hence,
this patch ensures that we don't list a node among its seeds in multi-node
clusters.
This also means that multi-node clusters will require manual intervention
during initialization, as none of the nodes will be able to successfully
bootstrap. To initialize a new multi-node cluster, we need to temporarily add
a node among its own seeds in order to break the symmetry and allow it to
fully start up. All other nodes will then normally bootstrap from this node.
Bug: T91617
Change-Id: Iaa3bbf07b6053e139dc9483f849f6d4acba1418b
---
M manifests/init.pp
M templates/cassandra.yaml.erb
2 files changed, 10 insertions(+), 4 deletions(-)
Approvals:
Alexandros Kosiaris: Verified; Looks good to me, approved
diff --git a/manifests/init.pp b/manifests/init.pp
index 2ea990d..bb7baa3 100644
--- a/manifests/init.pp
+++ b/manifests/init.pp
@@ -266,8 +266,8 @@
fail('storage_port must be a port number between 1 and 65535')
}
- if (empty($seeds)) {
- fail('seeds must not be empty')
+ if (!is_array($seeds) or empty($seeds)) {
+ fail('seeds must be an array and not be empty')
}
if (empty($data_file_directories)) {
diff --git a/templates/cassandra.yaml.erb b/templates/cassandra.yaml.erb
index c6c7d84..994bd53 100644
--- a/templates/cassandra.yaml.erb
+++ b/templates/cassandra.yaml.erb
@@ -262,8 +262,14 @@
parameters:
# seeds is actually a comma-delimited list of addresses.
# Ex: "<ip1>,<ip2>,<ip3>"
- - seeds: <%= Array(@seeds).join(',') %>
-
+ # Omit own host name / IP in multi-node clusters (see
+ # https://phabricator.wikimedia.org/T91617).
+ <% clean_seeds = @seeds.length > 1 ? @seeds.select { |x|
+ x != @hostname \
+ && x != @ipaddress \
+ && x != @fqdn
+ }.join(',') :
@seeds.join(',') %>
+ - seeds: <%= clean_seeds %>
# For workloads with more data than can fit in memory, Cassandra's
# bottleneck will be reads that need to fetch data from
# disk. "concurrent_reads" should be set to (16 * number_of_drives) in
--
To view, visit https://gerrit.wikimedia.org/r/195483
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Iaa3bbf07b6053e139dc9483f849f6d4acba1418b
Gerrit-PatchSet: 7
Gerrit-Project: operations/puppet/cassandra
Gerrit-Branch: master
Gerrit-Owner: GWicke <[email protected]>
Gerrit-Reviewer: Alexandros Kosiaris <[email protected]>
Gerrit-Reviewer: Eevans <[email protected]>
Gerrit-Reviewer: Filippo Giunchedi <[email protected]>
Gerrit-Reviewer: GWicke <[email protected]>
Gerrit-Reviewer: Ori.livneh <[email protected]>
Gerrit-Reviewer: Ottomata <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
