Mobrovac has uploaded a new change for review.
https://gerrit.wikimedia.org/r/278329
Change subject: kafka_config: Utility function for formatting configuration
variables
......................................................................
kafka_config: Utility function for formatting configuration variables
The Kafka module classes need the configuration in various formats. For
example, some need a hash describing the brokers, some need just the
list of their FQDNs, while others need a concatenated string of
host:port pairs. Starting from a cluster name, a hash of all of the
brokers in all of the clusters and a hash or list of zookeeper hosts,
the function returns one hash conveniently keyed and formatted ready to
be used.
Bug: T130371
Change-Id: I64dcd3cceaed749bbae6e88fdf588b79ddf49fa5
---
A lib/puppet/parser/functions/kafka_config.rb
1 file changed, 37 insertions(+), 0 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/puppet/kafka
refs/changes/29/278329/1
diff --git a/lib/puppet/parser/functions/kafka_config.rb
b/lib/puppet/parser/functions/kafka_config.rb
new file mode 100644
index 0000000..1a2256d
--- /dev/null
+++ b/lib/puppet/parser/functions/kafka_config.rb
@@ -0,0 +1,37 @@
+# == Function: kakfa_config(string cluster_name, hash|nil clusters, hash|array
zookeeper_hosts)
+#
+# Reworks various variables to be in a format suitable for supplying them
+# to the kafka module classes.
+#
+
+module Puppet::Parser::Functions
+ newfunction(:kafka_config, :type => :rvalue, :arity => 3) do |args|
+ cluster_name, clusters, zk_hosts = *args
+ # nil is sometimes passed as 'undef' by Puppet, so get around this
stupidity
+ clusters = Hash.new unless clusters.kind_of?(Hash)
+ zk_hosts = zk_hosts.keys.sort if zk_hosts.kind_of?(Hash)
+ cluster = clusters[cluster_name] || {
+ 'brokers' => {
+ lookupvar('fqdn').to_s => { 'id' => '1' }
+ }
+ }
+ brokers = cluster['brokers']
+ jmx_port = 9999
+ conf = {
+ 'brokers' => {
+ 'hash' => brokers,
+ 'array' => brokers.keys,
+ 'string' => brokers.map { |host, port| "#{host}:#{port || 9092}"
}.sort.join ',',
+ 'graphite' => brokers.keys.map { |b| "#{b.tr '.', '_'}_#{jmx_port}"
}.join ',',
+ 'size' => brokers.keys.size
+ },
+ 'jmx_port' => jmx_port,
+ 'zookeeper' => {
+ 'hosts' => zk_hosts,
+ 'chroot' => "/kafka/#{cluster_name}",
+ 'url' => "#{zk_hosts.join ','}/kafka/#{cluster_name}"
+ }
+ }
+ end
+end
+
--
To view, visit https://gerrit.wikimedia.org/r/278329
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I64dcd3cceaed749bbae6e88fdf588b79ddf49fa5
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet/kafka
Gerrit-Branch: master
Gerrit-Owner: Mobrovac <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
