This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-helm-chart.git
The following commit(s) were added to refs/heads/master by this push:
new 89f28bc Support mechanism to provide external zookeeper-server list
to build global/configuration zookeeper (#269)
89f28bc is described below
commit 89f28bca9c908e9a06e8939928b09ed331df662e
Author: Rajan Dhabalia <[email protected]>
AuthorDate: Tue Oct 18 15:41:43 2022 -0700
Support mechanism to provide external zookeeper-server list to build
global/configuration zookeeper (#269)
* Support mechanism to provide external zookeeper-server list to build
global/configuration zookeeper
* Add external zk example
* add external zk list into values.yaml
Fixes #268
### Motivation
Right now, [chart
dynamically](https://github.com/apache/pulsar-helm-chart/blob/master/charts/pulsar/templates/zookeeper-statefulset.yaml#L140)
creates zk cluster with zk pods initialized in the same namespace. However,
for global/configuration zookeeper, user requires to build zk clusters with
pods deployed in different namespaces. Therefore, user needs a mechanism to
pass an external list of zk-servers to the chart and build zk-cluster with pods
across different namespaces.
### Modification
- Chart should be considering zk-value's configuration for external
zookeeper and generate zk-configuration file with appropriate zk-server list
and unique id of that zookeeper.
This PR sets `ZOOKEEPER_SERVERS` value provided by user and also sets
override-value flag which will be used by
[generate-zookeeper-config.sh](https://github.com/apache/pulsar/blob/master/docker/pulsar/scripts/generate-zookeeper-config.sh)
to override external zk list in config file and assign appropriate id to the
host.
https://github.com/apache/pulsar/pull/15987 fixes
[generate-zookeeper-config.sh](https://github.com/apache/pulsar/blob/master/docker/pulsar/scripts/generate-zookeeper-config.sh)
changes.
### Result
- User can add `ZOOKEEPER_SERVERS` string into `zookeeper.configData` in
[Values.yaml](https://github.com/apache/pulsar-helm-chart/blob/master/charts/pulsar/values.yaml#L385)
file to override external zk-server list.
---
charts/pulsar/templates/zookeeper-statefulset.yaml | 17 +++++++++++++----
charts/pulsar/values.yaml | 2 ++
examples/values-zookeeper-aws.yaml | 3 +++
3 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/charts/pulsar/templates/zookeeper-statefulset.yaml
b/charts/pulsar/templates/zookeeper-statefulset.yaml
index 5590a40..a72d50b 100644
--- a/charts/pulsar/templates/zookeeper-statefulset.yaml
+++ b/charts/pulsar/templates/zookeeper-statefulset.yaml
@@ -138,10 +138,19 @@ spec:
containerPort: {{ .Values.zookeeper.ports.clientTls }}
{{- end }}
env:
- - name: ZOOKEEPER_SERVERS
- value:
- {{- $global := . }}
- {{ range $i, $e := until (.Values.zookeeper.replicaCount | int)
}}{{ if ne $i 0 }},{{ end }}{{ template "pulsar.fullname" $global }}-{{
$global.Values.zookeeper.component }}-{{ printf "%d" $i }}{{ end }}
+ - name: ZOOKEEPER_SERVERS
+ {{- if .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
+ value: {{ .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
+ {{- else }}
+ {{- $global := . }}
+ value: {{ range $i, $e := until (.Values.zookeeper.replicaCount |
int) }}{{ if ne $i 0 }},{{ end }}{{ template "pulsar.fullname" $global }}-{{
$global.Values.zookeeper.component }}-{{ printf "%d" $i }}{{ end }}
+ {{- end }}
+ - name: EXTERNAL_PROVIDED_SERVERS
+ {{- if .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
+ value: "true"
+ {{- else }}
+ value: "false"
+ {{- end }}
envFrom:
- configMapRef:
name: "{{ template "pulsar.fullname" . }}-{{
.Values.zookeeper.component }}"
diff --git a/charts/pulsar/values.yaml b/charts/pulsar/values.yaml
index a602edb..5d970e1 100644
--- a/charts/pulsar/values.yaml
+++ b/charts/pulsar/values.yaml
@@ -397,6 +397,8 @@ zookeeper:
## templates/zookeeper-configmap.yaml
##
configData:
+ # External zookeeper server list in case of global-zk list to create zk
cluster across zk deployed on different clusters/namespaces
+ # ZOOKEEPER_SERVERS:
"us-east1-pulsar-zookeeper-0.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-1.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-2.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-0.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-1.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pu
[...]
PULSAR_MEM: >
-Xms64m -Xmx128m
PULSAR_GC: >
diff --git a/examples/values-zookeeper-aws.yaml
b/examples/values-zookeeper-aws.yaml
index 8b870ef..2116f86 100644
--- a/examples/values-zookeeper-aws.yaml
+++ b/examples/values-zookeeper-aws.yaml
@@ -45,6 +45,9 @@ monitoring:
node_exporter: false
zookeeper:
+ configData:
+ # External zookeeper server list in case of global-zk list to create zk
cluster across zk deployed on different clusters/namespaces
+ # ZOOKEEPER_SERVERS:
"us-east1-pulsar-zookeeper-0.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-1.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-2.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-0.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-1.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pu
[...]
volumes:
# use a persistent volume or emptyDir
persistence: true
