This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
The following commit(s) were added to refs/heads/master by this push: new 83f78c3 Add a deployment method using docker-compose 83f78c3 is described below commit 83f78c3b8187f7f0e4a93c50055e2e08b0bdd4e3 Author: Khurrum Nasimm <khurrumnas...@gmail.com> AuthorDate: Mon Aug 13 11:11:36 2018 -0700 Add a deployment method using docker-compose Descriptions of the changes in this PR: ### Motivation Provide a deployment method using docker-compose. ### Changes - Use helm as the template engine to generate docker compose file - Deploy zookeeper in cluster mode in docker-compose - Manages port mapping to make it able to access out side of the docker-composed cluster. Author: Khurrum Nasimm <khurrumnas...@gmail.com> Reviewers: Enrico Olivelli <eolive...@gmail.com>, Sijie Guo <si...@apache.org> This closes #1530 from khurrumnasimm/docker_compose --- conf/zookeeper.conf | 4 - deploy/docker-compose/README.md | 61 +++++++ deploy/docker-compose/compose/Chart.yaml | 28 ++++ .../compose/templates/docker-compose.yml | 132 +++++++++++++++ deploy/docker-compose/compose/values.yaml | 45 +++++ deploy/docker-compose/docker-compose.yaml | 184 +++++++++++++++++++++ 6 files changed, 450 insertions(+), 4 deletions(-) diff --git a/conf/zookeeper.conf b/conf/zookeeper.conf index 407baeb..89c9851 100644 --- a/conf/zookeeper.conf +++ b/conf/zookeeper.conf @@ -37,10 +37,6 @@ dataLogDir=data/zookeeper/txlog # the port at which the clients will connect clientPort=2181 -# the port at which the admin will listen -adminPort=9990 -zookeeper.admin.enableServer=true - # limit on queued clients - default: 1000 globalOutstandingLimit=1000 diff --git a/deploy/docker-compose/README.md b/deploy/docker-compose/README.md new file mode 100644 index 0000000..e8dcb8a --- /dev/null +++ b/deploy/docker-compose/README.md @@ -0,0 +1,61 @@ +# Apache BookKeeper docker-compose + +## Requirements + +* Docker >= 16.10 +* Docker Compose >= 1.6.0 + +## Quick start + +```bash +$ git clone https://github.com/apache/bookkeeper.git +$ cd bookkeeper/deploy/docker-compose +$ docker-compose pull # Get the latest Docker images +$ docker-compose up -d +$ cd ../../ +$ bin/bkctl bookies list +$ bin/bkctl ledger simpletest +``` + +## Access Apache BookKeeper cluster + + +### Ledger Service + +You can use `zk://localhost:2181/ledgers` as metadataServiceUri to access ledger storage service. + +```bash +$ bin/bkctl -u 'zk://localhost:2181/ledgers' ledger simpletest +``` + +### DistributedLog + +You can use `distributedlog://localhost:2181/distributedlog` as dlog uri to access ledger storage service +using [distributedlog](http://bookkeeper.apache.org/docs/latest/api/distributedlog-api/) API. + +```bash +$ bin/dlog tool create -u 'distributedlog://localhost:2181/distributedlog' --prefix test-stream -e 0-99 +``` + +## Customize Apache BookKeeper Cluster + +### Install Helm + +[Helm](https://helm.sh) is used as a template render engine + +``` +curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash +``` + +Or if you use Mac, you can use homebrew to install Helm by `brew install kubernetes-helm` + +### Bring up Apache BookKeeper cluster + +```bash +$ git clone https://github.com/apache/bookkeeper.git +$ cd bookkeeper/deploy/docker-compose +$ vi compose/values.yaml # custom cluster size, docker image, port mapping etc +$ helm template compose > generated-docker-compose.yaml +$ docker-compose -f generated-docker-compose.yaml pull # Get the latest Docker images +$ docker-compose -f generated-docker-compose.yaml up -d +``` diff --git a/deploy/docker-compose/compose/Chart.yaml b/deploy/docker-compose/compose/Chart.yaml new file mode 100644 index 0000000..73bb4b2 --- /dev/null +++ b/deploy/docker-compose/compose/Chart.yaml @@ -0,0 +1,28 @@ +#/** +# * Licensed to the Apache Software Foundation (ASF) under one +# * or more contributor license agreements. See the NOTICE file +# * distributed with this work for additional information +# * regarding copyright ownership. The ASF licenses this file +# * to you under the Apache License, Version 2.0 (the +# * "License"); you may not use this file except in compliance +# * with the License. You may obtain a copy of the License at +# * +# * http://www.apache.org/licenses/LICENSE-2.0 +# * +# * Unless required by applicable law or agreed to in writing, software +# * distributed under the License is distributed on an "AS IS" BASIS, +# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# * See the License for the specific language governing permissions and +# * limitations under the License. +# */ + +apiVersion: v1 +description: apache-bookkeeper-docker-compose +name: apache-bookkeeper-docker-compose +version: 4.8.0 +home: https://github.com/apache/bookkeeper +sources: + - https://github.com/apache/bookkeeper/deploy/docker-compose +keywords: + - log storage + - stream storage diff --git a/deploy/docker-compose/compose/templates/docker-compose.yml b/deploy/docker-compose/compose/templates/docker-compose.yml new file mode 100644 index 0000000..fbc70be --- /dev/null +++ b/deploy/docker-compose/compose/templates/docker-compose.yml @@ -0,0 +1,132 @@ +{{- /* +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +*/ -}} +{{- $zkSize := .Values.zookeeper.size | int }} +{{- $bkSize := .Values.bookkeeper.size | int }} +{{- $zkAdminPort := .Values.zookeeper.adminPort | int }} +{{- $zkClientPort := .Values.zookeeper.clientPort | int }} +{{- $zkPeerPort := .Values.zookeeper.peerPort | int }} +{{- $zkLeaderPort := .Values.zookeeper.leaderPort | int }} +{{- $bookiePort := .Values.bookkeeper.bookiePort | int }} +{{- $bookieGrpcPort := .Values.bookkeeper.bookieGrpcPort | int }} +{{- $bookieHttpPort := .Values.bookkeeper.bookieHttpPort | int }} + +{{- define "zookeeper_servers" }} + {{- range until (.Values.zookeeper.size | int) }} + {{- if . -}} + , + {{- end -}} + zookeeper-{{ . }}:2181 + {{- end -}} +{{- end -}} + +{{- define "metadata_service_uri" }} + {{- range until (.Values.zookeeper.size | int) }} + {{- if . -}} + ; + {{- end -}} + zookeeper-{{ . }}:2181 + {{- end -}} +{{- end -}} + + +{{- define "zookeeper_server_list" }} + {{- $zk := dict "servers" (list) -}} + {{- range until (.Values.zookeeper.size | int) }} + {{- $noop := printf "server.%d=zookeeper-%d:%d:%d:participant;0.0.0.0:%d" . . ($.Values.zookeeper.peerPort | int) ($.Values.zookeeper.leaderPort | int) ($.Values.zookeeper.clientPort | int) | append $zk.servers | set $zk "servers" -}} + {{- end -}} + {{- join " " $zk.servers -}} +{{- end -}} + +version: '3' + +services: + {{- range until $zkSize }} + zookeeper-{{ . }}: + image: {{ $.Values.zookeeper.image }} + hostname: zookeeper-{{ . }} + command: ["zookeeper"] + {{- if eq $.Values.networkMode "host" }} + network_mode: host + {{- else }} + ports: + - "{{ add $zkAdminPort . 1000 }}:{{ $zkAdminPort }}" + - "{{ add $zkClientPort . }}:{{ $zkClientPort }}" + {{- end }} + volumes: + - {{ $.Values.dataDir }}/zookeeper-{{ . }}/data:/data/zookeeper/data + - {{ $.Values.dataDir }}/zookeeper-{{ . }}/txlog:/data/zookeeper/txlog + environment: + - ZK_dataDir=/data/zookeeper/data + - ZK_dataLogDir=/data/zookeeper/txlog + - ZK_clientPort={{ $zkClientPort }} + - ZK_ID={{ . }} + - ZK_SERVERS={{- template "zookeeper_server_list" $ }} + - ZK_standaloneEnabled=false + healthcheck: + test: ["CMD", "curl", "-s", "http://localhost:{{ $zkAdminPort }}/commands/stat"] + interval: 60s + timeout: 3s + retries: 60 + restart: on-failure + {{ end }} + + {{- range until $bkSize }} + bookie-{{ . }}: + image: {{ $.Values.bookkeeper.image }} + hostname: bookie-{{ . }} + {{- if eq $.Values.networkMode "host" }} + network_mode: host + {{- else }} + ports: + {{- if eq $.Values.networkMode "host" }} + - "{{ add $bookiePort . }}:{{ $bookiePort }}" + {{- else }} + - "{{ add $bookiePort . }}:{{ add $bookiePort . }}" + {{- end }} + - "{{ add $bookieHttpPort . }}:{{ $bookieHttpPort }}" + - "{{ add $bookieGrpcPort . }}:{{ $bookieGrpcPort }}" + {{- end }} + volumes: + - {{ $.Values.dataDir }}/bookie-{{ . }}/journal:/data/bookkeeper/journal + - {{ $.Values.dataDir }}/bookie-{{ . }}/ledgers:/data/bookkeeper/ledgers + environment: + - BK_zkServers={{- template "zookeeper_servers" $ }} + - BK_zkLedgersRootPath=/ledgers + - BK_metadataServiceUri=zk://{{- template "metadata_service_uri" $ }}/ledgers + - BK_DATA_DIR=/data/bookkeeper + {{- if eq $.Values.networkMode "host" }} + - BK_bookiePort={{ $bookiePort }} + {{- else }} + - BK_advertisedAddress={{ $.Values.advertisedAddr }} + - BK_bookiePort={{ add $bookiePort . }} + {{- end }} + - BK_httpServerEnabled=true + depends_on: + {{- range until $zkSize }} + - "zookeeper-{{.}}" + {{- end }} + healthcheck: + test: ["CMD", "curl", "-s", "http://localhost:{{ $bookieHttpPort }}/heartbeat"] + interval: 60s + timeout: 3s + retries: 60 + restart: on-failure + {{ end }} diff --git a/deploy/docker-compose/compose/values.yaml b/deploy/docker-compose/compose/values.yaml new file mode 100644 index 0000000..a3b3eda --- /dev/null +++ b/deploy/docker-compose/compose/values.yaml @@ -0,0 +1,45 @@ +#/** +# * Licensed to the Apache Software Foundation (ASF) under one +# * or more contributor license agreements. See the NOTICE file +# * distributed with this work for additional information +# * regarding copyright ownership. The ASF licenses this file +# * to you under the Apache License, Version 2.0 (the +# * "License"); you may not use this file except in compliance +# * with the License. You may obtain a copy of the License at +# * +# * http://www.apache.org/licenses/LICENSE-2.0 +# * +# * Unless required by applicable law or agreed to in writing, software +# * distributed under the License is distributed on an "AS IS" BASIS, +# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# * See the License for the specific language governing permissions and +# * limitations under the License. +# */ + +# data directories +dataDir: ./data + +# advertised address that bookies used for advertising themselves. +# host network mode is useless on Mac, so in order to let clients +# on the host accessing bookies in the docker network, we can set +# the advertisedAddr to let bookies advertised in a host network. +advertisedAddr: 127.0.0.1 + +# supported networkMode: bridge | host +# host network mode is useless on Mac +networkMode: bridge + +zookeeper: + size: 3 + image: apachebookkeeper/bookkeeper-current:latest + adminPort: 8080 + clientPort: 2181 + peerPort: 2888 + leaderPort: 3888 + +bookkeeper: + size: 3 + image: apachebookkeeper/bookkeeper-current:latest + bookiePort: 3181 + bookieGrpcPort: 4181 + bookieHttpPort: 8080 diff --git a/deploy/docker-compose/docker-compose.yaml b/deploy/docker-compose/docker-compose.yaml new file mode 100644 index 0000000..1e25df2 --- /dev/null +++ b/deploy/docker-compose/docker-compose.yaml @@ -0,0 +1,184 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +--- +# Source: apache-bookkeeper-docker-compose/templates/docker-compose.yml +version: '3' + +services: + zookeeper-0: + image: apachebookkeeper/bookkeeper-current:latest + hostname: zookeeper-0 + command: ["zookeeper"] + ports: + - "9080:8080" + - "2181:2181" + volumes: + - ./data/zookeeper-0/data:/data/zookeeper/data + - ./data/zookeeper-0/txlog:/data/zookeeper/txlog + environment: + - ZK_dataDir=/data/zookeeper/data + - ZK_dataLogDir=/data/zookeeper/txlog + - ZK_clientPort=2181 + - ZK_ID=0 + - ZK_SERVERS=server.0=zookeeper-0:2888:3888:participant;0.0.0.0:2181 server.1=zookeeper-1:2888:3888:participant;0.0.0.0:2181 server.2=zookeeper-2:2888:3888:participant;0.0.0.0:2181 + - ZK_standaloneEnabled=false + healthcheck: + test: ["CMD", "curl", "-s", "http://localhost:8080/commands/stat"] + interval: 60s + timeout: 3s + retries: 60 + restart: on-failure + + zookeeper-1: + image: apachebookkeeper/bookkeeper-current:latest + hostname: zookeeper-1 + command: ["zookeeper"] + ports: + - "9081:8080" + - "2182:2181" + volumes: + - ./data/zookeeper-1/data:/data/zookeeper/data + - ./data/zookeeper-1/txlog:/data/zookeeper/txlog + environment: + - ZK_dataDir=/data/zookeeper/data + - ZK_dataLogDir=/data/zookeeper/txlog + - ZK_clientPort=2181 + - ZK_ID=1 + - ZK_SERVERS=server.0=zookeeper-0:2888:3888:participant;0.0.0.0:2181 server.1=zookeeper-1:2888:3888:participant;0.0.0.0:2181 server.2=zookeeper-2:2888:3888:participant;0.0.0.0:2181 + - ZK_standaloneEnabled=false + healthcheck: + test: ["CMD", "curl", "-s", "http://localhost:8080/commands/stat"] + interval: 60s + timeout: 3s + retries: 60 + restart: on-failure + + zookeeper-2: + image: apachebookkeeper/bookkeeper-current:latest + hostname: zookeeper-2 + command: ["zookeeper"] + ports: + - "9082:8080" + - "2183:2181" + volumes: + - ./data/zookeeper-2/data:/data/zookeeper/data + - ./data/zookeeper-2/txlog:/data/zookeeper/txlog + environment: + - ZK_dataDir=/data/zookeeper/data + - ZK_dataLogDir=/data/zookeeper/txlog + - ZK_clientPort=2181 + - ZK_ID=2 + - ZK_SERVERS=server.0=zookeeper-0:2888:3888:participant;0.0.0.0:2181 server.1=zookeeper-1:2888:3888:participant;0.0.0.0:2181 server.2=zookeeper-2:2888:3888:participant;0.0.0.0:2181 + - ZK_standaloneEnabled=false + healthcheck: + test: ["CMD", "curl", "-s", "http://localhost:8080/commands/stat"] + interval: 60s + timeout: 3s + retries: 60 + restart: on-failure + + bookie-0: + image: apachebookkeeper/bookkeeper-current:latest + hostname: bookie-0 + ports: + - "3181:3181" + - "8080:8080" + - "4181:4181" + volumes: + - ./data/bookie-0/journal:/data/bookkeeper/journal + - ./data/bookie-0/ledgers:/data/bookkeeper/ledgers + environment: + - BK_zkServers=zookeeper-0:2181,zookeeper-1:2181,zookeeper-2:2181 + - BK_zkLedgersRootPath=/ledgers + - BK_metadataServiceUri=zk://zookeeper-0:2181;zookeeper-1:2181;zookeeper-2:2181/ledgers + - BK_DATA_DIR=/data/bookkeeper + - BK_advertisedAddress=127.0.0.1 + - BK_bookiePort=3181 + - BK_httpServerEnabled=true + depends_on: + - "zookeeper-0" + - "zookeeper-1" + - "zookeeper-2" + healthcheck: + test: ["CMD", "curl", "-s", "http://localhost:8080/heartbeat"] + interval: 60s + timeout: 3s + retries: 60 + restart: on-failure + + bookie-1: + image: apachebookkeeper/bookkeeper-current:latest + hostname: bookie-1 + ports: + - "3182:3182" + - "8081:8080" + - "4182:4181" + volumes: + - ./data/bookie-1/journal:/data/bookkeeper/journal + - ./data/bookie-1/ledgers:/data/bookkeeper/ledgers + environment: + - BK_zkServers=zookeeper-0:2181,zookeeper-1:2181,zookeeper-2:2181 + - BK_zkLedgersRootPath=/ledgers + - BK_metadataServiceUri=zk://zookeeper-0:2181;zookeeper-1:2181;zookeeper-2:2181/ledgers + - BK_DATA_DIR=/data/bookkeeper + - BK_advertisedAddress=127.0.0.1 + - BK_bookiePort=3182 + - BK_httpServerEnabled=true + depends_on: + - "zookeeper-0" + - "zookeeper-1" + - "zookeeper-2" + healthcheck: + test: ["CMD", "curl", "-s", "http://localhost:8080/heartbeat"] + interval: 60s + timeout: 3s + retries: 60 + restart: on-failure + + bookie-2: + image: apachebookkeeper/bookkeeper-current:latest + hostname: bookie-2 + ports: + - "3183:3183" + - "8082:8080" + - "4183:4181" + volumes: + - ./data/bookie-2/journal:/data/bookkeeper/journal + - ./data/bookie-2/ledgers:/data/bookkeeper/ledgers + environment: + - BK_zkServers=zookeeper-0:2181,zookeeper-1:2181,zookeeper-2:2181 + - BK_zkLedgersRootPath=/ledgers + - BK_metadataServiceUri=zk://zookeeper-0:2181;zookeeper-1:2181;zookeeper-2:2181/ledgers + - BK_DATA_DIR=/data/bookkeeper + - BK_advertisedAddress=127.0.0.1 + - BK_bookiePort=3183 + - BK_httpServerEnabled=true + depends_on: + - "zookeeper-0" + - "zookeeper-1" + - "zookeeper-2" + healthcheck: + test: ["CMD", "curl", "-s", "http://localhost:8080/heartbeat"] + interval: 60s + timeout: 3s + retries: 60 + restart: on-failure + +