joshfischer1108 opened a new issue #3633:
URL: https://github.com/apache/incubator-heron/issues/3633
**Describe the bug**
The apache bookeeper helm chart will not deploy correctly when using helm.
It's seems to be related a persistentVolumeClaim causing a failedScheduling
event.
```
Warning FailedScheduling 13m (x2 over 13m) default-scheduler 0/1
nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.
```
**To Reproduce**
Steps to reproduce the behavior:
### Generate helm charts
```
$ bazel build --config=darwin scripts/packages:generate-helm-charts
```
### Output is something like
```
Successfully packaged chart and saved it to:
/private/var/somepath/70e9bd6aaead2621044e8d790ac25401/somepath/org_apache_heron/heron-0.0.0.tgz
```
The actual tgz file of the helm chart will be located at the path above +
/heron-charts
```
$ open
/private/var/somepath/70e9bd6aaead2621044e8d790ac25401/somepath/org_apache_heron/heron-charts
```
### CD into the path above
```
$ cd path/above/heron-charts && tar -xvzf heron-0.0.0.tgz
```
### Change the name of the docker image in the chart
In the folder heron swap out the container name in the values.yaml
`heron/heron:0.0.0` should be changed to
`joshfischer1108/heron:debian10-ecd2b034c0`
`zkReplicas` has a value of `3` and can be changed to `1` to save resources
### start minikube
```
$ minikube start --memory=7168 --cpus=3 --disk-size=20g
```
### create heron namespace in kubernetes
```
$ kubectl create namespace heron
```
### Install the heron helm chart
Fromt the prevously mentioned directory above
`/private/var/somepath/70e9bd6aaead2621044e8d790ac25401/somepath/org_apache_heron/heron-charts`
execute the following command
```
$ helm install heron ./heron -n heron
```
the above command installs heron into kubernetes using helm and names the
deployment "heron" into the namespace "heron". It can take several minutes for
all the pods to start up.
you can check the status of the pods by executing the below:
```
$ kubectl get po -n heron
```
If the initial helm deployment is successful you will see something like
below.
#### Even though Helm is successfull we are not out of the woods yet.
```
NAME: heron
LAST DEPLOYED: Sat Nov 21 16:05:38 2020
NAMESPACE: heron
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
You can check the status of Heron by doing the following:
1. Start kubectl proxy
kubectl proxy -p 8001
2. Verify you can access the API server
curl
http://localhost:8001/api/v1/namespaces/heron/services/heron-apiserver:9000/proxy/api/v1/version
3. Verify you can access the heron ui
open
http://localhost:8001/api/v1/namespaces/heron/services/heron-ui:8889/proxy
4. Set your service url
heron config heron set service_url
http://localhost:8001/api/v1/namespaces/heron/services/heron-apiserver:9000/proxy
5. Submit an example topology
heron submit heron ~/.heron/examples/heron-api-examples.jar
org.apache.heron.examples.api.AckingTopology acking
6. Visit the heron ui and see if your example topology is running
open
http://localhost:8001/api/v1/namespaces/heron/services/heron-ui:8889/proxy
7. You can kill the example topology by
heron kill heron acking
```
After a minute or two zookeeper should start deploying itself and then
failures start to occur with bookeeper nodes
```
$ kubectl get po -n heron
NAME READY STATUS RESTARTS AGE
heron-bookie-0 0/1 Error 3 4m36s
heron-bookie-1 0/1 Pending 0 83s
heron-tools-5867d98b9-2hpnw 3/3 Running 0 4m36s
heron-zookeeper-0 1/1 Running 0 4m36s
```
## to check the status of the failing
```
$ kubectl describe po heron-bookie-0 -n heron
```
The truncated output of this command will be something like
```
Warning FailedScheduling 4m54s (x2 over 4m54s) default-scheduler 0/1
nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.
Normal Scheduled 4m52s default-scheduler
Successfully assigned heron/heron-bookie-0 to minikube
Normal Pulling 4m49s kubelet, minikube
Pulling image "joshfischer1108/heron:debian10-ecd2b034c0"
Normal Pulled 2m40s kubelet, minikube
Successfully pulled image "joshfischer1108/heron:debian10-ecd2b034c0" in
2m9.567024773s
Normal Created 2m39s kubelet, minikube
Created container bookie-init-container
Normal Started 2m39s kubelet, minikube
Started container bookie-init-container
Normal Pulling 2m14s kubelet, minikube
Pulling image "apache/bookkeeper:4.11.0"
Normal Created 111s kubelet, minikube
Created container bookie-format
Normal Pulled 111s kubelet, minikube
Successfully pulled image "apache/bookkeeper:4.11.0" in 23.603994497s
Normal Started 110s kubelet, minikube
Started container bookie-format
Normal Pulled 30s (x4 over 102s) kubelet, minikube
Container image "apache/bookkeeper:4.11.0" already present on machine
Normal Created 30s (x4 over 102s) kubelet, minikube
Created container bookie
Normal Started 29s (x4 over 101s) kubelet, minikube
Started container bookie
Warning BackOff 6s (x5 over 75s) kubelet, minikube
Back-off restarting failed container
```
I'm not very familiar with bookeeper. I remember others having trouble with
it in past.
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Operating System**
- OS: [e.g. iOS]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
