Here is the info of openwhisk pods before reinstalling chart: `kubectl get po
-n openwhisk`
```
NAME READY STATUS RESTARTS AGE
apigateway-7b87dd957f-nfm47 1/1 Running 0 2d
controller-0 0/1 Init:1/2 0 2d
couchdb-6b87ccfb78-9454z 1/1 Running 0 2d
init-couchdb-dd4nt 0/1 Completed 0 5d
invoker-j7smk 0/1 Init:CrashLoopBackOff 821 5d
invoker-p7jqq 0/1 Init:CrashLoopBackOff 1455 5d
kafka-0 1/1 Running 0 2d
nginx-598c75d7d6-j26jw 1/1 Running 0 2d
redis-5d77674f65-bnzpp 1/1 Running 0 5d
zookeeper-0 1/1 Running 0 5d
```
from above we can see that the couchdb pod `couchdb-6b87ccfb78-9454z` has ZERO
restarts. I think it means couchdb pod has not crashed yet?
I tried to delete and reinstall the chart. Here is the status. It shows that
this problem happened again. :(
`helm status openwhisk`
```
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
invoker-9nkwp 0/1 Init:CrashLoopBackOff 2 1m
invoker-gnns9 0/1 Init:CrashLoopBackOff 2 1m
apigateway-7b87dd957f-dh2sb 1/1 Running 0 1m
couchdb-6b87ccfb78-wvmj6 1/1 Running 0 1m
nginx-598c75d7d6-rmr86 1/1 Running 0 1m
redis-5d77674f65-qqm4q 1/1 Running 0 1m
controller-0 0/1 Init:1/2 0 1m
kafka-0 1/1 Running 0 1m
zookeeper-0 1/1 Running 0 1m
init-couchdb-4dbqc 1/1 Running 0 1m
install-catalog-gpfsf 0/1 Init:0/1 0 1m
install-routemgmt-tjzxt 0/1 Init:0/1 0 1m
```
The logs of couchdb:
`kubectl logs -n openwhisk couchdb-6b87ccfb78-wvmj6 | tail -20`
```
[notice] 2018-09-06T10:05:43.841086Z couchdb@couchdb0 <0.1792.0> ab7b974bb6
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[error] 2018-09-06T10:05:44.347501Z couchdb@couchdb0 emulator -------- Error in
process <0.1809.0> on node 'couchdb@couchdb0' with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,378}]},{mem3_shards,load_shards_from_disk...
[notice] 2018-09-06T10:05:44.347576Z couchdb@couchdb0 <0.333.0> --------
chttpd_auth_cache changes listener died database_does_not_exist at
mem3_shards:load_shards_from_db/6(line:403) <=
mem3_shards:load_shards_from_disk/1(line:378) <=
mem3_shards:load_shards_from_disk/2(line:407) <=
mem3_shards:for_docid/3(line:91) <= fabric_doc_open:go/3(line:38) <=
chttpd_auth_cache:ensure_auth_ddoc_exists/2(line:187) <=
chttpd_auth_cache:listen_for_changes/1(line:134)
[notice] 2018-09-06T10:05:44.846488Z couchdb@couchdb0 <0.1793.0> 07983bc56d
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[notice] 2018-09-06T10:05:45.851705Z couchdb@couchdb0 <0.1826.0> 7f011bbcff
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[notice] 2018-09-06T10:05:46.857345Z couchdb@couchdb0 <0.1827.0> 1e5d6c7034
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[notice] 2018-09-06T10:05:47.863583Z couchdb@couchdb0 <0.1859.0> 5aada43a57
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 2
[notice] 2018-09-06T10:05:48.868913Z couchdb@couchdb0 <0.1860.0> 2eb483dd98
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[error] 2018-09-06T10:05:49.348616Z couchdb@couchdb0 emulator -------- Error in
process <0.1893.0> on node 'couchdb@couchdb0' with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,378}]},{mem3_shards,load_shards_from_disk...
[notice] 2018-09-06T10:05:49.348700Z couchdb@couchdb0 <0.333.0> --------
chttpd_auth_cache changes listener died database_does_not_exist at
mem3_shards:load_shards_from_db/6(line:403) <=
mem3_shards:load_shards_from_disk/1(line:378) <=
mem3_shards:load_shards_from_disk/2(line:407) <=
mem3_shards:for_docid/3(line:91) <= fabric_doc_open:go/3(line:38) <=
chttpd_auth_cache:ensure_auth_ddoc_exists/2(line:187) <=
chttpd_auth_cache:listen_for_changes/1(line:134)
[notice] 2018-09-06T10:05:49.874551Z couchdb@couchdb0 <0.1892.0> 24e7be4da4
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[notice] 2018-09-06T10:05:50.884279Z couchdb@couchdb0 <0.1894.0> 3601177d80
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[notice] 2018-09-06T10:05:51.889791Z couchdb@couchdb0 <0.1926.0> f40a302de1
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[notice] 2018-09-06T10:05:52.898896Z couchdb@couchdb0 <0.1927.0> b333eb5725
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[notice] 2018-09-06T10:05:53.904828Z couchdb@couchdb0 <0.1959.0> dfd6484146
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[error] 2018-09-06T10:05:54.349466Z couchdb@couchdb0 emulator -------- Error in
process <0.1976.0> on node 'couchdb@couchdb0' with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,378}]},{mem3_shards,load_shards_from_disk...
[notice] 2018-09-06T10:05:54.349538Z couchdb@couchdb0 <0.333.0> --------
chttpd_auth_cache changes listener died database_does_not_exist at
mem3_shards:load_shards_from_db/6(line:403) <=
mem3_shards:load_shards_from_disk/1(line:378) <=
mem3_shards:load_shards_from_disk/2(line:407) <=
mem3_shards:for_docid/3(line:91) <= fabric_doc_open:go/3(line:38) <=
chttpd_auth_cache:ensure_auth_ddoc_exists/2(line:187) <=
chttpd_auth_cache:listen_for_changes/1(line:134)
[notice] 2018-09-06T10:05:54.910685Z couchdb@couchdb0 <0.1960.0> 1816533b33
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[notice] 2018-09-06T10:05:55.923100Z couchdb@couchdb0 <0.1993.0> ca7d4af1b6
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
[notice] 2018-09-06T10:05:56.928718Z couchdb@couchdb0 <0.1994.0> 3c1057e0b6
couchdb.openwhisk.svc.cluster.local:5984 172.22.69.14 undefined GET
/test_activations 404 ok 1
```
[ Full content available at:
https://github.com/apache/incubator-openwhisk-deploy-kube/issues/293 ]
This message was relayed via gitbox.apache.org for [email protected]
