Hi All,

We are trying to bring up DCAE core services and for some reason
*dcae-bootstrap*, *dcae-cloudify, deployment-handler* and
*dcae-policy-handler* PODs are hanging in INIT state for a very long time.

The observation is that *dcae-cloudify-manager* is stuck in *INIT* state
and the *deployment-handler* keeps waiting for *dcae-cloudify-manager* POD
to come up.

We are using ElAlto release to bring up DCAE services. We did try
to disable dcae-servicechange-handler, dcae-inventory-api, dcae-dashboard
and dcae-redis got us into the same issue.

Could someone point me to any known issues or workarounds to resolve this
one, below are the log snippet for your reference.

Thanks & Regards
Vivek

# DCAEGEN2 POD status

*dev-dcaegen2-dcae-bootstrap-7975648cdc-qsnx9                  0/1
Init:0/2     9
 146mdev-dcaegen2-dcae-cloudify-manager-6fd48c88bc-5gtst           0/1
Running      0          124m*
dev-dcaegen2-dcae-config-binding-service-bf5b8c5db-x4pcx      4/4
Running      0          2d1h
dev-dcaegen2-dcae-dashboard-7bc7cfcf86-fx2t4                  0/2
Init:0/2     66         2d1h
dev-dcaegen2-dcae-dashboard-pg-0                              1/1
Running      0          2d1h
dev-dcaegen2-dcae-dashboard-pg-1                              1/1
Running      0          2d1h
dev-dcaegen2-dcae-dashboard-pgpool-7455d9b99f-g5l92           1/1
Running      0          2d1h
dev-dcaegen2-dcae-dashboard-pgpool-7455d9b99f-g6k65           1/1
Running      0          2d1h
dev-dcaegen2-dcae-db-0                                        1/1
Running      0          2d1h
dev-dcaegen2-dcae-db-1                                        1/1
Running      0          2d1h
*dev-dcaegen2-dcae-deployment-handler-7855bbb99f-tbddd         0/2
Init:0/3     66         2d1h*
dev-dcaegen2-dcae-healthcheck-d45c4f896-s9tvw                 1/1
Running      0          2d1h
dev-dcaegen2-dcae-inv-pg-0                                    1/1
Running      0          2d1h
dev-dcaegen2-dcae-inv-pg-1                                    1/1
Running      0          2d1h
dev-dcaegen2-dcae-inv-pgpool-75cf498b5f-rcpkw                 1/1
Running      0          2d1h
dev-dcaegen2-dcae-inv-pgpool-75cf498b5f-vg9vp                 1/1
Running      0          2d1h
dev-dcaegen2-dcae-inventory-api-5855f56c7f-8nrrj              1/1
Running      0          2d1h
dev-dcaegen2-dcae-pgpool-7f64667b9-mpncs                      1/1
Running      0          2d1h
dev-dcaegen2-dcae-pgpool-7f64667b9-tsrz2                      1/1
Running      0          2d1h
*dev-dcaegen2-dcae-policy-handler-76ddbd4fc9-jvlfw             0/2
Init:0/3     66         2d1h*
dev-dcaegen2-dcae-redis-0                                     1/1
Running      0          2d1h
dev-dcaegen2-dcae-redis-1                                     1/1
Running      0          2d1h
dev-dcaegen2-dcae-redis-2                                     1/1
Running      0          2d1h
dev-dcaegen2-dcae-redis-3                                     1/1
Running      0          2d1h
dev-dcaegen2-dcae-redis-4                                     1/1
Running      0          2d1h
dev-dcaegen2-dcae-redis-5                                     1/1
Running      0          2d1h
dev-dcaegen2-dcae-servicechange-handler-775d98576b-twcr5      1/1
Running      0          2d1h

*# Cloudify manager logs struch in INIT state*
$KLOG dev-dcaegen2-dcae-cloudify-manager-6b88d488-2l5fg
+ '[' -d /cfy-persist ']'
++ ls -A /cfy-persist
+ '[' -z '' ']'
+ for d in '$PDIRS'
++ dirname /var/lib/pgsql/9.5/data
+ p=/var/lib/pgsql/9.5
+ mkdir -p /cfy-persist/var/lib/pgsql/9.5
+ cp -rp /var/lib/pgsql/9.5/data /cfy-persist/var/lib/pgsql/9.5
+ for d in '$PDIRS'
++ dirname /opt/manager/resources
+ p=/opt/manager
+ mkdir -p /cfy-persist/opt/manager
+ cp -rp /opt/manager/resources /cfy-persist/opt/manager
+ for d in '$PDIRS'
++ dirname /opt/mgmtworker/env/plugins
+ p=/opt/mgmtworker/env
+ mkdir -p /cfy-persist/opt/mgmtworker/env
+ cp -rp /opt/mgmtworker/env/plugins /cfy-persist/opt/mgmtworker/env
+ for d in '$PDIRS'
++ dirname /opt/mgmtworker/work/deployments
+ p=/opt/mgmtworker/work
+ mkdir -p /cfy-persist/opt/mgmtworker/work
+ cp -rp /opt/mgmtworker/work/deployments /cfy-persist/opt/mgmtworker/work
+ for d in '$PDIRS'
+ '[' -d /var/lib/pgsql/9.5/data ']'
+ mv /var/lib/pgsql/9.5/data /var/lib/pgsql/9.5/data-initial
++ dirname /var/lib/pgsql/9.5/data
+ ln -sf /cfy-persist//var/lib/pgsql/9.5/data /var/lib/pgsql/9.5
+ for d in '$PDIRS'
+ '[' -d /opt/manager/resources ']'
+ mv /opt/manager/resources /opt/manager/resources-initial
++ dirname /opt/manager/resources
+ ln -sf /cfy-persist//opt/manager/resources /opt/manager
+ for d in '$PDIRS'
+ '[' -d /opt/mgmtworker/env/plugins ']'
+ mv /opt/mgmtworker/env/plugins /opt/mgmtworker/env/plugins-initial
++ dirname /opt/mgmtworker/env/plugins
+ ln -sf /cfy-persist//opt/mgmtworker/env/plugins /opt/mgmtworker/env
+ for d in '$PDIRS'
+ '[' -d /opt/mgmtworker/work/deployments ']'
+ mv /opt/mgmtworker/work/deployments
/opt/mgmtworker/work/deployments-initial
++ dirname /opt/mgmtworker/work/deployments
+ ln -sf /cfy-persist//opt/mgmtworker/work/deployments /opt/mgmtworker/work
+ exec /sbin/init --log-target=journal

*# DCAE Deployment handler is waiting for clodify-manager to come up*
$KLOG dev-dcaegen2-dcae-deployment-handler-7855bbb99f-drrrh -c
dcae-deployment-handler-readiness
2020-04-12 15:02:44,401 - INFO - Checking if dcae-cloudify-manager  is ready
2020-04-12 15:02:45,961 - INFO - dcae-cloudify-manager is not ready.
2020-04-12 15:02:50,977 - INFO - Checking if dcae-cloudify-manager  is ready
2020-04-12 15:02:52,581 - INFO - dcae-cloudify-manager is not ready.
2020-04-12 15:02:57,600 - INFO - Checking if dcae-cloudify-manager  is ready
2020-04-12 15:02:59,216 - INFO - dcae-cloudify-manager is not ready.
2020-04-12 15:03:04,235 - INFO - Checking if dcae-cloudify-manager  is ready
2020-04-12 15:03:05,904 - INFO - dcae-cloudify-manager is not ready.
2020-04-12 15:03:10,923 - INFO - Checking if dcae-cloudify-manager  is ready

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#20620): https://lists.onap.org/g/onap-discuss/message/20620
Mute This Topic: https://lists.onap.org/mt/72967794/21656
Group Owner: [email protected]
Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub  
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Reply via email to