Hi All,
When deploying the ONAP Honolulu release, SDNC pods are not coming up and CDS
Health check is also Failing.
aarna@anod-master:~/oom/kubernetes/robot$ ./ete-k8s.sh onap health
++ export NAMESPACE=onap
++ NAMESPACE=onap
+++ kubectl --namespace onap get pods
+++ sed 's/ .*//'
+++ grep robot
++ POD=dev-robot-585fbf6b94-hkrlf
++ TAGS='-i health'
+++ dirname ./ete-k8s.sh
++ DIR=.
++ SCRIPTDIR=scripts/etescript
++ ETEHOME=/var/opt/ONAP
++ [[ health == \e\x\e\c\s\c\r\i\p\t ]]
+++ kubectl --namespace onap exec dev-robot-585fbf6b94-hkrlf -- bash -c 'ls -1q
/share/logs/ | wc -l'
++ export GLOBAL_BUILD_NUMBER=1
++ GLOBAL_BUILD_NUMBER=1
+++ printf %04d 1
++ OUTPUT_FOLDER=0001_ete_health
++ DISPLAY_NUM=91
++ VARIABLEFILES='-V /share/config/robot_properties.py'
++ VARIABLES='-v GLOBAL_BUILD_NUMBER:14498'
++ case $2 in
++ kubectl --namespace onap exec dev-robot-585fbf6b94-hkrlf --
/var/opt/ONAP/runTags.sh -V /share/config/robot_properties.py -v
GLOBAL_BUILD_NUMBER:14498 -d /share/logs/0001_ete_health -i health --display 91
Starting Xvfb on display :91 with res 1280x1024x24
Executing robot tests at log level TRACE
==============================================================================
Testsuites
==============================================================================
/var/opt/ONAP/robot/library/paramiko/transport.py:33:
CryptographyDeprecationWarning: Python 2 is no longer supported by the Python
core team. Support for it is now deprecated in cryptography, and will be
removed in the next release.
from cryptography.hazmat.backends import default_backend
Testsuites.Health-Check :: Test that ONAP components are available via basi...
==============================================================================
Basic A&AI Health Check | PASS |
------------------------------------------------------------------------------
Enhanced A&AI Health Check | PASS |
------------------------------------------------------------------------------
Basic AAF Health Check | PASS |
------------------------------------------------------------------------------
Basic AAF SMS Health Check | PASS |
------------------------------------------------------------------------------
Basic APPC Health Check | PASS |
------------------------------------------------------------------------------
Basic CLI Health Check | PASS |
------------------------------------------------------------------------------
Basic CLAMP Health Check | PASS |
------------------------------------------------------------------------------
Basic DCAE Health Check | PASS |
------------------------------------------------------------------------------
Basic DMAAP Data Router Health Check | PASS |
------------------------------------------------------------------------------
Basic DMAAP Message Router Health Check | PASS |
------------------------------------------------------------------------------
Basic DMAAP Bus Controller Health Check With Basic Auth | PASS |
------------------------------------------------------------------------------
Basic External API NBI Health Check | PASS |
------------------------------------------------------------------------------
Basic Microservice Bus Health Check | PASS |
------------------------------------------------------------------------------
Basic Multicloud API Health Check | PASS |
------------------------------------------------------------------------------
Basic Multicloud-pike API Health Check | PASS |
------------------------------------------------------------------------------
Basic Multicloud-starlingx API Health Check | PASS |
------------------------------------------------------------------------------
Basic Multicloud-titanium_cloud API Health Check | PASS |
------------------------------------------------------------------------------
Basic Multicloud-vio API Health Check | PASS |
------------------------------------------------------------------------------
Basic Multicloud-k8s API Health Check | PASS |
------------------------------------------------------------------------------
Basic OOF-Homing Health Check | PASS |
------------------------------------------------------------------------------
Basic OOF-OSDF Health Check | PASS |
------------------------------------------------------------------------------
Basic OOF-CMSO Health Check | PASS |
------------------------------------------------------------------------------
Basic Policy Health Check | PASS |
------------------------------------------------------------------------------
Enhanced Policy New Healthcheck | FAIL |
202 != 200
------------------------------------------------------------------------------
Basic Portal Health Check | PASS |
------------------------------------------------------------------------------
Basic SDC Health Check (DMaaP:UP)
| PASS |
------------------------------------------------------------------------------
Enhanced SDC Health Check | PASS |
------------------------------------------------------------------------------
*Basic SDNC Health Check [ WARN ]
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None))
after connection broken by
'NewConnectionError('<urllib3.connection.HTTPSConnection object at
0x7f60ec2a1410>: Failed to establish a new connection: [Errno 101] Network is
unreachable',)': /restconf/operations/SLI-API:healthcheck*
*[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None,
status=None)) after connection broken by
'NewConnectionError('<urllib3.connection.HTTPSConnection object at
0x7f60ec93a890>: Failed to establish a new connection: [Errno 101] Network is
unreachable',)': /restconf/operations/SLI-API:healthcheck*
*[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None,
status=None)) after connection broken by
'NewConnectionError('<urllib3.connection.HTTPSConnection object at
0x7f60ec93aa50>: Failed to establish a new connection: [Errno 101] Network is
unreachable',)': /restconf/operations/SLI-API:healthcheck*
*| FAIL |*
*Test timeout 20 seconds exceeded.*
------------------------------------------------------------------------------
*Enhanced SDNC Health Check [ WARN ]
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None))
after connection broken by
'NewConnectionError('<urllib3.connection.HTTPSConnection object at
0x7f60ec220590>: Failed to establish a new connection: [Errno 101] Network is
unreachable',)':
/restconf/config/GENERIC-RESOURCE-API:services/service/GRSIdummy123*
*| FAIL |*
*Test timeout 20 seconds exceeded.*
------------------------------------------------------------------------------
Basic SO Health Check | PASS |
------------------------------------------------------------------------------
Basic UseCaseUI API Health Check | PASS |
------------------------------------------------------------------------------
Basic VFC gvnfmdriver API Health Check | PASS |
------------------------------------------------------------------------------
Basic VFC huaweivnfmdriver API Health Check | PASS |
------------------------------------------------------------------------------
Basic VFC nslcm API Health Check | PASS |
------------------------------------------------------------------------------
Basic VFC vnflcm API Health Check | PASS |
------------------------------------------------------------------------------
Basic VFC vnfmgr API Health Check | PASS |
------------------------------------------------------------------------------
Basic VFC vnfres API Health Check | PASS |
------------------------------------------------------------------------------
Basic VFC ztevnfmdriver API Health Check | PASS |
------------------------------------------------------------------------------
Basic VID Health Check | PASS |
------------------------------------------------------------------------------
Basic VNFSDK Health Check | PASS |
------------------------------------------------------------------------------
Basic Multicloud-fcaps API Health Check | PASS |
------------------------------------------------------------------------------
Basic Modeling genericparser API Health Check | PASS |
------------------------------------------------------------------------------
Enhanced CDS Health Check [ WARN ]
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None))
after connection broken by
'NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7f60ec40cb10>: Failed to establish a new connection: [Errno -2] Name or
service not known',)': /api/v1/execution-service/health-check
[ WARN ] Retrying (Retry(total=1, connect=None, read=None, redirect=None,
status=None)) after connection broken by
'NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7f60ec8e6190>: Failed to establish a new connection: [Errno -2] Name or
service not known',)': /api/v1/execution-service/health-check
[ WARN ] Retrying (Retry(total=0, connect=None, read=None, redirect=None,
status=None)) after connection broken by
'NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7f60ef13ed50>: Failed to establish a new connection: [Errno -2] Name or
service not known',)': /api/v1/execution-service/health-check
| FAIL |
ConnectionError: HTTPConnectionPool(host='cds-blueprints-processor-http.onap',
port=8080): Max retries exceeded with url:
/api/v1/execution-service/health-check (Caused by
NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7f60ef13ea50>: Failed to establish a new connection: [Errno -2] Name or
service not known',))
------------------------------------------------------------------------------
Testsuites.Health-Check :: Test that ONAP components are available... | FAIL |
43 critical tests, 39 passed, 4 failed
43 tests total, 39 passed, 4 failed
==============================================================================
Testsuites | FAIL |
43 critical tests, 39 passed, 4 failed
43 tests total, 39 passed, 4 failed
==============================================================================
Output: /share/logs/0001_ete_health/output.xml
Log: /share/logs/0001_ete_health/log.html
Report: /share/logs/0001_ete_health/report.html
+++++++++++++++++++++++++++++++++++
aarna@anod-master:~/oom/kubernetes/robot$ kubectl get pods -n onap |grep sdnc
dev-sdnc-0 0/2 Init:0/6 0
13h
dev-sdnc-ansible-server-5d94f5478b-9qwqg 0/1 Init:1/2 77
13h
dev-sdnc-dbinit-job-bcdkv 0/1 Completed 0
13h
dev-sdnc-dgbuilder-7c446d8f57-d8xt8 1/1 Running 0
13h
dev-sdnc-dmaap-listener-7469df45b-d27hh 0/1 Init:1/2 0
13h
dev-sdnc-sdnrdb-init-job-6qbnn 0/1 Completed 0
13h
dev-sdnc-ueb-listener-9454f95b9-cdp2x 0/1 Init:1/2 0
13h
dev-sdnc-web-6994c84f6d-47lhk 0/1 Init:2/3 77
13h
dev-so-sdnc-adapter-5d6bb9569-wjjx6 2/2 Running 0
12h
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
*When I describe pod and see for events. It is complaining about "*
*MountVolume.SetUp failed for volume "certservice-tls-volume" : secrets
"oom-cert-service-client-tls-secret" not found* *"*
kubectl describe pod -n onap dev-sdnc-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedMount 8m3s (x389 over 13h) kubelet MountVolume.SetUp failed
for volume "certservice-tls-volume" : secrets
"oom-cert-service-client-tls-secret" not found
Warning FailedMount 2m5s (x345 over 13h) kubelet Unable to mount volumes
for pod "dev-sdnc-0_onap(0109ffc4-d426-11eb-9253-525400dc0302)": timeout
expired waiting for volumes to attach or mount for pod "onap"/"dev-sdnc-0".
list of unmounted volumes=[certservice-tls-volume]. list of unattached
volumes=[dev-sdnc-data localtime logs data-filebeat filebeat-conf
sdnc-logging-cfg-config bin config-input properties dev-sdnc-aaf-config
aaf-agent-certs aaf-add-config certservice-tls-volume cmpv2-certs-volume-0
default-token-zk4qs]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#23339): https://lists.onap.org/g/onap-discuss/message/23339
Mute This Topic: https://lists.onap.org/mt/83863580/21656
Mute #sdnc:https://lists.onap.org/g/onap-discuss/mutehashtag/sdnc
Group Owner: [email protected]
Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
