Not sure why you needed to roll the kafka version it was the zookeeper version that was the problem in https://jira.onap.org/browse/DMAAP-100 repository: docker.io image: cdposs/zookeeper:3.4.9
Try a distribution (./demo-k8s.sh onap healthdist) and see if things are running correctly on the SDC – DmaaP -> SO, AAI, SDNC(ueb listener) – it should be based on the results you are showing below. The timeout to DCAE is strange – you can adjust that value in the testsuite Kubectl -n onap exec -it [robot pod] bash Cd /var/opt/Op*/robot/testsuite Vi health_check.robot (may need to do apt-get update; apt-get install vim) Change the timeout at the top of that script: Test Timeout 10 second Brian From: Signorelli Marco <[email protected]> Sent: Wednesday, February 06, 2019 11:18 AM To: FREEMAN, BRIAN D <[email protected]>; [email protected] Subject: Re: [onap-discuss] [Casablanca] Kafka's pod in crashloobackoff Hi all, first of all thanks for the support. For some reason, the dns now works fine and resolves the dns queries in few ms. I have followed your suggestions for reinstalling a single component correctly. I have done the following steps: 1. Changed the kafka image version into ~/oom/kubernetes/dmaap/charts/message-router/charts/message-router-kafka/values.yaml. Is it correct? #repository: nexus3.onap.org:10001 #image: onap/dmaap/kafka01101:0.0.1 repository: docker.io image: wurstmeister/kafka:2.11-2.0.0 using an old version of kafka docker image. 2. Then I have reinstalled the dmaap and all come up correctly. Kafka's log shows that it is connected to zookeeper correctly and in fact by running the command "telnet message-router-zookeeper 2181" and then "stats"(from a pod i.e. dev-robot), it shows that kafka is connected to zookeeper: root@dev-robot-robot-646785574-9j6fd:/# telnet message-router-zookeeper 2181 Trying 10.42.89.16... Connected to message-router-zookeeper.onap.svc.cluster.local. Escape character is '^]'. stats Zookeeper version: 3.4.9-1757313, built on 08/23/2016 06:50 GMT Clients: /10.42.59.249:60376[1](queued=0,recved=7139,sent=7139) /10.42.59.249:60522[1](queued=0,recved=17281,sent=17281) /10.42.194.78:51264[0](queued=0,recved=1,sent=0) /10.42.59.249:60718[1](queued=0,recved=1836,sent=1850) /10.42.67.117:56168[1](queued=0,recved=5964,sent=5970) <------> this is the kafka client 3.After dmaap I deployed the AAI,SDC,SDNC,APPC. All the others components were already installed (before dmaap installation). Now I have only following pod in a non-running state: dev-aai-aai-data-router-5d55646cdc-6g9pg 1/2 CrashLoopBackOff 18 1h 10.42.189.96 sb4-k8s-10 <none> 4. I performed the following tests to see the healthcheck of all components : root@dev-robot-robot-646785574-9j6fd:/# curl http://10.43.180.208:3904/topics<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.43.180.208-3A3904_topics&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=ympMrJjsT_8m7nmLOi8lnW4IrkKGe3mly_IeUFftdn8&s=Hjx0W3p6cZJAVn56AQdEdCGahca4bG2NFZlDs4E_2xA&e=> {"topics": [ "ECOMP-PORTAL-OUTBOX-APP1",[email protected]> "__consumer_offsets", "ECOMP-PORTAL-OUTBOX-POL1", "SDC-DISTR-STATUS-TOPIC-AUTO", "SDC-DISTR-NOTIF-TOPIC-AUTO", "AAI-EVENT", "ECOMP-PORTAL-OUTBOX-VID1", "ECOMP-PORTAL-INBOX", "champRawEvents", "ECOMP-PORTAL-OUTBOX-SDC1", "APPC-TEST1", "ECOMP-PORTAL-OUTBOX-DBC1" ]} so all components of message-router work. Is it right? root@dev-robot-robot-646785574-9j6fd:/# curl http://10.42.165.132:8080/sdc2/rest/healthCheck<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.42.165.132-3A8080_sdc2_rest_healthCheck&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=ympMrJjsT_8m7nmLOi8lnW4IrkKGe3mly_IeUFftdn8&s=7L8RKc1NsdxwjAa9fz9wJZcHu4uNFm7txManXHbdBYA&e=> { "sdcVersion": "1.3.6", "siteMode": "unknown", "componentsInfo": [ { "healthCheckComponent": "BE", "healthCheckStatus": "UP", "version": "1.3.6", "description": "OK" }, { "healthCheckComponent": "TITAN", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "ES", "healthCheckStatus": "UP", "version": "1.3.6", "description": "OK" }, { "healthCheckComponent": "DE", "healthCheckStatus": "UP", "description": "OK" },[email protected]> { "healthCheckComponent": "CASSANDRA", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "ON_BOARDING", "healthCheckStatus": "UP", "version": "1.3.6", "description": "OK", "componentsInfo": [ { "healthCheckComponent": "ZU", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "BE", "healthCheckStatus": "UP", "version": "1.3.6", "description": "OK"[email protected]> }, { "healthCheckComponent": "CAS", "healthCheckStatus": "UP", "version": "2.1.17", "description": "OK" } ] }, { "healthCheckComponent": "DCAE", "healthCheckStatus": "UP", "version": "1.3.0", "description": "OK", "componentsInfo": [ { "healthCheckComponent": "BE", "healthCheckStatus": "UP", "version": "1.3.0", "description": "OK" }, { "healthCheckComponent": "TOSCA_LAB", "healthCheckStatus": "UP", "version": "1.3.0", "description": "OK" } ] } ] } and finally, I have run the robot-tests and 47/51 tests passed, except for the following: ------------------------------------------------------------------------------ Basic DCAE Health Check | FAIL | Test timeout 10 seconds exceeded. ------------------------------------------------------------------------------ Basic OOF-Homing Health Check | FAIL | Test timeout 10 seconds exceeded. --------------------------------------------------------------------------- Basic Holmes Rule Management API Health Check | FAIL | 502 != 200 ------------------------------------------------------------------------------ Basic Holmes Engine Management API Health Check | FAIL | 502 != 200 ------------------------------------------------------------------------------ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15368): https://lists.onap.org/g/onap-discuss/message/15368 Mute This Topic: https://lists.onap.org/mt/29665079/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
