Hi,
The issue with only dmaap-dr-node not coming up is not related to DMAAP-964<https://jira.onap.org/browse/DMAAP-964>. DMAAP-964 is an issue which occurs due to the mariadb not being instantiated correctly so dmaap-dr-prov and dmaap-dr-node do not come up. The issue we are now seeing with only dmaap-dr-node not coming up is due to AAF certs expiring. We are tracking that issue here DMAAP-1048<https://jira.onap.org/browse/DMAAP-1048>, we also reached out to the onap-seccom mailing list for any advice on extending or renewing the certs but haven't heard anything back yet. The link to that message is here - https://lists.onap.org/g/onap-seccom/topic/aaf_certificates_expired/29915899?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,29915899. I'm guessing that this issue has occurred before so any help or feedback would be appreciated. Regards, Conor ________________________________ From: [email protected] <[email protected]> on behalf of Michael O'Brien <[email protected]> Sent: Wednesday 20 February 2019 03:10:28 To: [email protected]; [email protected]; [email protected] Subject: Re: [onap-discuss] dev-dmaap-dmaap-dr-node pod does not come up due to https call to dev-dmaap-dmaap-dr-prov pod #dmaap #casablanca Guys seeing this again both on casablanca 20190218 and master 20180218 - both running only dmaap (nothing else deployed or deployed with aaf) LOG-898<https://jira.onap.org/browse/LOG-898> master onap onap-dmaap-dmaap-dr-node-57f77bd54d-q7v78 0/1 CrashLoopBackOff 9 26m 10.42.112.70 obriensystemsu0 <none> casablanca (3.0.1 essentially) onap onap-dmaap-dmaap-dr-node-57f77bd54d-kw8n7 0/1 Running 79 5h I am also seeing an issue with SDNC ansible on both the latest Casablanca and master – sdnc required dmaap and sdc to come up - note onap onap-sdnc-sdnc-ansible-server-7d595dd8-fvm95 0/1 CrashLoopBackOff 41 3h /michael From: [email protected] <[email protected]> On Behalf Of kranthi guttikonda Sent: Tuesday, February 19, 2019 2:07 PM To: [email protected]; [email protected] Subject: Re: [onap-discuss] dev-dmaap-dmaap-dr-node pod does not come up due to https call to dev-dmaap-dmaap-dr-prov pod #dmaap #casablanca Please take a look into https://jira.onap.org/browse/DMAAP-964 From: <[email protected]<mailto:[email protected]>> on behalf of D'Alessandro Alessandro Gerardo <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Tuesday, February 19, 2019 at 12:16 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: [onap-discuss] dev-dmaap-dmaap-dr-node pod does not come up due to https call to dev-dmaap-dmaap-dr-prov pod #dmaap #casablanca Hi all, I have deployed Casablanca and I have an issue with the highlighted pod dev-dmaap-dbc-pg-0 1/1 Running 0 30m 10.42.29.89 k8s-13 <none> dev-dmaap-dbc-pg-1 1/1 Running 0 27m 10.42.182.91 k8s-7 <none> dev-dmaap-dbc-pgpool-7b748d5894-88wb9 1/1 Running 0 30m 10.42.140.152 k8s-11 <none> dev-dmaap-dbc-pgpool-7b748d5894-pj8zv 1/1 Running 0 30m 10.42.183.217 k8s-6 <none> dev-dmaap-dmaap-bus-controller-6757c4c86-rv9zh 1/1 Running 0 30m 10.42.156.206 k8s-10 <none> dev-dmaap-dmaap-dr-db-bb4c67cfd-84c99 1/1 Running 0 30m 10.42.138.166 k8s-10 <none> dev-dmaap-dmaap-dr-node-5655ffbd55-gdfpx 0/1 Running 6 30m 10.42.30.95 k8s-6 <none> dev-dmaap-dmaap-dr-prov-66df46884f-sjd7x 1/1 Running 0 30m 10.42.13.150 k8s-7 <none> dev-dmaap-message-router-684b499dbc-8c6dm 1/1 Running 0 30m 10.42.191.38 k8s-11 <none> dev-dmaap-message-router-kafka-8466bf6864-t7gtq 1/1 Running 0 30m 10.42.32.83 k8s-4 <none> dev-dmaap-message-router-zookeeper-5bd997b466-dfmzl 1/1 Running 0 30m 10.42.25.154 k8s-2 <none> Log from dev-dmaap-dmaap-dr-node-5655ffbd55-gdfpx is: 16:03:22.198 ERROR [Node Configuration Timer] org.onap.dmaap.datarouter.node.NodeConfigManager - EELF0004E Configuration failed. java.net.UnknownHostException: dmaap-dr-prov - try again later. provurl:: https://dmaap-dr-prov:8443/internal/prov 16:03:22.200 ERROR [Node Configuration Timer] org.onap.dmaap.datarouter.node.NodeConfigManager - EELF0004E Configuration failed. java.net.UnknownHostException: dmaap-dr-prov - try again later. provurl:: https://dmaap-dr-prov:8443/internal/prov what I notice is that if the request comes from http it is satisfied while it is NOT if it comes from https please look at the details here below from the robot container: curl -v http://dmaap-dr-prov:8080/internal/prov { "feeds": [ ], "groups": [ ], "subscriptions": [ ], "parameters": { "ACTIVE_POD": "dmaap-dr-prov", "DELIVERY_INIT_RETRY_INTERVAL": 10, "DELIVERY_MAX_AGE": 86400, "DELIVERY_MAX_RETRY_INTERVAL": 3600, "DELIVERY_RETRY_RATIO": 2, "LOGROLL_INTERVAL": 300, "NODES": ["dmaap-dr-node"], "PROV_ACTIVE_NAME": "dmaap-dr-prov", "PROV_AUTH_ADDRESSES": ["dmaap-dr-prov","dmaap-dr-node"], "PROV_AUTH_SUBJECTS": [""], "PROV_DOMAIN": "onap", "PROV_MAXFEED_COUNT": 10000, "PROV_MAXSUB_COUNT": 100000, "PROV_NAME": "dmaap-dr-prov", "PROV_REQUIRE_CERT": "false", "PROV_REQUIRE_SECURE": "false", "STANDBY_POD": "", "_INT_VALUES": ["LOGROLL_INTERVAL","PROV_MAXFEED_COUNT","PROV_MAXSUB_COUNT","DELIVERY_INIT_RETRY_INTERVAL","DELIVERY_MAX_RETRY_INTERVAL","DELIVERY_RETRY_RATIO","DELIVERY_MAX_AGE"] }, "ingress": [ ], "egress": { }, "routing": [ ] curl - https://dmaap-dr-prov:8080/internal/prov THE SAME REQUEST OF DMAAP-DR-NODE * Trying 10.43.74.144... * TCP_NODELAY set * Connected to dmaap-dr-prov (10.43.74.144) port 8080 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/certs/ca-certificates.crt CApath: /etc/ssl/certs * TLSv1.2 (OUT), TLS handshake, Client hello (1): * error:1408F10B:SSL routines:ssl3_get_record:wrong version number * stopped the pause stream! * Closing connection 0 curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number I’m re-installing Casablanca after changing the file: root@rancher:~/oom/kubernetes/dmaap/charts/dmaap-bus-controller/resources/dmaap/onap.json<mailto:root@rancher:~/oom/kubernetes/dmaap/charts/dmaap-bus-controller/resources/dmaap/onap.json> as below: from "drProvUrl": "https://dmaap-dr-prov:8443<https://dmaap-dr-prov:8443/>" to "drProvUrl": "http://dmaap-dr-prov:8080<http://dmaap-dr-prov:8080/>" I’m wondering if someone else has already experienced the same issue and tested this change to share with me if there will be side effects. Thanks in advance Best regards, Alessandro This email and the information contained herein is proprietary and confidential and subject to the Amdocs Email Terms of Service, which you may review at https://www.amdocs.com/about/email-terms-of-service -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15653): https://lists.onap.org/g/onap-discuss/message/15653 Mute This Topic: https://lists.onap.org/mt/29919443/21656 Mute #dmaap: https://lists.onap.org/mk?hashtag=dmaap&subid=2740164 Mute #casablanca: https://lists.onap.org/mk?hashtag=casablanca&subid=2740164 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
