Hi,
To debug these issues, please login to the DCAE VM and run the following script to give me some debugging information about the current state of your deployment. Also the logs/error.log may contain specifics about the endpoint. It seems to be localhost:9998 which is the controller process. Carsten #!/bin/bash PW=$(grep OPENSTACK-PASSWORD /opt/app/dcae-controller/config.yaml | sed s/OPENSTACK-PASSWORD:.//) set -e echo ======= config.yaml cat /opt/app/dcae-controller/config.yaml | sed "s/$PW/XXXXXX/" echo ======= docker docker images docker ps -a ID=$(docker ps | grep dcae-controller: | cut -c1-12) echo ======= docker logs docker logs $ID 2>&1 echo ======= dcae-controller.sh.log docker exec $ID cat /opt/app/dcae-controller-platform-server/logs/dcae-controller.sh.log echo ======= reports docker exec -e GROOVY_HOME=/opt/app/groovy-2.4.6 $ID /opt/app/dcae-controller-platform-server/bin/dcae-controller.sh report -n /reports/dcae/vms docker exec -e GROOVY_HOME=/opt/app/groovy-2.4.6 $ID /opt/app/dcae-controller-platform-server/bin/dcae-controller.sh report -n /reports/dcae/service-instances echo ======= logs err docker exec $ID cat /opt/app/dcae-controller-platform-server/logs/controller-platform-server-controller.err | head -10000 | sed "s/$PW/XXXXXX/" echo ======= logs out docker exec $ID cat /opt/app/dcae-controller-platform-server/logs/controller-platform-server-controller.out | head -10000 From: Gaurav Gupta (c) [mailto:[email protected]] Sent: Wednesday, August 23, 2017 1:32 PM To: Ran Pollak <[email protected]>; LUND, CARSTEN (CARSTEN) <[email protected]> Cc: [email protected]; [email protected]; Ran Pollak <[email protected]>; Josef Reisinger <[email protected]> Subject: Re: [onap-discuss] Stabilized the ONAP master branch Ran , Carsten on Further analysis of DCAE Deployment failure . We saw following error for all our DCAE VM after running the deploy-service-instance bin/dcae-controller.sh deploy-service-instance -i $ZONE -s vm-docker-host-1 . but the issue is - these error do not have the information /Log indicating to Which IP Address and Port a socket connection is being setup . Can this be corrected to at least have IP Address and Port Number printed in the Logs .. java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:153) at java.net.SocketInputStream.read(SocketInputStream.java:122) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read1(BufferedInputStream.java:275) at java.io.BufferedInputStream.read(BufferedInputStream.java:334) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:707) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:650) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1371) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468) at org.openecomp.ncomp.sirius.manager.JavaHttpClient.httpBinaryTransaction(JavaHttpClient.java:168) at org.openecomp.ncomp.sirius.manager.AbstractClient.httpStringTransaction(AbstractClient.java:219) at org.openecomp.ncomp.sirius.manager.AbstractClient.httpJsonTransaction(AbstractClient.java:205) at org.openecomp.ncomp.sirius.manager.AbstractClient.operationPath2(AbstractClient.java:65) at org.openecomp.ncomp.sirius.manager.AbstractClient.operation(AbstractClient.java:97) at org.openecomp.ncomp.sirius.manager.AbstractClient$operation.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:110) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:146) at org.openecomp.ncomp.sirius.manager.console.Console.operation(Console.groovy:107) at org.openecomp.ncomp.sirius.manager.console.Console.operation(Console.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrap.invoke(PogoMetaMethodSite.java:187) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:68) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:110) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:138) at deploy-service-instance.run(deploy-service-instance.groovy:50) at groovy.lang.GroovyShell.runScriptOrMainOrTestOrRunnable(GroovyShell.java:261) at groovy.lang.GroovyShell.run(GroovyShell.java:522) at groovy.lang.GroovyShell.run(GroovyShell.java:511) at groovy.ui.GroovyMain.processOnce(GroovyMain.java:650) at groovy.ui.GroovyMain.run(GroovyMain.java:381) at groovy.ui.GroovyMain.process(GroovyMain.java:367) at groovy.ui.GroovyMain.processArgs(GroovyMain.java:126) at groovy.ui.GroovyMain.main(GroovyMain.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.groovy.tools.GroovyStarter.rootLoader(GroovyStarter.java:109) at org.codehaus.groovy.tools.GroovyStarter.main(GroovyStarter.java:131) with best regards gaurav ________________________________ From: Gaurav Gupta (c) Sent: 22 August 2017 19:06 To: Josef Reisinger Cc: [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]>; Ran Pollak Subject: Re: [onap-discuss] Stabilized the ONAP master branch Thanks Josef if you can send me the output of following script form your environment .It will be great . the other Info if you could share , PG and CDAP VM's have how many dockers running on each VM . #!/bin/bash PW=$(grep OPENSTACK-PASSWORD /opt/app/dcae-controller/config.yaml | sed s/OPENSTACK-PASSWORD:.//) set -e echo ======= config.yaml cat /opt/app/dcae-controller/config.yaml | sed "s/$PW/XXXXXX/" echo ======= docker docker images docker ps -a ID=$(docker ps | grep dcae-controller: | cut -c1-12) echo ======= docker logs docker logs $ID 2>&1 echo ======= dcae-controller.sh.log docker exec $ID cat /opt/app/dcae-controller-platform-server/logs/dcae-controller.sh.log echo ======= reports docker exec -e GROOVY_HOME=/opt/app/groovy-2.4.6 $ID /opt/app/dcae-controller-platform-server/bin/dcae-controller.sh report -n /reports/dcae/vms docker exec -e GROOVY_HOME=/opt/app/groovy-2.4.6 $ID /opt/app/dcae-controller-platform-server/bin/dcae-controller.sh report -n /reports/dcae/service-instances echo ======= logs err docker exec $ID cat /opt/app/dcae-controller-platform-server/logs/controller-platform-server-controller.err | head -10000 | sed "s/$PW/XXXXXX/" echo ======= logs out docker exec $ID cat /opt/app/dcae-controller-platform-server/logs/controller-platform-server-controller.out | head -10000 with best regards gaurav ________________________________ From: Josef Reisinger <[email protected]<mailto:[email protected]>> Sent: 22 August 2017 18:48 To: Gaurav Gupta (c) Cc: [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]>; Ran Pollak Subject: Re: [onap-discuss] Stabilized the ONAP master branch Hi Gaurav, from what I see in the logs attached to the linked email, I am getting a bit confused. This URL=http://10.110.208.162:5000/v3/tokens<https://urldefense.proofpoint.com/v2/url?u=http-3A__10.110.208.162-3A5000_v3_tokens&d=DwQF-g&c=LFYZ-o9_HUMeMTSQicvjIg&r=G7P6ajmMIva8VFVj0kle3avtPgyVjBbnnM53TmeR0u4&m=XlTpodrbPzzMYOtqMlXy0XQjTqhdq8ho5-Btx25r9Fo&s=7q87pzI726KMZflRkcHg66abElPbVMFnMymrdDPDs-I&e=> looks like you are trying to talk to the V3 keystone API - which is not (yet) supported while this REQUEST BODY: {"auth":{"RAX-KSKEY:apiKeyCredentials":{"username":"admin","apiKey":"yyyyyyy"}}} seem you are using the protocol to a Rackspace authentication server. and this "exception": "java.lang.IllegalArgumentException: !Absolute URI: null/servers", is a typical sign that the dcae_controller cannot talk to the Openstack keystone. I had a conversation about this topic earlier today, see https://lists.onap.org/pipermail/onap-discuss/2017-August/003663.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.onap.org_pipermail_onap-2Ddiscuss_2017-2DAugust_003663.html&d=DwMFAw&c=uilaK90D4TOVoH58JNXRgQ&r=ebJjFMpXijqZjbZCcbF7yJIq2ES6jM0Q-DEcP-qjjeI&m=eKT1FspovuiyRCAwmir9g-c0DCX6Ym0Bww5JClvpKQg&s=5vrN0A4sTw2UJqrUtVknUchQFsf52lXmmBG0_5KlMgk&e=> I can confirm that DCAE runs - i installed it on a 2nd environment yesterday, using docker 1.0 for all except DCAE and DCAE code 1.1 artifacts_version: 1.1.0-SNAPSHOT docker_version: 1.0-STAGING-latest gerrit_branch: master and I changed a line in the heat template for the dcae controller vm to use docker version 1.1 echo "__docker_version__" > /opt/config/docker_version.txt echo "1.1-STAGING-latest" > /opt/config/docker_version.txt Finally; Be patient. Be even more patient, I also saw three of the five docker container on collector and retried a couple of times. It seems as if the remaining two docker container are created _after_ pg & cdap are up. Mit freundlichen Grüßen / Kind regards Josef Reisinger From: "Gaurav Gupta (c)" <[email protected]<mailto:[email protected]>> To: Ran Pollak <[email protected]<mailto:[email protected]>> Cc: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: 22.08.2017 14:37 Subject: Re: [onap-discuss] Stabilized the ONAP master branch Sent by: [email protected]<mailto:[email protected]> ________________________________ Hello Ran We are trying to Deploy DACE on VIO . We have been successful in rest of the VM's and their dockers But have issues in Collector VM , PostGreSql and CDAP VM's . The situation is - on Collector VM - 3 dockers are up and on PostgreSql and CDAP VM - there is no dockers . our DCAE VM's have Floating and Private ip assigned to . from inside the VM We can reach to open internet . We have tried on both the Release Version and situation is same . Test.No Project Name Artifacts version docker version ONAPVersion/gerrit DCAE Version Observation 1 OPEN-ECOMP 1.1.0-SNAPSHOT 1.1-STAGING-latest master 1.1.0 DCAE's 5 Additional VMs are created. Collector VM downloads 2 Dockers but NO Containers start 2 onap 1.1.0-SNAPSHOT 1.0-STAGING-latest 1.0.0 1.1.0 DCAE's 5 Additional VMs are created. Collector VM downloads 6 Dockers and start 3 containers but the remaining dockers are not started Whatever we tried have been captured in the given link - . https://lists.onap.org/pipermail/onap-discuss/2017-July/002414.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.onap.org_pipermail_onap-2Ddiscuss_2017-2DJuly_002414.html&d=DwMFAw&c=uilaK90D4TOVoH58JNXRgQ&r=ebJjFMpXijqZjbZCcbF7yJIq2ES6jM0Q-DEcP-qjjeI&m=eKT1FspovuiyRCAwmir9g-c0DCX6Ym0Bww5JClvpKQg&s=tudkaLUNwG34hzYVGYwmf0cZ5Dh4qdq-9UeGH8gG4dw&e=>. with best regards gaurav ________________________________ From: Ran Pollak <[email protected]<mailto:[email protected]>> Sent: 22 August 2017 15:18 To: Gaurav Gupta (c) Cc: [email protected]<mailto:[email protected]> Subject: FW: [onap-discuss] Stabilized the ONAP master branch Thanks Gaurav! We attend to have a different format for the daily mail that will include all the Components with the relevant issues, so we will have DCAE as well. Also if you can send me the relevant info regarding DCAE issues that will be very helpful ! BR, Ran Pollak From: [email protected]<mailto:[email protected]>[mailto:[email protected]] On Behalf Of Gaurav Gupta (c) Sent: Tuesday, August 22, 2017 06:35 To: Ran Pollak <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> Subject: Re: [onap-discuss] Stabilized the ONAP master branch Hi Ran Pollak amazing intiative . May I request you to include DACE Init issues also in your list of items to be resolved . Do let me know if you need any specifics of the issues . with best regards Gaurav ________________________________ From: [email protected]<mailto:[email protected]><[email protected]<mailto:[email protected]>> on behalf of Ran Pollak <[email protected]<mailto:[email protected]>> Sent: 21 August 2017 21:34 To: [email protected]<mailto:[email protected]> Subject: [onap-discuss] Stabilized the ONAP master branch Hello ONAP Community Members, This is the first Daily status for the ONAP community regarding the status of the Master branch, this format will change into all the ONAP components with theirissues/Status and we will add also info regarding "health-check" scenarios. Part of our efforts to stabilize the 1.1 ONAP and certify Open loop, we are troubleshooting the issues and below is the issue we are now stuck at. We need more help on this since this is reported as high and still in Open state. Bug Module Issue State Defect Aging Resolution - AAI Unable to deploy model loader docker in AAI Closed NA Update the port numbers and restart the dockers - SO Unable to connect to AAI due to SSL Closed NA 1. Comment the certificate part in java.security file resides in /etc/java-8-openjdk/security 2. Update mso.asdc.json located in /etc/mso/config.d path. AAI-190<https://urldefense.proofpoint.com/v2/url?u=https-3A__jira.onap.org_browse_AAI-2D190&d=DwMFAg&c=uilaK90D4TOVoH58JNXRgQ&r=ebJjFMpXijqZjbZCcbF7yJIq2ES6jM0Q-DEcP-qjjeI&m=1OWfF7oUOY6tjenngZvE99gKYQYaGXwvFd0fd90XvxA&s=xj7XvnhCWnAzA1S8XZDhhZ5yA4XBFypyiptNHJNwkMk&e=> AAI AAI was not able to communicate with ASDC Closed NA Stop and remove the old dockers, place new dockers and update the java.security file like above. VID-38<https://urldefense.proofpoint.com/v2/url?u=https-3A__jira.onap.org_browse_VID-2D38&d=DwMFAg&c=uilaK90D4TOVoH58JNXRgQ&r=ebJjFMpXijqZjbZCcbF7yJIq2ES6jM0Q-DEcP-qjjeI&m=1OWfF7oUOY6tjenngZvE99gKYQYaGXwvFd0fd90XvxA&s=zkpfZpxYEtyVOV3qcSUdyZjVLTaBiIMIXGU135karQc&e=> VID Deploy button is not working in VID to distribute Model Open 10 days We have this issue reported and been followed up for more than ten days yet no update. Posted on community as well. Please, if you have something to add or ask please contactme. BR, Ran Pollak This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at https://www.amdocs.com/about/email-disclaimer<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.amdocs.com_about_email-2Ddisclaimer&d=DwMFAg&c=uilaK90D4TOVoH58JNXRgQ&r=ebJjFMpXijqZjbZCcbF7yJIq2ES6jM0Q-DEcP-qjjeI&m=1OWfF7oUOY6tjenngZvE99gKYQYaGXwvFd0fd90XvxA&s=9Ry6rgG1kK3uvv1a91PhMJNA7tZB3XqFLWFBzf7E7c8&e=> This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at https://www.amdocs.com/about/email-disclaimer<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.amdocs.com_about_email-2Ddisclaimer&d=DwMFAg&c=uilaK90D4TOVoH58JNXRgQ&r=ebJjFMpXijqZjbZCcbF7yJIq2ES6jM0Q-DEcP-qjjeI&m=M6_SFYQBgLnt0-YJbUhX_Ld-soEJRJvEI74s7PJr1DI&s=ufdJeC48slQpQNOyMLf30O-MqKmlxOQuwWqMXOPBjxI&e=>_______________________________________________ onap-discuss mailing list [email protected]<mailto:[email protected]> https://lists.onap.org/mailman/listinfo/onap-discuss<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.onap.org_mailman_listinfo_onap-2Ddiscuss&d=DwMFAw&c=uilaK90D4TOVoH58JNXRgQ&r=ebJjFMpXijqZjbZCcbF7yJIq2ES6jM0Q-DEcP-qjjeI&m=eKT1FspovuiyRCAwmir9g-c0DCX6Ym0Bww5JClvpKQg&s=w60EpoH42rBaAAnu9xjesIDLNMxwnx1F92gldNGg6hU&e=>
_______________________________________________ onap-discuss mailing list [email protected] https://lists.onap.org/mailman/listinfo/onap-discuss
