Folks, I run into more or less the same error, with the difference that "my" container dies after max 5 seconds. I need a second ONAP environment and after a number of tries with 1.1 and 1.0 last week, I came to the conclusion that trying a copy of my still existing ONAP environment would be the fastest route to target. I could close some of the VMs, but the aai (one-VM-version) refuses to start up with the above mentioned issue. I'll have a deeper look into the issues tomorrow.
Mit freundlichen Grüßen / Kind regards Josef Reisinger From: "Kapil Gupta (c)" <[email protected]> To: "[email protected]" <[email protected]>, "[email protected]" <[email protected]> Cc: "[email protected]" <[email protected]> Date: 18.09.2017 19:01 Subject: RE: [onap-discuss] AAI Error "model-loader docker exited with in 2 munities with error ModelLoader ERROR|failed to connect to url: /asdc/v1/distributionUebCluster" Hi Steven, Thanks for your suggestions. As I mentioned in trail email SDC all dockers up and running and health check is also good. However, all ONAP VMs(including SDC and AAI VM) currently running with version 1.0.0 in our lab. As per you It seems Model -Loader docker is only stable with version 1.1.0 Please confirm , should we move all ONAP VM from version 1.0.0 to version 1.1.0 or there is a way to fix with version 1.0.0. @Josef, please share your view as you setup ONAP lab with version 1.0.0 Thanks Kapil On 18 Sep 2017, at 9:48 PM, Steven Blimkie <[email protected]> wrote: Hello, In the 1.0.0 version (which you appear to be using), the model loader will shut itself down if it is unable to successfully connect with the SDC on startup. In the 1.1.0 version, the model loader container will remain up, and continue to retry. Based, on your error log, it appears that the ML encounters a connection timeout trying to reach the SDC (and subsequently shuts itself down). You?ll need to validate that the SDC is up and running, and that it is reachable from the ML container. Regards, Steve From: [email protected] [ mailto:[email protected]] On Behalf Of Kapil Gupta (c) Sent: Monday, September 18, 2017 3:40 AM To: [email protected] Subject: [onap-discuss] AAI Error "model-loader docker exited with in 2 munities with error ModelLoader ERROR|failed to connect to url: /asdc/v1/distributionUebCluster" Hi ALL, Problem Statement:- AAI model-loader docker exited within 2 munities with error ?ModelLoader ERROR|failed to connect to url: /asdc/v1/distributionUebCluster org.apache.http.conn.HttpHostConnectException: Connect to 10.1.1.21:8443? We also find this error on https://jira.onap.org/browse/AAI-170 but solution was not very clear to me, Please let me know where and how to fix it. We are using below configuration docker version 1.0-STAGING-latest artifacts_version 1.1.0-SNAPSHOT FYI Healthcheck of SDC dockers from AAI (curl http://10.1.1.21:8181/sdc1/rest/healthCheck) is good. Log attached in the end of email. AAI logs:- root@6c742bb86d8f :/opt/jetty/jetty-distribution-9.3.9.v20160517/logs/AAI-ML# cat error.log 2017-09-18T12:02:58.961Z||main|ModelLoader||org.openecomp.modelloader.service.ModelLoaderService||INFO|MDLSVC0001I|MDLSVC0001I Loading configuration | 2017-09-18T12:02:58.968Z||main|ModelLoader||org.openecomp.sdc.impl.DistributionClientImpl||INFO|DistributionClient - init 2017-09-18T12:05:06.408Z||main|ModelLoader|| org.openecomp.sdc.impl.DistributionClientImpl||ERROR|failed to connect to url: /asdc/v1/distributionUebCluster org.apache.http.conn.HttpHostConnectException: Connect to 10.1.1.21:8443 [/10.1.1.21] failed: Connection timed out (Connection timed out) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) ~[httpclient-4.5.jar:4.5] at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) ~[httpclient-4.5.jar:4.5] at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) ~[httpclient-4.5.jar:4.5] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.jar:4.5] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[httpclient-4.5.jar:4.5] at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[httpclient-4.5.jar:4.5] at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.jar:4.5] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.5.jar:4.5] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.5.jar:4.5] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) ~[httpclient-4.5.jar:4.5] at org.openecomp.sdc.http.HttpAsdcClient.getRequest(HttpAsdcClient.java:278) ~[sdc-distribution-client-1.0.0.jar:na] at org.openecomp.sdc.http.AsdcConnectorClient.performAsdcServerRequest(AsdcConnectorClient.java:143) [sdc-distribution-client-1.0.0.jar:na] at org.openecomp.sdc.http.AsdcConnectorClient.getServerList(AsdcConnectorClient.java:94) [sdc-distribution-client-1.0.0.jar:na] at org.openecomp.sdc.impl.DistributionClientImpl.initUebServerList(DistributionClientImpl.java:324) [sdc-distribution-client-1.0.0.jar:na] at org.openecomp.sdc.impl.DistributionClientImpl.init(DistributionClientImpl.java:240) [sdc-distribution-client-1.0.0.jar:na] at org.openecomp.modelloader.service.ModelLoaderService.init(ModelLoaderService.java:95) [classes/:na] at org.openecomp.modelloader.service.ModelLoaderService.start(ModelLoaderService.java:72) [classes/:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_141] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_141] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_141] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_141] CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6c742bb86d8f nexus3.onap.org:10001/openecomp/model-loader:1.0-STAGING-latest "/bin/sh -c /opt/j..." 3 days ago Exited (1) 5 minutes ago model-loader-service 4f879f9086fb nexus3.onap.org:10001/openecomp/ajsc-aai:1.0-STAGING-latest "/bin/sh -c ./star..." 3 days ago Up 2 days aai-service c033269baf00 nexus3.onap.org:10001/aaidocker/aai-hbase-1.2.3 "/bin/bash /opt/en..." 3 days ago Up 2 days hbase-1.2.3 root@onap-vm-aai:~# root@onap-vm-aai:~# docker images REPOSITORY TAG IMAGE ID CREATED SIZE nexus3.onap.org:10001/openecomp/ajsc-aai 1.0-STAGING-latest 3d71430ebe01 3 days ago 1.35GB nexus3.onap.org:10001/openecomp/model-loader 1.0-STAGING-latest ed1067ebafbd 3 days ago 635MB nexus3.onap.org:10001/aaidocker/aai-hbase-1.2.3 latest aba535a6f8b5 9 months ago 1.56GB root@onap-vm-aai:/opt# curl http://10.1.1.21:8181/sdc1/rest/healthCheck { "sdcVersion": "1.0.0", "siteMode": "unknown", "componentsInfo": [ { "healthCheckComponent": "BE", "healthCheckStatus": "UP", "version": "1.0.0", "description": "OK" }, { "healthCheckComponent": "ES", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "TITAN", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "DE", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "FE", "healthCheckStatus": "UP", "version": "1.0.0", "description": "OK" } ] Thanks Kapil From: Kapil Gupta (c) Sent: 14 September 2017 19:02 To: 'Michael O'Brien' <[email protected]>; Josef Reisinger < [email protected]> Cc: [email protected] Subject: vFW demo setup issue with AAI Hi Michael and Josef, Thanks for your information. We are still struggling with some issues with Model-Loader with ASDC communication as well as error in aai-service container .please suggest ideas to move further . @ Josef :- still we are not able to login in aai-service container with root user as we are getting Permission denied . root@onap-vm-aai:/opt# docker exec -it aai-service bash -u root bash: root: Permission denied @Michael:- We redeploy all AAI-dockers after deleting all AAI docker and images and comment below lines in ? model-loader-service? docker at /etc/java-8-openjdk/security/java.security * jdk.certpath.disabledAlgorithms=MD2, MD5, SHA1 jdkCA & usage TLSServer, \ * jdk.jar.disabledAlgorithms=MD2, MD5, RSA keySize < 1024 * jdk.tls.disabledAlgorithms=SSLv3, RC4, MD5withRSA, DH keySize < 768, \ But model-loader-service docker does not remain stable and exits within 2 minute . CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 532e18c6d5b1 nexus3.onap.org:10001/openecomp/model-loader:1.0-STAGING-latest "/bin/sh -c /opt/j..." About an hour ago Exited (1) 14 seconds ago model-loader-service 0fb66d3ae3ea nexus3.onap.org:10001/openecomp/ajsc-aai:1.0-STAGING-latest "/bin/sh -c ./star..." About an hour ago Up About an hour aai-service 93e45d4892c0 nexus3.onap.org:10001/aaidocker/aai-hbase-1.2.3 "/bin/bash /opt/en..." About an hour ago Up About an hour hbase-1.2.3 Error Logs from model-loader-service docker is:- 2017-09-14T17:39:35.916Z||main|ModelLoader||org.openecomp.sdc.http.AsdcConnectorClient ||ERROR|status from ASDC is org .openecomp.sdc.http.HttpAsdcResponse@3f91b517 2017-09-14T17:39:35.917Z||main|ModelLoader||org.openecomp.sdc.http.AsdcConnectorClient ||ERROR|DistributionClientResultImpl [responseStatus=ASDC_CONNECTION_FAILED, responseMessage=ASDC server problem] 2017-09-14T17:39:35.927Z||main|ModelLoader||org.openecomp.modelloader.service.ModelLoaderService||ERROR|MDLSVC2001E|MDLSVC2001E Unable to register with ASDC: Failed to initialize distribution client: ASDC server problem| 2017-09-14T17:39:35.927Z||main|ModelLoader||org.openecomp.modelloader.service.ModelLoaderService||INFO|MDLSVC0002I|MDLSVC0002I Stopping distribution client| 2017-09-14T17:39:35.927Z||main|ModelLoader||org.openecomp.sdc.impl.DistributionClientImpl||INFO|stop DistributionClient 2017-09-14T17:44:59.073Z||main|ModelLoader||org.openecomp.modelloader.service.ModelLoaderService||INFO|MDLSVC0001I|MDLSVC0001I Loading configuration | 2017-09-14T17:44:59.079Z||main|ModelLoader||org.openecomp.sdc.impl.DistributionClientImpl||INFO|DistributionClient - init HealthCheck of SDC seems good as we checked SDC health from AAI as well as from SDC VMs. SDC Health check from AAI:- root@onap-vm-aai:/opt# curl http://10.1.1.21:8181/sdc1/rest/healthCheck | grep -c "Ok" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 673 100 673 0 0 334k 0 --:--:-- --:--:-- --:--:-- 657k 0 root@onap-vm-aai:/opt# curl http://10.1.1.21:8181/sdc1/rest/healthCheck { "sdcVersion": "1.0.0", "siteMode": "unknown", "componentsInfo": [ { "healthCheckComponent": "BE", "healthCheckStatus": "UP", "version": "1.0.0", "description": "OK" }, { "healthCheckComponent": "ES", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "TITAN", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "DE", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "FE", "healthCheckStatus": "UP", "version": "1.0.0", "description": "OK" } ] SDC Health check from SD CVM with /data/scripts/ docker_health.sh root@onap-vm-sdc:/data/scripts# ./docker_health.sh { "cluster_name" : "SDC-ES-AUTO", "status" : "green", "timed_out" : false, "number_of_nodes" : 1, "number_of_data_nodes" : 1, "active_primary_shards" : 4, "active_shards" : 4, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 } BE health-Check: { "sdcVersion": "1.0.0", "siteMode": "unknown", "componentsInfo": [ { "healthCheckComponent": "BE", "healthCheckStatus": "UP", "version": "1.0.0", "description": "OK" }, { "healthCheckComponent": "ES", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "TITAN", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "DE", "healthCheckStatus": "UP", "description": "OK" } ] } FE health-Check: { "sdcVersion": "1.0.0", "siteMode": "unknown", "componentsInfo": [ { "healthCheckComponent": "BE", "healthCheckStatus": "UP", "version": "1.0.0", "description": "OK" }, { "healthCheckComponent": "ES", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "TITAN", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "DE", "healthCheckStatus": "UP", "description": "OK" }, { "healthCheckComponent": "FE", "healthCheckStatus": "UP", "version": "1.0.0", "description": "OK" } ] } check user existance: OK root@onap-vm-sdc:/data/scripts# Thanks Kapil From: Michael O'Brien [mailto:[email protected]] Sent: 14 September 2017 09:49 To: Josef Reisinger <[email protected]>; Kapil Gupta (c) < [email protected]> Cc: [email protected] Subject: RE: [onap-discuss] vFW demo setup issue Retesting a fix for the SDC onboarding issue (security related) - was pushed between the 5th-8th (our rackspace reference deployment was built late on the 7th so we may have missed the merge) https://jira.onap.org/browse/SDC-264 https://lists.onap.org/pipermail/onap-discuss/2017-September/004587.html https://lists.onap.org/pipermail/onap-discuss/2017-September/004351.html If this works then we should be able to move past service creation into (2 of 3) vf creation for the vFW. /michael From: Michael O'Brien Sent: Wednesday, September 13, 2017 18:31 To: 'Josef Reisinger' <[email protected]>; Kapil Gupta (c) < [email protected]> Cc: [email protected] Subject: RE: [onap-discuss] vFW demo setup issue Kapil, Hi, I just did a service distribution in a 20170907 rackspace build (1.1) as we need a distributed failure scenario for testing logs ? Stela was with me and noted that even though the service distributes ?ok? ? if you check the monitor screen you will see several AAI rest calls that fail (I will raise an SDC bug and put it on the status page later today). If distribution fails then you will have issues running demo.sh init ? because parent nodes will not be on AAI properly ? you will get a 404. Also talking with Susan ? who broke out a lot of the rest calls in the robot scripts ? init creates the cloud-region first before creating and associating the ?Demonstration? customer ? running these in sequence got us further ? but we still fail when attempting to deploy the service (have not reached vf or vf-modele creation yet) https://jira.onap.org/browse/SDC-334 https://wiki.onap.org/display/DW/ONAP+master+branch+Stabilization We need return to and fully break out what is being posted/put out of the 3 VID and 3 Robot calls during service creation - https://wiki.onap.org/display/DW/UCA-20+OSS+JAX-RS+2+Client#UCA-20OSSJAX-RS2Client-Prerequisites /michael From: Josef Reisinger [mailto:[email protected]] Sent: Wednesday, September 13, 2017 04:48 To: Kapil Gupta (c) <[email protected]> Cc: bharath thiruveedula <[email protected]>; ROSE, DANIEL V < [email protected]>; Michael O'Brien <[email protected]>; LUND, CARSTEN (CARSTEN) <[email protected]>; [email protected]; [email protected] Subject: RE: [onap-discuss] vFW demo setup issue Kapil, I tried to compare the logs with a fresh log from my instance and it seems as there is still some issue with AAI. Are you sure that you see no errors from the model-loader container in AAI from "putTool.sh"? I remember that was the root of all eval... Mit freundlichen Grüßen / Kind regards Josef Reisinger From: "Kapil Gupta (c)" <[email protected]> To: Josef Reisinger <[email protected]>, " [email protected]" <[email protected]>, "ROSE, DANIEL V" <[email protected]> Cc: bharath thiruveedula <[email protected]>, " [email protected]" <[email protected]>, "LUND, CARSTEN (CARSTEN)" <[email protected]>, "[email protected]" < [email protected]> Date: 12.09.2017 09:46 Subject: RE: [onap-discuss] vFW demo setup issue Hi ALL, To debug '500' does not match '^(201|200? ) while running ?demo.sh init? from Robot VM , we are using below link. https://wiki.onap.org/questions/3245760/robotvm-demo.sh-failing-with-ssl-error after apply below changes , Now I getting error '404' does not match '^(201|412)$' while running ./demo.sh init from robot vm. ---- logs ?InitDemo.zip? attached 1.stop your hbase container and restarting like this: docker run -itd --net=host --hostname localhost --name=hbase wc9368/aai-hbase-1.2.3 2 stop and remove the aai docker container and then start the aai docker container again docker stop aai-service && docker rm aai-service docker run -itd \ --name=aai-service \ --net=host \ -e AAI_REPO_PATH=r/aai \ -e AAI_CHEF_ENV=simpledemo \ -e AAI_CHEF_LOC=/var/chef/aai-data/environments \ -e docker_gitbranch=master \ nexus3.onap.org:10001/openecomp/ajsc-aai:1.0-STAGING-latest Please help us to debug this further and let me know if you want any other log to see. OutPut of :- root@onap-vm-aai:/opt# nc -z -v aai.hbase.simpledemo.openecomp.org 2181 nc: getaddrinfo: Name or service not known root@onap-vm-aai:~# cat /etc/hosts 127.0.0.1 localhost # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts 127.0.0.1 onap-vm-aai 10.110.208.215 policy.api.simpledemo.openecomp.org 10.110.208.216 portal.api.simpledemo.openecomp.org 10.1.1.21 sdc.api.simpledemo.openecomp.org 10.110.208.220 vid.api.simpledemo.openecomp.org Thanks Kapil From: Josef Reisinger [mailto:[email protected]] Sent: 08 September 2017 15:19 To: Kapil Gupta (c) <
_______________________________________________ onap-discuss mailing list [email protected] https://lists.onap.org/mailman/listinfo/onap-discuss
