Hi Ruoyu,
If you do docker logs testconfig_aai.hbase.simpledemo.onap.org_1 in aai vm1
INFO 01:40:39 Compacted 4 sstables to
[/var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-5,].
5,904 bytes to 5,740 (~97% of original) in 596ms = 0.009185MB/s. 4 total
partitions merged to 1. Partition merge counts were {4:1, }
INFO 01:40:46 No gossip backlog; proceeding
INFO 01:40:47 Netty using native Epoll event loop
INFO 01:40:47 Using Netty Version:
[netty-buffer=netty-buffer-4.0.44.Final.452812a,
netty-codec=netty-codec-4.0.44.Final.452812a,
netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a,
netty-codec-http=netty-codec-http-4.0.44.Final.452812a,
netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a,
netty-common=netty-common-4.0.44.Final.452812a,
netty-handler=netty-handler-4.0.44.Final.452812a,
netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb,
netty-transport=netty-transport-4.0.44.Final.452812a,
netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a,
netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a,
netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a,
netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a]
INFO 01:40:47 Starting listening for CQL clients on /0.0.0.0:9042...
INFO 01:40:47 Binding thrift service to /0.0.0.0:9160
INFO 01:40:47 Listening for thrift clients...
If you see logs like this, then your cassandra container in vm2 is fine.
It is the connectivity from aai vm1 docker container to aai vm2 that seems to
be the issue.
You can verify this by running this command:
docker run -it --entrypoint=nc
nexus3.onap.org:10001/onap/aai-resources:1.2-STAGING-latest<https://urldefense.proofpoint.com/v2/url?u=http-3A__nexus3.onap.org-3A10001_onap_aai-2Dresources-3A1.2-2DSTAGING-2Dlatest&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=f44eG3iZaja2ozEA2yRZnQ&m=JNIwkeG3Z4-KHc2apnKE6YZK7h4AK1QnEjGgUk3m0dA&s=XankMg-oLBpcT2QNOKmkhdiG17Ty86d9a0jQJFgGn_M&e=>
-z -v
aai.hbase.simpledemo.onap.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__aai.hbase.simpledemo.onap.org&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=f44eG3iZaja2ozEA2yRZnQ&m=JNIwkeG3Z4-KHc2apnKE6YZK7h4AK1QnEjGgUk3m0dA&s=8YsOCeUnhmMI7rim6jtcJMcFYmyEoNXyJLTf5vW6iFo&e=>
9160
You can check if from the docker container within vm1 is able to reach vm2 via
the hostname by running the above command
If you are unable to do so, please check your dns servers.
If you are able to ping the hostname from aai vm1 to vm2 successsfully without
overwriting the /etc/hosts and are unable to do from within docker container of
vm1 to vm2, then the docker setup is the problem.
A manual workaround would be to add this to docker-compose-app.yml in
/opt/test-config in aai vm1.
extra_hosts:
-
"aai.hbase.simpledemo.openecomp.org:10.12.25.83<http://aai.hbase.simpledemo.openecomp.org:10.12.25.83>"
Where you overwrite the ip address to your aai vm2 ip address to get around
this issue.
For permanently solve the issue, see if the problem is the dns server or the
docker not inheriting the dns server correctly and properly fix it.
Thanks,
Harish
From: Ying, Ruoyu [mailto:[email protected]]
Sent: Tuesday, June 05, 2018 9:03 PM
To: KAJUR, HARISH V <[email protected]>
Cc: [email protected]; FORSYTH, JAMES <[email protected]>
Subject: RE: [aai] Exception faced when setup aai inst1 through docker
Hi Harish,
Any other suggestions?
Best Regards,
Ruoyu
From: Ying, Ruoyu
Sent: Tuesday, June 5, 2018 9:46 AM
To: 'KAJUR, HARISH V' <[email protected]<mailto:[email protected]>>
Cc: [email protected]<mailto:[email protected]>; FORSYTH,
JAMES <[email protected]<mailto:[email protected]>>
Subject: RE: [aai] Exception faced when setup aai inst1 through docker
Hi Harish,
I'm able to ping the host from aai inst1. So the dns is able to resolve the
host name to the aai inst2 ip address.
ubuntu@onap-aai-inst1:/opt/test-config$ ping aai.hbase.simpledemo.onap.org
PING vm2.aai.simpledemo.onap.org (10.0.1.2) 56(84) bytes of data.
64 bytes from 10.0.1.2: icmp_seq=1 ttl=64 time=1.39 ms
64 bytes from 10.0.1.2: icmp_seq=2 ttl=64 time=0.734 ms
64 bytes from 10.0.1.2: icmp_seq=3 ttl=64 time=0.824 ms
And also I can see the docker containers running inside aai inst2, which looks
like this:
ubuntu@onap-aai-inst2:~$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
23b8f67f5c1c elasticsearch:2.4.1 "/docker-entrypoin..." 3 days ago
Up 3 days 0.0.0.0:9200->9200/tcp, 9300/tcp elasticsearch
43a2605bb71b cassandra:2.1 "docker-entrypoint..." 3 days ago
Up 3 days
test-config_aai.hbase.simpledemo.onap.org_1
Do you have any method to measure whether the Cassandra is working fine?
Thanks.
Best Regards,
Ruoyu
From: KAJUR, HARISH V [mailto:[email protected]]
Sent: Tuesday, June 5, 2018 9:36 AM
To: Ying, Ruoyu <[email protected]<mailto:[email protected]>>;
[email protected]<mailto:[email protected]>; FORSYTH, JAMES
<[email protected]<mailto:[email protected]>>
Subject: RE: [aai] Exception faced when setup aai inst1 through docker
Hi Ruoyu,
Can you please try to do a ping aai.hbase.simpledemo.onap.org from the aai vm1?
If it doesn't work, it seems that the issue is some kind of dns issue that's
causing it.
Check the dns servers and see if they are up and then run this command to see
if your aai vm1 server is using the dns server:
cat /etc/resolv.conf
The dns server for onap should show up here
If everything on the networking side looks good between aai vm1 and aai vm2,
then please go to aai vm2 and check if the docker containers are running there?
Thanks,
Harish
From: Ying, Ruoyu [mailto:[email protected]]
Sent: Monday, June 04, 2018 9:01 PM
To: [email protected]<mailto:[email protected]>; KAJUR,
HARISH V <[email protected]<mailto:[email protected]>>; FORSYTH, JAMES
<[email protected]<mailto:[email protected]>>
Subject: [aai] Exception faced when setup aai inst1 through docker
Hi all,
I'm trying to setup AAI env through docker and I've already setup the
aai-inst2. But when I try to bring up aai-inst1, I faced an error like this:
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at
org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at
org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at
org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:594)
Caused by: java.lang.ExceptionInInitializerError
at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:98)
at org.onap.aai.dbgen.GenTester.main(GenTester.java:118)
... 8 more
Caused by: java.lang.RuntimeException: Failed to instantiate graphs
at org.onap.aai.dbmap.AAIGraph.<init>(AAIGraph.java:83)
at org.onap.aai.dbmap.AAIGraph.<init>(AAIGraph.java:55)
at org.onap.aai.dbmap.AAIGraph$Helper.<clinit>(AAIGraph.java:88)
... 10 more
Caused by: java.lang.IllegalArgumentException: Could not instantiate
implementation:
org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxStoreManager
at
org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:69)
at
org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:477)
at
org.janusgraph.diskstorage.Backend.getStorageManager(Backend.java:409)
at
org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1376)
at
org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164)
at
org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133)
at
org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:113)
at org.onap.aai.dbmap.AAIGraph.loadGraph(AAIGraph.java:110)
at org.onap.aai.dbmap.AAIGraph.<init>(AAIGraph.java:80)
... 12 more
Caused by: java.lang.reflect.InvocationTargetException
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:58)
... 20 more
Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary
failure in storage backend
at
org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxStoreManager.ensureKeyspaceExists(AstyanaxStoreManager.java:619)
at
org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxStoreManager.<init>(AstyanaxStoreManager.java:314)
... 25 more
Caused by: com.netflix.astyanax.connectionpool.exceptions.PoolTimeoutException:
PoolTimeoutException:
[host=aai.hbase.simpledemo.onap.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__aai.hbase.simpledemo.onap.org&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=f44eG3iZaja2ozEA2yRZnQ&m=SrmK0BqP1xPP2MTuS-HPmAl6viUzpbZTYHtHh5eU_KA&s=yla8QxOXKGEgdAFg5-q6e-kl_VZMsKKNLK-stF5f7mw&e=>(aai.hbase.simpledemo.onap.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__aai.hbase.simpledemo.onap.org&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=f44eG3iZaja2ozEA2yRZnQ&m=SrmK0BqP1xPP2MTuS-HPmAl6viUzpbZTYHtHh5eU_KA&s=yla8QxOXKGEgdAFg5-q6e-kl_VZMsKKNLK-stF5f7mw&e=>):9160,
latency=10001(10001), attempts=1]Timed out waiting for connection
at
com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.waitForConnection(SimpleHostConnectionPool.java:231)
at
com.netflix.astyanax.connectionpool.impl.SimpleHostConnectionPool.borrowConnection(SimpleHostConnectionPool.java:198)
at
com.netflix.astyanax.connectionpool.impl.RoundRobinExecuteWithFailover.borrowConnection(RoundRobinExecuteWithFailover.java:84)
at
com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:117)
at
com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:352)
at
com.netflix.astyanax.thrift.ThriftClusterImpl.executeSchemaChangeOperation(ThriftClusterImpl.java:146)
at
com.netflix.astyanax.thrift.ThriftClusterImpl.internalCreateKeyspace(ThriftClusterImpl.java:321)
at
com.netflix.astyanax.thrift.ThriftClusterImpl.addKeyspace(ThriftClusterImpl.java:294)
at
org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxStoreManager.ensureKeyspaceExists(AstyanaxStoreManager.java:614)
... 26 more
Does anyone know what's the problem? Thanks.
Best Regards,
Ruoyu
_______________________________________________
onap-discuss mailing list
[email protected]
https://lists.onap.org/mailman/listinfo/onap-discuss
