Hi, We are trying to install cassablanca on kubernetes and we are facing issues with AAI. AAi is not able to run because of issue with graphadmin, Can you let us know if we have missed some configuration ? We followed the same sequence that you have specified above. All other components "consul ,msb, dmaap, dcaegen2, aaf, robot" are up and running.
The following error is seen in "aai-aai-graphadmin-create-db-schema-mjc4z" JOB. Project Build Version: 1.0.1 chown: changing ownership of '/opt/app/aai-graphadmin/resources/application.properties': Read-only file system chown: changing ownership of '/opt/app/aai-graphadmin/resources/etc/appprops/aaiconfig.properties': Read-only file system chown: changing ownership of '/opt/app/aai-graphadmin/resources/etc/appprops/janusgraph-cached.properties': Read-only file system chown: changing ownership of '/opt/app/aai-graphadmin/resources/etc/appprops/janusgraph-realtime.properties': Read-only file system chown: changing ownership of '/opt/app/aai-graphadmin/resources/etc/auth/aai_keystore': Read-only file system chown: changing ownership of '/opt/app/aai-graphadmin/resources/localhost-access-logback.xml': Read-only file system chown: changing ownership of '/opt/app/aai-graphadmin/resources/logback.xml': Read-only file system Wed Apr 24 16:09:21 IST 2019 Starting /opt/app/aai-graphadmin/bin/createDBSchema.sh ---- NOTE --- about to open graph (takes a little while)--------; Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:595) Caused by: java.lang.ExceptionInInitializerError at org.onap.aai.dbmap.AAIGraph.getInstance(AAIGraph.java:103) at org.onap.aai.schema.GenTester.main(GenTester.java:126) ... 8 more Caused by: java.lang.RuntimeException: Failed to instantiate graphs at org.onap.aai.dbmap.AAIGraph.<init>(AAIGraph.java:85) at org.onap.aai.dbmap.AAIGraph.<init>(AAIGraph.java:57) at org.onap.aai.dbmap.AAIGraph$Helper.<clinit>(AAIGraph.java:90) ... 10 more Caused by: org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:57) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:159) at org.janusgraph.diskstorage.configuration.backend.KCVSConfiguration.get(KCVSConfiguration.java:100) at org.janusgraph.diskstorage.configuration.BasicConfiguration.isFrozen(BasicConfiguration.java:106) at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1394) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:164) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:133) at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:113) at org.onap.aai.dbmap.AAIGraph.loadGraph(AAIGraph.java:115) at org.onap.aai.dbmap.AAIGraph.<init>(AAIGraph.java:82) ... 12 more Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT1M at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:101) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:55) ... 21 more Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxKeyColumnValueStore.getNamesSlice(AstyanaxKeyColumnValueStore.java:161) at org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxKeyColumnValueStore.getNamesSlice(AstyanaxKeyColumnValueStore.java:115) at org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxKeyColumnValueStore.getSlice(AstyanaxKeyColumnValueStore.java:104) at org.janusgraph.diskstorage.configuration.backend.KCVSConfiguration$1.call(KCVSConfiguration.java:103) at org.janusgraph.diskstorage.configuration.backend.KCVSConfiguration$1.call(KCVSConfiguration.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:148) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:162) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:69) ... 22 more Caused by: com.netflix.astyanax.connectionpool.exceptions.TokenRangeOfflineException: TokenRangeOfflineException: [host=10.233.66.28(10.233.66.28):9160, latency=1(1), attempts=1]UnavailableException() at com.netflix.astyanax.thrift.ThriftConverter.ToConnectionPoolException(ThriftConverter.java:165) at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:65) at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:28) at com.netflix.astyanax.thrift.ThriftSyncConnectionFactoryImpl$ThriftConnection.execute(ThriftSyncConnectionFactoryImpl.java:153) at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:119) at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:352) at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4.execute(ThriftColumnFamilyQueryImpl.java:538) at org.janusgraph.diskstorage.cassandra.astyanax.AstyanaxKeyColumnValueStore.getNamesSlice(AstyanaxKeyColumnValueStore.java:159) ... 29 more Caused by: UnavailableException() at org.apache.cassandra.thrift.Cassandra$multiget_slice_result$multiget_slice_resultStandardScheme.read(Cassandra.java:14687) at org.apache.cassandra.thrift.Cassandra$multiget_slice_result$multiget_slice_resultStandardScheme.read(Cassandra.java:14633) at org.apache.cassandra.thrift.Cassandra$multiget_slice_result.read(Cassandra.java:14559) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_multiget_slice(Cassandra.java:741) at org.apache.cassandra.thrift.Cassandra$Client.multiget_slice(Cassandra.java:725) at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4$1.internalExecute(ThriftColumnFamilyQueryImpl.java:544) at com.netflix.astyanax.thrift.ThriftColumnFamilyQueryImpl$4$1.internalExecute(ThriftColumnFamilyQueryImpl.java:541) at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:60) ... 35 more Failed to run the tool /opt/app/aai-graphadmin/bin/createDBSchema.sh successfully Failed to run the createDBSchema.sh Status of pods kubectl get pods -n onap NAME READY STATUS RESTARTS AGE tcs-aaf-aaf-cm-76696c8bcf-sqqpw 1/1 Running 0 7h tcs-aaf-aaf-cs-6c69c87d44-vbtk7 1/1 Running 0 7h tcs-aaf-aaf-fs-5fd8c8bd8d-fbbwb 1/1 Running 0 7h tcs-aaf-aaf-gui-777fb85d96-vtwnf 1/1 Running 0 7h tcs-aaf-aaf-hello-774f4b6f5-lb6t4 1/1 Running 0 7h tcs-aaf-aaf-locate-fbd8f454b-lgkjd 1/1 Running 0 7h tcs-aaf-aaf-oauth-77fcbf54cd-trdxx 1/1 Running 0 7h tcs-aaf-aaf-service-9f76d4746-4knwv 1/1 Running 0 7h tcs-aaf-aaf-sms-5f764cf7cf-vxl7g 1/1 Running 0 7h tcs-aaf-aaf-sms-quorumclient-0 1/1 Running 0 7h tcs-aaf-aaf-sms-quorumclient-1 1/1 Running 0 7h tcs-aaf-aaf-sms-quorumclient-2 1/1 Running 0 6h tcs-aaf-aaf-sms-vault-0 2/2 Running 3 7h tcs-aaf-aaf-sshsm-distcenter-gmbwl 0/1 Completed 0 7h tcs-aaf-aaf-sshsm-testca-69v6k 0/1 Init:Error 0 7h tcs-aaf-aaf-sshsm-testca-6ghcc 0/1 Init:Error 0 7h tcs-aaf-aaf-sshsm-testca-rz58d 0/1 Init:Error 0 7h tcs-aaf-aaf-sshsm-testca-ttbq7 0/1 Init:Error 0 7h tcs-aaf-aaf-sshsm-testca-xht79 0/1 Completed 0 6h tcs-aaf-aaf-sshsm-testca-z6sfw 0/1 Init:Error 0 7h tcs-aai-aai-5cbbdb4ff5-4hwpb 0/1 Init:0/1 45 7h tcs-aai-aai-babel-68fc787d74-pvdtr 2/2 Running 0 1d tcs-aai-aai-cassandra-0 1/1 Running 1 1d tcs-aai-aai-champ-5b64bd67c7-qnx4j 1/2 Running 0 1d tcs-aai-aai-data-router-6654597bdd-6nfmw 2/2 Running 6 1d tcs-aai-aai-elasticsearch-68bcf6c8fd-9stb4 1/1 Running 0 1d tcs-aai-aai-gizmo-d548b4d9-cvf6d 2/2 Running 0 1d tcs-aai-aai-graphadmin-67f5965fb7-88dwz 0/2 Init:0/1 165 1d tcs-aai-aai-graphadmin-create-db-schema-mjc4z 0/1 Error 0 1d tcs-aai-aai-graphadmin-create-db-schema-p5vms 0/1 Error 0 1d tcs-aai-aai-graphadmin-create-db-schema-sbhsk 0/1 Error 0 1d tcs-aai-aai-graphadmin-create-db-schema-wqw29 0/1 Error 0 1d tcs-aai-aai-graphadmin-create-db-schema-zrmpv 0/1 Error 0 1d tcs-aai-aai-modelloader-6dfbcd7596-9dd9s 2/2 Running 0 1d tcs-aai-aai-resources-588998b4ff-qnlvd 0/2 Init:0/1 165 1d tcs-aai-aai-search-data-57666c9494-n24bc 2/2 Running 0 1d tcs-aai-aai-sparky-be-7db4b8dcf-qr29r 0/2 Init:0/1 0 1d tcs-aai-aai-spike-6f9f5f5c9d-fxcrs 2/2 Running 0 1d tcs-aai-aai-traversal-7df69d5885-72kj5 0/2 Init:0/1 165 1d tcs-aai-aai-traversal-update-query-data-gr2w8 0/1 Init:0/1 165 1d tcs-consul-consul-69d7c64bdd-wms5l 1/1 Running 0 1d tcs-consul-consul-server-0 1/1 Running 0 1d tcs-consul-consul-server-1 1/1 Running 0 1d tcs-consul-consul-server-2 1/1 Running 0 1d tcs-dcaegen2-dcae-bootstrap-6b6bb89cd5-5vhmv 1/1 Running 0 1d tcs-dcaegen2-dcae-cloudify-manager-6b6f59fc66-k79c9 1/1 Running 0 1d tcs-dcaegen2-dcae-db-0 1/1 Running 0 1d tcs-dcaegen2-dcae-db-1 1/1 Running 0 1d tcs-dcaegen2-dcae-healthcheck-5fc6d94989-ch5kn 1/1 Running 0 1d tcs-dcaegen2-dcae-pgpool-77b844664d-5hd4c 1/1 Running 0 1d tcs-dcaegen2-dcae-pgpool-77b844664d-xxnh8 1/1 Running 0 1d tcs-dcaegen2-dcae-redis-0 1/1 Running 0 1d tcs-dcaegen2-dcae-redis-1 1/1 Running 0 1d tcs-dcaegen2-dcae-redis-2 1/1 Running 0 1d tcs-dcaegen2-dcae-redis-3 1/1 Running 0 1d tcs-dcaegen2-dcae-redis-4 1/1 Running 0 1d tcs-dcaegen2-dcae-redis-5 1/1 Running 0 1d tcs-dmaap-dbc-pg-0 1/1 Running 0 1d tcs-dmaap-dbc-pg-1 1/1 Running 0 1d tcs-dmaap-dbc-pgpool-57d7b76446-qgrcs 1/1 Running 0 1d tcs-dmaap-dbc-pgpool-57d7b76446-qphtf 1/1 Running 0 1d tcs-dmaap-dmaap-bus-controller-7567b865b7-x7vtz 1/1 Running 0 1d tcs-dmaap-dmaap-dr-db-655587488d-2j5b2 1/1 Running 1 1d tcs-dmaap-dmaap-dr-node-649659c584-ctfjs 1/1 Running 0 1d tcs-dmaap-dmaap-dr-prov-595cd8bc55-6kjtv 1/1 Running 6 1d tcs-dmaap-message-router-5f7b985c88-dj8j2 1/1 Running 0 1d tcs-dmaap-message-router-kafka-678fb8558b-hc768 1/1 Running 0 1d tcs-dmaap-message-router-zookeeper-54bb8cd9cf-kl55w 1/1 Running 0 1d tcs-esr-esr-gui-699d9f579b-hrhk9 1/1 Running 0 7h tcs-esr-esr-server-85d8c5f57-vj7m4 2/2 Running 0 7h tcs-msb-kube2msb-6db5fd8c85-xrbs7 1/1 Running 0 1d tcs-msb-msb-consul-66445944b6-phjpk 1/1 Running 0 1d tcs-msb-msb-discovery-6bd858b659-42xbp 2/2 Running 0 1d tcs-msb-msb-eag-78fbb94cc9-7cn2p 2/2 Running 0 1d tcs-msb-msb-iag-5d45c9999b-lwcb6 2/2 Running 0 1d tcs-oof-cmso-db-0 0/1 CrashLoopBackOff 105 7h tcs-oof-music-cassandra-0 1/1 Running 0 7h tcs-oof-music-cassandra-1 1/1 Running 0 7h tcs-oof-music-cassandra-2 1/1 Running 0 7h tcs-oof-music-cassandra-job-config-z28dh 0/1 Completed 0 7h tcs-oof-music-tomcat-8f64f65d8-4pbnp 1/1 Running 0 7h tcs-oof-music-tomcat-8f64f65d8-dl58x 1/1 Running 0 7h tcs-oof-music-tomcat-8f64f65d8-fmrjn 0/1 Init:0/3 0 7h tcs-oof-oof-557c8c677d-rjb2b 0/1 Init:0/2 45 7h tcs-oof-oof-cmso-service-5467475444-ppmd7 0/1 Init:0/2 45 7h tcs-oof-oof-has-api-5d79d97fb7-nzfkx 0/1 Init:0/3 45 7h tcs-oof-oof-has-controller-658bbb894-m9dqf 0/1 Init:0/3 45 7h tcs-oof-oof-has-data-5575788564-zbv4x 0/1 Init:0/4 45 7h tcs-oof-oof-has-healthcheck-k5swr 0/1 Init:0/1 45 7h tcs-oof-oof-has-onboard-bvfsz 0/1 Init:0/2 45 7h tcs-oof-oof-has-reservation-9fd696d8d-dbw2k 0/1 Init:0/4 45 7h tcs-oof-oof-has-solver-7fd7878df9-dn4xw 0/1 Init:0/4 45 7h tcs-oof-zookeeper-0 1/1 Running 0 7h tcs-oof-zookeeper-1 1/1 Running 0 7h tcs-oof-zookeeper-2 1/1 Running 0 7h tcs-robot-robot-86d89ffdb9-xd65b 1/1 Running 0 1d Regards, Yogi -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16815): https://lists.onap.org/g/onap-discuss/message/16815 Mute This Topic: https://lists.onap.org/mt/31418159/21656 Mute #aai: https://lists.onap.org/mk?hashtag=aai&subid=2740164 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
