Hi all, In a new El-Alto ONAP installation (using OOM Git commit e9ea0c35279af0ca244366a9730a1966cbb600d3, tag: 5.0.1-ONAP), I was only able to access SDC-BE catalog API via HTTPS (from both outside and inside of Kubernetes cluster).
Ext-API NBI POD needs to communicate with SDC-BE catalog upon receiving and processing a service order. By default, NBI POD configuration accesses SDC-BE via http://sdc-be:8080. So I changed the k8s deployment configuration for SDC_HOST to target https://sdc-be:8443 instead. Now the problem is that upon POD re-creation of NBI, it fails to register with MSB. MSB looks fine in the ONAP health. It is not due to config change I did because I reverted the change and it still cannot register with MSB. Could anyone please help identifying the issue? Unfortunately the logs (MSB Consul logs) are not being very helpful in identifying the root cause. Also, shouldn't NBI default configuration target HTTPS for SDC-BE since it can only be accessed that way? Do I miss anything else here? NBI POD logs 2019-11-07 09:23:51.081 INFO 6 --- [ main] o.o.nbi.apis.servicecatalog.SdcClient : SDC GET url : https://sdc-be.onap:8443/sdc/v1/catalog/services/{id}/metadata 2019-11-07 09:23:51.081 INFO 6 --- [ main] o.o.nbi.apis.servicecatalog.SdcClient : SDC FIND url : https://sdc-be.onap:8443/sdc/v1/catalog/services 2019-11-07 09:23:51.081 INFO 6 --- [ main] o.o.nbi.apis.servicecatalog.SdcClient : SDC HealthCheck : https://sdc-be.onap:8443/sdc2/rest/healthCheck 2019-11-07 09:23:51.118 INFO 6 --- [ main] o.o.n.a.h.s.d.CheckDMaaPEventsManager : DMaaP Get Events url : http://message-router.onap:3904/events/$topic/$consumergroup/$consumerid?timeout=$timeout 2019-11-07 09:23:51.883 INFO 6 --- [ main] org.onap.nbi.Application : Started Application in 9.755 seconds (JVM running for 10.628) 2019-11-07 09:23:51.892 INFO 6 --- [ main] org.onap.nbi.ServiceRegisterRunner : Registering with msb discovery (msb-discovery.onap:10081): - host: [10.42.177.209] - port: [8080] - name: [nbi] - version: [v4] - url: [/nbi/api/v4] - path: [] - protocol: [REST]g - visualRange: [1] - enableSSL: [false] 2019-11-07 09:23:51.892 INFO 6 --- [ main] org.onap.nbi.ServiceRegisterRunner : Registration with msb discovery (attempt 1/1) 2019-11-07 09:23:51.895 INFO 6 --- [ main] o.o.m.s.httpclient.msb.MSBServiceClient : msb service info:msbSvrAddress:msb-discovery.onap:10081 2019-11-07 09:23:52.127 ERROR 6 --- [ main] o.o.m.s.httpclient.msb.MSBServiceClient : msb service info:msbSvrAddress:msb-discovery.onap:10081 org.onap.msb.sdk.discovery.common.RouteException: register consul service fail:400 at org.onap.msb.sdk.discovery.util.HttpClientUtil.httpPostWithJSON(HttpClientUtil.java:54) ~[msb-java-sdk-1.2.5.jar!/:na] at org.onap.msb.sdk.discovery.MSBService.registerMicroServiceInfo(MSBService.java:118) ~[msb-java-sdk-1.2.5.jar!/:na] at org.onap.msb.sdk.discovery.MSBService.registerMicroServiceInfo(MSBService.java:72) ~[msb-java-sdk-1.2.5.jar!/:na] at org.onap.msb.sdk.httpclient.msb.MSBServiceClient.lambda$registerMicroServiceInfo$1(MSBServiceClient.java:66) ~[msb-java-sdk-1.2.5.jar!/:na] at org.onap.msb.sdk.httpclient.msb.MSBServiceClient.invokeMsb(MSBServiceClient.java:50) ~[msb-java-sdk-1.2.5.jar!/:na] at org.onap.msb.sdk.httpclient.msb.MSBServiceClient.registerMicroServiceInfo(MSBServiceClient.java:66) ~[msb-java-sdk-1.2.5.jar!/:na] at org.onap.nbi.ServiceRegisterRunner.run(ServiceRegisterRunner.java:127) ~[classes!/:na] at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:780) ~[spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE] at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:764) ~[spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:319) ~[spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1214) ~[spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1203) ~[spring-boot-2.1.7.RELEASE.jar!/:2.1.7.RELEASE] at org.onap.nbi.Application.run(Application.java:30) ~[classes!/:na] at org.onap.nbi.Application.main(Application.java:26) ~[classes!/:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:na] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:na] at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na] at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) ~[app.jar:na] at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) ~[app.jar:na] at org.springframework.boot.loader.Launcher.launch(Launcher.java:51) ~[app.jar:na] at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52) ~[app.jar:na] 2019-11-07 09:23:52.128 WARN 6 --- [ main] org.onap.nbi.ServiceRegisterRunner : Registration with msb discovery (attempt 1/1) FAILED. MSB Discovery POD logs 2019-11-07 09:23:52 097 ERROR [org.onap.msb.sdclient.resources.MicroServiceResource][dw-39 - POST /api/microservices/v1/services?createOrUpdate=true] - **** json string:org.onap.msb.sdclient.core.MicroServiceInfo@889eb19 2019-11-07 09:23:52 097 INFO [org.onap.msb.sdclient.resources.MicroServiceResource][dw-39 - POST /api/microservices/v1/services?createOrUpdate=true] - [POST REQUEST] Request IP:10.42.177.209,Request Param:[createOrUpdate]true,Request Body:{"serviceName":"nbi","version":"v4","url":"/nbi/api/v4","protocol":"REST","visualRange":"1","lb_policy":"","publish_port":null,"namespace":"","network_plane_type":null,"host":"","path":"","enable_ssl":false,"nodes":[{"ip":"10.42.177.209","port":"8080","lb_server_params":null,"checkType":"HTTP","checkUrl":"/nbi/api/v4/status","checkInterval":null,"checkTimeOut":null,"ttl":"","tls_skip_verify":true,"ha_role":""}],"metadata":null,"labels":null} 2019-11-07 09:23:52 107 ERROR [org.onap.msb.sdclient.wrapper.util.HttpClientUtil][dw-39 - POST /api/microservices/v1/services?createOrUpdate=true] - 400 2019-11-07 09:23:52 107 ERROR [org.onap.msb.sdclient.wrapper.ConsulServiceWrapper][dw-39 - POST /api/microservices/v1/services?createOrUpdate=true] - register consul service throw exception java.lang.Exception: register consul service fail:400 at org.onap.msb.sdclient.wrapper.ConsulServiceWrapper.saveMicroServiceInstance(ConsulServiceWrapper.java:719) at org.onap.msb.sdclient.resources.MicroServiceResource.addMicroService(MicroServiceResource.java:120) at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) at io.dropwizard.jetty.NonblockingServletHolder.handle(NonblockingServletHolder.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623) at io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:35) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:45) at io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:39) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:239) at io.dropwizard.jetty.ContextRoutingHandler.handle(ContextRoutingHandler.java:38) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703) at io.dropwizard.jetty.BiDiGzipHandler.handle(BiDiGzipHandler.java:67) at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56) at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) 10.42.177.209 - - [07/Nov/2019:09:23:52 +0000] 46f3d597c0895ddbc27c0ddfc617a31d "POST /api/microservices/v1/services?createOrUpdate=true HTTP/1.1" 500 request_processing_time:0.030 upstream_response_time:0.030 "-" "Apache-HttpClient/4.5.9 (Java/11.0.4)" "-" Best Regards Stavros -------------------------------------------------------- E-mail : [email protected]<mailto:[email protected]> BBA E2E SDAN @Athens Fixed Networks NOKIA -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#19368): https://lists.onap.org/g/onap-discuss/message/19368 Mute This Topic: https://lists.onap.org/mt/45617343/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
