goooogs opened a new issue #6635: URL: https://github.com/apache/skywalking/issues/6635
Please answer these questions before submitting your issue. - Why do you submit this issue? - [x] Question or discussion - [x] Bug - [ ] Requirement - [ ] Feature or performance improvement ___ ### Question - What do you want to know? I have two isolated VPC: VPC1 and VPC2. skywalking was deployed in VPC1 (kubernetes + istio), and I'm located in VPC2, access skywalking with these addresses: * UI: https://skywalking-ui.example.com * GRPC over https: skywalking-oap-grpc.example.com:443 * REST: https://skywalking-oap-rest.example.com before startup my java application, i use swctl to check skywalking status: ``` swctl --base-url https://skywalking-oap-rest.example.com/graphql --grpcAddr skywalking-oap-grpc.example.com:443 checkHealth --grpcTLS INFO OAP modules are healthy INFO OAP gRPC is healthy ``` it looks fine. Then i startup my java application: ``` SW_AGENT_COLLECTOR_BACKEND_SERVICES=skywalking-oap-grpc.example.com:443 \ SW_AGENT_NAME=test::app1 \ SW_AGENT_FORCE_TLS=true \ java -javaagent:/agent/skywalking-agent.jar -jar /app1.jar ``` But i got apm agent error ``` ERROR 2021-03-28 10:30:27:764 SkywalkingAgent-7-JVMService-consume-0 JVMMetricsSender : send JVM metrics to Collector fail. org.apache.skywalking.apm.dependencies.io.grpc.StatusRuntimeException: UNAVAILABLE: io exception at org.apache.skywalking.apm.dependencies.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:262) at org.apache.skywalking.apm.dependencies.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243) at org.apache.skywalking.apm.dependencies.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156) at org.apache.skywalking.apm.network.language.agent.v3.JVMMetricReportServiceGrpc$JVMMetricReportServiceBlockingStub.collect(JVMMetricReportServiceGrpc.java:181) at org.apache.skywalking.apm.agent.core.jvm.JVMMetricsSender.run(JVMMetricsSender.java:82) at org.apache.skywalking.apm.util.RunnableWithExceptionProtection.run(RunnableWithExceptionProtection.java:33) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: java.io.IOException: Connection reset by peer at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method) at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276) at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233) at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223) at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:358) at org.apache.skywalking.apm.dependencies.io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253) at org.apache.skywalking.apm.dependencies.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133) at org.apache.skywalking.apm.dependencies.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350) at org.apache.skywalking.apm.dependencies.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148) at org.apache.skywalking.apm.dependencies.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) at org.apache.skywalking.apm.dependencies.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) at org.apache.skywalking.apm.dependencies.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) at org.apache.skywalking.apm.dependencies.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) at org.apache.skywalking.apm.dependencies.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at org.apache.skywalking.apm.dependencies.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.skywalking.apm.dependencies.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more ``` Why swctl checkHealth can pass, but it really not with grpc over https? I hope skywalking grpc over https works fine for apm agent. ___ ### Bug - Which version of SkyWalking, OS, and JRE? SkyWalking: apache-skywalking-apm-es6-8.4.0 swctl: version 0.6.0 OS: MacOS Catalina 10.15.4 JRE: openjdk version "11.0.8" 2020-07-14, 64-Bit - Which company or project? - What happened? If possible, provide a way to reproduce the error. e.g. demo application, component version. ```yaml apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: annotations: name: my-gateway namespace: istio-system spec: selector: istio: ingressgateway servers: - hosts: - '*.example.com' port: name: http number: 80 protocol: HTTP - hosts: - '*.example.com' port: name: https number: 443 protocol: HTTPS tls: mode: SIMPLE privateKey: /etc/istio/ingressgateway-certs/tls.key serverCertificate: /etc/istio/ingressgateway-certs/tls.crt --- apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: skywalking-oap-grpc spec: hosts: - skywalking-oap-grpc.example.com gateways: - istio-system/my-gateway http: - route: - destination: host: skywalking-oap.default.svc.cluster.local port: number: 11800 ``` ___ ### Requirement or improvement - Please describe your requirements or improvement suggestions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
