zeroall edited a comment on issue #2969: OutOfMemoryError by com.google.common.reflect.ClassPath.from URL: https://github.com/apache/skywalking/issues/2969#issuecomment-506927724 Hi Wu Sheng, I took a look at Skywalking with the Arthas tool and got some information: ``` $ dashboard ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON 1 main main 5 RUNNABLE 98 0:33 false false 17 AsyncAppender-Worker-arthas-cache.result.A system 9 WAITING 0 0:0 false true 15 Attach Listener system 9 RUNNABLE 0 0:0 false true 11 Curator-ConnectionStateManager-0 main 5 TIMED_WAITING 0 0:0 false true 14 Curator-Framework-0 main 5 WAITING 0 0:0 false true 3 Finalizer system 8 WAITING 0 0:0 false true 2 Reference Handler system 10 WAITING 0 0:0 false true 4 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true 26 Timer-for-arthas-dashboard-4a94f150-5fea-4 system 10 RUNNABLE 0 0:0 false true 25 as-command-execute-daemon system 10 TIMED_WAITING 0 0:0 false true 19 job-timeout system 9 TIMED_WAITING 0 0:0 false true 13 main-EventThread main 5 WAITING 0 0:0 false true 12 main-SendThread(192.168.1.214:2181) main 5 RUNNABLE 0 0:0 false true 20 nioEventLoopGroup-2-1 system 10 RUNNABLE 0 0:0 false false Memory used total max usage GC heap 677M 954M 954M 70.97% gc.ps_scavenge.count 29 ps_eden_space 33M 205M 212M 16.01% gc.ps_scavenge.time(ms) 788 ps_survivor_space 0K 67584K 67584K 0.00% gc.ps_marksweep.count 2 ps_old_gen 647M 683M 683M 94.78% gc.ps_marksweep.time(ms) 554 nonheap 31M 32M -1 97.82% code_cache 5M 5M 240M 2.46% metaspace 22M 23M -1 96.89% Runtime os.name Linux os.version 3.10.0-957.21.3.el7.x86_64 java.version 1.8.0_201 java.home /opt/jdk1.8.0_201/jre systemload.average 0.38 processors 4 uptime 35s ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON 1 main main 5 RUNNABLE 97 0:37 false false 26 Timer-for-arthas-dashboard-4a94f150-5fea-4 system 10 RUNNABLE 1 0:0 false true 17 AsyncAppender-Worker-arthas-cache.result.A system 9 WAITING 0 0:0 false true 15 Attach Listener system 9 RUNNABLE 0 0:0 false true 11 Curator-ConnectionStateManager-0 main 5 TIMED_WAITING 0 0:0 false true 14 Curator-Framework-0 main 5 WAITING 0 0:0 false true 3 Finalizer system 8 WAITING 0 0:0 false true 2 Reference Handler system 10 WAITING 0 0:0 false true 4 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true 25 as-command-execute-daemon system 10 TIMED_WAITING 0 0:0 false true 19 job-timeout system 9 TIMED_WAITING 0 0:0 false true 13 main-EventThread main 5 WAITING 0 0:0 false true 12 main-SendThread(192.168.1.214:2181) main 5 RUNNABLE 0 0:0 false true 20 nioEventLoopGroup-2-1 system 10 RUNNABLE 0 0:0 false false Memory used total max usage GC heap 755M 954M 954M 79.19% gc.ps_scavenge.count 29 ps_eden_space 73M 205M 212M 34.48% gc.ps_scavenge.time(ms) 788 ps_survivor_space 0K 67584K 67584K 0.00% gc.ps_marksweep.count 6 ps_old_gen 682M 683M 683M 99.97% gc.ps_marksweep.time(ms) 1778 nonheap 31M 32M -1 98.05% code_cache 6M 6M 240M 2.54% metaspace 22M 23M -1 98.00% Runtime os.name Linux os.version 3.10.0-957.21.3.el7.x86_64 java.version 1.8.0_201 java.home /opt/jdk1.8.0_201/jre systemload.average 0.43 processors 4 uptime 40s ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON 1 main main 5 RUNNABLE 99 0:41 false false 17 AsyncAppender-Worker-arthas-cache.result.A system 9 WAITING 0 0:0 false true 15 Attach Listener system 9 RUNNABLE 0 0:0 false true 11 Curator-ConnectionStateManager-0 main 5 TIMED_WAITING 0 0:0 false true 14 Curator-Framework-0 main 5 WAITING 0 0:0 false true 3 Finalizer system 8 WAITING 0 0:0 false true 2 Reference Handler system 10 WAITING 0 0:0 false true 4 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true 26 Timer-for-arthas-dashboard-4a94f150-5fea-4 system 10 RUNNABLE 0 0:0 false true 25 as-command-execute-daemon system 10 TIMED_WAITING 0 0:0 false true 19 job-timeout system 9 TIMED_WAITING 0 0:0 false true 13 main-EventThread main 5 WAITING 0 0:0 false true 12 main-SendThread(192.168.1.214:2181) main 5 RUNNABLE 0 0:0 false true 20 nioEventLoopGroup-2-1 system 10 RUNNABLE 0 0:0 false false Memory used total max usage GC heap 853M 954M 954M 89.40% gc.ps_scavenge.count 29 ps_eden_space 170M 205M 212M 80.43% gc.ps_scavenge.time(ms) 788 ps_survivor_space 0K 67584K 67584K 0.00% gc.ps_marksweep.count 12 ps_old_gen 682M 683M 683M 99.97% gc.ps_marksweep.time(ms) 3008 nonheap 31M 33M -1 95.90% code_cache 5M 6M 240M 2.36% metaspace 22M 23M -1 97.18% Runtime os.name Linux os.version 3.10.0-957.21.3.el7.x86_64 java.version 1.8.0_201 java.home /opt/jdk1.8.0_201/jre systemload.average 0.48 processors 4 uptime 45s ID NAME GROUP PRIORITY STATE %CPU TIME INTERRUPTED DAEMON 1 main main 5 RUNNABLE 99 0:43 false false 17 AsyncAppender-Worker-arthas-cache.result.A system 9 WAITING 0 0:0 false true 15 Attach Listener system 9 RUNNABLE 0 0:0 false true 11 Curator-ConnectionStateManager-0 main 5 TIMED_WAITING 0 0:0 false true 14 Curator-Framework-0 main 5 WAITING 0 0:0 false true 3 Finalizer system 8 WAITING 0 0:0 false true 2 Reference Handler system 10 WAITING 0 0:0 false true 4 Signal Dispatcher system 9 RUNNABLE 0 0:0 false true 26 Timer-for-arthas-dashboard-4a94f150-5fea-4 system 10 RUNNABLE 0 0:0 false true 25 as-command-execute-daemon system 10 TIMED_WAITING 0 0:0 false true 19 job-timeout system 9 TIMED_WAITING 0 0:0 false true 13 main-EventThread main 5 WAITING 0 0:0 false true 12 main-SendThread(192.168.1.214:2181) main 5 TIMED_WAITING 0 0:0 false true 20 nioEventLoopGroup-2-1 system 10 RUNNABLE 0 0:0 false false Memory used total max usage GC heap 881M 954M 954M 92.35% gc.ps_scavenge.count 29 ps_eden_space 198M 205M 212M 93.73% gc.ps_scavenge.time(ms) 788 ps_survivor_space 0K 67584K 67584K 0.00% gc.ps_marksweep.count 25 ps_old_gen 682M 683M 683M 99.97% gc.ps_marksweep.time(ms) 5618 nonheap 31M 33M -1 95.69% code_cache 5M 6M 240M 2.33% metaspace 22M 23M -1 97.33% Runtime os.name Linux os.version 3.10.0-957.21.3.el7.x86_64 java.version 1.8.0_201 java.home /opt/jdk1.8.0_201/jre systemload.average 1.00 processors 4 uptime 50s jvm RUNTIME --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- MACHINE-NAME 12887@centos0 JVM-START-TIME 2019-06-29 12:56:05 MANAGEMENT-SPEC-VERSION 1.2 SPEC-NAME Java Virtual Machine Specification SPEC-VENDOR Oracle Corporation SPEC-VERSION 1.8 VM-NAME Java HotSpot(TM) 64-Bit Server VM VM-VENDOR Oracle Corporation VM-VERSION 25.201-b09 INPUT-ARGUMENTS -Xms1G -Xmx1G -Doap.logDir=/root/data/skywalking/apache-skywalking-apm-bin/logs CLASS-PATH /root/data/skywalking/apache-skywalking-apm-bin/oap-libs/zookeeper-3.4.10.jar:/root/data/skywalking/apache-skywalking-ap m-bin/oap-libs/zipkin-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/zipkin-2.9.1.ja r:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/telemetry-prometheus-6.1.0.jar:/root/data/skywalking/apache-s kywalking-apm-bin/oap-libs/telemetry-api-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/t-digest-3.2 .jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/swagger-annotations-1.5.12.jar:/root/data/skywalking/apach e-skywalking-apm-bin/oap-libs/storage-zipkin-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/s torage-jdbc-hikaricp-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/storage-jaeger-plugin-6.1 .0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/storage-elasticsearch-plugin-6.1.0.jar:/root/data/skywal king/apache-skywalking-apm-bin/oap-libs/snakeyaml-1.18.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/slf4 j-api-1.7.25.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/skywalking-trace-receiver-plugin-6.1.0.jar:/ro ot/data/skywalking/apache-skywalking-apm-bin/oap-libs/skywalking-sharing-server-plugin-6.1.0.jar:/root/data/skywalking/a pache-skywalking-apm-bin/oap-libs/skywalking-register-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking- apm-bin/oap-libs/skywalking-mesh-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/skyw alking-jvm-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/skywalking-clr-receiver-pl ugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/simpleclient_httpserver-0.6.0.jar:/root/data/sky walking/apache-skywalking-apm-bin/oap-libs/simpleclient_hotspot-0.6.0.jar:/root/data/skywalking/apache-skywalking-apm-bi n/oap-libs/simpleclient_common-0.6.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/simpleclient-0.6.0.jar :/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/sharding-jdbc-core-2.0.3.jar:/root/data/skywalking/apache-skyw alking-apm-bin/oap-libs/server-starter-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/server-core-6. 1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/server-alarm-plugin-6.1.0.jar:/root/data/skywalking/apa che-skywalking-apm-bin/oap-libs/retrofit-2.3.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/reflectasm-1 .11.7.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/receiver-proto-6.1.0.jar:/root/data/skywalking/apache -skywalking-apm-bin/oap-libs/reactive-streams-1.0.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/rank-ev al-client-6.3.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/query-graphql-plugin-6.1.0.jar:/root/data/s kywalking/apache-skywalking-apm-bin/oap-libs/proto-google-common-protos-1.0.0.jar:/root/data/skywalking/apache-skywalkin g-apm-bin/oap-libs/protobuf-java-3.4.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/parent-join-client-6 .3.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/opencensus-contrib-grpc-metrics-0.12.3.jar:/root/data/ skywalking/apache-skywalking-apm-bin/oap-libs/opencensus-api-0.12.3.jar:/root/data/skywalking/apache-skywalking-apm-bin/ oap-libs/okio-1.13.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/okhttp-ws-2.7.5.jar:/root/data/skywalk ing/apache-skywalking-apm-bin/oap-libs/okhttp-3.9.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/okhttp- 2.7.5.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-transport-4.1.27.Final.jar:/root/data/skywalkin g/apache-skywalking-apm-bin/oap-libs/netty-tcnative-boringssl-static-2.0.7.Final.jar:/root/data/skywalking/apache-skywal king-apm-bin/oap-libs/netty-resolver-4.1.27.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-han dler-proxy-4.1.27.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-handler-4.1.27.Final.jar:/roo t/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-common-4.1.27.Final.jar:/root/data/skywalking/apache-skywalki ng-apm-bin/oap-libs/netty-codec-socks-4.1.27.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-co dec-http-4.1.27.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-codec-http2-4.1.27.Final.jar:/r oot/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-codec-4.1.27.Final.jar:/root/data/skywalking/apache-skywalk ing-apm-bin/oap-libs/netty-buffer-4.1.27.Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/netty-3.10.5 .Final.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/mysql-connector-java-5.1.47.jar:/root/data/skywalkin g/apache-skywalking-apm-bin/oap-libs/lucene-suggest-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/l ucene-spatial-extras-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-spatial-7.3.1.jar:/root/d ata/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-spatial3d-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm -bin/oap-libs/lucene-sandbox-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-queryparser-7.3.1 .jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-queries-7.3.1.jar:/root/data/skywalking/apache-skyw alking-apm-bin/oap-libs/lucene-misc-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-memory-7.3 .1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-join-7.3.1.jar:/root/data/skywalking/apache-skywa lking-apm-bin/oap-libs/lucene-highlighter-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-grou ping-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/lucene-core-7.3.1.jar:/root/data/skywalking/apac he-skywalking-apm-bin/oap-libs/lucene-backward-codecs-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs /lucene-analyzers-common-7.3.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/logging-interceptor-2.7.5.ja r:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/log4j-slf4j-impl-2.9.0.jar:/root/data/skywalking/apache-skywa lking-apm-bin/oap-libs/log4j-over-slf4j-1.7.25.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/log4j-core-2 .9.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/log4j-api-2.9.0.jar:/root/data/skywalking/apache-skywa lking-apm-bin/oap-libs/log4j-1.2.16.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/library-util-6.1.0.jar: /root/data/skywalking/apache-skywalking-apm-bin/oap-libs/library-server-6.1.0.jar:/root/data/skywalking/apache-skywalkin g-apm-bin/oap-libs/library-module-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/library-client-6.1. 0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/library-buffer-6.1.0.jar:/root/data/skywalking/apache-sky walking-apm-bin/oap-libs/kotlin-stdlib-1.1.60.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/kotlin-reflec t-1.1.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jsr305-1.3.9.jar:/root/data/skywalking/apache-skywa lking-apm-bin/oap-libs/jopt-simple-4.6.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/joda-time-2.9.9.jar: /root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jna-4.5.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/o ap-libs/jline-0.9.94.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jetty-util-9.4.2.v20170220.jar:/root/d ata/skywalking/apache-skywalking-apm-bin/oap-libs/jetty-servlet-9.4.2.v20170220.jar:/root/data/skywalking/apache-skywalk ing-apm-bin/oap-libs/jetty-server-9.4.2.v20170220.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jetty-sec urity-9.4.2.v20170220.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jetty-io-9.4.2.v20170220.jar:/root/da ta/skywalking/apache-skywalking-apm-bin/oap-libs/jetty-http-9.4.2.v20170220.jar:/root/data/skywalking/apache-skywalking- apm-bin/oap-libs/jcl-over-slf4j-1.7.25.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/javax.servlet-api-3. 1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/java-dataloader-2.0.2.jar:/root/data/skywalking/apache- skywalking-apm-bin/oap-libs/jaeger-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/ja ckson-module-kotlin-2.8.8.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson-mapper-asl-1.9.13.jar:/ro ot/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson-datatype-jdk8-2.9.5.jar:/root/data/skywalking/apache-skywa lking-apm-bin/oap-libs/jackson-datatype-guava-2.9.5.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson -dataformat-yaml-2.8.10.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson-dataformat-smile-2.8.10.jar :/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson-dataformat-cbor-2.8.10.jar:/root/data/skywalking/apach e-skywalking-apm-bin/oap-libs/jackson-databind-2.9.5.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackso n-core-asl-1.9.13.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/jackson-core-2.9.5.jar:/root/data/skywalk ing/apache-skywalking-apm-bin/oap-libs/jackson-annotations-2.9.5.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap -libs/httpcore-nio-4.4.5.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/httpcore-4.4.5.jar:/root/data/skyw alking/apache-skywalking-apm-bin/oap-libs/httpclient-4.5.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/ httpasyncclient-4.1.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/hppc-0.7.1.jar:/root/data/skywalking/ apache-skywalking-apm-bin/oap-libs/HikariCP-3.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/HdrHistog ram-2.1.9.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/guava-20.0.jar:/root/data/skywalking/apache-skywa lking-apm-bin/oap-libs/gson-2.8.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/grpc-stub-1.15.1.jar:/roo t/data/skywalking/apache-skywalking-apm-bin/oap-libs/grpc-protobuf-lite-1.15.1.jar:/root/data/skywalking/apache-skywalki ng-apm-bin/oap-libs/grpc-protobuf-1.15.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/grpc-netty-1.15.1. jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/grpc-core-1.15.1.jar:/root/data/skywalking/apache-skywalkin g-apm-bin/oap-libs/grpc-context-1.15.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/groovy-2.4.5-indy.ja r:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/graphql-java-tools-5.2.3.jar:/root/data/skywalking/apache-sky walking-apm-bin/oap-libs/graphql-java-8.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/generated-analysi s-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/exporter-6.1.0.jar:/root/data/skywalking/apache-sky walking-apm-bin/oap-libs/error_prone_annotations-2.2.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/envo y-metrics-receiver-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/elasticsearch-x-content-6.3 .2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/elasticsearch-secure-sm-6.3.2.jar:/root/data/skywalking/ apache-skywalking-apm-bin/oap-libs/elasticsearch-rest-high-level-client-6.3.2.jar:/root/data/skywalking/apache-skywalkin g-apm-bin/oap-libs/elasticsearch-rest-client-6.3.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/elastics earch-core-6.3.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/elasticsearch-cli-6.3.2.jar:/root/data/sky walking/apache-skywalking-apm-bin/oap-libs/elasticsearch-6.3.2.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-l ibs/curator-x-discovery-4.0.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/curator-recipes-4.0.1.jar:/ro ot/data/skywalking/apache-skywalking-apm-bin/oap-libs/curator-framework-4.0.1.jar:/root/data/skywalking/apache-skywalkin g-apm-bin/oap-libs/curator-client-4.0.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/converter-jackson-2 .3.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/consul-client-1.2.6.jar:/root/data/skywalking/apache-s kywalking-apm-bin/oap-libs/commons-pool-1.5.4.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/commons-lang3 -3.7.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/commons-io-2.6.jar:/root/data/skywalking/apache-skywal king-apm-bin/oap-libs/commons-dbcp-1.4.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/commons-codec-1.11.j ar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/cluster-zookeeper-plugin-6.1.0.jar:/root/data/skywalking/apa che-skywalking-apm-bin/oap-libs/cluster-standalone-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap- libs/cluster-kubernetes-plugin-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/cluster-consul-plugin- 6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/client-java-proto-2.0.0.jar:/root/data/skywalking/apa che-skywalking-apm-bin/oap-libs/client-java-api-2.0.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/clien t-java-2.0.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/caffeine-2.6.2.jar:/root/data/skywalking/apach e-skywalking-apm-bin/oap-libs/bcprov-jdk15on-1.59.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/bcprov-ex t-jdk15on-1.59.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/bcpkix-jdk15on-1.59.jar:/root/data/skywalkin g/apache-skywalking-apm-bin/oap-libs/apm-util-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/apm-net work-6.1.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/apm-datacarrier-6.1.0.jar:/root/data/skywalking/ apache-skywalking-apm-bin/oap-libs/antlr4-runtime-4.7.1.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/ann otations-13.0.jar:/root/data/skywalking/apache-skywalking-apm-bin/oap-libs/animal-sniffer-annotations-1.17.jar:/root/dat a/skywalking/apache-skywalking-apm-bin/oap-libs/aggs-matrix-stats-client-6.3.2.jar:/root/data/skywalking/apache-skywalki ng-apm-bin/config:.:/opt/java/lib/dt.jar:/opt/java/lib/tools.jar BOOT-CLASS-PATH /opt/jdk1.8.0_201/jre/lib/resources.jar:/opt/jdk1.8.0_201/jre/lib/rt.jar:/opt/jdk1.8.0_201/jre/lib/sunrsasign.jar:/opt/j dk1.8.0_201/jre/lib/jsse.jar:/opt/jdk1.8.0_201/jre/lib/jce.jar:/opt/jdk1.8.0_201/jre/lib/charsets.jar:/opt/jdk1.8.0_201/ jre/lib/jfr.jar:/opt/jdk1.8.0_201/jre/classes LIBRARY-PATH /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CLASS-LOADING --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- LOADED-CLASS-COUNT 4478 TOTAL-LOADED-CLASS-COUNT 4478 UNLOADED-CLASS-COUNT 0 IS-VERBOSE false --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- COMPILATION --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- NAME HotSpot 64-Bit Tiered Compilers TOTAL-COMPILE-TIME 8097(ms) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- GARBAGE-COLLECTORS --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- PS Scavenge 29/788(ms) [count/time] PS MarkSweep 301/49445(ms) [count/time] --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- MEMORY-MANAGERS --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CodeCacheManager Code Cache Metaspace Manager Metaspace Compressed Class Space PS Scavenge PS Eden Space PS Survivor Space PS MarkSweep PS Eden Space PS Survivor Space PS Old Gen --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- MEMORY --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- HEAP-MEMORY-USAGE 1000865792(954.50 MiB)/1073741824(1.00 GiB)/1000865792(954.50 MiB)/930960160(887.83 MiB) [committed/init/max/used] NO-HEAP-MEMORY-USAGE 34603008(33.00 MiB)/2555904(2.44 MiB)/-1(-1 B)/33061200(31.53 MiB) [committed/init/max/used] PENDING-FINALIZE-COUNT 0 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- OPERATING-SYSTEM --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- OS Linux ARCH amd64 PROCESSORS-COUNT 4 LOAD-AVERAGE 4.47 VERSION 3.10.0-957.21.3.el7.x86_64 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- THREAD --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- COUNT 18 DAEMON-COUNT 12 PEAK-COUNT 20 STARTED-COUNT 23 DEADLOCK-COUNT 0 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- FILE-DESCRIPTOR --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- MAX-FILE-DESCRIPTOR-COUNT 65536 OPEN-FILE-DESCRIPTOR-COUNT 244 Affect(row-cnt:0) cost in 55728 ms. $ $ Connection closed by foreign host. Then I looked at the oap-lib directory, which contains the following files: # root @ centos0 in ~/data/skywalking/apache-skywalking-apm-bin/oap-libs [13:10:10] $ ls aggs-matrix-stats-client-6.3.2.jar httpcore-4.4.5.jar mysql-connector-java-5.1.47.jar animal-sniffer-annotations-1.17.jar httpcore-nio-4.4.5.jar netty-3.10.5.Final.jar annotations-13.0.jar jackson-annotations-2.9.5.jar netty-buffer-4.1.27.Final.jar antlr4-runtime-4.7.1.jar jackson-core-2.9.5.jar netty-codec-4.1.27.Final.jar apm-datacarrier-6.1.0.jar jackson-core-asl-1.9.13.jar netty-codec-http2-4.1.27.Final.jar apm-network-6.1.0.jar jackson-databind-2.9.5.jar netty-codec-http-4.1.27.Final.jar apm-util-6.1.0.jar jackson-dataformat-cbor-2.8.10.jar netty-codec-socks-4.1.27.Final.jar bcpkix-jdk15on-1.59.jar jackson-dataformat-smile-2.8.10.jar netty-common-4.1.27.Final.jar bcprov-ext-jdk15on-1.59.jar jackson-dataformat-yaml-2.8.10.jar netty-handler-4.1.27.Final.jar bcprov-jdk15on-1.59.jar jackson-datatype-guava-2.9.5.jar netty-handler-proxy-4.1.27.Final.jar caffeine-2.6.2.jar jackson-datatype-jdk8-2.9.5.jar netty-resolver-4.1.27.Final.jar client-java-2.0.0.jar jackson-mapper-asl-1.9.13.jar netty-tcnative-boringssl-static-2.0.7.Final.jar client-java-api-2.0.0.jar jackson-module-kotlin-2.8.8.jar netty-transport-4.1.27.Final.jar client-java-proto-2.0.0.jar jaeger-receiver-plugin-6.1.0.jar okhttp-2.7.5.jar cluster-consul-plugin-6.1.0.jar java-dataloader-2.0.2.jar okhttp-3.9.0.jar cluster-kubernetes-plugin-6.1.0.jar javax.servlet-api-3.1.0.jar okhttp-ws-2.7.5.jar cluster-standalone-plugin-6.1.0.jar jcl-over-slf4j-1.7.25.jar okio-1.13.0.jar cluster-zookeeper-plugin-6.1.0.jar jetty-http-9.4.2.v20170220.jar opencensus-api-0.12.3.jar commons-codec-1.11.jar jetty-io-9.4.2.v20170220.jar opencensus-contrib-grpc-metrics-0.12.3.jar commons-dbcp-1.4.jar jetty-security-9.4.2.v20170220.jar parent-join-client-6.3.2.jar commons-io-2.6.jar jetty-server-9.4.2.v20170220.jar protobuf-java-3.4.0.jar commons-lang3-3.7.jar jetty-servlet-9.4.2.v20170220.jar proto-google-common-protos-1.0.0.jar commons-pool-1.5.4.jar jetty-util-9.4.2.v20170220.jar query-graphql-plugin-6.1.0.jar consul-client-1.2.6.jar jline-0.9.94.jar rank-eval-client-6.3.2.jar converter-jackson-2.3.0.jar jna-4.5.1.jar reactive-streams-1.0.2.jar curator-client-4.0.1.jar joda-time-2.9.9.jar receiver-proto-6.1.0.jar curator-framework-4.0.1.jar jopt-simple-4.6.jar reflectasm-1.11.7.jar curator-recipes-4.0.1.jar jsr305-1.3.9.jar retrofit-2.3.0.jar curator-x-discovery-4.0.1.jar kotlin-reflect-1.1.1.jar server-alarm-plugin-6.1.0.jar elasticsearch-6.3.2.jar kotlin-stdlib-1.1.60.jar server-core-6.1.0.jar elasticsearch-cli-6.3.2.jar library-buffer-6.1.0.jar server-starter-6.1.0.jar elasticsearch-core-6.3.2.jar library-client-6.1.0.jar sharding-jdbc-core-2.0.3.jar elasticsearch-rest-client-6.3.2.jar library-module-6.1.0.jar simpleclient-0.6.0.jar elasticsearch-rest-high-level-client-6.3.2.jar library-server-6.1.0.jar simpleclient_common-0.6.0.jar elasticsearch-secure-sm-6.3.2.jar library-util-6.1.0.jar simpleclient_hotspot-0.6.0.jar elasticsearch-x-content-6.3.2.jar log4j-1.2.16.jar simpleclient_httpserver-0.6.0.jar envoy-metrics-receiver-plugin-6.1.0.jar log4j-api-2.9.0.jar skywalking-clr-receiver-plugin-6.1.0.jar error_prone_annotations-2.2.0.jar log4j-core-2.9.0.jar skywalking-jvm-receiver-plugin-6.1.0.jar exporter-6.1.0.jar log4j-over-slf4j-1.7.25.jar skywalking-mesh-receiver-plugin-6.1.0.jar generated-analysis-6.1.0.jar log4j-slf4j-impl-2.9.0.jar skywalking-register-receiver-plugin-6.1.0.jar graphql-java-8.0.jar logging-interceptor-2.7.5.jar skywalking-sharing-server-plugin-6.1.0.jar graphql-java-tools-5.2.3.jar lucene-analyzers-common-7.3.1.jar skywalking-trace-receiver-plugin-6.1.0.jar groovy-2.4.5-indy.jar lucene-backward-codecs-7.3.1.jar slf4j-api-1.7.25.jar grpc-context-1.15.1.jar lucene-core-7.3.1.jar snakeyaml-1.18.jar grpc-core-1.15.1.jar lucene-grouping-7.3.1.jar storage-elasticsearch-plugin-6.1.0.jar grpc-netty-1.15.1.jar lucene-highlighter-7.3.1.jar storage-jaeger-plugin-6.1.0.jar grpc-protobuf-1.15.1.jar lucene-join-7.3.1.jar storage-jdbc-hikaricp-plugin-6.1.0.jar grpc-protobuf-lite-1.15.1.jar lucene-memory-7.3.1.jar storage-zipkin-plugin-6.1.0.jar grpc-stub-1.15.1.jar lucene-misc-7.3.1.jar swagger-annotations-1.5.12.jar gson-2.8.1.jar lucene-queries-7.3.1.jar t-digest-3.2.jar guava-20.0.jar lucene-queryparser-7.3.1.jar telemetry-api-6.1.0.jar HdrHistogram-2.1.9.jar lucene-sandbox-7.3.1.jar telemetry-prometheus-6.1.0.jar HikariCP-3.1.0.jar lucene-spatial3d-7.3.1.jar zipkin-2.9.1.jar hppc-0.7.1.jar lucene-spatial-7.3.1.jar zipkin-receiver-plugin-6.1.0.jar httpasyncclient-4.1.2.jar lucene-spatial-extras-7.3.1.jar zookeeper-3.4.10.jar httpclient-4.5.2.jar lucene-suggest-7.3.1.jar ``` [email protected] From: [email protected] Date: 2019-06-29 12:02 To: reply+abtzq63yj4gngmgzzia6r753eqhh3evbnhhbxdmihi Subject: Re: Re: [apache/skywalking] OutOfMemoryError by com.google.common.reflect.ClassPath.from (#2969) Hi Wu Sheng: I ran skywalking in another linux environment, and it worked. Jdk: java version "1.8.0_201" Java(TM) SE Runtime Environment (build 1.8.0_201-b09) Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode) But when using the same JDK to run in the faulty machine, it will not work. It should be a problem of the OS system. I unzipped skywalking to the directory /root/data/skywalking/apache-skywalking-apm-bin/， and changed the following file: application.yml log4j2.xml I put the document in the attachment. [email protected] From: 吴晟 Wu Sheng Date: 2019-06-29 11:33 To: apache/skywalking CC: zeroall; Author Subject: Re: [apache/skywalking] OutOfMemoryError by com.google.common.reflect.ClassPath.from (#2969) This is new to me. We have just tested the features in different
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
