zxz666 opened a new issue, #9349: URL: https://github.com/apache/skywalking/issues/9349
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar issues. ### Apache SkyWalking Component OAP server (apache/skywalking) ### What happened hi,some errors logs when i startup skywalking 9.1.0. ### What you expected to happen # # There is insufficient memory for the Java Runtime Environment to continue. # Cannot create worker GC thread. Out of system resources. # Possible reasons: # The system is out of physical RAM or swap space # The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap # Possible solutions: # Reduce memory load on the system # Increase physical memory or swap space # Check if swap backing store is full # Decrease Java heap size (-Xmx/-Xms) # Decrease number of Java threads # Decrease Java thread stack sizes (-Xss) # Set larger code cache with -XX:ReservedCodeCacheSize= # JVM is running with Unscaled Compressed Oops mode in which the Java heap is # placed in the first 4GB address space. The Java Heap base address is the # maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress # to set the Java Heap base and to place the Java Heap above 4GB virtual address. # This output file may be truncated or incomplete. # # Out of Memory Error (workerManager.hpp:70), pid=1, tid=1 # # JRE version: (11.0.15+10) (build ) # Java VM: OpenJDK 64-Bit Server VM (11.0.15+10, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64) # Core dump will be written. Default location: /skywalking/core.1 # --------------- S U M M A R Y ------------ Command Line: -Xms2G org.apache.skywalking.oap.server.starter.OAPServerStartUp Host: Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz, 4 cores, 15G, Ubuntu 22.04 LTS Time: Mon Jul 18 13:59:22 2022 CST elapsed time: 0.004166 seconds (0d 0h 0m 0s) --------------- T H R E A D --------------- Current thread (0x0000558d4e315000): JavaThread "Unknown thread" [_thread_in_vm, id=1, stack(0x00007ffc44e7f000,0x00007ffc4567c000)] Stack: [0x00007ffc44e7f000,0x00007ffc4567c000], sp=0x00007ffc45676410, free space=8157k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0xf424ca] VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x1ca V [libjvm.so+0xf4338b] VMError::report_and_die(Thread*, char const*, int, unsigned long, VMErrorType, char const*, __va_list_tag*)+0x2b V [libjvm.so+0x6cf005] report_vm_out_of_memory(char const*, int, unsigned long, VMErrorType, char const*, ...)+0xd5 V [libjvm.so+0xf6f58b] AbstractWorkGang::add_workers(unsigned int, bool)+0x14b V [libjvm.so+0x7aa8c5] G1CollectedHeap::G1CollectedHeap(G1CollectorPolicy*)+0x5b5 V [libjvm.so+0x79c3b9] G1Arguments::create_heap()+0x69 V [libjvm.so+0xefdc11] Universe::initialize_heap()+0x31 V [libjvm.so+0xefdec6] universe_init()+0x46 V [libjvm.so+0x8a4207] init_globals()+0x57 V [libjvm.so+0xed5537] Threads::create_vm(JavaVMInitArgs*, bool*)+0x357 V [libjvm.so+0x960262] JNI_CreateJavaVM+0x52 C [libjli.so+0x49af] JavaMain+0x8f C [libjli.so+0x9d94] CallJavaMainInNewThread+0x74 "hs_err_pid1.log" 362L, 22259C number of Java threads # Decrease Java thread stack sizes (-Xss) # Set larger code cache with -XX:ReservedCodeCacheSize= # JVM is running with Unscaled Compressed Oops mode in which the Java heap is # placed in the first 4GB address space. The Java Heap base address is the # maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress # to set the Java Heap base and to place the Java Heap above 4GB virtual address. # This output file may be truncated or incomplete. # # Out of Memory Error (workerManager.hpp:70), pid=1, tid=1 # # JRE version: (11.0.15+10) (build ) # Java VM: OpenJDK 64-Bit Server VM (11.0.15+10, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64) # Core dump will be written. Default location: /skywalking/core.1 # --------------- S U M M A R Y ------------ Command Line: -Xms2G org.apache.skywalking.oap.server.starter.OAPServerStartUp Host: Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz, 4 cores, 15G, Ubuntu 22.04 LTS Time: Mon Jul 18 13:59:22 2022 CST elapsed time: 0.004166 seconds (0d 0h 0m 0s) --------------- T H R E A D --------------- Current thread (0x0000558d4e315000): JavaThread "Unknown thread" [_thread_in_vm, id=1, stack(0x00007ffc44e7f000,0x00007ffc4567c000)] Stack: [0x00007ffc44e7f000,0x00007ffc4567c000], sp=0x00007ffc45676410, free space=8157k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0xf424ca] VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x1ca V [libjvm.so+0xf4338b] VMError::report_and_die(Thread*, char const*, int, unsigned long, VMErrorType, char const*, __va_list_tag*)+0x2b V [libjvm.so+0x6cf005] report_vm_out_of_memory(char const*, int, unsigned long, VMErrorType, char const*, ...)+0xd5 V [libjvm.so+0xf6f58b] AbstractWorkGang::add_workers(unsigned int, bool)+0x14b V [libjvm.so+0x7aa8c5] G1CollectedHeap::G1CollectedHeap(G1CollectorPolicy*)+0x5b5 V [libjvm.so+0x79c3b9] G1Arguments::create_heap()+0x69 V [libjvm.so+0xefdc11] Universe::initialize_heap()+0x31 V [libjvm.so+0xefdec6] universe_init()+0x46 V [libjvm.so+0x8a4207] init_globals()+0x57 V [libjvm.so+0xed5537] Threads::create_vm(JavaVMInitArgs*, bool*)+0x357 V [libjvm.so+0x960262] JNI_CreateJavaVM+0x52 C [libjli.so+0x49af] JavaMain+0x8f C [libjli.so+0x9d94] CallJavaMainInNewThread+0x74 C [libjli.so+0x637d] ContinueInNewThread+0x4d C [libjli.so+0x7a00] JLI_Launch+0x1470 C [java+0xa73] --------------- P R O C E S S --------------- Threads class SMR info: _java_thread_list=0x0000558d4e2d8770, length=0, elements={ Java Threads: ( => current thread ) Other Threads: =>0x0000558d4e315000 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=1, stack(0x00007ffc44e7f000,0x00007ffc4567c000)] Threads with active compile tasks: VM state:not at safepoint (not fully initialized) VM Mutex/Monitor currently owned by a thread: None [error occurred during error reporting (printing compressed oops mode), id 0xb, SIGSEGV (0xb) at pc=0x00007f10f9afc543] GC Heap History (0 events): No events Deoptimization events (0 events): No events Classes redefined (0 events): No events Internal exceptions (0 events): No events Events (2 events): Event: 0.002 Protecting memory [0x00007ffc44e7f000,0x00007ffc44e83000] with protection modes 0 Event: 0.002 Loaded shared library /opt/java/openjdk/lib/libzip.so Dynamic libraries: 558d4d2a6000-558d4d2a7000 r-xp 00000000 fd:11 30937083 /opt/java/openjdk/bin/java 558d4d4a7000-558d4d4a8000 r--p 00001000 fd:11 30937083 /opt/java/openjdk/bin/java 558d4d4a8000-558d4d4a9000 rw-p 00002000 fd:11 30937083 /opt/java/openjdk/bin/java 558d4e2d2000-558d4e355000 rw-p 00000000 00:00 0 [heap] 7f10e2d20000-7f10e30dd000 rw-p 00000000 00:00 0 7f10e30dd000-7f10e334d000 rwxp 00000000 00:00 0 7f10e334d000-7f10e366d000 ---p 00000000 00:00 0 7f10e366d000-7f10e38dd000 rwxp 00000000 00:00 0 7f10e38dd000-7f10eaba5000 ---p 00000000 00:00 0 7f10eaba5000-7f10eae15000 rwxp 00000000 00:00 0 7f10eae15000-7f10f20dd000 ---p 00000000 00:00 0 7f10f20dd000-7f10f7c00000 r--s 00000000 fd:11 30937404 /opt/java/openjdk/lib/modules 7f10f7c00000-7f10f7c07000 r-xp 00000000 fd:11 30937403 /opt/java/openjdk/lib/libzip.so 7f10f7c07000-7f10f7e07000 ---p 00007000 fd:11 30937403 /opt/java/openjdk/lib/libzip.so 7f10f7e07000-7f10f7e08000 r--p 00007000 fd:11 30937403 /opt/java/openjdk/lib/libzip.so 7f10f7e08000-7f10f7e09000 rw-p 00008000 fd:11 30937403 /opt/java/openjdk/lib/libzip.so ### How to reproduce I use docker-compose to start up skywalking. version: '3' services: skywalking-oap: image: apache/skywalking-oap-server:9.1.0 container_name: skywalking-oap restart: always ports: - 11800:11800 - 12800:12800 environment: SW_STORAGE: elasticsearch SW_STORAGE_ES_CLUSTER_NODES: xx.xx.xx.xx:9200 SW_HEALTH_CHECKER: default SW_TELEMETRY: prometheus TZ: Asia/Shanghai JAVA_OPTS: "-Xms4096m -Xmx4096m" healthcheck: test: ["CMD", "./bin/swctl", "ch"] interval: 30s timeout: 10s retries: 3 start_period: 40s skywalking-ui: image: apache/skywalking-ui:9.1.0 container_name: skywalking-ui depends_on: - skywalking-oap links: - skywalking-oap restart: always ports: - 3080:8080 environment: SW_OAP_ADDRESS: xx.xx.xx.xx:12800 ### Anything else And i think have enough memory in centos. total used free shared buff/cache available Mem: 15615 3356 8493 0 3766 11929 Swap: 0 0 0 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
