业务场景：
每隔几分钟执行一次批量任务：通过 sql 从 s3 中读数据，经过处理后将数据写入到 clickhouse 和 kafka
执行一定次数后 task-manager 会报 OutOfMemoryError: Metaspace 问题
环境信息：
flink 版本：1.12.2
部署方式： standalone kubernetes session 模式
依赖信息：
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka_2.11</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.5</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-jdbc_2.11</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>ru.yandex.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<version>0.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-parquet_2.11</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-json</artifactId>
<version>${flink.version}</version>
</dependency>
heap dump1:
Leak Suspects
System Overview
Leaks
Overview
Problem Suspect 1
21 instances of "org.apache.flink.util.ChildFirstClassLoader", loaded
by "sun.misc.Launcher$AppClassLoader @
0x73b2d42e0" occupy 29,656,880 (41.16%) bytes.
Biggest instances:
org.apache.flink.util.ChildFirstClassLoader @ 0x73ca2a1e8 - 1,474,760 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73d2af820 - 1,474,168 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73cdcaa10 - 1,474,160 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73cf6aab0 - 1,474,160 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73d1111d8 - 1,474,160 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73d2bb108 - 1,474,128 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73de202e0 - 1,474,120 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73dadc778 - 1,474,112 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73d5f70e8 - 1,474,064 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73d93aa38 - 1,474,064 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73e179638 - 1,474,064 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73dc80418 - 1,474,056 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73dfcda60 - 1,474,056 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73e4bcd38 - 1,474,056 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73d6006e8 - 1,474,032 (2.05%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73c7d2ad8 - 1,461,944 (2.03%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73ca1bb98 - 1,460,752 (2.03%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73bf203f0 - 1,460,744 (2.03%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73e3284a8 - 1,445,232 (2.01%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73e65de00 - 1,445,232 (2.01%)
bytes.
Keywords
org.apache.flink.util.ChildFirstClassLoader
sun.misc.Launcher$AppClassLoader @ 0x73b2d42e0
Details »
Problem Suspect 2
34,407 instances of "org.apache.flink.core.memory.HybridMemorySegment",
loaded by "sun.misc.Launcher$AppClassLoader @
0x73b2d42e0" occupy 7,707,168 (10.70%) bytes.
Keywords
org.apache.flink.core.memory.HybridMemorySegment
sun.misc.Launcher$AppClassLoader @ 0x73b2d42e0
Details »
heap dump2:
Leak Suspects
System Overview
Leaks
Overview
Problem Suspect 1
21 instances of "org.apache.flink.util.ChildFirstClassLoader", loaded
by "sun.misc.Launcher$AppClassLoader @
0x73b2d42e0" occupy 26,061,408 (30.68%) bytes.
Biggest instances:
org.apache.flink.util.ChildFirstClassLoader @ 0x73e9e9930 - 1,474,224 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73edce0b8 - 1,474,224 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73f1ad7d0 - 1,474,168 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73f3e5118 - 1,474,168 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73f5d3fe0 - 1,474,168 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73ebd8d28 - 1,474,160 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73efc00c0 - 1,474,160 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73e2251a8 - 1,474,136 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73cc24af0 - 1,474,064 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73cdca3e0 - 1,474,064 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73cf6f860 - 1,474,064 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73d114768 - 1,474,064 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73ca6f878 - 1,474,056 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73d2b7640 - 1,474,056 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73d2c1d80 - 1,474,040 (1.74%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73c7e2868 - 1,469,720 (1.73%)
bytes.
org.apache.flink.util.ChildFirstClassLoader @ 0x73bf34a98 - 1,460,808 (1.72%)
bytes.
Keywords
org.apache.flink.util.ChildFirstClassLoader
sun.misc.Launcher$AppClassLoader @ 0x73b2d42e0
Details »
Problem Suspect 2
4 instances
of "org.apache.flink.streaming.runtime.tasks.OneInputStreamTask", loaded
by "sun.misc.Launcher$AppClassLoader @
0x73b2d42e0" occupy 11,644,200 (13.71%) bytes.
Biggest instances:
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask @ 0x73e2d0cb0 -
4,364,536 (5.14%) bytes.
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask @ 0x73d62fb88 -
3,643,576 (4.29%) bytes.
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask @ 0x73dae0270 -
3,635,952 (4.28%) bytes.
Keywords
sun.misc.Launcher$AppClassLoader @ 0x73b2d42e0
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask
Details »
