Given the numbers above from JCMD. You think I should be ok with 2GB
metaspace? That was captured while all jobs where running on the cluster
for that 1 node.
I set it to 2GB. But none of the above numbers indicated max 2GB metaspace.
On Mon., Dec. 27, 2021, 10:47 a.m. John Smith,
wrote:
> Ok
Ok all settings above are for smaller dev cluster and I'm experimenting to
set metasize to 2GB. It runs same jobs as production just less volume in
terms of data.
The below snapshot of JCMD are of a slightly bigger task manager and the
active cluster... It also once in a while does metaspace so
Yes standalone cluster. 3 zoo, 3 job, 3 tasks.
The task managers have taskslots at double core. So 2*4
I think metaspace of 2GB is ok. I'll try to get some jcmd stats.
The jobs are fairly straight forward ETL they read from Kafka, do some json
parsing, using vertx.io json parser and either
Hi John,
Sounds to me you have a Flink standalone cluster deployed directly on
physical hosts. If that is the case, use `t.m.flink.size` instead of
`t.m.process.size`. The latter does not limit the overall memory
consumption of the processes, and is only used for calculating how much
non-JVM
Ok I tried taskmanager.memory.process.size: 7168m
It's worst, the task manager can barely start before it throws
java.lang.OutOfMemoryError: Metaspace
I will try...
taskmanager.memory.flink.size: 5120m
taskmanager.memory.jvm-metaspace.size: 2048m
On Sun, 26 Dec 2021 at 19:46, John Smith
Hi running Flink 1.10
I have
taskmanager.memory.flink.size: 6144m
taskmanager.memory.jvm-metaspace.size: 1024m
taskmanager.numberOfTaskSlots: 8
parallelism.default: 1
1- The host has a physical ram of 8GB. I'm better off just to configure
"taskmanager.memory.process.size" as 7GB and let flink
