Hi, Theodor, The description in https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/deployment/config/#memory-configuration map help you to config the memory for flink.
> Flink tries to shield users as much as possible from the complexity of > configuring the JVM for data-intensive processing. In most cases, users > should only need to set the values taskmanager.memory.process.size or > taskmanager.memory.flink.size (depending on how the setup), and possibly > adjusting the ratio of JVM heap and Managed Memory via > taskmanager.memory.managed.fraction. The other options below can be used > for performance tuning and fixing memory related errors. > I think maybe you should set the taskmanager.memory.process.size or taskmanager.memory.flink.size, or increase the memory of the container. Hope this helps! Best, Hang Theodor Wübker <theo.wueb...@inside-m2m.de> 于2023年2月2日周四 23:37写道： > Hello everyone, > > I have a Standalone Custer running in a docker-swarm with a very simple > docker-compose configuration [3]. When I run my job there with a > parallelism greater than one, I get an out of memory error. Nothing out of > the ordinary, so I wanted to increase the JVM heap. I did that by setting > ‘taskmanager.memory.task.heap.size’ according to [1]. However the > taskmanager would not start, throwing an Exception saying that this > configuration clashes with the configured total process memory - even > though I had not configured that at all. Due to this, I could also not set > the total Flink memory. > Now I wonder, why did the TM tell me that the total process memory is > already configured? Also, in [2] I read that the cluster should not even > start when neither total Flink memory nor total process memory are > specified - which, as you can see in my configuration, I have not done [3]. > > Maybe someone can enlighten me, why it looks like I can’t configure the > memory properly? Thanks :) > > -Theo > > [1] > https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/memory/mem_setup_tm/#configure-heap-and-managed-memory > > [2] > https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/memory/mem_setup/#configure-total-memory > > [3] The compose configuration: > > jobmanager: > image: flink:1.16.0 > command: jobmanager > environment: > - TASK_MANAGER_NUMBER_OF_TASK_SLOTS=2 > - | > FLINK_PROPERTIES= > jobmanager.rpc.address: jobmanager > > > taskmanager-01: > image: flink:1.16.0 > depends_on: > - jobmanager > command: taskmanager > environment: > - | > FLINK_PROPERTIES= > jobmanager.rpc.address: jobmanager > taskmanager.numberOfTaskSlots: 2 >
