[ https://issues.apache.org/jira/browse/FLINK-30081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642100#comment-17642100 ]
Mingliang Liu commented on FLINK-30081: --------------------------------------- Thanks [~gaoyunhaii] It makes perfect sense to me now those options are not honored by the mini-cluster. But it is still not ideal to fail the local job instead of just ignoring those options. In my day job, many users develop, test and debug locally based on the same configurations. They override only if required (e.g. parallelism). We pass the configurations via Spring Boot environments, but I guess using flink-conf.yaml file would have similar issues. Suppose the base flink-conf.yaml file has different min/max values, it is a burden for users to override those config values for local testing which turns out to be ignored. We can either relax the check in the configuration, or just remove them from configuration in mini-cluster environment. Thoughts? > Local executor can not accept different jvm-overhead.min/max values > ------------------------------------------------------------------- > > Key: FLINK-30081 > URL: https://issues.apache.org/jira/browse/FLINK-30081 > Project: Flink > Issue Type: Improvement > Components: Runtime / Configuration > Affects Versions: 1.16.0 > Reporter: Mingliang Liu > Priority: Major > > In local executor, it's not possible to set different values for > {{taskmanager.memory.jvm-overhead.max}} and > {{{}taskmanager.memory.jvm-overhead.min{}}}. The same problem for > {{taskmanager.memory.network.max}} and {{{}taskmanager.memory.network.min{}}}. > Sample code to reproduce: > {code:java} > Configuration conf = new Configuration(); > conf.setString(TaskManagerOptions.JVM_OVERHEAD_MIN.key(), "1GB"); > conf.setString(TaskManagerOptions.JVM_OVERHEAD_MAX.key(), "2GB"); > StreamExecutionEnvironment.createLocalEnvironment(conf) > .fromElements("Hello", "World") > .executeAndCollect() > .forEachRemaining(System.out::println); {code} > The failing exception is something like: > {code:java} > Exception in thread "main" java.lang.IllegalArgumentException > at org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:122) > at > org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.calculateTotalProcessMemoryFromComponents(TaskExecutorResourceUtils.java:182) > at > org.apache.flink.runtime.taskexecutor.TaskExecutorMemoryConfiguration.create(TaskExecutorMemoryConfiguration.java:119) > {code} > I think the problem was that we expect the max and min to equal, but local > executor did not reset them correctly? -- This message was sent by Atlassian Jira (v8.20.10#820010)