GJL commented on issue #7185: [FLINK-10884] [yarn/mesos] adjust container memory param to set a safe margin from offheap memory URL: https://github.com/apache/flink/pull/7185#issuecomment-494271894 Sorry, I am very late to the game. For YARN TaskManager containers, _-XX:MaxDirectMemorySize_ is set to `offHeapSizeMB`: https://github.com/apache/flink/blob/3fba21a0ce5534b8c95a2bb9e8547b0061e5bb8f/flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/BootstrapTools.java#L430-L433 We could introduce an offheap cutoff. However, I think @NicoK's comment addresses a valid point here: > Having an actual limit (as is used through ContaineredTaskManagerParameters) will only result in a different error in case you are hitting it: java.lang.OutOfMemoryError instead of being killed by YARN/Mesos/etc. That may be better for diagnosing it but doesn't solve the root cause of something using up too much memory. The bottom line is that unless the TM's offheap memory allocations respect _-XX: MaxDirectMemorySize_, you will probably run into another error. Please correct me if I am wrong. @wg1026688210: Did you try out your changes in production, and did it solve your problem? Were you able to work around your problem otherwise, e.g., by increasing the total container memory?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
