[
https://issues.apache.org/jira/browse/FLINK-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416225#comment-16416225
]
Kirill A. Korinskiy commented on FLINK-8809:
--------------------------------------------
[~greghogan] yes, I discovered this "infinite" value after OOM Killer killed
java with flink, and the loop that I explained before started on the system.
After I decreased it ti to reasonable value about month ago, everything looks
normal until today.
My application use a lot of States and as result memory, and it runs on two
type of nodes with 64 and 96 GB ram.
Java memory settings (in megabytes):
- 64: jvm_memory=50790; direct_memory=2048
- 96: jvm_memory=76455; direct_memory=3072
It is enough to consume up to 1-1.5 Gbit from kafka and produce to kafka
results at my case.
Idea with {{jdk.nio.maxCachedBufferSize}} is interesting but it adds one more
settings to this configuration, and after my changes it stoped try to allocate
more memory that system has.
I can say that I see easy way to solve this by:
- adding comfortable configuration settings;
- setup it to something likes 2-3-4 Gb by default.
As result users may easy change this value if it doesn't big enough over
configuration, without understanding how start scripts works and find
workaround that I explained at the issue.
> Decrease maximum value of DirectMemory at default config
> --------------------------------------------------------
>
> Key: FLINK-8809
> URL: https://issues.apache.org/jira/browse/FLINK-8809
> Project: Flink
> Issue Type: Bug
> Components: TaskManager
> Reporter: Kirill A. Korinskiy
> Priority: Major
>
> Good day!
>
> Have I can see since this
> [commit|https://github.com/apache/flink/commit/6c44d93d0a9da725ef8b1ad2a94889f79321db73]
> TaskManager uses 8,388,607 terabytes as maximum out of heap memory. I guess
> that not any system has so much memory and it may be a reason to kill java
> process by OOM Killer.
>
> I suggest to decrease this value to reasonable value by default.
>
> Right now I see only one way to overstep this hardcoded value: setup
> FLINK_TM_HEAP to 0, and specified heap size by hand over
> FLINK_ENV_JAVA_OPTS_TM.
> Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
