Hi,
Yes, GC is still a major concern. Even G1 has a hard time dealing with
>64GB heap in our experience.
To mitigate, we run multiple TMs with smaller heap per machine, and use
RocksDBStateBackend.
Best regards,
Kien
On 10/18/2017 4:40 PM, Marchant, Hayden wrote:
I read in the Flink documentation that the TaskManager runs all tasks within
its own JVM, and that the recommendation is to set the taskmanager.heap.mb to
be as much as is available on the server. I have a very large server with 192GB
so thinking of giving most of it to the Task Manager.
I recall that there are concerns with long stop-the-world garbage collection
pauses about allocating too much memory to a JVM - is this still a concern with
G1 ?
Thanks,
Hayden Marchant
