Hello guys, When you enable ExternalShuffleService (spark-shuffle) in NodeManager, there are no suggestions of increasing NM heap size in Spark docs or anywhere else, shouldn't we include this in spark's documentation?
I have seen NM take a lot of memory 5+ gb with default 1g, and in case of its GC pauses spark can become very slow when tasks are doing shuffle. I don't think users are aware of NM becoming bottleneck. Sincerely, Artur Sukhenko -- -- Artur Sukhenko
