Rajesh Balamohan created TEZ-3248:
-------------------------------------
Summary: Allowing one fetcher to go past thresholds could cause
OOM in corner cases (in OrderedShuffle)
Key: TEZ-3248
URL: https://issues.apache.org/jira/browse/TEZ-3248
Project: Apache Tez
Issue Type: Improvement
Reporter: Rajesh Balamohan
Assignee: Rajesh Balamohan
In ordered shuffle, tez allows exactly one fetcher to go beyond the thresholds
before stalling the fetchers. This works in most of the cases, but in some
corner cases could lead to OOM. It might be good to trigger mem to disk merge
before stalling the threads to release memory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
