Jonathan Eagles created TEZ-3954:
------------------------------------
Summary: Reduce Tez Shuffle Handler Memory needs for holding
TezIndexRecords
Key: TEZ-3954
URL: https://issues.apache.org/jira/browse/TEZ-3954
Project: Apache Tez
Issue Type: Bug
Reporter: Jonathan Eagles
Assignee: Jonathan Eagles
Tez Shuffle Handler holds on to entire SpillRecord to better accommodate for
range fetch operation. For single reduce fetch holding on to spillRecord can
overwhelm memory if Index Cache is overrun. In the case of single fetch, only
keep the single tez index record needed and for range fetch, keep the entire
spill record in memory, but free it after it is needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
