Rajesh Balamohan created TEZ-4208:
-------------------------------------
Summary: Pipelinesorter uses single SortSpan after spill
Key: TEZ-4208
URL: https://issues.apache.org/jira/browse/TEZ-4208
Project: Apache Tez
Issue Type: Bug
Reporter: Rajesh Balamohan
Attachments: q67_sorter.log
Though it could have created multiple spans, tez always uses the first span
after spill. It is quite possible that other spans are bigger compared to the
first one, due to progressive space allocation. Fixing this would help in
reducing the number of spills (depending on the jobs) and lesser load for
indexcache entries (as lesser number of files have to be opened).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
