Gopal V created TEZ-2217:
----------------------------
Summary: The min-held-containers constraint is not enforced during
query runtime
Key: TEZ-2217
URL: https://issues.apache.org/jira/browse/TEZ-2217
Project: Apache Tez
Issue Type: Bug
Affects Versions: 0.6.0, 0.7.0
Reporter: Gopal V
Assignee: Bikas Saha
The min-held containers constraint is respected during query idle times, but is
not respected when a query is actually in motion.
The AM releases unused containers during dag execution without checking for
min-held containers.
{code}
2015-03-20 15:41:53,475 INFO [DelayedContainerManager]
rm.YarnTaskSchedulerService: Container's idle timeout expired. Releasing
container, containerId=container_1424502260528_1348_01_000013,
containerExpiryTime=1426891313264, idleTimeoutMin=5000
2015-03-20 15:41:53,475 INFO [DelayedContainerManager]
rm.YarnTaskSchedulerService: Releasing unused container:
container_1424502260528_1348_01_000013
{code}
This is actually useful only after the AM has received a soft pre-emption
message, doing it on an idle cluster slows down one of the most common query
patterns in BI systems.
{code}
create temporary table smalltable as ...;
select ... bigtable JOIN smalltable ON ...;
{code}
The smaller query in the beginning throws away the pre-warmed capacity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
