Gopal V created TEZ-1086:
----------------------------
Summary: Allow Tez sessions to decay smoothly instead of timeouts
Key: TEZ-1086
URL: https://issues.apache.org/jira/browse/TEZ-1086
Project: Apache Tez
Issue Type: New Feature
Reporter: Gopal V
With hiveserver2's session sharing, we hit a problem with session container
delays.
If the timeout is too small, like 2 seconds, there is no real reuse of
containers, even for small queries which hit the server.
If the timeout is too big, like 2 minutes, then a big query will hold onto idle
containers for way too long causing smaller queries to stall.
An ideal middle ground would be to keep some containers around, while getting
rid of a majority of containers owned by the session.
A simple heuristic could be to set the fractional-life of a session container
to 2 seconds.
If the fraction is 0.5 (i.e half-life), then 50% of the idle capacity would
become available for the small query in 2 seconds.
This could work better for the JIT as the mechanism can sort containers by
their launchtime and kill off the newer ones always (or pick 1 per node to
keep).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
