Hello qiang The difference between a session and the earlier AM pool is that the earlier version supported running only one DAG before the AM shutdown. A session supports running DAGs in serial.
The limitation of being able to run only one DAG in one AM meant there was a launch cost of the AM to be accounted for. The initial implementation of an AM pool was just to try out a proof of concept to see whether an AM pool helped remove this overhead. Over time, with the help of sessions and container reuse, the additional performance benefit that an AM pool brought in became less and less. Furthermore, there was a lot more complexity in terms of keeping an AM Pool service up and running, making sure that it would work properly with YARN ( and queue allocation/utilization ). From a security point of view, an AM pool can be problematic. Pre-launching AMs implies that they need to either be run as a super user or all jobs submitted to the AM have to be run as the same user. There are probably other reasons which I might have missed. Hope the above helps. Let us know if you need more clarifications. — Hitesh On Dec 25, 2014, at 5:54 AM, SkaterXu <[email protected]> wrote: > question for why not use tez 1.0app master pool? > i have seen docs in horton works mentioned to start a app master pool in > 2.0doc. but why later tez use session mode app master? what is the > difference? > > regards, > qiang
