Re: How to setup the max memory for my big Hive SQL which is on MapReduce of Yarn

2017-06-06 Thread Stephen Sprague
have you researched the yarn schedulers? namely the capacity and fair schedulers? those are the places where resource limits can be easily defined. On Mon, Jun 5, 2017 at 9:25 PM, Chang.Wu <583424...@qq.com> wrote: > My Hive engine is MapReduce and Yarn. What my urgent need is to limit the > mem

How to setup the max memory for my big Hive SQL which is on MapReduce of Yarn

2017-06-05 Thread Chang.Wu
My Hive engine is MapReduce and Yarn. What my urgent need is to limit the memory usage of my big sql so that my bigger sql will run a longer time instead of using up all the resource of queue or even all the resource of my whole yarn cluster at once. But I cannot find any solution to define my