Thanks for your response. I get you point, We always take 3 or 4 impalad as coordinator(in about 30 impalad cluster), I do not find any isolation about CPU. does Impala community have some plans about this ?
2017-05-09 12:33 GMT+08:00 Matthew Jacobs <[email protected]>: > Hi Yu, > > Yes, you can use the fair-scheduler.xml and llama-site.xml files as > you described. Admission control is the best mechanism that Impala has > right now, unfortunately there isn't a way to get hard resource > isolation between queries. > > I don't understand your problem though, are you saying that more > queries are running than you expect? That can happen when there are > many coordinators because admission control isn't centralized -- it > operates as part of the coordinator. We often recommend using fewer > coordinators to avoid this "over admission". > > On Wed, May 3, 2017 at 7:25 PM, yu feng <[email protected]> wrote: > > Thanks for your response. I have used Impala's admission control in our > > impala cluster, though we have not deployed CM, we set fair-scheduler.xml > > and llama-site.xml and I think it can finish the same goal comparing to > > using CM. > > > > I find in admission control, what we can do is limit Max Running Queries > > and Max Memory Usage for every queries. and in some scenario， we have > > to overselling the max memory, such as every impalad has 128GB and we set > > up two pools, one is max running=32 and max memory =4GB, and another is > max > > running=16 and max memory = 2GB, when more and more queries running. we > > have no ideas about whether the cluster have any problems, maybe we can > do > > some tests. > > > > what is more, Without Llama, we can not use cgroups achieve isolation > > between queries(impala run all queries in one process using different > > threads). I always think this kind of isolation is too softly. do you > have > > some suggestion? > > > > Thanks a lot >
