There is no explicit CPU isolation yet. Providing CPU isolation (between queries, users, etc.) is something we certainly want for Impala but nobody has started working on yet.
On Tue, May 9, 2017 at 2:45 AM, yu feng <[email protected]> wrote: > Thanks for your response. I get you point, We always take 3 or 4 impalad > as coordinator(in about 30 impalad cluster), I do not find any isolation about > CPU. does Impala community have some plans about this ? > > 2017-05-09 12:33 GMT+08:00 Matthew Jacobs <[email protected]>: > >> Hi Yu, >> >> Yes, you can use the fair-scheduler.xml and llama-site.xml files as >> you described. Admission control is the best mechanism that Impala has >> right now, unfortunately there isn't a way to get hard resource >> isolation between queries. >> >> I don't understand your problem though, are you saying that more >> queries are running than you expect? That can happen when there are >> many coordinators because admission control isn't centralized -- it >> operates as part of the coordinator. We often recommend using fewer >> coordinators to avoid this "over admission". >> >> On Wed, May 3, 2017 at 7:25 PM, yu feng <[email protected]> wrote: >> > Thanks for your response. I have used Impala's admission control in our >> > impala cluster, though we have not deployed CM, we set fair-scheduler.xml >> > and llama-site.xml and I think it can finish the same goal comparing to >> > using CM. >> > >> > I find in admission control, what we can do is limit Max Running Queries >> > and Max Memory Usage for every queries. and in some scenario， we have >> > to overselling the max memory, such as every impalad has 128GB and we set >> > up two pools, one is max running=32 and max memory =4GB, and another is >> max >> > running=16 and max memory = 2GB, when more and more queries running. we >> > have no ideas about whether the cluster have any problems, maybe we can >> do >> > some tests. >> > >> > what is more, Without Llama, we can not use cgroups achieve isolation >> > between queries(impala run all queries in one process using different >> > threads). I always think this kind of isolation is too softly. do you >> have >> > some suggestion? >> > >> > Thanks a lot >>
