Hi Roberto, There's not really a good way to achieve exactly that at the moment that I can think of. One of the challenges is that Impala, right now, is quite opportunistic in the way it uses resources - if there are unused cores on the system, queries will often create new threads to take advantage of them. This makes it hard for admission control to anticipate how much CPU a query will use ahead of time. We have plans to make CPU consumption more deterministic (https://issues.apache.org/jira/browse/IMPALA-3902), which in turn would let admission control distinguish between the two kinds of queries you describe more automatically.
I've recently been focusing a lot on some of these admission control and resource management problems, so I'm optimistic that there will be a lot of improvements in the pipeline to make it easier to solve problems like this. If the "resource-intensive" and "resource-light" queries come from different workloads, it is possible to set up different resource pools with different concurrency limits and route the queries to those. It doesn't really sound like that's the problem you're trying to solve though - Tim On Wed, Feb 21, 2018 at 11:16 AM, Roberto Cerioni, Paulo < [email protected]> wrote: > Hello, > > > I have set admission control for Impala and I was able to create multiple > queues and limit the resources of each queue. > > For example, if we have 3 queries that require 100mb to run each, > submitted to a queue with 300mb maximum memory, then the 3 queries will be > admitted immediately. However, if the runtime of each query normally is 1 > second, all finish at the same time, but with 3 seconds runtime each. > > > If we allow only 1 query to run in the queue at once using admission > control, that query will finish in 1 second, but what I want is to be able > to run as much queries as the queue resources allow in parallel without > compromising the runtime. > > > Thanks, > > Paulo; > > >
