For a given query the logic for determining the memory that will be
required from admission is:
- if the query has mem_limit use that
- otherwise, use memory estimates from the planner

A query may be assigned a mem_limit by:
- taking the default mem_limit from the pool it was submitted to (this is
the recommended practice)
- manually setting one for the query (in case you want to override the pool
default for a single query)

In that setup, the memory estimates from the planner are irrelevant for
admission decisions and only serve for informational purposes.
Please do not read too much into the memory estimates from the planner.
They can be totally wrong (like your 8TB example).


On Fri, Feb 23, 2018 at 3:47 AM, Jeszy <jes...@gmail.com> wrote:

> Again, the 8TB estimate would not be relevant if the query had a mem_limit
> set.
> I think all that we discussed is covered in the docs, but if you feel
> like specific parts need clarification, please file a jira.
>
> On 23 February 2018 at 11:51, Fawze Abujaber <fawz...@gmail.com> wrote:
> > Sorry for  asking many questions, but i see your answers are closing the
> > gaps that i cannot find in the documentation.
> >
> > So how we can explain that there was an estimate for 8T per node and
> impala
> > decided to submit this query?
> >
> > My goal that each query running beyond the actual limit per node to fail
> (
> > and this is what i setup in the default memory per node per pool) an want
> > all other queries to be queue and not killed, so what i understand that i
> > need to setup the max queue query to unlimited and the queue timeout to
> > hours.
> >
> > And in order to reach that i need to setup the default memory per node
> for
> > each pool and setting either max concurrency or the max memory per pool
> that
> > will help to measure the max concurrent queries that can run in specific
> > pool.
> >
> > I think reaching this goal will close all my gaps.
> >
> >
> >
> > On Fri, Feb 23, 2018 at 11:49 AM, Jeszy <jes...@gmail.com> wrote:
> >>
> >> > Do queuing query or not is based on the prediction which based on the
> >> > estimate and of course the concurrency that can run in a pool.
> >>
> >> Yes, it is.
> >>
> >> > If I have memory limit per pool and memory limit per node for a pool,
> so
> >> > it
> >> > can be used to estimate number of queries that can run concurrently,
> is
> >> > this
> >> > also based on the prediction and not the actual use.
> >>
> >> Also on prediction.
> >
> >
>

Reply via email to