[
https://issues.apache.org/jira/browse/DRILL-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Padma Penumarthy updated DRILL-5290:
------------------------------------
Description:
Currently, DrillOperatorTable which contains standard SQL operators and
functions and Drill User Defined Functions (UDFs) (built-in and dynamic) gets
built for each query as part of creating QueryContext. This is an expensive
operation ( ~30 msec to build) and allocates ~2M on heap for each query. For
high throughput, low latency operational queries, we quickly run out of heap
memory, causing JVM hangs. Build operator table once during startup for static
built-in functions and save in DrillbitContext, so we can reuse it across
queries.
Provide a system/session option to not use dynamic UDFs so we can use the
operator table saved in DrillbitContext and avoid building each time.
was:
Currently, DrillOperatorTable which contains standard SQL operators and
functions and Drill User Defined Functions (UDFs) (built-in and dynamic) gets
built for each query as part of creating QueryContext. This is an expensive
operation ( ~30 msec to build) and allocates ~2M on heap for each query. For
high throughput, concurrent low latency operational queries, we quickly run out
of heap memory, causing JVM hangs. Build operator table once during startup for
static built-in functions and save in DrillbitContext, so we can reuse it
across queries.
Provide an system/session option to not use dynamic UDFs so we can use the
operator table saved in DrillbitContext and avoid building each time.
> Provide an option to build operator table once for built-in static functions
> and reuse it across queries.
> ---------------------------------------------------------------------------------------------------------
>
> Key: DRILL-5290
> URL: https://issues.apache.org/jira/browse/DRILL-5290
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.9.0
> Reporter: Padma Penumarthy
> Assignee: Padma Penumarthy
> Fix For: 1.10
>
>
> Currently, DrillOperatorTable which contains standard SQL operators and
> functions and Drill User Defined Functions (UDFs) (built-in and dynamic) gets
> built for each query as part of creating QueryContext. This is an expensive
> operation ( ~30 msec to build) and allocates ~2M on heap for each query. For
> high throughput, low latency operational queries, we quickly run out of heap
> memory, causing JVM hangs. Build operator table once during startup for
> static built-in functions and save in DrillbitContext, so we can reuse it
> across queries.
> Provide a system/session option to not use dynamic UDFs so we can use the
> operator table saved in DrillbitContext and avoid building each time.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
