[
https://issues.apache.org/jira/browse/DRILL-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881938#comment-15881938
]
ASF GitHub Bot commented on DRILL-5290:
---------------------------------------
Github user ppadma commented on the issue:
https://github.com/apache/drill/pull/757
Thanks Sudheesh, Paul and Arina for the review. Original thought was it is
not worth the effort to keep operator table in sync with changes in dynamic
UDFs. This option is meant to be used for short running operational queries,
which most likely will not need dynamic UDF support (well, at least for now).
For long running analytic queries, this is not considered an issue.
However, I am looking into implementing Sudheesh's suggestion. I will
provide an update soon.
> Provide an option to build operator table once for built-in static functions
> and reuse it across queries.
> ---------------------------------------------------------------------------------------------------------
>
> Key: DRILL-5290
> URL: https://issues.apache.org/jira/browse/DRILL-5290
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.9.0
> Reporter: Padma Penumarthy
> Assignee: Padma Penumarthy
> Labels: doc-impacting
> Fix For: 1.10.0
>
>
> Currently, DrillOperatorTable which contains standard SQL operators and
> functions and Drill User Defined Functions (UDFs) (built-in and dynamic) gets
> built for each query as part of creating QueryContext. This is an expensive
> operation ( ~30 msec to build) and allocates ~2M on heap for each query. For
> high throughput, low latency operational queries, we quickly run out of heap
> memory, causing JVM hangs. Build operator table once during startup for
> static built-in functions and save in DrillbitContext, so we can reuse it
> across queries.
> Provide a system/session option to not use dynamic UDFs so we can use the
> operator table saved in DrillbitContext and avoid building each time.
> *Please note, changes are adding new option exec.udf.use_dynamic which needs
> to be documented.*
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
