[
https://issues.apache.org/jira/browse/KUDU-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177996#comment-17177996
]
ASF subversion and git services commented on KUDU-3181:
-------------------------------------------------------
Commit c444e8dcbc9dbdef3d599fe4858d740ff91b2045 in kudu's branch
refs/heads/master from Li Zhiming
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=c444e8d ]
KUDU-3181: put a bound on the queue of compilation manager
Add flag codegen_queue_capacity to config max queue size.
Without the bound, when codegen cache hit rate is low, the queue
could accumulate tons of entries, each of which retains
a copy of schema meta data, thus consuming a lot of memeory.
Change-Id: Ia85b5818d1a5ea3ea978808900dd9c9e0cc0784e
Reviewed-on: http://gerrit.cloudera.org:8080/16330
Reviewed-by: Andrew Wong <[email protected]>
Tested-by: Andrew Wong <[email protected]>
Reviewed-by: Alexey Serbin <[email protected]>
> Compilation manager queue may have too many tasks
> -------------------------------------------------
>
> Key: KUDU-3181
> URL: https://issues.apache.org/jira/browse/KUDU-3181
> Project: Kudu
> Issue Type: Bug
> Components: codegen
> Reporter: Li Zhiming
> Priority: Major
> Attachments: heap.svg
>
>
> When a client frequently scanning for thousands of diffrent columns, the
> code_cache_hits rate is quite low. Then compilation task is frequently
> submitted to queue, but the compiler manager thread cannot consumes the queue
> quick enough. The queue could accumulate tons of entries, each of which
> retains a copy of schema meta data, so a lot of memeory is consumed for a
> long time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
