[
https://issues.apache.org/jira/browse/CALCITE-1808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16193455#comment-16193455
]
Jesus Camacho Rodriguez commented on CALCITE-1808:
--------------------------------------------------
bq. Is each query creating a new RelMetadataProvider?
Yes, each query creates a new provider. Do you think sharing the provider
across queries (making the provider static) might cause any issues, e.g., while
compiling queries concurrently?
bq. And does it use object identity?
At first, we thought a solution would be to use weak keys, but they will not
work here, since then the cache will use identity (==) equality to compare them
instead of equals(), and key equality for this cache is based on equals().
Even if we might be able to share the provider among queries, I think setting a
max size for the cache via {{maximumSize}} method to a reasonable albeit high
value (num of metadata providers * max num of concurrent queries?) could be
good since it would prevent any OOM issues if a system using Calcite does not
share the provider among queries. What do you think?
> JaninoRelMetadataProvider loading cache might cause OOM error
> -------------------------------------------------------------
>
> Key: CALCITE-1808
> URL: https://issues.apache.org/jira/browse/CALCITE-1808
> Project: Calcite
> Issue Type: Bug
> Components: core
> Reporter: Jagruti Varia
> Assignee: Julian Hyde
> Attachments: calcite-retained-strong-refs.png
>
>
> JaninoRelMetadataProvider has a static handler cache with size increasing
> over time for long running HS2, ending up causing OOM errors.
> {code:java}
> /** Cache of pre-generated handlers by provider and kind of metadata.
> * For the cache to be effective, providers should implement identity
> * correctly. */
> private static final LoadingCache<Key, MetadataHandler> HANDLERS =
> CacheBuilder.newBuilder().build(
> new CacheLoader<Key, MetadataHandler>() {
> public MetadataHandler load(@Nonnull Key key) {
> //noinspection unchecked
> return load3(key.def, key.provider.handlers(key.def),
> key.relClasses);
> }
> });
> ...
> /** Key for the cache. */
> private static class Key {
> public final MetadataDef def;
> public final RelMetadataProvider provider;
> public final ImmutableList<Class<? extends RelNode>> relClasses;
> ...
> {code}
> The lifecycle for providers is per query and we have multiple providers
> instantiated on the lifecycle of a query. The entries are retained in the
> cache even when query planning has finished.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
