[
https://issues.apache.org/jira/browse/CALCITE-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973809#comment-15973809
]
Jinfeng Ni commented on CALCITE-1748:
-------------------------------------
CALCITE-911 added {{SimpleCalciteSchema}}, which delegates the caching of
implicit subschema/table/functions to the wrapped {{Schema}} instance, in stead
of keeping the cache in CalciteSchema.
What [~cshi] tries to solve in Calcite-1748 and DRILL-5089 is to allow
dynamically loading and expanding CalciteSchema tree. Currently, Drill will
build a CalciteSchema tree including all the enabled storage plugin. This would
significantly slow down the query planning, as reported in DRILL-5089. That's
why [~cshi] wants to make {{CalciteCatalogReader.getSchema()}} extendable, so
that he can essentially make {{getSchema()}} behave like {{getOrAddSchema()}}.
[~julianhyde], do you mean this Schema SPI [1]? If I understand correctly,
the subschema obtained from {{Schema.getSubSchema()}} is for implicit schemas
in {{CalciteSchema class}}. I feel it might be possible to go with this way,
by making {{Schema.getSubSchema()}} behave like {{getOrAdd}}. Is this what you
suggested? Also, I'm not clear the difference between explicit sub schema and
implicit sub schema. (Currently, Drill creates the CalciteSchema tree by
treating each subschema as explicit schema). Would appreciate if you can
provide some suggestion.
1. https://calcite.apache.org/apidocs/org/apache/calcite/schema/Schema.html
> Make CalciteCatalogReader.getSchema extendable to support dynamically load
> schema tree - getSchema need to be set to protected to allow overriding
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-1748
> URL: https://issues.apache.org/jira/browse/CALCITE-1748
> Project: Calcite
> Issue Type: Bug
> Reporter: Chunhui Shi
> Assignee: Julian Hyde
>
> In system like Drill, there is a need to load partial schema (e.g. for only
> one storage plugin) only when needed. Since Drill has no way to get a full
> available schema tree before hand, nor could Drill cache available schema for
> a storage plugin(e.g. Hive, MongoDB) since the storage plugin may not have
> notification mechanism to update Schema tree timely.
>
> The proposed fix is to load schema dynamically as shown in
> https://issues.apache.org/jira/browse/DRILL-5089
> To achieve this, we need to make CalciteCatalogReader.getSchema to be
> protected so it could be overridden by derived class while the derived class
> can reuse other functionalities in CalciteCatalogReader class
> private CalciteSchema getSchema(Iterable<String> schemaNames,
> SqlNameMatcher nameMatcher)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
