[
https://issues.apache.org/jira/browse/CALCITE-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964875#comment-15964875
]
Chunhui Shi commented on CALCITE-1748:
--------------------------------------
Calcite has the assumption that a full schema tree is always available. This is
true in traditional database systems. Since for these systems, they could
always cache a global full schema tree -- The name space for tables/schema is
in control of these systems. When there are updates of the name space, their
SQL engine could have the guarantee that they always have up-to-date schema
tree. But this is not the case for Drill. So if Calcite could allow dynamically
searching and loading schema, that would give more flexibility to Calcite's
consumers like Drill.
Calcite-911 is to solve the cache issue since Drill wants to load schema on fly
for each query. I opened this JIRA was for this: when Drill load schema on the
fly, Drill does not need to load irrelevant schemas - Drill will load a portion
of schema only when it is needed -- when getSchema is called. To make getting
schema to be extendable would create more flexibility for query engines like
Drill that were designed to be separated from storage layer.
> Make CalciteCatalogReader.getSchema extendable to support dynamically load
> schema tree - getSchema need to be set to protected to allow overriding
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-1748
> URL: https://issues.apache.org/jira/browse/CALCITE-1748
> Project: Calcite
> Issue Type: Bug
> Reporter: Chunhui Shi
> Assignee: Julian Hyde
>
> In system like Drill, there is a need to load partial schema (e.g. for only
> one storage plugin) only when needed. Since Drill has no way to get a full
> available schema tree before hand, nor could Drill cache available schema for
> a storage plugin(e.g. Hive, MongoDB) since the storage plugin may not have
> notification mechanism to update Schema tree timely.
>
> The proposed fix is to load schema dynamically as shown in
> https://issues.apache.org/jira/browse/DRILL-5089
> To achieve this, we need to make CalciteCatalogReader.getSchema to be
> protected so it could be overridden by derived class while the derived class
> can reuse other functionalities in CalciteCatalogReader class
> private CalciteSchema getSchema(Iterable<String> schemaNames,
> SqlNameMatcher nameMatcher)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
