[
https://issues.apache.org/jira/browse/CALCITE-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307861#comment-14307861
]
Julian Hyde commented on CALCITE-584:
-------------------------------------
Here is how I see the relationship between "scannable" and "queryable". There
are several dichotomies:
* ScannableTable (and FilterableTable and ProjectableFilterableTable) vs
QueryableTable SPIs;
* BINDABLE vs ENUMERABLE conventions;
* Interpreter vs generated-java-Enumerables engines;
* runtime decisions vs planner rules vs;
* user-defined functions vs built-in functions;
* interpreted vs compiled;
* weakly typed vs strongly typed;
* extensions vs built-in.
For each dichotomy, one side naturally aligns with the "scannable" philosophy
and one with "queryable", but things are not as rigid as they first appear.
1. Corresponding to the table SPIs are two calling conventions: BINDABLE and
ENUMERABLE. Their results look similar -- Enumerables -- but don't be deceived.
BINDABLE runs using an interpreter and returns Object[]s, whereas ENUMERABLE
runs by generating a Java AST, compiling it, and generating an Enumerable over
Object[], synthetic classes, boxed primitives (e.g. java.lang.Integer) or empty
List.
2. The calling conventions correspond to runtime engines: The Interpreter
implements BINDABLE, and the generated-Java-enumerable engine implements
ENUMERABLE. However, Interpreter is not the only thing that can produce
BINDABLE.
3. Both engines (interpreter and enumerable) will implement 100% of the core
algebra. The goals of the interpreter are simplicity of its code and
correctness. Performance is not a goal. (Except - see point 6 - it should be
possible and reasonably efficient to use a small Interpreter to convert a
bindable table to enumerable convention.)
4. I plan to move the org.apache.calcite.adapter.enumerable out of the core
module into a new module, enumerable. Core will no longer depend on enumerable.
This simplifies Calcite for users who just want to use it as a planning engine,
but yes, adds complexity elsewhere.
5. The test suite needs to continue to run. I do not want to refactor the test
suite (at least, not right now). Since the test suite is mostly in core, the
test suite will use the Interpreter by default. You can also run test suite
from the enumerable module, using the enumerable engine, and it should also
succeed. (Tests that explicitly depend on one convention, e.g. use explain,
will need to be made conditional.)
6. Using a bindable table SPI (e.g. ScannableTable) does not imply that you use
the interpreter for the whole plan. My preferred mode would be to instantiate a
small Interpreter to invoke the bindable table, handle any projects & filters
that it refuses at run time, and convert to enumerable convention. This mode of
operation will be as efficient as using QueryableTable for many data sources.
7. Implementations of built-in functions are today very dependent on code
generation. This allows say a NOT NULL INTEGER value to be represented as an
int, logical short-cuts, and common sub-expression elimination. I do not
propose to change that in the short term (for example: BindableFilter is
implemented as FilterNode, FilterNode uses Interpreter.compile to convert the
RexNode into a Scalar, and the JaninoRexCompiler invokes RexToLixTranslator
followed by Janino).
8. For aggregate functions (and windowed aggregates), I propose to support only
user-defined aggregates in the interpreter. It is straightforward to create
UDAs that implement the built-ins (SUM, COUNT etc.)
> Allow TableMacro to return Table other than TranslatableTable
> -------------------------------------------------------------
>
> Key: CALCITE-584
> URL: https://issues.apache.org/jira/browse/CALCITE-584
> Project: Calcite
> Issue Type: Bug
> Reporter: Julian Hyde
> Assignee: Julian Hyde
>
> The TableMacro.apply method used to return Table but in
> https://github.com/apache/incubator-calcite/commit/aa1f0983c126c23466deb990c96d6ff2dffd908c
> this changed to TranslatableTable. It seems reasonable to allow TableMacro
> to return other sub-types of Table, for example ScannableTable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
