[
https://issues.apache.org/jira/browse/CALCITE-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680871#comment-14680871
]
Julian Hyde commented on CALCITE-809:
-------------------------------------
I think your patch is a big improvement, so I have pushed it to my branch
https://github.com/julianhyde/incubator-calcite/tree/new-master and I will
merge to apache/master as soon as we re-open after release 1.4.
However, the method Sink.setSourceEnumerable doesn't seem right. I want to
remove it ASAP.
I wonder whether, rather than creating EnumerableProxySink, we should create a
different kind of Source when we are reading from a table (especially a table
that happens to be infinite, i.e. a stream).
Also, the Interpreter should have an execution mode that runs each operator in
a thread, rather than assuming that each Node.run() method will run to
completion in a reasonable (or even finite) time.
> TableScan does not support large/infinite scans
> -----------------------------------------------
>
> Key: CALCITE-809
> URL: https://issues.apache.org/jira/browse/CALCITE-809
> Project: Calcite
> Issue Type: Bug
> Components: core
> Reporter: Jesse Yates
> Assignee: Julian Hyde
> Fix For: 1.5.0-incubating
>
> Attachments: calcite-803-v0.patch, calcite-803-v1.patch
>
>
> When running a simple query (e.g. select stream * from orders) and the
> StreamableTable returning a Enumerable<Object[]> backed by an infinite
> stream, you end up quickly trying to store all the values of the stream in a
> ListSink in a TableScanNode.
> From Julian:
> {quote}
> You're hitting the interpreter "cheap and dirty"
> implementation of TableScan. I made the interpreter the simplest thing
> that could possibly work, so I made every operator build a list. (I
> know, I know. Enumerable uses iterators, and other implementations do
> even better. But I wanted to fit it into one page of code.)
> ...
> The solution will be either to fix the interpreter to use iterators
> (or similar) rather than lists, or to recognize that a query is
> infinite and not use the interpreter.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
