Hi, I've only been using Calcite for a short while and am trying to hook up my own streaming table. The problem I'm running into seems be, at its core, an impedance mismatch.
Streams, by their very nature, are expected to be infinite (this what Julian is getting at in the stream tutorial <http://calcite.incubator.apache.org/docs/stream.html>). As such, they should send incremental results to along to the ResultSet. However, when running a simple query (e.g. select stream * from orders) and the StreamableTable returning a Enumerable<Object[]> from an infinite stream using the standard Linq4j tools, you end up quickly trying to store all the values of the stream in a ListSink in a TableScanNode. <https://github.com/apache/incubator-calcite/blob/3e50232b681e8dadb921580ee6f3e0376dd0f664/core/src/main/java/org/apache/calcite/interpreter/TableScanNode.java#L73> It seems like the TableScanNode needs to be made stream aware and the sink needs to forward results onward. Here is a simple test <https://github.com/jyates/incubator-calcite/commit/7111a0ed8456a567bd846660485bf509d77614fc> that verifies an infinite stream infinitely adds data to the ListSink (drop a breakpoint in TableScanNode to see it in action). As I mentioned, I'm still pretty new to Calcite, so any pointers to my being completely wrong would be much appreciated (or I'm happy to follow up in a JIRA if this looks like a bug). Thanks, Jesse Yates
