[
https://issues.apache.org/jira/browse/CALCITE-849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997869#comment-14997869
]
Jesse Yates commented on CALCITE-849:
-------------------------------------
FYI, [~julianhyde]. Is this kinda what you were thinking?
Put together a simple case as as branch [on
github|https://github.com/jyates/calcite/tree/non-matching-stream-849].
Basically, lets you configure a policy for the cooperative statement checking
when creating the result set. When running the query, the policy is attached to
the statement and then sent down in a special field in the DataContext. That
policy is then used to create a simple wrapper enumerator around the one
returned from the Interpreter in the generated code (Baz).
The code still needs support for:
- setting the number of rows to use before pausing
- prepared statements
For those who don't read linq4j natively, the patch adds line 13 to the
generated code for the simple 'select' case:
{code}
/* 12 */ public final org.apache.calcite.linq4j.Enumerator
baseEnumerator = _interpreter.enumerator();
/* 13 */ public final org.apache.calcite.linq4j.Enumerator
wrappedEnumerator =
((org.apache.calcite.jdbc.cooperative.CooperativeIterationPolicy)
root.get("COOPERATIVE_EXEC_POLICY")).apply(baseEnumerator);
/* 14 */ public void reset() {
{code}
Would love comments, suggestions etc. This was the best I could come up with
the solves the simple case, but isn't wildly invasive. I don't know how much it
will solve the case where people will be creating their own plans, but at least
the code is pretty simple and the policy is in place.
> Streams/Slow iterators dont close on statement close
> ----------------------------------------------------
>
> Key: CALCITE-849
> URL: https://issues.apache.org/jira/browse/CALCITE-849
> Project: Calcite
> Issue Type: Bug
> Reporter: Jesse Yates
> Assignee: Julian Hyde
> Fix For: 1.5.0
>
> Attachments: calcite-849-bug.patch
>
>
> This is easily seen when querying an infinite stream with a clause that
> cannot be matched
> {code}
> select stream PRODUCT from orders where PRODUCT LIKE 'noMatch';
> select stream * from orders where PRODUCT LIKE 'noMatch';
> {code}
> The issue arises when accessing the results in a multi-threaded context. Yes,
> its not a good idea (and things will break, like here). However, this case
> feels like it ought to be an exception.
> Suppose you are accessing a stream and have a query that doesn't match
> anything on the stream for a long time. Because of the way a ResultSet is
> built, the call to executeQuery() will hang until the first matching result
> is received. In that case, you might want to cancel the query because its
> taking so long. You also want the thing that's accessing the stream (the
> StreamTable implementation) to cancel the querying/collection - via a call to
> close on the passed iterator/enumerable.
> Since the first result was never generated, the ResultSet was never returned
> to the caller. You can get around this by using a second thread and keeping a
> handle to the creating statement. When you go to close that statement though,
> you end up not closing the cursor (and the underlying iterables/enumberables)
> because it never finished getting created.
> It gets even more problematic if you are use select * as the iterable doesn't
> finish getting created in the AvaticaResultSet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
