[
https://issues.apache.org/jira/browse/DRILL-6149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360141#comment-16360141
]
Paul Rogers commented on DRILL-6149:
------------------------------------
Let's think about this one just a bit. Drill follows SQL semantics. I SQL,
there is a global state and a session state. Each query has its own state
(represented by a cursor, though Drill does not have this concept.) There is no
place in SQL for query-level options.
What may be a bit confusing is that, when executing a fragment, there are
query-level options. So doesn't this contradict what was just said above?
No, it doesn't. Query-level options are an implementation detail for the fact
that Drill is a distributed system.
The user maintains a session at a single Drillbit: the Foreman. Drill does not
have the idea of a user session as a global object. It could. We've discussed
the following scenario. Suppose the user logs into Foreman A, issues a bunch of
session options, and runs a query. Foreman A dies and the user connection
automatically routes to Foreman B. Cool, we have HA. Well, not quite. The
user's session on A was lost, the session on B will run queries without the
previously-set session options, resulting in wrong results. To make this work,
we'd need a global "state store" for user session artifacts. We don't have
that, so there user session is local to the Foreman.
Now, consider what happens when we execute a query. Forman A sends fragments to
Drillbits B, C, and D. How, though, does a fragment running in B get access to
the session options held on A without a global session state?
The answer is that we use a "closure": the query closes around the session
options in effect at the time the query was launched. In piratical terms, we
copy the session options into the physical plan so that each fragment runs with
the set of options in effect when the query started.
Note that this solution also solves the concurrency problem that would result
if we did have a global session state and the query referenced that state. I
set option x to 10 and start a long-running query. I then set option x to 20.
Which value should the query use: the original one or the new one? Our answer
is: the original one. That is, we implicitly use an MVCC solution in which
queries use values in effect when queries start. Copying the values creates the
"frozen" version.
Note further that fragments NEVER reference session options: they only
reference the "frozen" options in the query and global options. (This means, by
the way, that we do have race conditions for global options -- we don't use
MVCC for them. This is probably a bug.)
So, back to the proposal in this ticket:
* We have no SQL syntax for setting a query-only option. Perhaps we could add
transactions to identify which ALTER SESSION commands go with a query and which
with a session. But, this would be very awkward.
* Session options are not very useful alway because Tableau and other tools
don't have a way to set them. We really need something like a persistent user
profile that can span sessions.
* So, we do not need to clean up session/query option confusion because there
isn't any -- at least not in the code.
Now, if the purpose here is to do code cleanup, then that's fine; perhaps make
that clearer in the title and description.
> Support Query Level Options
> ---------------------------
>
> Key: DRILL-6149
> URL: https://issues.apache.org/jira/browse/DRILL-6149
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Timothy Farkas
> Assignee: Venkata Jyothsna Donapati
> Priority: Major
>
> Currently the line between session options and query level options are
> blurred. Ideally a session level option can be set and persisted for the
> duration of a user's session. A query level option can be set for a query.
> Right now there is no explicit query level option. A query level option is a
> session option whose validator has a non zero ttl. We should change this to
> have options that belong to the Query Scope as defined in the
> OptionDefinition. The ttl should also be configurable by the user in their
> **alter set** command.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
