[
https://issues.apache.org/jira/browse/DRILL-7871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17378798#comment-17378798
]
ASF GitHub Bot commented on DRILL-7871:
---------------------------------------
paul-rogers commented on pull request #2251:
URL: https://github.com/apache/drill/pull/2251#issuecomment-877870693
This one is difficult. We are reviewing code under time pressure when the
design itself raises many questions. Let's start with the design. First, it is
too short, it leaves many issues unspecified. So, I have to infer them from
reviewing the code. As noted before, the storage plugin registry is complex; so
having to reverse-engineer a design at the code level makes the process slower
than it might otherwise be.
This PR adds two separate features: a new options level and changes to the
plugin registry. Since these are independent concepts, they should appear as
separate PRs: one for the options, another for plugins. That will reduce the
cognitive load on us poor reviewers.
So, the design of options:
> But it appears different users have the common Storage Plugins and System
Options.
> * There is no possibility for Drillbit configuring individually for every
user and persistent between sessions (Note: SessionOptions are removed after
closing session)
The above is by design. **System** options are just that: options for the
system. They are meant to be set by the administrator to control the Drill
cluster as a whole. Drill also has *Session* options: these are per user
session. If we want per-user option settings, then we need a new level: *User*
options.
Options form a hierarchy. At present that hierarchy, from most to least
specific, is:
* Per-query options *or* session options
* System options
* Options configured in the Drill config system.
It sounds like you want to add a new layer: user options:
* Per-query options **or**
* Session option
* Per-user option
* System options...
Suppose we have per-user system options: I set my queue length to 10 items,
you set your queue length to 100 items. But, the queue is a global resource:
which is used? This is just one example of how intractable "per-user system
options" is as a concept. So, we need actual user options.
One obvious choice is simply to persist the session options. I login and
change `foo` to 10. Drill simply persists that change so that next time I log
in, `foo` is 10. Easy, but is that the right answer? Drill has a serious flaw
that the behavior of any query depends on options: JSON modes, all-text mode
and so on. That was done as a quick and dirty short-term solution to ship the
product, but is still with us and still causing issues. If I set an option to
"all text mode", and run my query, I get a result. If I share that query with
you, but you don't set the all-text option, the query will produce different
results (or, probably, fail.)
So, to "fix" options, we need to push those options into the query (with
hints, say), or we need options associated with a query, as having per-user
options won't solve the "options associated with a query" problem.
So, we probably do not want to just persist session options as they are used
to control specific queries. Instead, we'd want to add a new system: `SET USER
OPTION ...` so that the hierarchy is as above.
The design is silent on many important questions:
* What is the new hierarchy?
* How does *the* (not an) administrator set truly-global options if all
users can set their own?
* How does the user set, review and clear the user options?
* Suppose a user is logged in twice. In session 1 they change `foo` to 10.
Do we expect session 2 (on a different Drillbit) to see that change? This is a
**very** difficult concurrency question. Of course they do. But, when? In the
middle of query planning? At the start of the next SQL statement? After a time
period? Only at the next login? If concurrently, how will one Drillbit track
changes made to the user options by another Drillbit? How can the use a "force
resync" to keep options synchronized?
* Can the admin see the per-user options? If so, how? `SHOW OPTIONS FOR USER
'bob'`? Remember, the admin is tasked with policing the system, they **must**
be able to see all options.
* How do we solve the current ambiguity of true system options (the query
queuing options mentioned above) vs. user/session options? At present, the
solution is ad-hoc and we just trust that most options are not changed. If a
naive user can change any option, we've got issues. Do we need a permissions
table to say which users can change which options? (And, categorize true system
options as read-only at the user and session levels.)
* You will need `USER` version of all of the option commands.
* You will need complete tests that demonstrate that the hierarchy works as
expected.
The good news is that, in terms of implementation, Drill's option system is
already hierarchical, so adding a new layer should be very easy.
All of this is complex. It should be done as its own PR along with a
*detailed* design.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@drill.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> StoragePluginStore instances for different users
> ------------------------------------------------
>
> Key: DRILL-7871
> URL: https://issues.apache.org/jira/browse/DRILL-7871
> Project: Apache Drill
> Issue Type: New Feature
> Components: Security
> Affects Versions: 1.18.0
> Reporter: Vitalii Diravka
> Assignee: Vitalii Diravka
> Priority: Major
>
> Different users should have their own storage plugin configs to have access
> to own storages only. The feature can be based on Drill User Impersonation
> model
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
