vdiravka commented on a change in pull request #2251:
URL: https://github.com/apache/drill/pull/2251#discussion_r664845699
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistry.java
##########
@@ -21,19 +21,18 @@
import java.util.Map;
import java.util.Set;
+import org.apache.calcite.schema.SchemaPlus;
Review comment:
> Are we controlling access to the "connectors" (code) or to the
"instances" (configs?)
* connectors and configs
> If we control the connectors, do we load new copies of the connector
metadata per user?
* yes, we do
> Can a user see the common set of system-wide configs? Or, only their own
private configs?
* only their private
> If system wide, how do we handle name conflicts? I add a "foo" plugin,
later the admin adds a system-wide "foo" or visa-versa.
* answered above
> Can a private plugin be promoted to system level? Is that a copy/paste
operation? With the user logged in as different windows? Or, is there a button
to do the operation?
* there is no such functionality yet
> When is the per-user information created? As you know, on the web, the
user session comes into existence for every single REST call. Plugin setup is
pretty heavy-weight. Is the user config cached between requests? If so, when do
we expire the user information if the user then becomes idle?
* the users plugins is created and placed within the user session. The
caching mechanism is absolutely the same as for for the system plugins registry
(`ephemeralPlugins` is kept within every session `UserStoragePluginRegistry`
inheritor of the `StoragePluginRegistryImpl`). If the UserSession is closed,
the `ephemeralPlugins` is persisted to ZK similar to the global plugins
approach.
> The registry handles updates. The current code is rather awful since there
is no coordination among Drillbits: multiple drillbits could come up at the
same time and they can all decide to do the upgrade. Adding users complicates
the effort. Will a user session trigger a plugin upgrade? How do we handle
races?
* Any plugins update or any query run is performed via the client session. A
new User Session can handle only own `UserStoragePluginRegistry` instance.
Handling races mechanism is the same as for `StoragePluginRegistryImpl`, but it
is event simpler - the user within one session can't update plugins and run
query simultaneously.
> This registry handles a subtle, obscure case. Suppose (before this
change), I run a query that users the "foo" plugin. Before the query starts
running (but after planning), I rush in and delete the "foo" config. What
happens? As it turns out, Drill saves the old config so that queries continue
to work. This works because there is only one registry. If we add per-user
registries, we'd need to pass the user info with the exec plan so that the
readers grab the user's version of the config, including any cached,
recently-deleted configs. (Distributed systems are complex!)
* for this case the query can fail indeed. But do we expect it to be
finished?! I don't think so. as the config was deleted and it is fine that the
query based on this config is failed.
The global idea is to have separate `StoragePluginRegistryImpl` per
UserSession and to keep all other functionality without changes. It allows to
avoid any unexpected behavior. Absolutely the same story with
`SystemOptionManager`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
