vdiravka commented on pull request #2251: URL: https://github.com/apache/drill/pull/2251#issuecomment-879480858
Hi @paul-rogers, I started design for system options from adding the new additional layer. But this approach is much more complex and we should cover all the cases you described. The design for the feature proposed by me is other: we just don't use global system options at all (in case `separate_workspace` is enabled). Every user has it's own system options. It gives a great functionality, which drill required for a long time, I think. We used session options for this purpose, but it can't be serialized. I also abandon the idea to Store Session options for the same reason, it has other concept and couples with a Query. About plugins the same thing, I tried to keep things as simple as it is possible (it corresponds to your last sentence). * I agree we can factor out system plugins and use that shared plugins for all users. It is minor improvement, it allows to keep less things in memory. This is not a blocker for this PR. I will create a ticket and proceed working on it after finishing this PR. * Also I agree with it is nice to have Group/Organization level of plugins and `System Options`. I noted about it in Design document. But it is much more complex improvement, requires introducing a lot of new rules and grants. I thought to consider working on it after finishing on this PR. Because now separate plugins and options can provide all things needed for users, but setting up is not so comfortable as it can be with Group/Organization level of these instances. And there are more similar instances are placed in memory now (separate instances for Groups/Organizations will improve this too) * About concurrency and distributed systems issues, they are all the same as it is now. Since event now one user can run query and other can edit the plugin config. Now the picture is even better, since two user can't sue common plugins registry or system options. Whey their own ones. They don't impact each other. About dividing the PR I also thought about it initially, because leverage of System Options is other than plugins, so a lot of things for options is other. But options and plugins are combined with a common idea - to have the separate workspace for users: plugin configs, system options and to see their own query profiles only. And the approach for implementation is very similar - to replace the global instances with their own user-session ones. So I can here to divide the PR in two parts or just in two commits for easier review. But I think we can also go with one PR and commit. So let me know do you like the above approach for this feature with a two (or more) further improvements or possibly you want to bring mail discussion on this topic with other devs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
