There are some examples of using internal (undocumented) configuration options in Drill. You simply don't put the option in drill-module.conf or in the SystemOptionManager. Adding any (arbitrary) key name in drill-override.conf automatically gives you an undocumented boot time option name. Or provide the options as a System property. See [1] as an example. This does show up in a query to sys.boot. However, if no one knows the name then it cannot be set (except by those that read the code, and nothing is hidden from them :) ), and it will not show up in the sys.boot either. The only case this is not safe is with security parameters, and even then it is a system administrator mistake to set an undocumented security parameter and leave it exposed. My $0.02
[1] https://github.com/apache/drill/blob/master/exec/memory/base/src/main/java/org/apache/drill/exec/memory/BoundsChecking.java On Tue, Aug 15, 2017 at 5:09 PM, Paul Rogers <[email protected]> wrote: > Hi Tim, > > As you noted, it might be easier to have a single options system (that is, > one for runtime system/session) options rather than two. If we have two, > we’d need to create two system tables, two session tables and two bundles > of options sent along with each query, and two sets of SQL ALTER commands. > We’d need two different internal APIs to get/set the options. Doing so > would rapidly become a fairly large project! > > On the other hand, if all we want to do is hide runtime options in the web > UI by default, we can do this using a metadata tag as part of the option > validator (defined in ExecConstants.java). In the iterator that lists the > options, we’d just skip any that are tagged as “internal” (say). > > Of course, if we did that, we’d need to add another table that lists the > internal runtime options. We’d need this if, say, user Fred has an issue > and posts to this list. Expert Barney says, “Hh, you need to change > such-and-so option.” How would Fred confirm he’d actually done so? He’d > need to query the internals options table. > > Another solution is simply to make the setting into a config (boot time) > option. If the setting is so obscure that users should never set it, is it > really worth having it in the system/session (runtime) option system? > > A different, but related, issue relates to config options. The config > system is hugely useful: we use it for user-set boot time options and to > externalize a large number of Drill constants. All that material shows up > if the user queries the boot options table using SQL. Do we want to filter > out the internal options from that system as well? If we do, maybe we can > use drill-override-example.conf for the filter: if an option appears in the > example file, then it is meant to be set by users and should appear in the > UI. If not, it is a private externalized constant that should not appear in > the UI. > > Thanks, > > - Paul > > > On Aug 15, 2017, at 4:34 PM, Timothy Farkas <[email protected]> wrote: > > > > Hello All, > > > > Boaz has proposed the idea of internal options. Currently all the > options are visible to to the user when they do: > > > > select * from sys.options. > > > > We would like internal options to be not visible to the user by default. > Possibly requiring them to do: > > > > select * from internal.options > > > > in order to see them, but internal options could be set in the same way > as other system options. The intention would be to add new options we > weren't comfortable with exposing to the end user as internal options. > After an internal option and its corresponding feature is considered stable > it could become a standard system option. > > > > This feature would be implemented on top of Jyosthna's feature > https://github.com/apache/drill/pull/868 > > > > Paul has suggested this could be accomplished by adding metadata to > options in the SystemOptionManager, and distinguishing between standard an > internal options based on that meta data. > > > > If you have any feedback about how the feature should look like to the > user or be implemented. Please let us know. > > > > Thanks, > > Tim > > > >
