Yeah we should only be looking for and only be reporting (if we choose to report to the user) a specific set of env variables. Anything else should be ignored.Should be an enum or constants somewhere listing what solr cares about, and we should ignore or be blind to anything else.
Perhaps we'd like to have a ConfigParams (or whatever) enum that has methods returning the env, sysprop, bin/solr arg, configFile and zkLocation that can be used to provide each possible configuration option (for things that are single value or short list, obviously an entire schema probably would not be setable by sysprop :) )? The return type of those methods could be Optional<>() since we neither have all of those for everything any time soon, and not all of them will make sense in all cases. zkLocation is a bit tricky and nebulous since it's probably a zk path and a JSON path or Xpath combined and relative to the chroot which itself is a potential config param, some stuff to think through there. On Thu, Nov 11, 2021 at 3:49 PM Ilan Ginzburg <ilans...@gmail.com> wrote: > Houston made a very valid comment back then on the placement plugin > support of environment variables (dropped as a consequence). > > > https://issues.apache.org/jira/browse/SOLR-15019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17286680#comment-17286680 > > It could be possible to unintentionally leak node data that should be kept > secret if Solr is allowed to freely access (random?) environment variables > as part of configuration. > > Something to keep in mind. > > Ilan > > > On Thu 11 Nov 2021 at 20:12, Eric Pugh <ep...@opensourceconnections.com> > wrote: > >> Agreed! >> >> I’ve noticed that in the Play Framework, you can configure everything via >> a property based configuration file, however it makes it easy to override >> the property file via another one, or via an ENV variables: >> >> db.default.username="smui" >> db.default.username=${?SMUI_DB_USER} >> >> Which turns out to be very liberating! >> >> >> On Nov 11, 2021, at 2:09 PM, Jan Høydahl <jan....@cominvent.com> wrote: >> >> +1 to a roundup of env and props across the board. I think SIP 11 is on >> the track of something. But can be done independent of this. >> >> Jan Høydahl >> >> 11. nov. 2021 kl. 17:44 skrev Gus Heck <gus.h...@gmail.com>: >> >> >> I guess all I mean is that it shouldn't be only sysprops. Enabling >> sysprops, Env vars etc seems fine but we need to clearly document >> precedence among any/all options. What is convenient varies from case to >> case and in a perfect world what I'd like to see is full support across >> each style (files, zk, props, env vars) with consistent and obvious naming >> and well documented resolution order. >> >> What I don't like is a little bit of env vars for some stuff, props for >> others, files for yet more stuff and some unclear aggregation of that >> showing up in zk... (or maybe some of it not showing up anywhere code could >> check it...) >> >> On Thu, Nov 11, 2021 at 11:07 AM Houston Putman <houstonput...@gmail.com> >> wrote: >> >>> I agree with Jan, when thinking about making Solr as cloud friendly as >>> possible EnvVars and (to a lesser extent) sysProps are much preferable than >>> having a setting in the solr.xml. >>> This is because it's easier to customize EnvVars per-node, while >>> customizing a config file is much harder, as those tend to be static and >>> shared across a whole environment. >>> >>> Also thanks for linking that SIP Jan, very applicable. >>> >>> - Houston >>> >>> On Fri, Nov 5, 2021 at 5:19 PM Jan Høydahl <jan....@cominvent.com> >>> wrote: >>> >>>> Thinking of these roles as labels, I think sysProps and envVars are the >>>> two universal methods, and nothing wrong with that. >>>> I keep trying to think cloud native and container, so having excellent >>>> 1st class support for env.vars for such configs is a priority to me. >>>> Most tools, CI-environments etc have built-in support for env.vars, and >>>> so it makes sense to me. >>>> >>>> See >>>> https://cwiki.apache.org/confluence/display/SOLR/SIP-11+Uniform+cluster-level+configuration+API >>>> for some interesting ideas around cluster/node level config. >>>> >>>> See >>>> >>>> 5. nov. 2021 kl. 15:04 skrev Gus Heck <gus.h...@gmail.com>: >>>> >>>> Agree better to something other than sysprops. an arg in the start >>>> script would be friendlier than -D props which generally are irritatingly >>>> verbose and expose too much implementation. >>>> >>>> We lack a config file per level. solr.xml does double duty as global >>>> and per-node depending on how it's used (zk or filesystem). >>>> >>>> Config file names are confusing too. Our file names are legacy of >>>> non-cloud mode I think, and we really should at some point (10.x?) rework >>>> configs to be cluster.xml, node.xml, collection.xml (formerly >>>> solrconfig.xml) and schema.xml (and maybe support something other than xml, >>>> but that's not nearly as important as clarity in naming, and having >>>> features) >>>> >>>> But this is all straying way off topic and should have its own SIP if >>>> someone seems to have time for it :) >>>> >>>> On Thu, Nov 4, 2021 at 6:07 PM Shawn Heisey <elyog...@elyograg.org> >>>> wrote: >>>> >>>>> On 11/4/21 2:51 PM, Noble Paul wrote: >>>>> > The SIP can be boiled down to the following >>>>> > >>>>> > * *Tag a node with a label (role) using a system property* >>>>> > ** Use the placement plugin to whitelist/block list certain nodes* >>>>> > ** Publish the roles through an API* >>>>> >>>>> >>>>> In general, for Solr, do we like the idea of having things controlled >>>>> by >>>>> system properties? >>>>> >>>>> I would think solr.xml would be the right place to configure this, >>>>> except that people can and probably do put solr.xml in zookeeper, >>>>> which >>>>> would mean every system would have the SAME solr.xml, and we're back >>>>> to >>>>> system properties as a way to customize solr.xml on each system. >>>>> >>>>> I have never used system properties to configure Solr. When I >>>>> customize >>>>> the config, I will often remove property substitutions from it and go >>>>> with explicit settings. My general opinion about system properties is >>>>> that if they're going to be used, they should DIRECTLY configure the >>>>> application, not be sent in via property substitution in a config >>>>> file. >>>>> I've never liked the way our default configs use that paradigm. It >>>>> means you cannot look at the config and know exactly how things are >>>>> configured, without finding out whether system properties have been >>>>> set. >>>>> >>>>> What color do others think that bikeshed should be painted? >>>>> >>>>> Thanks, >>>>> Shawn >>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org >>>>> For additional commands, e-mail: dev-h...@solr.apache.org >>>>> >>>>> >>>> >>>> -- >>>> http://www.needhamsoftware.com (work) >>>> http://www.the111shift.com (play) >>>> >>>> >>>> >> >> -- >> http://www.needhamsoftware.com (work) >> http://www.the111shift.com (play) >> >> >> _______________________ >> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467 >> | http://www.opensourceconnections.com | My Free/Busy >> <http://tinyurl.com/eric-cal> >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed >> <https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> >> This e-mail and all contents, including attachments, is considered to be >> Company Confidential unless explicitly stated otherwise, regardless >> of whether attachments are marked as such. >> >> -- http://www.needhamsoftware.com (work) http://www.the111shift.com (play)
