On Thu, Feb 4, 2021 at 12:23 PM Tomás Fernández Löbbe <tomasflo...@gmail.com> wrote:
> The point I was trying to make is that, having a single configset loading > from both, local and zk may be confusing for the user and cause issues that > may be difficult to track: Which file is Solr really reading right now? is > it the local one or the remote one? Is there a local one in a node or not? > is it being correctly overridden? How do I ensure that I always have a > local version of a file to override the remote? > Fair point -- it is less clear than today. I suppose anything we come up with will be :-) > So, I'm thinking that if we want to support this feature, a cleaner > approach could be to just have a type of configset that's defined as > "local", and then it belongs to the local filesystem. We can just prevent a > node from starting if it's supposed to have a configset that doesn't have. > It's 100% clear where a config file is being read from, etc. Maybe the > "configOverlay.json" is an exception and should live in ZooKeeper (and > never locally) for the config API to work, but having just "default to > local when a file is not in ZooKeeper" just confuses things IMO. > Hmmm, okay. While I agree configOverlay.json & params.json would always belong in ZK... for the rest, it's debatable. Can we get the schema there too if it's a "managed schema"? What about resource files (e.g. synonyms)? Whatever the answers are there, it would solve my primary motivation -- an easier upgrade path, at least where I work. I spoke with Ilan a couple weeks ago about this and he proposed an interesting idea: Put a simple version number on the configSet, and let them live in either ZK or local. The greater version number chooses which wins; the other is ignored. This is somewhat similar to your idea. Still... I'd prefer some way to establish defaults for specific configuration elements that live on the node, while letting the other aspects continue to reside in ZK (or have the option of resolving local as well). In my mind, this is just about making Solr's existing defaults in the code become configurable. It's a different way of looking at things than saying where does this or that file live. For example, imagine a node resident default configSet that is effectively the default that all configSets are overlayed on top of. Field types, analyzers, merge policies, request handlers -- it could define whatever it felt is needed. Then the ZK part is what is specific to a configSet for a given search app, and it doesn't need to specify the organization-wide settings. My original proposal doesn't quite do this directly because I thought of a cheap hack in concert with some other Solr features that'd suffice for my aims. But maybe I should propose more explicitly a node-local default configSet, designed to make setting defaults simple/easy in one place and specific to a node. One might call this configSet inheritance. I think it would lead to configSets that are simpler to read/maintain because they would only contain what an app needs, and not the organization-wide needs and/or Solr defaults. WDYT?
