Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-02-16 Thread David Smiley
On Thu, Feb 4, 2021 at 12:23 PM Tomás Fernández Löbbe wrote: > The point I was trying to make is that, having a single configset loading > from both, local and zk may be confusing for the user and cause issues that > may be difficult to track: Which file is Solr really reading right now? is > it

Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-02-05 Thread Gus Heck
> > I'd prefer it being an explicit fallback or resolution order instead of > hardcoded magick. > I.e. able to configure a configset search path such as ["local", "zk", > "somethingelse"]. This would make resource loader prefer local files even > if they exist in ZK. > > This actually winds up

Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-02-04 Thread Jan Høydahl
Hi, I can see need for such flexibility, but I'm also worried that we complicate things and make debugging harder etc. If I'm not mistaken, the current logic in ZkResourceLoader is to first look in ZK, and if it is not found, look in local disk(?). I'd prefer it being an explicit fallback or

Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-02-04 Thread Gus Heck
It sounds like the issue is that we need both a "per node config" and a "per collection" config. This could all be in zookeeper, and with a clear well documented precedence order (node wins) for any attributes that overlap... would even make sense to have names for nodes that were not literal

Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-02-04 Thread Tomás Fernández Löbbe
> Ehh; I am not suggesting that configSets belong local, which would be a step backwards -- we put them in ZK for a reason right now :-) I'm suggesting we have both for the same configSet, where the deployer can choose which element is node resident vs cluster/ZK resident. Thanks to existing

Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-01-26 Thread David Smiley
On Tue, Jan 26, 2021 at 1:27 PM Tomás Fernández Löbbe wrote: > Thanks for bringing this up, David. I thought about this same situation > before, but I think I never convinced myself in one way or another :p. As I > mentioned in many other emails, I think the infrastructure and the node >

Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-01-26 Thread Tomás Fernández Löbbe
Thanks for bringing this up, David. I thought about this same situation before, but I think I never convinced myself in one way or another :p. As I mentioned in many other emails, I think the infrastructure and the node configuration (such as solr.xml) needs to be local (at least, needs to be able

Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-01-25 Thread David Smiley
I'm not entirely sure how to react to the feedback. Maybe in listing multiple benefits and a follow-on proposal, I inadvertently opened doors to distracting points. I know I can be guilty of scope creep. My proposal has no impact on where JARs go, and so let's not discuss lib directories, the

Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-01-24 Thread Ilan Ginzburg
An aspect that would be interesting to consider IMO is upgrade and configuration changes. For example a collection in use across Solr version upgrade might require different configuration (config set) with the old and new Solr versions. Solr itself can require changes in config across updates.

Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-01-23 Thread Gus Heck
I'm in agreement with Eric here that fewer ways (or at least a clearer default way) of supplying resources would be better. Additionally, it should be easy to specify that this resource that I've shared should be loaded on a per SolrCore or per node basis (or even better per collection present on

Re: [DISCUSS] ConfigSet ZK to file system fallback

2021-01-22 Thread Eric Pugh
There is a lot in here ;-). With the caveat that I don’t have recent experience that many of you do with massive solr clusters, I think that we need to commit to fewer, not more, ways of maintaining the supporting resources that these clusters need.. I’d like to see ways of managing our