I'm +1 on the general idea, but I'm not convinced about the mutable/immutable separation.
Do we not think it is valid to modify a single config(set) that affects multiple collections? I can imagine a case where my data with the same config(set) is partitioned into many different collections, whether by date, sorted order, etc. that all use the same underlying config(set). Let's say I have collections partitioned by month and I decide I want to add another field; I don't want to have to modify jan/schema feb/schema mar/schema etc. I just want to modify the single underlying config(set). You can imagine having a configset API that let's me do that, so if I wanted to modify a single collection's config I would call: jan/schema but if i wanted to modify the underlying config(set) I would call: configset/month_partitioned_config My point is this: if the problem is that it is confusing to have configsets modified when you make collection-level calls, then we should fix that (I'm 100% in agreement with that, btw). You can fix that by having a configset and a per-collection diff; defining the configset as immutable doesn't solve the problem, only locks us into a implementation that doesn't support the use case above. I'm not even saying we should implement a configset API, only that defining this as an immutable vs mutable implementation blocks us from doing that. TLDR: we should think about this as configset base vs per-collection diff, not as immutable base vs per-collection mutable. Thoughts? Greg On Tue, May 19, 2015 at 10:52 AM, Tomás Fernández Löbbe < tomasflo...@gmail.com> wrote: > I created https://issues.apache.org/jira/browse/SOLR-7570 > > On Fri, May 15, 2015 at 10:31 AM, Alan Woodward <a...@flax.co.uk> wrote: > >> +1 >> >> A nice way of doing it would be to make it part of the SolrResourceLoader >> interface. The ZK resource loader could check in the collection-specific >> zknode first, and then under configs/, and we could add a writeResource() >> method that writes to the collection-specific node as well. Then all >> config I/O goes via the resource loader, and we have a way of keeping >> certain parts immutable. >> >> On 15 May 2015, at 17:39, Tomás Fernández Löbbe <tomasflo...@gmail.com> >> wrote: >> >> I agree about differentiating the mutable part (configoverlay, generated >> schema, etc) and the immutable (the configset) , but I think it would be >> better if the mutable part is placed under /collections/x/..., otherwise >> "/configs" would have a mix of ConfigSets and collection-specific >> configuration. >> >> On Fri, May 15, 2015 at 6:38 AM, Noble Paul <noble.p...@gmail.com> wrote: >> >>> I think this needs more discussion >>> >>> When a collection is created we should have two things >>> >>> an immutable part and a mutable part >>> >>> for instance my collection name is "x" and it uses schemaless example >>> conf >>> >>> I must now have two conf dirs >>> >>> configs/schemaless and >>> configs/x >>> >>> all the mutable stuff goes to configs/x >>> >>> and config/schemaless remains immutable >>> >>> >>> >>> On Tue, May 12, 2015 at 2:23 AM, Tomás Fernández Löbbe < >>> tomasflo...@gmail.com> wrote: >>> >>>> I think this is fine.I don't think we need a new concept of "config >>>> templates", we just need to make it clear that the configset used to create >>>> the collection is not modified by Solr, and that any change done via API >>>> only affects the single collection where the config command is issued. >>>> >>>> I guess the schema API should start using something like configoverlay, >>>> or maybe persist the updated schema to this new path? >>>> >>>> Tomás >>>> >>>> On Fri, May 8, 2015 at 10:28 PM, Noble Paul <noble.p...@gmail.com> >>>> wrote: >>>> >>>>> I agree with you on the point that it causes confusion. >>>>> >>>>> My suggestion would be to have something called "config templates" and >>>>> they are immutable . So , we don't need a configset API >>>>> each collection have it's own conf folder . >>>>> >>>>> So, when a collection is created we should go ahead and create a >>>>> corresponding conf dir. >>>>> >>>>> Ideally, it should not copy over all configs from it's template. It >>>>> should just store the configoverlay.json, params.json in the collection's >>>>> conf directory and inherit the rest from the template >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Sat, May 9, 2015 at 9:35 AM, Tomás Fernández Löbbe < >>>>> tomasflo...@gmail.com> wrote: >>>>> >>>>>> I think the concept of ConfigSets has become a bit confusing with the >>>>>> Config APIs (I'm thinking in SolrCloud mode here). Solr requires that a >>>>>> configset is pushed to ZooKeeper before creating a collection that uses >>>>>> it. >>>>>> It supports multiple collections using the same configset, which I think >>>>>> is >>>>>> great. You could also have a couple of configsets that no collection is >>>>>> currently using (who knows, maybe one that was recently deprecated, or >>>>>> that >>>>>> will be used soon, etc). This gives me the idea that configsets are a >>>>>> separate entity than the collection, not just a collection's >>>>>> configuration. >>>>>> >>>>>> Config APIs allow you to operate on a collection to add handlers, >>>>>> change settings, etc. The problem is that you are not really applying the >>>>>> changes to the collection but to the complete configset. All collections >>>>>> using it will get the changes, and all of them will be reloaded after a >>>>>> change. >>>>>> >>>>>> Shouldn't those APIs be at a different level/outside the collection? >>>>>> Maybe a configset API? Or, maybe the configs (for example, the >>>>>> configoverlay.json) should only apply to the collection where the API >>>>>> call >>>>>> was made and not to other collections using the configset? >>>>>> >>>>>> Tomás >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> ----------------------------------------------------- >>>>> Noble Paul >>>>> >>>> >>>> >>> >>> >>> -- >>> ----------------------------------------------------- >>> Noble Paul >>> >> >> >> >
