>> but I think it would be better if the mutable part is placed under >> /collections/x/..., otherwise "/configs"
it makes sense. The problem we have is managedschema currently writes to the same If we could change the managedschema behavior somehow it would have been better On Fri, May 22, 2015 at 10:21 PM, Tomás Fernández Löbbe <tomasflo...@gmail.com> wrote: >> TLDR: we should think about this as configset base vs per-collection diff, >> not as immutable base vs per-collection mutable. > > Makes sense, I was mostly thinking of it being immutable from the current > Config APIs. Editing a configset for multiple collection is a valid and > useful feature, the problem is doing that from inside one collection's API > call. > >> So then the question becomes, do we want an API that can *also* make >> collection-specific changes to a shared config? > > If we feel there is no need for collection-specific config changes, I'm OK, > but again, the API should be outside of the collection, like a Configset > API. The "generate configset based on X" should also be a command of this > API. In addition, this could allow users to edit a configset that's not > currently being used by any collection. > > Tomás > > > On Fri, May 22, 2015 at 7:10 AM, Yonik Seeley <ysee...@gmail.com> wrote: >> >> Makes sense Greg. >> >> Just looking at it from the ZK perspective (APIs aside), the original >> idea behind referencing a config set by name was so that you could >> change it in one place and everyone relying on it would get the >> changes. >> >> If one wants collections to have separate independent config sets they >> can already do that. >> >> So then the question becomes, do we want an API that can *also* make >> collection-specific changes to a shared config? >> >> An alternative would be a command to make a copy of a config set, and >> a command to switch a specific collection to use that new config set. >> Then any further changes would be collection specific. That's sort of >> like SOLR-5955 - config templates - but you can "template" off of any >> other config set, at any point in time. Actually, that type of >> functionality seems generally useful regardless. >> >> -Yonik >> >> >> On Thu, May 21, 2015 at 8:07 PM, Gregory Chanan <gcha...@cloudera.com> >> wrote: >> > I'm +1 on the general idea, but I'm not convinced about the >> > mutable/immutable separation. >> > >> > Do we not think it is valid to modify a single config(set) that affects >> > multiple collections? I can imagine a case where my data with the same >> > config(set) is partitioned into many different collections, whether by >> > date, >> > sorted order, etc. that all use the same underlying config(set). Let's >> > say >> > I have collections partitioned by month and I decide I want to add >> > another >> > field; I don't want to have to modify >> > jan/schema >> > feb/schema >> > mar/schema >> > etc. >> > >> > I just want to modify the single underlying config(set). You can >> > imagine >> > having a configset API that let's me do that, so if I wanted to modify a >> > single collection's config I would call: >> > jan/schema >> > but if i wanted to modify the underlying config(set) I would call: >> > configset/month_partitioned_config >> > >> > My point is this: if the problem is that it is confusing to have >> > configsets >> > modified when you make collection-level calls, then we should fix that >> > (I'm >> > 100% in agreement with that, btw). You can fix that by having a >> > configset >> > and a per-collection diff; defining the configset as immutable doesn't >> > solve >> > the problem, only locks us into a implementation that doesn't support >> > the >> > use case above. I'm not even saying we should implement a configset >> > API, >> > only that defining this as an immutable vs mutable implementation blocks >> > us >> > from doing that. >> > >> > TLDR: we should think about this as configset base vs per-collection >> > diff, >> > not as immutable base vs per-collection mutable. >> > >> > Thoughts? >> > Greg >> > >> > >> > On Tue, May 19, 2015 at 10:52 AM, Tomás Fernández Löbbe >> > <tomasflo...@gmail.com> wrote: >> >> >> >> I created https://issues.apache.org/jira/browse/SOLR-7570 >> >> >> >> On Fri, May 15, 2015 at 10:31 AM, Alan Woodward <a...@flax.co.uk> >> >> wrote: >> >>> >> >>> +1 >> >>> >> >>> A nice way of doing it would be to make it part of the >> >>> SolrResourceLoader >> >>> interface. The ZK resource loader could check in the >> >>> collection-specific >> >>> zknode first, and then under configs/, and we could add a >> >>> writeResource() >> >>> method that writes to the collection-specific node as well. Then all >> >>> config >> >>> I/O goes via the resource loader, and we have a way of keeping certain >> >>> parts >> >>> immutable. >> >>> >> >>> On 15 May 2015, at 17:39, Tomás Fernández Löbbe >> >>> <tomasflo...@gmail.com> >> >>> wrote: >> >>> >> >>> I agree about differentiating the mutable part (configoverlay, >> >>> generated >> >>> schema, etc) and the immutable (the configset) , but I think it would >> >>> be >> >>> better if the mutable part is placed under /collections/x/..., >> >>> otherwise >> >>> "/configs" would have a mix of ConfigSets and collection-specific >> >>> configuration. >> >>> >> >>> On Fri, May 15, 2015 at 6:38 AM, Noble Paul <noble.p...@gmail.com> >> >>> wrote: >> >>>> >> >>>> I think this needs more discussion >> >>>> >> >>>> When a collection is created we should have two things >> >>>> >> >>>> an immutable part and a mutable part >> >>>> >> >>>> for instance my collection name is "x" and it uses schemaless example >> >>>> conf >> >>>> >> >>>> I must now have two conf dirs >> >>>> >> >>>> configs/schemaless and >> >>>> configs/x >> >>>> >> >>>> all the mutable stuff goes to configs/x >> >>>> >> >>>> and config/schemaless remains immutable >> >>>> >> >>>> >> >>>> >> >>>> On Tue, May 12, 2015 at 2:23 AM, Tomás Fernández Löbbe >> >>>> <tomasflo...@gmail.com> wrote: >> >>>>> >> >>>>> I think this is fine.I don't think we need a new concept of "config >> >>>>> templates", we just need to make it clear that the configset used to >> >>>>> create >> >>>>> the collection is not modified by Solr, and that any change done via >> >>>>> API >> >>>>> only affects the single collection where the config command is >> >>>>> issued. >> >>>>> >> >>>>> I guess the schema API should start using something like >> >>>>> configoverlay, >> >>>>> or maybe persist the updated schema to this new path? >> >>>>> >> >>>>> Tomás >> >>>>> >> >>>>> On Fri, May 8, 2015 at 10:28 PM, Noble Paul <noble.p...@gmail.com> >> >>>>> wrote: >> >>>>>> >> >>>>>> I agree with you on the point that it causes confusion. >> >>>>>> >> >>>>>> My suggestion would be to have something called "config templates" >> >>>>>> and >> >>>>>> they are immutable . So , we don't need a configset API >> >>>>>> each collection have it's own conf folder . >> >>>>>> >> >>>>>> So, when a collection is created we should go ahead and create a >> >>>>>> corresponding conf dir. >> >>>>>> >> >>>>>> Ideally, it should not copy over all configs from it's template. It >> >>>>>> should just store the configoverlay.json, params.json in the >> >>>>>> collection's >> >>>>>> conf directory and inherit the rest from the template >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> On Sat, May 9, 2015 at 9:35 AM, Tomás Fernández Löbbe >> >>>>>> <tomasflo...@gmail.com> wrote: >> >>>>>>> >> >>>>>>> I think the concept of ConfigSets has become a bit confusing with >> >>>>>>> the >> >>>>>>> Config APIs (I'm thinking in SolrCloud mode here). Solr requires >> >>>>>>> that a >> >>>>>>> configset is pushed to ZooKeeper before creating a collection that >> >>>>>>> uses it. >> >>>>>>> It supports multiple collections using the same configset, which I >> >>>>>>> think is >> >>>>>>> great. You could also have a couple of configsets that no >> >>>>>>> collection is >> >>>>>>> currently using (who knows, maybe one that was recently >> >>>>>>> deprecated, or that >> >>>>>>> will be used soon, etc). This gives me the idea that configsets >> >>>>>>> are a >> >>>>>>> separate entity than the collection, not just a collection's >> >>>>>>> configuration. >> >>>>>>> >> >>>>>>> Config APIs allow you to operate on a collection to add handlers, >> >>>>>>> change settings, etc. The problem is that you are not really >> >>>>>>> applying the >> >>>>>>> changes to the collection but to the complete configset. All >> >>>>>>> collections >> >>>>>>> using it will get the changes, and all of them will be reloaded >> >>>>>>> after a >> >>>>>>> change. >> >>>>>>> >> >>>>>>> Shouldn't those APIs be at a different level/outside the >> >>>>>>> collection? >> >>>>>>> Maybe a configset API? Or, maybe the configs (for example, the >> >>>>>>> configoverlay.json) should only apply to the collection where the >> >>>>>>> API call >> >>>>>>> was made and not to other collections using the configset? >> >>>>>>> >> >>>>>>> Tomás >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> -- >> >>>>>> ----------------------------------------------------- >> >>>>>> Noble Paul >> >>>>> >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> ----------------------------------------------------- >> >>>> Noble Paul >> >>> >> >>> >> >>> >> >> >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> > -- ----------------------------------------------------- Noble Paul --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
