I'm perhaps slightly conservative with respect to configuration, but I'm not fond of hidden configuration that I can't see. What I don't like is looking at a config file and not seeing the full story. That means i have to read the config and ALSO go read some part of the documentation that I've failed to memorize, and probably need to google to find to be fully aware of what's going on.... (and no I didn't like it when some standard stuff disappeared from solrconfig.xml a while back either). Small changes of course seem reasonable, but the further we drift into implicit things, especially if we get a collection of several implicit things described in various disparate parts of the manual the more cryptic the system becomes. That's my opinion, YMMV.
-Gus On Thu, Jan 3, 2019 at 2:57 PM David Smiley <david.w.smi...@gmail.com> wrote: > Broadly, you refer to "locale" issues. Solr's way of dealing with this > today is with optional & configurable use of URPs. The schema-less / > data-driven mode has some of these enabled; you can see it in the > solrconfig.xml including many date formats. You can look into that for > further info if you like. The primitive field types are not locale > sensitive. > > Update: It's looking like 8.0 will only employ this implicit field type > mechanism for _nest_path_ which probably won't be in the default schema. > Assuming it isn't, then it'll only be documented in the context of this > particular feature. It'd be nice to see the scope of fields expanded and > at that juncture it could/should be more broadly documented. That can wait > to people have energy to do it. > > On Sun, Dec 30, 2018 at 4:54 AM Jörn Franke <jornfra...@gmail.com> wrote: > >> Hi David, >> >> I now get the idea and yes this makes sense. It would require though some >> tutorial or best practices, eg overriding a platform data type may make not >> so much sense - it may confuse new developers in an existing project that >> know Solr, but then get a platform type that has not the default behavior. >> >> Could you deal with different languages in platform types? Eg for dates >> it does not seem a problem, because Solr expects only one specific type of >> date that needs to be somehow converted beforehand (maybe that conversion >> could be also part of a platform type), but decimals are different in some >> languages or Boolean values. >> >> Am 30.12.2018 um 07:01 schrieb David Smiley <david.w.smi...@gmail.com>: >> >> Thanks for your thoughtful response Jörn! >> ... >> On Sat, Dec 29, 2018 at 4:14 AM Jörn Franke <jornfra...@gmail.com> wrote: >> >>> I think it is a good idea, but I see some potential complexity for >>> “deployment” of collections. For instance, in environments where Solr is >>> used as a shared platform amongst several stakeholders, every time you >>> deploy/modify a collection you need to take care that the platform types >>> exist. If it exists in the Test environment then i need to make sure that >>> it exists as well in acceptance/production. The problem is that the >>> platform type could have been defined by somebody else who has not yet (eg >>> due to project/sprint delays) not updated the other environments. Another >>> issue is if I move to another Solr cluster in the same environment. Then, I >>> have to make sure that all platform types move with me. >>> >> >> RE "the platform type could have been defined by somebody else": I'm not >> imagining it'd be configurable, thus the "somebody else" is the Solr >> project/committers. >> >> Otherwise, I think I get your point, but perhaps I don't. It's the same >> point for *any* use of some new feature of Solr. If you use some new >> feature, you have to take care that all Solr instances you deploy your >> configuration to can handle that new feature. That's a fairly generic >> point that would apply to just about anything in Solr. >> >> >>> A (minor) issue is that platform types may change (for whatever reasons) >>> and that then potentially all collections have to be reindexed or we have >>> different versions of the same platform type making things not easier. >>> >> >> Yes it's possible. Though I think that point is apart from the feature I >> propose. You're saying that you might want to use an "int" field and then >> one day realize you want some newer/better definition of what an "int" is >> (e.g. trie -> points). Sure. That's true wether the field type is >> explicit or implicit. There's nothing stopping you from explicitly >> defining the field type if you want to; the names would not be reserved. If >> you want to stick with your current index running the new Solr version, >> then you would keep luceneMatchVersion what it was, which would effectively >> retain the interpretation of the implicit field types. >> >> >>> Currently we have all our Schema definitions in a version management >>> system (we use the Schema API but the JSON requests are out there) so that >>> projects can inspire from each other. Needless to say, that careful type >>> engineering requires also some documentation on technical design and may be >>> indeed very Collection specific. >>> >>> Another issue could be that a platform type may also imply a certain >>> platform solrconfig.xml (eg lib directive etc). >>> >> >> I'm imagining platform types would be basic primitive types (int, >> boolean, etc. and some special situations like in the issue I referenced). >> They would not depend on contrib libs... though I could imagine one day an >> evolution of this in which a contrib could somehow auto-add implicit field >> types. >> >> >>> I am not sure yet what are the exact benefits of referring to types of >>> other collections in the Solr runtime itself instead of having a version >>> system and letting projects decide if they want to adapt types of other >>> collections, but maybe I am overlooking something here. >>> >> >> The notion of implicit field types is not a cross-config >> (cross-collection) thing. Implicit field types are nothing more than >> built-in shortcuts. >> >> I recall one of my very early observations of Solr's schema was of >> surprise to see primitive types defined in the schema. Consider in SQL DDL >> statements that refer to varchar and such. Your DDL doesn't need to define >> what a varchar is! >> >> Happy New Year, >> ~ David >> >> Am 28.12.2018 um 17:36 schrieb David Smiley <david.w.smi...@gmail.com>: >>> >>> While working on https://issues.apache.org/jira/browse/SOLR-12768 it >>> occurred to me that it would be nice if Solr had implicitly defined field >>> types. This would allow you to define a field in your schema that refers >>> to a type that is *not* also in your schema -- at least not explicitly >>> (need not explicitly be put in your schema.xml if classic, or need not be >>> passed to schema manipulation API if you use that). The idea would be that >>> these types would be Solr platform provided field types that need not be >>> defined by you. >>> >>> There are multiple ways this loose idea might be conceived / imagined >>> into a concrete proposal. >>> >>> (A) The main idea I'm kicking around right now is that Solr would _not_ >>> throw an error at the moment of reading your field definition that it >>> doesn't see your type... instead it would see it's a platform type (via >>> some built-in hard-coded registry) and then register that type on the fly. >>> So if you were to read the schema then you'd see it. In this way, it's >>> kind of a shortcut. Platform field types that you don't actually refer to >>> will never end up being put into your schema. >>> >>> (B) A schema could pre-initialize with the platform/implicit types. >>> This is the simplest idea but I don't like it because you may not even need >>> some of these types. I'm not going to go down this path now but wanted to >>> mention it. >>> >>> I'm exploring (A) right now... I'm hoping to do this for at least a >>> "_nest_path_" field in support of nested documents in 8.0, but conceivably >>> the idea would be expanded to lots of things in our base schema right now >>> (int, str, etc.) >>> -- >>> Lucene/Solr Search Committer (PMC), Developer, Author, Speaker >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: >>> http://www.solrenterprisesearchserver.com >>> >>> -- >> Lucene/Solr Search Committer (PMC), Developer, Author, Speaker >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: >> http://www.solrenterprisesearchserver.com >> >> -- > Lucene/Solr Search Committer (PMC), Developer, Author, Speaker > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: > http://www.solrenterprisesearchserver.com > -- http://www.the111shift.com
