I choose to be more optimistic wrt «Solr committers» ability to integrate new and changed Lucene APIs in Solr. You do not need to be a Lucene committer in order to learn how to USE the Lucene APIs, and I believe there are several «Solr committers» who already posess those skills and are pretty deep in Lucene already. Hopefully they are interested in doing lucene upgrades for Solr, even if that some times includes implementing support for a new fieldType (points vs trie), getting rid of index-time-boost features etc. I may even attempt some of those tasks myself for the areas of Lucene API I am comfortable with.
Jan > 13. mai 2020 kl. 16:24 skrev Doug Turnbull > <dturnb...@opensourceconnections.com>: > > Jason, I hear your arguments and think of them FOR a split > > This might sound a bit harsh, but maybe Lucene devs helping with Solr has let > Solr off the hook a bit too much? I actually like the fact that the split > causes Solr to figure out it's own situation and focus on its problems. > > Regardless of the split or not, Solr is going to sink or swim based on the > efforts of Solr committers, not Lucene committers. I don't think Lucene > committers are going to be the ones to really address the systemic issues > with Solr. If anything, I imagine they are "let me fix this so the code > compiles" level of maintenance. > > "Falling behind Lucene" is counterbalanced to me with "Should Solr be on > cutting-edge Lucene?" > > I'd be OK with a stable, robust Solr that got 1-2 major versions behind > Lucene, but was rock-solid with a lower barrier to entry... > > On Wed, May 13, 2020 at 10:07 AM Jason Gerlowski <gerlowsk...@gmail.com > <mailto:gerlowsk...@gmail.com>> wrote: > Wanted to add my two cents to the mix, though I'm a little late as the > vote has already progressed pretty far. > > I'm against a split. From the points raised, I agree that Lucene has > much to gain. But Solr has a lot to lose. > > Lucene devs would be freed from keeping Solr usage up to date. That's > a great improvement for Lucene itself. But that burden doesn't > disappear - it's just being moved to a different (smaller) group of > committers - who by definition don't know Lucene as well, and are less > suited to the task. (Lucene devs still might help post-split, but > given that avoiding this burden is one of the arguments made above for > a split, it seems unwise to assume how much this generosity will > continue.) > > One likely result is that Solr will fall behind Lucene. Possibly > permanently behind. Lucene folks are doing great work to improve > perf, add features etc. so falling behind is a Very Bad Thing. To > Solr, Lucene is not the same as Jetty or Jackson which Solr can fall > behind on without significant detriment. Lucene and the core search > functionality it offers is what brings people to Solr (or Elastic). > Putting ourselves in a position to fall behind on Lucene does a huge > disservice to our users, and loses Solr one of its greatest > advantages. > > I hope that in the case of a split, the Solr community would rise to > the occasion and prevent this. But my personal judgement is that it's > unlikely. I hate to be negative, and I hope to be proven wrong, but > that's how things look to me. We (Solr folks) have a bad track record > of addressing things with less-tangible, less-sellable benefits. Take > our ongoing test flakiness woes and SolrCloud instability issues as > examples: both are serious threats to the project, both have been > around for years, and both are here to stay for the foreseeable > future. > > If conditions were different in a way that made "falling behind" less > likely, I'd be all for a split. But given (1) our recent track record > of addressing these sort of issues, (2) our test flakiness which will > make identifying "Lucene snapshot upgrade" bugs exceedingly difficult, > and (3) the current economic conditions which may make it harder for > committers to negotiate time from their employers to work on Lucene > updates...now seems like a bad time to attempt a split. It will harm > Solr more than it helps Lucene. > > On Tue, May 12, 2020 at 3:37 PM Namgyu Kim <kng0...@gmail.com > <mailto:kng0...@gmail.com>> wrote: > > > > It's hard to make a decision because it seems to have pros and cons. > > Basically, I agree to separate but there are some questions. > > So I don't not vote right now. > > > > 1) Release version > > Currently, versions of Lucene and Solr are aligned, how will they be > > managed in the future? > > Other people took Elasticsearch as an example... But it was an independent > > project from the beginning. > > So there is no problem with the Lucene version. (Elasticsearch 7.7 and > > Lucene 8.5.1) > > I'm sure if we make solr as an independent project, it will make cracks > > about the version structure. (like Lucene 8.6.2 and Solr 8.9.1) > > But it's also strange to suddenly start a new version of the Solr. (Solr > > 1.0) > > Of course it's a matter of adaption, but it's likely to cause some > > confusion for existing users. > > > > 2) Complementary relationship > > When Lucene and Solr are built together, Solr can always maintain the > > latest Lucene. > > In my personal opinion, it's a great advantage of Solr. > > Because Solr doesn't have to suffer from Lucene API changes and has latest > > library. > > But it will be difficult if Solr becomes independent. > > If Solr tracks the master branch of Lucene on separate repository(project), > > can it always check and reflect Lucene's API changes? > > > > On Tue, May 12, 2020 at 10:12 PM Doug Turnbull > > <dturnb...@opensourceconnections.com > > <mailto:dturnb...@opensourceconnections.com>> wrote: > >> > >> I'll give a perspective that comes more from the user's / "market" point > >> of view as at OSC we onboard lots of new organizations into Solr. > >> > >> - Most new users incorrectly think of Solr as an independent Apache > >> project, and many will have little knowledge or awareness of Lucene itself > >> until given the full history of Lucene, Solr, Elasticsearch... or they > >> have to dive into the code/write a plugin > >> > >> - Most orgs / managers think in terms of "Solr" (as in "Solr" vs > >> "Elasticsearch" vs "Vespa, etc). So the starting point for new devs / > >> folks is from the Solr angle > >> > >> - Lucene, when discussed, is understood more colloquially as a Solr > >> dependency > >> > >> - If someone brings down the code to do some kind of work or > >> investigation, there's typically surprise that Lucene and Solr are bundled > >> together. > >> > >> - There's further surprise as the projects are indeed so different: Lucene > >> and Solr tests, for example look little alike. They seem to have different > >> coding syles / practices. One has more server-like and distributed system > >> concerns; the other is clearly a low-level library for doing search work... > >> > >> I personally have a hard time explaining to new users the rationale for > >> keeping these together, and it only increases the barrier to entry (to > >> both projects) to have this added complexity of two very different code > >> bases munged together. > >> > >> Just my 2 cents... > >> -Doug > >> > >> On Tue, May 12, 2020 at 7:30 AM Alan Woodward <romseyg...@gmail.com > >> <mailto:romseyg...@gmail.com>> wrote: > >>> > >>> One advantage I find with the way Elasticsearch and Lucene interact is > >>> that ES doesn’t depend on the master branch. We upgrade our master > >>> branch frequently to keep up to date with the latest release branch, and > >>> that lets us find regressions or API problems pretty quickly, but it also > >>> insulates us from having to make big changes immediately. I find this > >>> really useful for things like deprecations. Let’s say we deprecate a > >>> particular API in the release branch, and remove it entirely in master. > >>> Currently, that means Solr needs to immediately switch over to the new > >>> API in its master branch. But the whole point of doing deprecations > >>> first is that it gives users time to find issues with the replacements - > >>> if we find that the replacement API doesn’t quite fit in ES, we have time > >>> to work out either how to change our code, or to improve the new API, but > >>> because the deprecated version is still there we’re not blocked from > >>> upgrading and getting other improvements. Solr, meanwhile, may end up > >>> with a hacky workaround because that’s what got tests passing for the > >>> Lucene developer; or worse, we end up just copying the deprecated API > >>> wholesale into Solr and abandoning it there - witness TrieField or > >>> UninvertingReader. > >>> > >>> > On 11 May 2020, at 19:05, Atri Sharma <a...@apache.org > >>> > <mailto:a...@apache.org>> wrote: > >>> > > >>> > My two cents: > >>> > > >>> > As a Lucene heavy developer, I have several found maintaining Solr > >>> > dependencies while making large changes a bit cumbersome. I believe > >>> > Lucene and Solr should exist in a symbiotic relationship but not > >>> > tightly coupled with each other. > >>> > > >>> > > >>> > On Mon, May 11, 2020 at 7:22 PM Erik Hatcher <erik.hatc...@gmail.com > >>> > <mailto:erik.hatc...@gmail.com>> wrote: > >>> >> > >>> >> Without reading much or replying to any specific points made on this > >>> >> thread, here's my raw thoughts on this age-old topic.... (finally > >>> >> coming out of my cocoon after taking things in for a bit) > >>> >> > >>> >> Solr is a search -server- with distributed capabilities, that > >>> >> leverages the magic of Lucene underneath. Solr depends on Lucene, is > >>> >> a consumer of it. Lucene is a tight search -library- with little to > >>> >> no external dependencies. Their purposes and end-users are different. > >>> >> > >>> >> I was never really for the grand unification of Lucene and Solr back > >>> >> in the day because: > >>> >> > >>> >> - Solr's developer experience would be greatly streamlined, faster, > >>> >> cleaner, leaner, and focused > >>> >> - Having Lucene change when Solr doesni't (yet) adapt to those changes > >>> >> leads to confusion and inconsistency, loose wires hanging out of the > >>> >> wall unconnected or duct taped together > >>> >> - It simply makes sense to keep Lucene versioned and tightly > >>> >> controlled for upgrades, various testing configurations varying Lucene > >>> >> versions, within Solr > >>> >> - Solr could have a very concerted upgrade effort for Lucene > >>> >> capability jumps, with a focused upgrade effort at the > >>> >> changed/improved/added touch points just like other dependencies > >>> >> within Solr (like Tika and Jetty) > >>> >> > >>> >> Those points all kinda say the same thing.... Solr depends on > >>> >> "lucene.jar" and I'm in the camp that thinks Solr and Lucene > >>> >> development, communities, and end-users/consumers would all greatly > >>> >> benefit from a fancy new TLP and focused community for solr.apache.org > >>> >> <http://solr.apache.org/> and a tight(er) relationship with the Lucene > >>> >> community as an involved and vested consumer. > >>> >> > >>> >> Erik > >>> >> > >>> > > >>> > > >>> > -- > >>> > Regards, > >>> > > >>> > Atri > >>> > Apache Concerted > >>> > > >>> > --------------------------------------------------------------------- > >>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > >>> > <mailto:dev-unsubscr...@lucene.apache.org> > >>> > For additional commands, e-mail: dev-h...@lucene.apache.org > >>> > <mailto:dev-h...@lucene.apache.org> > >>> > > >>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > >>> <mailto:dev-unsubscr...@lucene.apache.org> > >>> For additional commands, e-mail: dev-h...@lucene.apache.org > >>> <mailto:dev-h...@lucene.apache.org> > >>> > >> > >> > >> -- > >> Doug Turnbull | CTO | OpenSource Connections, LLC | 240.476.9983 > >> Author: Relevant Search; Contributor: AI Powered Search > >> This e-mail and all contents, including attachments, is considered to be > >> Company Confidential unless explicitly stated otherwise, regardless of > >> whether attachments are marked as such. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > <mailto:dev-unsubscr...@lucene.apache.org> > For additional commands, e-mail: dev-h...@lucene.apache.org > <mailto:dev-h...@lucene.apache.org> > > > > -- > Doug Turnbull | CTO | OpenSource Connections > <http://opensourceconnections.com/>, LLC | 240.476.9983 > Author: Relevant Search <http://manning.com/turnbull>; Contributor: AI > Powered Search <http://aipoweredsearch.com/> > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such.
