Jason, I hear your arguments and think of them FOR a split This might sound a bit harsh, but maybe Lucene devs helping with Solr has let Solr off the hook a bit too much? I actually like the fact that the split causes Solr to figure out it's own situation and focus on its problems.
Regardless of the split or not, Solr is going to sink or swim based on the efforts of Solr committers, not Lucene committers. I don't think Lucene committers are going to be the ones to really address the systemic issues with Solr. If anything, I imagine they are "let me fix this so the code compiles" level of maintenance. "Falling behind Lucene" is counterbalanced to me with "Should Solr be on cutting-edge Lucene?" I'd be OK with a stable, robust Solr that got 1-2 major versions behind Lucene, but was rock-solid with a lower barrier to entry... On Wed, May 13, 2020 at 10:07 AM Jason Gerlowski <gerlowsk...@gmail.com> wrote: > Wanted to add my two cents to the mix, though I'm a little late as the > vote has already progressed pretty far. > > I'm against a split. From the points raised, I agree that Lucene has > much to gain. But Solr has a lot to lose. > > Lucene devs would be freed from keeping Solr usage up to date. That's > a great improvement for Lucene itself. But that burden doesn't > disappear - it's just being moved to a different (smaller) group of > committers - who by definition don't know Lucene as well, and are less > suited to the task. (Lucene devs still might help post-split, but > given that avoiding this burden is one of the arguments made above for > a split, it seems unwise to assume how much this generosity will > continue.) > > One likely result is that Solr will fall behind Lucene. Possibly > permanently behind. Lucene folks are doing great work to improve > perf, add features etc. so falling behind is a Very Bad Thing. To > Solr, Lucene is not the same as Jetty or Jackson which Solr can fall > behind on without significant detriment. Lucene and the core search > functionality it offers is what brings people to Solr (or Elastic). > Putting ourselves in a position to fall behind on Lucene does a huge > disservice to our users, and loses Solr one of its greatest > advantages. > > I hope that in the case of a split, the Solr community would rise to > the occasion and prevent this. But my personal judgement is that it's > unlikely. I hate to be negative, and I hope to be proven wrong, but > that's how things look to me. We (Solr folks) have a bad track record > of addressing things with less-tangible, less-sellable benefits. Take > our ongoing test flakiness woes and SolrCloud instability issues as > examples: both are serious threats to the project, both have been > around for years, and both are here to stay for the foreseeable > future. > > If conditions were different in a way that made "falling behind" less > likely, I'd be all for a split. But given (1) our recent track record > of addressing these sort of issues, (2) our test flakiness which will > make identifying "Lucene snapshot upgrade" bugs exceedingly difficult, > and (3) the current economic conditions which may make it harder for > committers to negotiate time from their employers to work on Lucene > updates...now seems like a bad time to attempt a split. It will harm > Solr more than it helps Lucene. > > On Tue, May 12, 2020 at 3:37 PM Namgyu Kim <kng0...@gmail.com> wrote: > > > > It's hard to make a decision because it seems to have pros and cons. > > Basically, I agree to separate but there are some questions. > > So I don't not vote right now. > > > > 1) Release version > > Currently, versions of Lucene and Solr are aligned, how will they be > managed in the future? > > Other people took Elasticsearch as an example... But it was an > independent project from the beginning. > > So there is no problem with the Lucene version. (Elasticsearch 7.7 and > Lucene 8.5.1) > > I'm sure if we make solr as an independent project, it will make cracks > about the version structure. (like Lucene 8.6.2 and Solr 8.9.1) > > But it's also strange to suddenly start a new version of the Solr. (Solr > 1.0) > > Of course it's a matter of adaption, but it's likely to cause some > confusion for existing users. > > > > 2) Complementary relationship > > When Lucene and Solr are built together, Solr can always maintain the > latest Lucene. > > In my personal opinion, it's a great advantage of Solr. > > Because Solr doesn't have to suffer from Lucene API changes and has > latest library. > > But it will be difficult if Solr becomes independent. > > If Solr tracks the master branch of Lucene on separate > repository(project), can it always check and reflect Lucene's API changes? > > > > On Tue, May 12, 2020 at 10:12 PM Doug Turnbull < > dturnb...@opensourceconnections.com> wrote: > >> > >> I'll give a perspective that comes more from the user's / "market" > point of view as at OSC we onboard lots of new organizations into Solr. > >> > >> - Most new users incorrectly think of Solr as an independent Apache > project, and many will have little knowledge or awareness of Lucene itself > until given the full history of Lucene, Solr, Elasticsearch... or they have > to dive into the code/write a plugin > >> > >> - Most orgs / managers think in terms of "Solr" (as in "Solr" vs > "Elasticsearch" vs "Vespa, etc). So the starting point for new devs / folks > is from the Solr angle > >> > >> - Lucene, when discussed, is understood more colloquially as a Solr > dependency > >> > >> - If someone brings down the code to do some kind of work or > investigation, there's typically surprise that Lucene and Solr are bundled > together. > >> > >> - There's further surprise as the projects are indeed so different: > Lucene and Solr tests, for example look little alike. They seem to have > different coding syles / practices. One has more server-like and > distributed system concerns; the other is clearly a low-level library for > doing search work... > >> > >> I personally have a hard time explaining to new users the rationale for > keeping these together, and it only increases the barrier to entry (to both > projects) to have this added complexity of two very different code bases > munged together. > >> > >> Just my 2 cents... > >> -Doug > >> > >> On Tue, May 12, 2020 at 7:30 AM Alan Woodward <romseyg...@gmail.com> > wrote: > >>> > >>> One advantage I find with the way Elasticsearch and Lucene interact is > that ES doesn’t depend on the master branch. We upgrade our master branch > frequently to keep up to date with the latest release branch, and that lets > us find regressions or API problems pretty quickly, but it also insulates > us from having to make big changes immediately. I find this really useful > for things like deprecations. Let’s say we deprecate a particular API in > the release branch, and remove it entirely in master. Currently, that > means Solr needs to immediately switch over to the new API in its master > branch. But the whole point of doing deprecations first is that it gives > users time to find issues with the replacements - if we find that the > replacement API doesn’t quite fit in ES, we have time to work out either > how to change our code, or to improve the new API, but because the > deprecated version is still there we’re not blocked from upgrading and > getting other improvements. Solr, meanwhile, may end up with a hacky > workaround because that’s what got tests passing for the Lucene developer; > or worse, we end up just copying the deprecated API wholesale into Solr and > abandoning it there - witness TrieField or UninvertingReader. > >>> > >>> > On 11 May 2020, at 19:05, Atri Sharma <a...@apache.org> wrote: > >>> > > >>> > My two cents: > >>> > > >>> > As a Lucene heavy developer, I have several found maintaining Solr > >>> > dependencies while making large changes a bit cumbersome. I believe > >>> > Lucene and Solr should exist in a symbiotic relationship but not > >>> > tightly coupled with each other. > >>> > > >>> > > >>> > On Mon, May 11, 2020 at 7:22 PM Erik Hatcher <erik.hatc...@gmail.com> > wrote: > >>> >> > >>> >> Without reading much or replying to any specific points made on > this thread, here's my raw thoughts on this age-old topic.... (finally > coming out of my cocoon after taking things in for a bit) > >>> >> > >>> >> Solr is a search -server- with distributed capabilities, that > leverages the magic of Lucene underneath. Solr depends on Lucene, is a > consumer of it. Lucene is a tight search -library- with little to no > external dependencies. Their purposes and end-users are different. > >>> >> > >>> >> I was never really for the grand unification of Lucene and Solr > back in the day because: > >>> >> > >>> >> - Solr's developer experience would be greatly streamlined, faster, > cleaner, leaner, and focused > >>> >> - Having Lucene change when Solr doesni't (yet) adapt to those > changes leads to confusion and inconsistency, loose wires hanging out of > the wall unconnected or duct taped together > >>> >> - It simply makes sense to keep Lucene versioned and tightly > controlled for upgrades, various testing configurations varying Lucene > versions, within Solr > >>> >> - Solr could have a very concerted upgrade effort for Lucene > capability jumps, with a focused upgrade effort at the > changed/improved/added touch points just like other dependencies within > Solr (like Tika and Jetty) > >>> >> > >>> >> Those points all kinda say the same thing.... Solr depends on > "lucene.jar" and I'm in the camp that thinks Solr and Lucene development, > communities, and end-users/consumers would all greatly benefit from a fancy > new TLP and focused community for solr.apache.org and a tight(er) > relationship with the Lucene community as an involved and vested consumer. > >>> >> > >>> >> Erik > >>> >> > >>> > > >>> > > >>> > -- > >>> > Regards, > >>> > > >>> > Atri > >>> > Apache Concerted > >>> > > >>> > --------------------------------------------------------------------- > >>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > >>> > For additional commands, e-mail: dev-h...@lucene.apache.org > >>> > > >>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > >>> For additional commands, e-mail: dev-h...@lucene.apache.org > >>> > >> > >> > >> -- > >> Doug Turnbull | CTO | OpenSource Connections, LLC | 240.476.9983 > >> Author: Relevant Search; Contributor: AI Powered Search > >> This e-mail and all contents, including attachments, is considered to > be Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- *Doug Turnbull **| CTO* | OpenSource Connections <http://opensourceconnections.com>, LLC | 240.476.9983 Author: Relevant Search <http://manning.com/turnbull>; Contributor: *AI Powered Search <http://aipoweredsearch.com>* This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
