Re: [VOTE] Solr to become a top-level Apache project (TLP)
For the reasons I included in my message on the discussion thread ( https://mail-archives.apache.org/mod_mbox/lucene-dev/202005.mbox/%3CCAPsCRSWds%3Dctvf837BFRXqyPxGLJ-5a%2B3hCa6HwrA8FFR5MQkA%40mail.gmail.com%3E) I'm voting in favor of this. I hope the actual execution of it will follow the path of a code/release split for a few versions before a more permanent project split, but if not I think the benefits of a split will still be seen, although perhaps down a bumpier road. +1 (binding) On Sat, May 16, 2020 at 9:44 PM Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > +1 (binding) > > On Tue, May 12, 2020 at 1:07 PM Dawid Weiss wrote: > >> Dear Lucene and Solr developers! >> >> According to an earlier [DISCUSS] thread on the dev list [2], I am >> calling for a vote on the proposal to make Solr a top-level Apache >> project (TLP) and separate Lucene and Solr development into two >> independent entities. >> >> To quickly recap the reasons and consequences of such a move: it seems >> like the reasons for the initial merge of Lucene and Solr, around 10 >> years ago, have been achieved. Both projects are in good shape and >> exhibit signs of independence already (mailing lists, committers, >> patch flow). There are many technical considerations that would make >> development much easier if we move Solr out into its own TLP. >> >> We discussed this issue [2] and both PMC members and committers had a >> chance to review all the pros and cons and express their views. The >> discussion showed that there are clearly different opinions on the >> matter - some people are in favor, some are neutral, others are >> against or not seeing the point of additional labor. Realistically, I >> don't think reaching 100% level consensus is going to be possible -- >> we are a diverse bunch with different opinions and personalities. I >> firmly believe this is the right direction hence the decision to put >> it under the voting process. Should something take a wrong turn in the >> future (as some folks worry it may), all blame is on me. >> >> Therefore, the proposal is to separate Solr from under Lucene TLP, and >> make it a TLP on its own. The initial structure of the new PMC, >> committer base, git repositories and other managerial aspects can be >> worked out during the process if the decision passes. >> >> Please indicate one of the following (see [1] for guidelines): >> >> [ ] +1 - yes, I vote for the proposal >> [ ] -1 - no, I vote against the proposal >> >> Please note that anyone in the Lucene+Solr community is invited to >> express their opinion, though only Lucene+Solr committers cast binding >> votes (indicate non-binding votes in your reply, please). >> >> The vote will be active for a week to give everyone a chance to read >> and cast a vote. >> >> Dawid >> >> [1] https://www.apache.org/foundation/voting.html >> [2] >> https://lists.apache.org/thread.html/rfae2440264f6f874e91545b2030c98e7b7e3854ddf090f7747d338df%40%3Cdev.lucene.apache.org%3E >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> > > -- > Regards, > Shalin Shekhar Mangar. >
Re: [DISCUSS] Lucene-Solr split (Solr promoted to TLP)
There’s something enticing when thinking of Lucene and Solr as independent codebases. I’ve always thought of Lucene as core search (indexing, analysis, tokenization, etc…) and Solr as a search experience. Lucene is more a library (or set of libraries) used by applications providing search experiences. Solr is just one of those applications - it provides the experience of search as a service, and feels focused on making search approachable and palatable to search novices. The work I put into streaming expressions was born out of a desire to more widely expose search functionality. The streaming API drew me in because it exposed a new way to interact with core search functionality, and expressions came out of wanting to make it all easy to use for end users. I didn’t, and don’t, give a whole lot of thought to the internals of Lucene. I like a good user experience and I see Solr as an application trying to provide that. I do, however, have concerns about the long-term impact of a split. Lucene is able to set a very explicit N-1 backward compatibility policy because it can have less immediate concern for the downstream user. And this is not to denigrate Lucene at all - in fact I agree with that policy for core search functionality. If and when incompatible changes lead to significant gains they can and are made. Inefficient older ways are not brought forward further than necessary. Solr has to be concerned with their end users, who may be relative search novices, when considering backward incompatible changes. More thought is given to the experience and impact of upgrades. How are those issues dealt with across replicas and shards? What will happen in a cloud made up of lucene indexes of varying versions? My concern revolves around what happens if (when) Solr falls behind Lucene. Will it ever be able to catch up? There’s an argument to be made that Solr being a consistent N versions behind Lucene has some value to the Solr project. But, what happens if Solr gets a slower release cadence? Will it fall further and further behind? Will its inability to use the latest and greatest in Lucene be the impetus for a community splitting fork? Will a new search application come along without the legacy concerns of Solr and become a more enticing option? Perhaps, to all of that. I can’t really say. What I can say is I don’t think it’s appropriate to stifle the growth, or in this case the change, of a community because of fear of the unknown. Yes, I am worried that a project split will lead to trouble and issues for Solr, and some of those fears are born out of how I know my company uses Solr. But I also think a lot of good could come out of a split. It’d be exciting to see how a Lucene community advances the state of the art of core search, and how a Solr community provides a clean and easily digestible search experience to end users. Will Lucene become more embeddable? Will Solr become more plug-n-play? I’m a fan of Christine’s suggestion of first executing a code and release split and later, after seeing the impact of such a split, decide on a project split. Full disclosure, Christine and I work at the same company. I think independent codebases will in the end benefit both, though I do agree there is more inherent and immediate risk to Solr. - Dennis On Fri, May 15, 2020 at 4:03 AM Dawid Weiss wrote: > Hi Christine! > > > * After a while (perhaps with Lucene 10.0 or perhaps at some other > natural point) we re-arrive at the "together or separate" question. If > splitting worked well then Solr promotion to TLP could be a natural next > step > > My whole point is that I think the split is by large already there: > the mailing lists, the issues, the codebase (git constitutes common > storage but the build system and nearly anything else pretty much > independent with Solr consuming Lucene artifacts). I also believe the > will to separate the projects has been with (some of) us for a long > time and postponing this decision won't change anything. > > Dawid > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
Re: [VOTE] Solr to become a top-level Apache project (TLP)
+1 (binding) On Tue, May 12, 2020 at 1:07 PM Dawid Weiss wrote: > Dear Lucene and Solr developers! > > According to an earlier [DISCUSS] thread on the dev list [2], I am > calling for a vote on the proposal to make Solr a top-level Apache > project (TLP) and separate Lucene and Solr development into two > independent entities. > > To quickly recap the reasons and consequences of such a move: it seems > like the reasons for the initial merge of Lucene and Solr, around 10 > years ago, have been achieved. Both projects are in good shape and > exhibit signs of independence already (mailing lists, committers, > patch flow). There are many technical considerations that would make > development much easier if we move Solr out into its own TLP. > > We discussed this issue [2] and both PMC members and committers had a > chance to review all the pros and cons and express their views. The > discussion showed that there are clearly different opinions on the > matter - some people are in favor, some are neutral, others are > against or not seeing the point of additional labor. Realistically, I > don't think reaching 100% level consensus is going to be possible -- > we are a diverse bunch with different opinions and personalities. I > firmly believe this is the right direction hence the decision to put > it under the voting process. Should something take a wrong turn in the > future (as some folks worry it may), all blame is on me. > > Therefore, the proposal is to separate Solr from under Lucene TLP, and > make it a TLP on its own. The initial structure of the new PMC, > committer base, git repositories and other managerial aspects can be > worked out during the process if the decision passes. > > Please indicate one of the following (see [1] for guidelines): > > [ ] +1 - yes, I vote for the proposal > [ ] -1 - no, I vote against the proposal > > Please note that anyone in the Lucene+Solr community is invited to > express their opinion, though only Lucene+Solr committers cast binding > votes (indicate non-binding votes in your reply, please). > > The vote will be active for a week to give everyone a chance to read > and cast a vote. > > Dawid > > [1] https://www.apache.org/foundation/voting.html > [2] > https://lists.apache.org/thread.html/rfae2440264f6f874e91545b2030c98e7b7e3854ddf090f7747d338df%40%3Cdev.lucene.apache.org%3E > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- Regards, Shalin Shekhar Mangar.
Re: [DISCUSS] 8.5.2 Release?
Done. On Sat, May 16, 2020 at 1:49 PM Tomás Fernández Löbbe wrote: > Mike, I'm about to merge https://issues.apache.org/jira/browse/SOLR-14471 > > On Fri, May 15, 2020 at 11:51 AM Mike Drob wrote: > >> Thanks for pointing those commits out to me, Noble. I cherry-picked the >> doap and backcompat index changes to branch_8 and master. >> >> As for the releases... >> >> mdrob-imp:/tmp/lucene $ svn log | head >> >> r39590 | noble | 2020-05-13 21:02:20 -0500 (Wed, 13 May 2020) | 1 line >> >> Stop mirroring 7.7.2 releases >> >> r39589 | noble | 2020-05-13 21:02:02 -0500 (Wed, 13 May 2020) | 1 line >> >> Stop mirroring 7.7.3 releases >> >> >> I suspect this shouldn't have happened, and I was going to revert (svn >> reverse merge...) commit r39589 here to fix that, but there is waaay too >> much stuff in there that shouldn't be there, like a bunch of maven >> artifacts. Can you take a look at this and clean it up? As of right now, >> 7.7.3 is missing completely from >> https://projects.apache.org/json/foundation/releases.json (which >> continues to block the releaseWizard script). >> >> Thanks, >> Mike >> >> On Fri, May 15, 2020 at 2:56 AM Jan Høydahl >> wrote: >> >>> ./poll-mirrors.py -version 7.7.3 >>> >>> 1 ↵ 11084 09:51:08 >>> >>> 2020-05-15 09:52:38 >>> Polling 204 Apache Mirrors and Maven Central... >>> >>> .X.XX..XXX.X >>> >>> 7.7.3 is downloadable from Maven Central >>> 7.7.3 is downloadable from 5/204 Apache Mirrors (2.45%) >>> Sleeping for 262 seconds... >>> >>> The RM job is not done until it is done… >>> >>> Note that you need to apply >>> https://issues.apache.org/jira/browse/LUCENE-9288 to the poll-mirrors >>> script for it to work. Do you have any idea why only 5 mirrors are updated >>> Noble? >>> >>> Jan >>> >>> 14. mai 2020 kl. 19:41 skrev Mike Drob : >>> >>> Noble, >>> >>> We're still missing a few: >>> >>> >>> https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo#ReleaseTodo-UpdatetheprojectDOAPfiles >>> >>> https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo#ReleaseTodo-GenerateBackcompatIndexes >>> >>> Also, the release is not on the mirrors - >>> https://lucene.apache.org/core/downloads.html >>> The link to >>> https://www.apache.org/dyn/closer.lua/lucene/java/7.7.3/lucene-7.7.3-src.tgz >>> doesn't resolve to anything... >>> >>> Mike >>> >>> On Wed, May 13, 2020 at 9:18 PM Noble Paul wrote: >>> I just finished all the steps given in https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo Please let me know if anything is missing. I still don't see the solr release details in https://lucene.apache.org/ How do i do it? On Thu, May 14, 2020 at 10:47 AM Noble Paul wrote: > > I'll fix them today > > On Thu, May 14, 2020, 9:19 AM Mike Drob wrote: >> >> Thanks. I’ve found one small change for the release script so far, I plan to batch all my notes together and commit them at the end of the process. >> >> Right now I think I’m waiting on a few final steps from 7.7.3 to complete and then we’ll be ready to roll with 8.5.2 >> >> On Wed, May 13, 2020 at 4:59 PM Jan Høydahl wrote: >>> >>> Mike, I merged latest changes to releaseWizard to branch_8_5 as well. So you’ll be first to test the new instructions for updating the web site. So please be prepared for discovering quirks in that part of the releaseWizard. >>> >>> Jan >>> >>> > 7. mai 2020 kl. 19:13 skrev Mike Drob : >>> > >>> > Devs, >>> > >>> > I know that we had 8.5.1 only a few weeks ago, but with the fix for LUCENE-9350 I think we should consider another bug-fix. I know that without it I will be explicitly recommending several users to stay off of 8.5.x on their upgrade plans. There are some pretty scary looking charts on SOLR-14428 that describe the impact of the bug in more detail. >>> > >>> > I'd be happy to volunteer as RM for this, would probably be looking at trying to get it a vote started sometime next week. >>> > >>> > Thanks, >>> > Mike >>> > >>> > >>> > https://issues.apache.org/jira/browse/SOLR-14428 >>> > https://issues.apache.org/jira/browse/LUCENE-9350 >>> > >>> > - >>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> > For additional commands, e-mail: dev-h...@lucene.apache.org >>> > >>> >>> >>>
Re: Lots of failures lately for lucene.index.TestBackwardsCompatability.testAllVersionsTested
OK, I’ll give it a rest until tomorrow and look again, thanks! > On May 16, 2020, at 4:54 PM, Simon Willnauer > wrote: > > I thinks it’s fixed now. The 7.7.3 version was missing. > > Simon > > >> On 16. May 2020, at 22:45, Erick Erickson wrote: >> >> Unfortunately the seed doesn’t reproduce, and I tried beasting it without >> getting any fails in 700 iterations (and counting). >> >> Here’s one example, I see three others in the last couple of hours. >> >> I’ve done zero investigation into where these are coming from, but I did >> notice there started being a lot of them starting 2-3 (?) days ago. >> >> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Windows/1134/ >> Java: 64bit/jdk-11.0.6 -XX:+UseCompressedOops -XX:+UseParallelGC >> >> 6 tests failed. >> FAILED: >> org.apache.lucene.index.TestBackwardsCompatibility.testAllVersionsTested >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Lots of failures lately for lucene.index.TestBackwardsCompatability.testAllVersionsTested
I thinks it’s fixed now. The 7.7.3 version was missing. Simon > On 16. May 2020, at 22:45, Erick Erickson wrote: > > Unfortunately the seed doesn’t reproduce, and I tried beasting it without > getting any fails in 700 iterations (and counting). > > Here’s one example, I see three others in the last couple of hours. > > I’ve done zero investigation into where these are coming from, but I did > notice there started being a lot of them starting 2-3 (?) days ago. > > Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Windows/1134/ > Java: 64bit/jdk-11.0.6 -XX:+UseCompressedOops -XX:+UseParallelGC > > 6 tests failed. > FAILED: > org.apache.lucene.index.TestBackwardsCompatibility.testAllVersionsTested > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [DISCUSS] 8.5.2 Release?
Mike, I'm about to merge https://issues.apache.org/jira/browse/SOLR-14471 On Fri, May 15, 2020 at 11:51 AM Mike Drob wrote: > Thanks for pointing those commits out to me, Noble. I cherry-picked the > doap and backcompat index changes to branch_8 and master. > > As for the releases... > > mdrob-imp:/tmp/lucene $ svn log | head > > r39590 | noble | 2020-05-13 21:02:20 -0500 (Wed, 13 May 2020) | 1 line > > Stop mirroring 7.7.2 releases > > r39589 | noble | 2020-05-13 21:02:02 -0500 (Wed, 13 May 2020) | 1 line > > Stop mirroring 7.7.3 releases > > > I suspect this shouldn't have happened, and I was going to revert (svn > reverse merge...) commit r39589 here to fix that, but there is waaay too > much stuff in there that shouldn't be there, like a bunch of maven > artifacts. Can you take a look at this and clean it up? As of right now, > 7.7.3 is missing completely from > https://projects.apache.org/json/foundation/releases.json (which > continues to block the releaseWizard script). > > Thanks, > Mike > > On Fri, May 15, 2020 at 2:56 AM Jan Høydahl wrote: > >> ./poll-mirrors.py -version 7.7.3 >> >> 1 ↵ 11084 09:51:08 >> >> 2020-05-15 09:52:38 >> Polling 204 Apache Mirrors and Maven Central... >> >> .X.XX..XXX.X >> >> 7.7.3 is downloadable from Maven Central >> 7.7.3 is downloadable from 5/204 Apache Mirrors (2.45%) >> Sleeping for 262 seconds... >> >> The RM job is not done until it is done… >> >> Note that you need to apply >> https://issues.apache.org/jira/browse/LUCENE-9288 to the poll-mirrors >> script for it to work. Do you have any idea why only 5 mirrors are updated >> Noble? >> >> Jan >> >> 14. mai 2020 kl. 19:41 skrev Mike Drob : >> >> Noble, >> >> We're still missing a few: >> >> >> https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo#ReleaseTodo-UpdatetheprojectDOAPfiles >> >> https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo#ReleaseTodo-GenerateBackcompatIndexes >> >> Also, the release is not on the mirrors - >> https://lucene.apache.org/core/downloads.html >> The link to >> https://www.apache.org/dyn/closer.lua/lucene/java/7.7.3/lucene-7.7.3-src.tgz >> doesn't resolve to anything... >> >> Mike >> >> On Wed, May 13, 2020 at 9:18 PM Noble Paul wrote: >> >>> I just finished all the steps given in >>> https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo >>> >>> Please let me know if anything is missing. >>> I still don't see the solr release details in https://lucene.apache.org/ >>> >>> How do i do it? >>> >>> >>> On Thu, May 14, 2020 at 10:47 AM Noble Paul >>> wrote: >>> > >>> > I'll fix them today >>> > >>> > On Thu, May 14, 2020, 9:19 AM Mike Drob wrote: >>> >> >>> >> Thanks. I’ve found one small change for the release script so far, I >>> plan to batch all my notes together and commit them at the end of the >>> process. >>> >> >>> >> Right now I think I’m waiting on a few final steps from 7.7.3 to >>> complete and then we’ll be ready to roll with 8.5.2 >>> >> >>> >> On Wed, May 13, 2020 at 4:59 PM Jan Høydahl >>> wrote: >>> >>> >>> >>> Mike, I merged latest changes to releaseWizard to branch_8_5 as >>> well. So you’ll be first to test the new instructions for updating the web >>> site. So please be prepared for discovering quirks in that part of the >>> releaseWizard. >>> >>> >>> >>> Jan >>> >>> >>> >>> > 7. mai 2020 kl. 19:13 skrev Mike Drob : >>> >>> > >>> >>> > Devs, >>> >>> > >>> >>> > I know that we had 8.5.1 only a few weeks ago, but with the fix >>> for LUCENE-9350 I think we should consider another bug-fix. I know that >>> without it I will be explicitly recommending several users to stay off of >>> 8.5.x on their upgrade plans. There are some pretty scary looking charts on >>> SOLR-14428 that describe the impact of the bug in more detail. >>> >>> > >>> >>> > I'd be happy to volunteer as RM for this, would probably be >>> looking at trying to get it a vote started sometime next week. >>> >>> > >>> >>> > Thanks, >>> >>> > Mike >>> >>> > >>> >>> > >>> >>> > https://issues.apache.org/jira/browse/SOLR-14428 >>> >>> > https://issues.apache.org/jira/browse/LUCENE-9350 >>> >>> > >>> >>> > >>> - >>> >>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> >>> > For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>> > >>> >>> >>> >>> >>> >>> - >>> >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>> >>> >>> >>> -- >>>
Lots of failures lately for lucene.index.TestBackwardsCompatability.testAllVersionsTested
Unfortunately the seed doesn’t reproduce, and I tried beasting it without getting any fails in 700 iterations (and counting). Here’s one example, I see three others in the last couple of hours. I’ve done zero investigation into where these are coming from, but I did notice there started being a lot of them starting 2-3 (?) days ago. Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Windows/1134/ Java: 64bit/jdk-11.0.6 -XX:+UseCompressedOops -XX:+UseParallelGC 6 tests failed. FAILED: org.apache.lucene.index.TestBackwardsCompatibility.testAllVersionsTested - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Solr to become a top-level Apache project (TLP)
+1 (binding) for Solr to become TLP (and for separating the two projects from each other) Regards, Doron -- Sent from: https://lucene.472066.n3.nabble.com/Lucene-Java-Developer-f564358.html - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
