> I think the danger is high to treat this branch as a black box (or an "all or > nothing").
True Ilan. Ideally, I would like a few of us to study the code & start pulling in changes we are confident of (even to 8x branch, why not). We cannot burden a single developer to do everything. This cannot be a task just for one or 2 devs. We all will have to work together to decompose the changes and digest them into master. I can do my bit. But, I'm sure we may hit a point where certain changes cannot be isolated and absorbed. We will have to collectively make a call, how to absorb them On Tue, Oct 6, 2020 at 9:00 PM Ishan Chattopadhyaya <[email protected]> wrote: > > > I'm willing to help and I believe others will too if the amount of work for > contributing is reasonable (i.e. not a three months effort). > > I looked into the possibility of doing so. To me, it seemed to be that it is > very hard to do so: possibly 1 year project for me. Problem is that it is > hard to pull out a particular class of improvements (say thread management > improvement) and have all tests pass with it (because tests have gotten > extensive improvements of their own) and also observe the effect of the > improvement. IIUC, every improvement to Solr seemed to require many > iterations to get the tests happy. I remember Mark telling me that it may not > even be possible for him to do something like that (i.e. bring all changes > into master as tiny pieces). > > What I volunteered to do, however, is to decompose roughly all the general > improvements into smaller, manageable commits. However, making sure all tests > pass at every commit point is beyond my capability. > > On Tue, 6 Oct, 2020, 3:10 pm Ilan Ginzburg, <[email protected]> wrote: >> >> Another option to integrate this work into the main code line would be to >> understand what changes have been made and where (Mark's descriptions in >> Slack go in the right way but are still too high level), and then port or >> even redo them in main, one by one. >> >> I think the danger is high to treat this branch as a black box (or an "all >> or nothing"). Using the merging itself to change our understanding and >> increase our knowledge of what was done can greatly reduce the risk. >> >> We do develop new features in Solr 9 without beta releasing them, so if we >> port Mark's improvements by small chunks (and maybe in the process decide >> that some should not be ported or not now) I don't see why this can't >> integrate to become like other improvements done to the code. If specific >> changes do require a beta release, do that release from master and pick the >> right moment. >> >> I'm willing to help and I believe others will too if the amount of work for >> contributing is reasonable (i.e. not a three months effort). This requires >> documenting the changes done in that branch, pointing to where these changes >> happened and then picking them up one by one and porting them more or less >> independently of each other. We might only port a subset of changes by the >> time 9.0 is released, that's fine we can continue in following releases. >> >> My 2 cents... >> Ilan >> >> Le mar. 6 oct. 2020 à 09:56, Noble Paul <[email protected]> a écrit : >>> >>> Yes, A docker image will definitely help. I wasn't trying to downplay that >>> >>> On Tue, Oct 6, 2020 at 6:55 PM Ishan Chattopadhyaya >>> <[email protected]> wrote: >>> > >>> > >>> > > Docker is not a big requirement for large scale installations. Most of >>> > > them already have their own install scripts. Availability of docker is >>> > > not important for them. If a user is only encouraged to install Solr >>> > > because of a docker image , most likely they are not running a large >>> > > enough cluster >>> > >>> > I disagree, Noble. Having a docker image us going to be useful to some >>> > clients, with complex usecases. Great point, David! >>> > >>> > On Tue, 6 Oct, 2020, 1:09 pm Ishan Chattopadhyaya, >>> > <[email protected]> wrote: >>> >> >>> >> As I said, I'm *personally* not confident in putting such a big >>> >> changeset into master that wasn't vetted in a real user environment >>> >> widely. I have, in the past, done enough bad things to Solr (directly or >>> >> indirectly), and I don't want to repeat the same. Also, I'll be very >>> >> uncomfortable if someone else did so. >>> >> >>> >> Having said this, if someone else wants to port the changes over to >>> >> master *without first getting enough real world testing*, feel free to >>> >> do so, and I can focus my efforts elsewhere. >>> >> >>> >> On Tue, 6 Oct, 2020, 9:22 am Tomás Fernández Löbbe, >>> >> <[email protected]> wrote: >>> >>> >>> >>> I was thinking (and I haven’t flushed it out completely but will throw >>> >>> the idea) that an alternative approach with this timeline could be to >>> >>> cut 9x branch around November/December? And then you could merge into >>> >>> master, it would have the latest changes from master plus the ref >>> >>> branch changes. From there any nightly build could be use to help >>> >>> test/debug. >>> >>> >>> >>> That said I don’t know for sure what are the changes in the branch that >>> >>> do not belong in 9. The problem with them being 10x only is that >>> >>> backports would potentially be more difficult for all the life of 9. >>> >>> >>> >>> On Mon, Oct 5, 2020 at 4:54 PM Noble Paul <[email protected]> wrote: >>> >>>> >>> >>>> >I don't think it can be said what committers do and don't do with >>> >>>> >regards to running Solr. All of us would answer this differently and >>> >>>> >at different points in time. >>> >>>> >>> >>>> " I have run it in one large cluster, so it is certified to be bug >>> >>>> free/stable" I don't think it's a reasonable approach. We need as much >>> >>>> feedback from our users because each of them stress Solr in a >>> >>>> different way. This is not to suggest that committers are not doing >>> >>>> testing or their tests are not valid. When I talk to the committers >>> >>>> out here they say they do not see any performance stability issues at >>> >>>> all. But, my client reports issues on a day to day basis. >>> >>>> >>> >>>> >>> >>>> >>> >>>> > Definitely publish a Docker image BTW -- it's the best way to try >>> >>>> > out any software. >>> >>>> >>> >>>> Docker is not a big requirement for large scale installations. Most of >>> >>>> them already have their own install scripts. Availability of docker is >>> >>>> not important for them. If a user is only encouraged to install Solr >>> >>>> because of a docker image , most likely they are not running a large >>> >>>> enough cluster >>> >>>> >>> >>>> >>> >>>> >>> >>>> On Tue, Oct 6, 2020, 6:30 AM David Smiley <[email protected]> wrote: >>> >>>>> >>> >>>>> Thanks so much for your responses Ishan... I'm getting much more >>> >>>>> information in this thread than my attempts to get questions answered >>> >>>>> on the JIRA issue months ago. And especially, thank you for >>> >>>>> volunteering for the difficult porting efforts! >>> >>>>> >>> >>>>> Tomas said: >>> >>>>>> >>> >>>>>> I do agree with the previous comments that calling it "Solr 10" >>> >>>>>> (even with the "-alpha") would confuse users, maybe use "reference"? >>> >>>>>> or maybe something in reference to SOLR-14788? >>> >>>>> >>> >>>>> >>> >>>>> I have the opposite opinion. This word "reference" is baffling to me >>> >>>>> despite whatever Mark's explanation is. I like the justification >>> >>>>> Ishan gave for 10-alpha and I don't think I could re-phrase his >>> >>>>> justification any better. *If* the release was _not_ official (thus >>> >>>>> wouldn't show up in the usual places anyone would look for a >>> >>>>> release), I think it would alleviate that confusion concern even >>> >>>>> more, although I think "alpha" ought to be enough of a signal not to >>> >>>>> use it without digging deeper on what's going on. >>> >>>>> >>> >>>>> Alex then Ishan said: >>> >>>>>> >>> >>>>>> > Maybe we could release it to >>> >>>>>> > committers community first and dogfood it "internally"? >>> >>>>>> >>> >>>>>> Alex: It is meaningless. Committers don't run large scale >>> >>>>>> installations. We barely even have time to take care of running unit >>> >>>>>> tests before destabilizing our builds. We are not the right >>> >>>>>> audience. However, we all can anyway check out the branch and start >>> >>>>>> playing with it, even without a release. There are orgs that don't >>> >>>>>> want to install any code that wasn't officially released; this >>> >>>>>> release is geared towards them (to help us test this at their scale). >>> >>>>> >>> >>>>> >>> >>>>> I don't think it can be said what committers do and don't do with >>> >>>>> regards to running Solr. All of us would answer this differently and >>> >>>>> at different points in time. From time to time, though not at >>> >>>>> present, I've been well positioned to try out a new version of Solr >>> >>>>> in a stage/test environment to see how it goes. (Putting on my >>> >>>>> Salesforce metaphorical hat...) Even though I'm not able to deploy it >>> >>>>> in a realistic way today, I'm able to run a battery of tests to see >>> >>>>> if one of the features we depend on have changed or is broken. >>> >>>>> That's useful feedback to an alpha release! And even though I'm >>> >>>>> saying I'm not well positioned to try out some new Solr release in a >>> >>>>> production-ish setting now, it's something I could make a good case >>> >>>>> for internally since upgrades take a lot of effort where I work. >>> >>>>> It's in our interest for SolrCloud to be very stable (of course). >>> >>>>> >>> >>>>> Regardless, I think what you're driving at Ishan is that you want an >>> >>>>> "official" release -- one that goes through the whole ceremony. You >>> >>>>> believe that people would be more likely to use it. I think all we >>> >>>>> need to do is announce (similar to a real release) that there is some >>> >>>>> unofficial alpha distribution and that we want to solicit your >>> >>>>> feedback -- basically, help us find bugs. Definitely publish a >>> >>>>> Docker image BTW -- it's the best way to try out any software. I'm >>> >>>>> -0 on doing an official release for alpha software because it's >>> >>>>> unnecessary to achieve the goals and somewhat confusing. I think the >>> >>>>> Solr 4 alpha/beta situation was different -- it was not some fork a >>> >>>>> committer was maintaining; it was the master branch of its time, and >>> >>>>> it was destined to be the very next release, not some possible future >>> >>>>> release. >>> >>>>> >>> >>>>> ~ David Smiley >>> >>>>> Apache Lucene/Solr Search Developer >>> >>>>> http://www.linkedin.com/in/davidwsmiley >>> >>> >>> >>> -- >>> ----------------------------------------------------- >>> Noble Paul >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> -- ----------------------------------------------------- Noble Paul --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
