I’ve picked up a few of the improvements and brought them over to main branch, fixed a few busy waits, improved some concurrent access, minor stuff really. I’m currently busy trying to digest and experiment with the jetty changes Mark mentioned earlier, so hopefully that’s another piece that can come in early. There’s some more components, but I don’t want to promise anything since it’s all competing with other priorities as well.
Mike On Tue, Jun 8, 2021 at 5:43 PM Jan Høydahl <[email protected]> wrote: > +1 to apply same code formatting on the ref branch to be able to compare. > > Solr still suffers from sloow and unstable tests. I whsh there were some > lessons learnt from the ref-branch that could be ported over. I don't know > if you separated out "Integration" tests from unit tests or something that > we could bring over. Or got rid of some more bad busy-wait / sleep > patterns? Or perhaps getting rid of unnecessary CloudCluster > initializations, unneccessary waiting for cluster state or whatever? If not > copy-pastable, having the biggest wins written down as short recipies of > how to fix the test, would help a lot I guess. Or perhaps the test speedup > all depends on the rest of the improvements made in non-test code... That's > what it sounds like to me, that you cannot have an improvement in > isolation, you need to do it all in one go :( > > Jan > > 8. jun. 2021 kl. 23:42 skrev David Smiley <[email protected]>: > > Sorry to hear that Mark. I hope it might be useful for little bits 'n > pieces. Occasionally I'm looking at some class, maybe a test, and I > quickly do a comparison to the ref branch to wonder what you did there. > There is a grand source code reformatting that is underway; we should apply > that processing to that branch so that the comparisons remain useful. > > Uwe: In light of Mark's comments, I think you should disable and/or remove > the CI build jobs. > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Tue, Jun 8, 2021 at 1:27 PM Mark Miller <[email protected]> wrote: > >> At this point I doubt the ref branch is going to help anyone much other >> than me as the need arises from any assignment i may have or in sharing >> information with others as the need arises (I put together a presentation >> on Jetty from it for others on my team a bit ago.) >> >> Extracting anything but isolated help for what I may need from it is >> currently beyond my ability. >> >> * It address and fixes thousands of bugs and issues of varying importance >> and interconnectedness. Where would you even start. >> >> * It fully fully embraces, fixes and extends HTTP2 support, async, async >> IO, Jetty. Doing that at all reasonably requires a tremendous amount of >> work as each is very sensitive to getting things very close to right all >> over the map. Even after spending ridiculous amounts of time on core >> issues, that work is heavy to make solid. >> >> * It speeds up and hardens test after test after test. Ridiculous parts >> and efforts involved, connected to everything else. >> >> * It minimizes resources and objects and GC tremendously everywhere, >> moves more off heap, mmaps transaction log files, parallelizes tons more, >> and leaves Lucene at the top of garbage generation stack (though Lucene is >> certainly reasonable there). Again, where do you start. >> >> * And then it does some things that build on having everything else below >> it. >> >> Other than a few items and hanging chads, it does or opens up whatever I >> ever wanted for SolrCloud. And the effort is not simply each of the many >> many items - it’s the crazy pain staking time and care working through all >> of the issues and connectedness and surface area for everything. >> >> When I get some time to set it up for others, it will offer an alternate >> view of what Solr could do, but ive imagined for a while it will mostly >> offer me things and occasionally some sharing and demoing for what some >> others have been interested in. And as I have things that I have to do, it >> provides me with a working map and model of what the issues are, what I may >> need to get around them, and where the current stuff stands. >> >> It’s what I would do with SolrCloud. It’s focused on what I think the >> problems are, what I think matters to the end result. I don’t even often >> see strong alignment on that with anyone other than silently across >> timelines with Dat. Always find something I hadn’t noticed from Dat and >> think, man, that guy is on my page. >> >> >> MRM >> >> On Fri, Jun 4, 2021 at 8:51 AM Jan Høydahl <[email protected]> wrote: >> >>> Hi, >>> >>> We have an upcoming committers meeting soon, and will likely touch on >>> the topic of Reference Branch. >>> As we could probably waste the whole meeting on that one topic, I'm >>> starting this thread to "warm up". >>> >>> As for me, I'm totally not up to date on what the status is. I don't >>> even know if anyone have been looking at the branch at all lately. >>> *Is there anyone who can give a short status update on what the state is*, >>> who is working on it, what is the next steps, what are the >>> challenges/blockers etc? >>> >>> As 9.0 is coming up, and the project plans for further changes, perhaps >>> even reorganization of the git folder structure, move to Java 17 on main >>> branch etc etc, I think there is still a potential window between now and 6 >>> months ahead where porting code from ref-branch is still doable. After that >>> it will become more and more problematic. >>> >>> Jan >>> >> -- >> - Mark >> >> http://about.me/markrmiller >> > >
