Hmm, quite some ideas there :-) One step at a time, though I'm not sure I agree with the destination you fantasized about. Would the next step "just" be a matter of splitting out the parts of SolrDispatchFilter that are not very filter-y and creating a SolrServlet from that? Don't need to retain any dubious things like excludePatterns or "passthrough". Thinking of Solr 10 here; more freedom to do something different that works but might break <1% of users for some obscure thing. FWIW I'd be happy to be a code reviewer. I tagged you for a review on something nearby the other day.
On Tue, Oct 22, 2024 at 5:35 PM Gus Heck <gus.h...@gmail.com> wrote: > Converting Solr to a servlet, or even a query servlet, an update servlet > and an admin servlet because those are really very separate operations has > been on my mind for a long time. The overlap among those operations is all > in things like authentication or tracing that should be reusable > ServletFilters, is one of those things I have long thought about, and these > thoughts are part of why I moved CoreContainer Initialization out of the > very bloated SolrDispatchFilter. Having CoreContainer exist before any > processing code is initialized or invoked was a critical first step to any > disassembly into a series of reusable filters and/or separation of > concerns. There's an even more radical possibility of making CoreContainer > a container (jetty) level resource, and separating the update/admin/query > bits into entirely distinct web applications... any of which might not be > installed in a particular running jetty. (assuming use sort of replication > to transfer indexes to installations hosting the query app... ) That's > pretty radical though, and servletization (TM) and separation of concerns > come first anyway. > > My recent web-socket stuff is also a sideways exploration of ways to get a > separate thing running inside our code base. Originally, I'd hoped to make > it a separate servlet, but we don't have any notion in our code base of how > to twiddle/tweak/adjust what's deployed via web.xml. I spent some time > thinking about the possibility of web-fragment.xml usage, but there's an > important tension there with startup time and classpath scanning. Our > classpath is gigantic so I'm loath to unset the metadata-complete=true > < > https://github.com/apache/solr/blob/main/solr/webapp/web/WEB-INF/web.xml#L22 > > > and so it wound up in the module world instead... > > I hack on things related to this occasionally, but it's a big project that > likely will take me a very long time to work through on my own unless I get > lucky and find a sponsor that allows me to work on it full time... > > -Gus > > On Tue, Oct 22, 2024 at 4:51 PM David Smiley <dsmi...@apache.org> wrote: > > > I suspect the choice of a ServletFilter vs Servlet was done for reasons > > that might have made sense forever-ago. It has always been weird. Does > it > > still make sense? Maybe Hossman remembers. > > > > Why I'm bringing this up: > > I heard of HTTP 404 responses coming from a Solr server I run, but didn't > > see the request in our Solr logs. It turns out that requests into Solr > to > > a non-existent core are not logged by Solr. A 404 made sense of course, > > and that's what happened, so there's no bug but the logging situation is > > disappointing. By code inspection, I see that SolrDispatchFilter will do > > the PASSTHROUGH case if it can't find the core. Jetty ultimately gets it > > (default servlet?) and returns the 404 and HTML in our "error404.html". > > Furthermore, I have also recently seen error scenarios where Solr does > > handle the request but doesn't log it *unless* it's a 500 (see > > ResponseUtils). Shouldn't we want at least one *Solr* log message for > > every request to Solr and no matter the HTTP status? I'm aware Jetty has > > logs but it's a patchwork between looking at both. > > > > ~ David Smiley > > Apache Lucene/Solr Search Developer > > http://www.linkedin.com/in/davidwsmiley > > > > > -- > http://www.needhamsoftware.com (work) > https://a.co/d/b2sZLD9 (my fantasy fiction book) >
