+1, thanks David. On Sat, 28 Sept, 2024, 1:16 am David Smiley, <dsmi...@apache.org> wrote:
> I just want to follow-up here to let everyone know I have a PR and it > includes a fundamental change in the nature of the Executor to be > blocking. Ideally someone would benchmark it. But even if that doesn't > happen, I'm inclined to merge to fix the RejectedExecutionException > problem. This is an experimental feature, after all. > > On Wed, Aug 21, 2024 at 10:12 AM David Smiley <dsmi...@apache.org> wrote: > > > Thanks; I filed https://issues.apache.org/jira/browse/SOLR-17414 > > > > On Tue, Aug 20, 2024 at 2:56 PM Ishan Chattopadhyaya > > <ichattopadhy...@gmail.com> wrote: > > > > > > Thanks, I'll take a look. > > > > > > On Tue, 20 Aug, 2024, 11:22 pm David Smiley, <dsmi...@apache.org> > wrote: > > > > > > > Since the new multiThreaded search feature landed, I see a new test > > > > failure involving "RejectedExecutionException" being thrown: > > > > > > > > > > > https://ge.apache.org/s/5ack462ji4mlu/tests/task/:solr:core:test/details/org.apache.solr.search.TestRealTimeGet/testStressGetRealtime?top-execution=1 > > > > It is thrown at a low level in Lucene building TermStates > > > > concurrently. I doubt the problem is specific to that test > > > > (TestRealTimeGet) but that test might induce more activity than most > > > > tests, thus crossing some thresholds like the queue size -- > apparently > > > > 1000. > > > > > > > > ** I don't think we should be throwing a RejectedExecutionException > > > > when running a Search query **. > > > > Have others explored this topic further and have ideas to share, even > > > > including ruled out ideas? Ishan, Noble, ? > > > > > > > > I'd prefer to avoid catching this exception to take some other action > > > > -- would feel too much like a hack. I'm optimistic a more elegant > > > > solution without doing this can be found. > > > > > > > > Perhaps SynchronousQueue? Maybe I can rule this out based on a > little > > > > test I did, as it throws a RejectedExecutionException (instead of > > > > blocking), which I'd rather not catch. > > > > > > > > Perhaps a LinkedBlockingQueue (thus infinite queue size) but have an > > > > ExecutorService subclass that observes the queue size to see if a > > > > threshold is reached and if so then runs the task directly? This way > > > > we never reject and the caller threads receive back pressure by doing > > > > the work that they would have done anyway with multiThreaded=false ! > > > > There is plenty of precedent for an ExecutorService that runs the > task > > > > in the caller thread -- I'm thinking of Lucene's > > > > SameThreadExecutorService and Solr's SimpleFacets.directExecutor. > > > > > > > > ~ David Smiley > > > > Apache Lucene/Solr Search Developer > > > > http://www.linkedin.com/in/davidwsmiley > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > > > > For additional commands, e-mail: dev-h...@solr.apache.org > > > > > > > > > > >
