> > It's there so "anyUnhandledExceptions" can be called; >
I will check the code again, but I remember that after commenting it, the only compile errors I saw were from MergeThread adding the exception ... perhaps I'm missing something, so I'll re-check the code. I understand your point now - merging is an internal process to IW, therefore there's no real user to notify on errors (e.g., even if IW knew there is an error, what would it do exactly?), and I guess keep trying to execute the merges is reasonable (while throwing the exceptions further - in hope that the user code will catch those and do something with them). BTW, what will happen if I encounter such exceptions, that are thrown repeatedly, and shutdown the JVM? I guess the index will not be in a corrupt state, right? The next time I'll open it, it should be in a state prior to the merge, or at least prior to the merge that failed? I think that sleeping in case of exceptions makes sense .. in case of IO errors that are temporary, this will not spawn threads endlessly, and sleeping will give an opportunity for the IO problem to resolve. In case of bugs, which are supposed to be detected during test time, it should give the developer a chance to kill the process relatively quickly. On Tue, Apr 28, 2009 at 2:39 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > On Tue, Apr 28, 2009 at 8:28 AM, Shai Erera <ser...@gmail.com> wrote: > > Every merge hit the exception, yes. > > > > And actually, the exceptions list is not used anywhere besides MT adding > the > > exception to the list. That's why I was curious why it's there. > > It's there so "anyUnhandledExceptions" can be called; we could add a > getter so that an app could query the CMS to see if there were > exceptions? But, an app can also subclass CMS & override > handleMergeException to do something. > > > I still think we should protect this case somehow, because even if it > hits a > > disk-full exception, there's no point continuing to run infinitely? > > The disk full could clear up, eg if something external was trying to > copy a massive file onto the same disk, IW could hit disk full, then > the copy would fail and remove the partially copied massive file, and > lots of space becomes available again. > > > So maybe > > before spwaning the next thread, check the exceptions list and if it goes > > over a certain threshold (10?) fail? > > But what does "fail" mean? Stop doing any merges forever, for this IW > instance? That seems dangerous. EG maybe the massive merge will > fail, but little merges can proceed. Also, stopping merging like this > spooks me because you might see a few exceptions, but then they stop > and you think all is good but in fact way too many segments are piling > up. > > We already "fail" now, in that the thread that was doing the merge > will throw an exception up to the JRE's default thread exception > handler. It's just that we then let MergePolicy select that merge > again. > > I like the "sleep on exception" because 1) you will continue to see > that exceptions are being thrown, but 2) it won't saturate your CPU. > > Mike > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org > >
