Hi Amin, I'm looking at way too much stuff, it's very welcome that you look at it. I didn't assign it to me as I've too much other stuff already assigned. Sometimes I do some experiments locally but I do only commit if it's looking good, most of the time I need to bring something up here for feedback: help and discussion are always welcome.
Cheers, Sanne 2010/1/9 Amin Mohammed-Coleman <ami...@gmail.com>: > Hi Sanne > Thanks for the advice! I'll take a look at it. > > Cheers > Amin > > On Sat, Jan 9, 2010 at 1:34 PM, Sanne Grinovero <sanne.grinov...@gmail.com> > wrote: >> >> Hi Amin, >> I've been looking a bit in this, but didn't take any action as we >> didn't discuss any strategy, so glad you join in propose something. >> I didn't think of jms, it's ok we provide some way for the end user to >> override whatever we provide, but I think we should just provide >> the basic stuff: logging the error or propagate it back when possible. >> And then add some extensibility like plugin loading the usual way. >> >> Originally (Hibernate Search 3.0.x) the changes made to different >> indexes were serialized, so when using the "sync" backend the same >> process would have applied changes to the index and, in case of >> exceptions, this would have been propagated up to user's code making >> it impossible to go unnoticed, or at least moving responsibility of >> handling it to the developer. >> When using the "async" mode for backend a different thread would have >> handled the indexwriter, so in case of an exception the error would >> have killed the separate thread, go unnoticed, and as we use a >> ThreadPool a new thread would have been spawned to replace the failed >> one. >> >> Since Search 3.1.x both "sync" and "async" use a separate thread - to >> be able to apply changes to different indexes in parallel - so >> exceptions are going unnoticed by user code even in "sync" mode. Look >> into "run" method of >> org.hibernate.search.backend.impl.lucene.LuceneBackendQueueProcessor, >> it creates a org.hibernate.search.backend.impl.lucene.QueueProcessors >> where most of this logic resides. >> This impl will use either runAllAsync() or runAllWaiting() depending >> on backend configuration (async/sync). The Async version just >> schedules the different tasks, the sync one is going to do the same >> but will wait for all of them to have finished before returning >> control, making sure to implement the sync behaviour even while using >> several threads to perform it. I was inspired by >> >> java.util.concurrent.AbstractExecutorService.invokeAll(Collection<Callable<T>> >> tasks), as mentioned in code comments, which is ignoring >> ExecutionException, so this empty catch block should be used now to do >> something. >> >> I'd suggest to provide this defaults: >> >> * for sync backend: >> by default: rethrow the exception (like old behaviour) >> configurable alternative: log it >> >> * for async backend: >> log it (it's not useful to rethrow as nobody is listening on it) >> >> To handle these cases only you would add the log statement, an rethrow >> it, fixing the QueueProcessors code. >> To give full control to the end user of what to do, I'd suggest to let >> him specify an implementationf of >> java.lang.Thread.UncaughtExceptionHandler >> and so we can set that on the thread. >> The backend is using the default JVM "Executors.newFixedThreadPool( 1 >> );" (initialized at >> org.hibernate.search.backend.impl.lucene.PerDPResources:51) >> but we could change that to use the Search Executor factory >> "org.hibernate.search.batchindexing.Executors" I later added for the >> batchbackend. >> The nice thingy for this Executor is that it customizes the thread >> names so you can easily spot Search's threads in monitoring/debugging >> tools; >> we could have a setUncaughtExceptionHandler( userImplementation ) in >> org.hibernate.search.batchindexing.Executors:85. >> If you change it there, the same handler would be used to manage >> errors in the batch indexer, so you solve both problems at once. >> Our "log the error" implementation would be the nice default for an >> exception handler, but still I'd like to make sure the user code will >> get the error propagated when >> using async mode. >> >> Cheers, >> Sanne >> >> >> 2010/1/9 Amin Mohammed-Coleman <ami...@gmail.com>: >> > Hi All >> > >> > Emmanuel asked me to look at this issue (HSearch 421) where exceptions >> > happening in backend process which are going unnoticed. I was wondering >> > if >> > I could get some advice/thoughts on how to tackle the problem. The >> > issue >> > mentioned providing the user the option to decide how to handle >> > exceptions >> > (for example queues, logs, etc), so I'm guessing there needs to be some >> > custom option that the user will need to set up, maybe something like >> > this? >> > >> > exception_handling_strategy=jms >> > exception_handling_strategy_jms_queue= >> > >> > or if they wanted to log the exceptions: >> > >> > exception_handling_strategy=log >> > >> > or the user could create a custom class which implements a particular >> > interface to handle exceptions >> > >> > exception_handling_strategy=custom >> > exception_handling_custom_class=ExceptionHandling >> > >> > I could be completely wrong in the above approach and therefore would be >> > grateful for any input. >> > >> > >> > Cheers >> > Amin >> > _______________________________________________ >> > hibernate-dev mailing list >> > hibernate-dev@lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > > > _______________________________________________ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
