ManifoldCFException.INTERRUPTED should only be thrown if thread has been asked to shut down. It is basically the equivalent of InterruptedException in the ManifoldCF world. If the thread sees it it exits immediately.
Karl On Tue, Oct 9, 2012 at 10:29 AM, Maciej Liżewski <[email protected]> wrote: > Well... that is possible :) > what exactly ManifoldCFException.INTERRUPTED is doing that could cause > such effects? > > > 2012/10/9 Karl Wright <[email protected]>: >> What JVM are you using? Because frankly this cannot logically happen. >> >> The only other possibility is that your code is somehow throwing >> ManifoldCFExceptions of type ManifoldCFException.INTERRUPTED. >> >> >> Karl >> >> >> On Tue, Oct 9, 2012 at 10:20 AM, Maciej Liżewski >> <[email protected]> wrote: >>> 2012/10/9 Karl Wright <[email protected]>: >>>> "- all worker threads are gone," ??? >>>> >>>> Really?? >>> >>> yes... really.. this is why I am also writing that this is strange... >>> this is list of currently active threads: >>> system >>> Reference Handler Waiting >>> Finalizer Waiting >>> Signal Dispatcher Running >>> Attach Listener Running >>> main >>> main Waiting >>> Timer-0 Evaluating... >>> HashSessionScavenger-0 Waiting >>> HashSessionScavenger-1 Evaluating... >>> HashSessionScavenger-2 Waiting >>> qtp1792381350-157 Evaluating... >>> qtp1792381350-158 Waiting >>> qtp1792381350-159 Evaluating... >>> qtp1792381350-161 Waiting >>> Job start thread Waiting >>> Startup thread Waiting >>> Delete startup thread Waiting >>> Finisher thread Waiting >>> Job notification thread Waiting >>> Job delete thread Waiting >>> Stuffer thread Waiting >>> Expire stuffer thread Waiting >>> Set priority thread Waiting >>> Expiration thread '0' Waiting >>> Expiration thread '1' Waiting >>> Expiration thread '2' Waiting >>> Expiration thread '3' Waiting >>> Expiration thread '4' Waiting >>> Expiration thread '5' Evaluating... >>> Expiration thread '6' Waiting >>> Expiration thread '7' Waiting >>> Expiration thread '8' Waiting >>> Expiration thread '9' Evaluating... >>> Document cleanup stuffer thread Waiting >>> Document cleanup thread '0' Waiting >>> Document cleanup thread '1' Waiting >>> Document cleanup thread '2' Waiting >>> Document cleanup thread '3' Evaluating... >>> Document cleanup thread '4' Waiting >>> Document cleanup thread '5' Waiting >>> Document cleanup thread '6' Evaluating... >>> Document cleanup thread '7' Waiting >>> Document cleanup thread '8' Evaluating... >>> Document cleanup thread '9' Waiting >>> Document delete stuffer thread Waiting >>> Document delete thread '0' Waiting >>> Document delete thread '1' Waiting >>> Document delete thread '2' Evaluating... >>> Document delete thread '3' Waiting >>> Document delete thread '4' Evaluating... >>> Document delete thread '5' Waiting >>> Document delete thread '6' Waiting >>> Document delete thread '7' Waiting >>> Document delete thread '8' Waiting >>> Document delete thread '9' Evaluating... >>> Job reset thread Waiting >>> Seeding thread Waiting >>> Idle cleanup thread Evaluating... >>> Connection pool reaper Sleeping >>> qtp1792381350-160 Running >>> qtp1792381350-162 Running >>> qtp1792381350-163 Running >>> qtp1792381350-156 Evaluating... >>> derby.daemons >>> derby.rawStoreDaemon Waiting >>> >>> but still UI shows: >>> >>> Name Status Start Time End Time Documents Active >>> Processed >>> Restart Pause Abort msol-x Running Tue Oct 09 15:15:29 CEST >>> 2012 5689 10 5689 >>> >>> (10 documents left active to process and status = "running") >>> >>> I checked the code of connector and there are only >>> "ManifoldCFException" throwed... >>> >>> >>>> >>>> I can think of no scenario where the worker threads disappear except >>>> if shutdown of the agents process is attempted and fails. >>>> WorkerThread catches all Throwable's and logs them and repeats in a >>>> loop. The only kind of exception that can cause the thread to exit is >>>> an InterruptedException. >>>> >>>> This isn't making a heck of a lot of sense... >>>> >>>> Karl >>>> >>>> >>>> On Tue, Oct 9, 2012 at 9:58 AM, Maciej Liżewski >>>> <[email protected]> wrote: >>>>> Just looking at threads... but nothing special. >>>>> - all worker threads are gone, >>>>> - stuffer thread runs in a loop but finds nothing to do... >>>>> - other threads just waits on 'sleep' commands. >>>>> >>>>> is there any particular thread I should look at? >>>>> >>>>> I could guess that there was some exception (maybe in my connector, >>>>> but I could not repeat it) that was not handled and some worker thread >>>>> just disappeared... >>>>> >>>>> How to enable debug logs so I could see verbose output from core >>>>> functions? >>>>> >>>>> >>>>> 2012/10/9 Karl Wright <[email protected]>: >>>>>> FWIW, getting thread dumps from the process running the agents process >>>>>> when it is "hung" may (or may not) help determine the underlying >>>>>> clause. >>>>>> >>>>>> Karl >>>>>> >>>>>> On Tue, Oct 9, 2012 at 9:21 AM, Karl Wright <[email protected]> wrote: >>>>>>> What is your deployment model? Is this a multiprocess deployment? >>>>>>> What database are you using? >>>>>>> >>>>>>> There are various load tests for each database, which do far more than >>>>>>> 7000 documents. I am concerned that you are seeing this because of >>>>>>> some kind of cross-process synchronization issues, which might occur >>>>>>> (for instance) if you are using a multiprocess environment with a >>>>>>> single-process properties.xml file. >>>>>>> >>>>>>> Karl >>>>>>> >>>>>>> On Tue, Oct 9, 2012 at 9:12 AM, Maciej Liżewski >>>>>>> <[email protected]> wrote: >>>>>>>> Ok... it is not a getMaxDocumentRequest issue, because I was able to >>>>>>>> get it even with getMaxDocumentRequest=1. Seems it occurs when >>>>>>>> indenxing large sets of documents (in my case ~7000). It also happened >>>>>>>> once for CIFS connecotr (with samba share)... >>>>>>>> >>>>>>>> result is like this: >>>>>>>> >>>>>>>> Name Status Start Time End Time Documents Active >>>>>>>> Processed >>>>>>>> Restart Pause Abort Mantis Running Tue Oct 09 13:56:59 CEST >>>>>>>> 2012 5689 1600 4400 >>>>>>>> >>>>>>>> there is "active" documents count 1600 for about an hour now but there >>>>>>>> is no server load and nothing changes... seems that it is hanging >>>>>>>> somwhere inside manifold core. >>>>>>>> >>>>>>>> also - when hitting abort - nothing happens (job process remains in >>>>>>>> "aborting" state)... >>>>>>>> >>>>>>>> Problem is that it happens irregularly (sometime 10 documents, >>>>>>>> sometime 1600 and sometime all documents are indexed). Tried to check >>>>>>>> that locally but on first pass everything went ok... really strange... >>>>>>>> >>>>>>>> >>>>>>>> 2012/10/3 Karl Wright <[email protected]>: >>>>>>>>> Hi Maciej, >>>>>>>>> >>>>>>>>> It sounds like your loop condition must be somehow incorrect. You may >>>>>>>>> not receive the full number of documents specified by >>>>>>>>> getMaxDocumentRequest(), but rather a number less than that. >>>>>>>>> >>>>>>>>> We have a number of connectors that use document batches > 1, e.g. the >>>>>>>>> LiveLink connector, so this is likely not the problem. >>>>>>>>> >>>>>>>>> I'd recommend adding System.out.println() diagnostics to see exactly >>>>>>>>> what is happening inside both getDocumentVersions() and >>>>>>>>> processDocuments(). >>>>>>>>> >>>>>>>>> Karl >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Oct 3, 2012 at 4:30 PM, Maciej Liżewski >>>>>>>>> <[email protected]> wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I have noticed strange problem with Connector (new one I am >>>>>>>>>> developing >>>>>>>>>> right now) and getMaxDocumentRequest parameter. >>>>>>>>>> When it returns 1 (default) everything seems ok, but when I set it to >>>>>>>>>> anything higher (5, 10, 20) indexing job does not end but hangs when >>>>>>>>>> there is only getMaxDocumentRequest documents left (when it should >>>>>>>>>> process 5 documents in a row - 5 documents stays "active") >>>>>>>>>> All document related functions seem written ok (they all iterate >>>>>>>>>> throug passed arrays), there are no exceptions thrown (at least I do >>>>>>>>>> not see any in console). >>>>>>>>>> >>>>>>>>>> What can be wrong and what should I look at to? any ideas? >>>>>>>>>> >>>>>>>>>> By the way - the new connector is for Mantis Bug tracker to index >>>>>>>>>> issues. >>>>>>>>>> >>>>>>>>>> TIA >>>>>>>>>> Redguy
