Ok Having looked over the code again the only obvious possible issue I can see is the following scenario:
1 - App launches a reader thread using a thread pool - reader is assigned to thread X 2 - Reader thread fails and releases thread X to the pool 3 - App launches a writer thread using a thread pool - writer is assigned to thread X 4 - Writer never detects the reader failure because it is on the same thread as the reader originally was and so the writer thread is considered alive I.e. it may be a timing issue combined with a thread scheduling issue. I haven't attempted to construct a test case to test this theory - I will do shortly - but this is the only obvious failure condition I can see from the code. Of course there may be others, stack traces/profiler output/etc from when this occurs in your production scenario might help here. If this is the case then scheduling the reader and writer threads on different thread pools would avoid any possibility of this happening. I'll reply again to the thread once I've had chance to concoct a test case for this scenario. 2.11.0 is the latest stable release version but the PipedRDFIterator code has not changed for the last couple of releases AFAIK so I doubt bumping the version will make much difference. Rob On 10/2/13 10:50 AM, "Norman Walsh" <[email protected]> wrote: >Rob Vesse <[email protected]> writes: >> I've never seen the email you quote before nor does it appear to be in >>the >> mailing list archives, are you sure you actually sent it to the list? > >Indeed, there it is, stuck in my outbound mail queue for seven days. Alas. > >> The PipedRDFIterator is supposed to handle errors sensibly, a minimal >> reproducible test case with sample data where it does not would be >>helpful > >Naturally, all my attempts to write such a case have failed. But it >consistently goes off into the weeds in the big, multi-threaded >production app. *sigh* > >I'll keep hacking at it. > >> Also what version of ARQ are you using? > >2.10.0 according to the POM. > > Be seeing you, > norm > >-- >Norman Walsh <[email protected]> | It would not be better if things >http://nwalsh.com/ | happened to men just as they wished.-- > | Heraclitus
