--- On Thu, 4/1/10, Adam Heath <[email protected]> wrote: > Adrian Crum wrote: > > --- On Thu, 4/1/10, Adam Heath <[email protected]> > wrote: > >> Adrian Crum wrote: > >>> I multi-threaded the data load by having one > thread > >> parse the XML files > >>> and put the results in a queue. Another > thread > >> services the queue and > >>> loads the data. I also multi-threaded the > EECAs - but > >> that has an issue > >>> I need to solve. > >> We need to be careful with that. > EntitySaxReader > >> supports reading > >> extremely large data files; it doesn't read the > entire > >> thing into > >> memory. So, any such event dispatch system > needs to > >> keep the parsing > >> from getting to far ahead. > > > > http://java.sun.com/javase/6/docs/api/java/util/concurrent/BlockingQueue.html > > Not really. That will block the calling thread when > no data is available.
Yeah, really. 1. Construct a FIFO queue, fire up n consumers to service the queue. 2. Consumers block, waiting for queue elements. 3. Producer adds elements to queue. Consumers unblock. 4. Queue reaches capacity, producer blocks, waiting for room. 5. Consumers empty the queue. 6. Goto step 2.
