--- On Thu, 4/1/10, Adam Heath <[email protected]> wrote: > Adrian Crum wrote: > > --- On Thu, 4/1/10, Adam Heath <[email protected]> > wrote: > >> Adrian Crum wrote: > >>> --- On Thu, 4/1/10, Adam Heath <[email protected]> > >> wrote: > >>>> Adrian Crum wrote: > >>>>> I multi-threaded the data load by > having one > >> thread > >>>> parse the XML files > >>>>> and put the results in a queue. > Another > >> thread > >>>> services the queue and > >>>>> loads the data. I also multi-threaded > the > >> EECAs - but > >>>> that has an issue > >>>>> I need to solve. > >>>> We need to be careful with that. > >> EntitySaxReader > >>>> supports reading > >>>> extremely large data files; it doesn't > read the > >> entire > >>>> thing into > >>>> memory. So, any such event dispatch > system > >> needs to > >>>> keep the parsing > >>>> from getting to far ahead. > >>> http://java.sun.com/javase/6/docs/api/java/util/concurrent/BlockingQueue.html > >> Not really. That will block the calling > thread when > >> no data is available. > > > > Yeah, really. > > > > 1. Construct a FIFO queue, fire up n consumers to > service the queue. > > 2. Consumers block, waiting for queue elements. > > 3. Producer adds elements to queue. Consumers > unblock. > > 4. Queue reaches capacity, producer blocks, waiting > for room. > > 5. Consumers empty the queue. > > 6. Goto step 2. > > And that's a blocking algo, which is bad. > > If you only have a limited number of threads, then anytime > one of them > blocks, the thread becomes unavailable to do real work. > > What needs to happen in these cases is that the thread > removes it self > from the thread pool, and the consumer thread then had to > resubmit the > producer. > > The whole point of SEDA is to not have unbounded resource > usage. If a > thread gets blocked, then that implies that another new > thread will be > needed to keep the work queue proceeding.
Why Events Are A Bad Idea (for high-concurrency servers) - http://capriccio.cs.berkeley.edu/pubs/threads-hotos-2003.pdf An interesting refutation to SEDA.
