--- On Thu, 4/1/10, Adam Heath <[email protected]> wrote: > Adrian Crum wrote: > > --- On Thu, 4/1/10, Adam Heath <[email protected]> > wrote: > >> Adrian Crum wrote: > >>> --- On Thu, 4/1/10, Adam Heath <[email protected]> > >> wrote: > >>>> Adrian Crum wrote: > >>>>> I multi-threaded the data load by > having one > >> thread > >>>> parse the XML files > >>>>> and put the results in a queue. > Another > >> thread > >>>> services the queue and > >>>>> loads the data. I also multi-threaded > the > >> EECAs - but > >>>> that has an issue > >>>>> I need to solve. > >>>> We need to be careful with that. > >> EntitySaxReader > >>>> supports reading > >>>> extremely large data files; it doesn't > read the > >> entire > >>>> thing into > >>>> memory. So, any such event dispatch > system > >> needs to > >>>> keep the parsing > >>>> from getting to far ahead. > >>> http://java.sun.com/javase/6/docs/api/java/util/concurrent/BlockingQueue.html > >> Not really. That will block the calling > thread when > >> no data is available. > > > > Yeah, really. > > > > 1. Construct a FIFO queue, fire up n consumers to > service the queue. > > 2. Consumers block, waiting for queue elements. > > 3. Producer adds elements to queue. Consumers > unblock. > > 4. Queue reaches capacity, producer blocks, waiting > for room. > > 5. Consumers empty the queue. > > 6. Goto step 2. > > And that's a blocking algo, which is bad.
Huh? You just asked for a blocking algorithm: "So, any such event dispatch system needs to keep the parsing from getting to far ahead." > The whole point of SEDA is to not have unbounded resource > usage. If a > thread gets blocked, then that implies that another new > thread will be > needed to keep the work queue proceeding. You lost me again. I thought we were talking about entity import/export - not SEDA.
