[LARM] using SEDA, pros and cons

[David Worms]

This email was originaly send earlier today, but I got an delivery error so I'm sending it again to a new address (using old email address). It was also send to the lucene-dev mailing list

In order to move our project to the next level, it has been discussed the use of an SEDA architecture.

With the help of the Excalibur / Event library, I created a simplified version of Larm which use SEDA as its backbone architecture. The goal was not to make it pretty or even working, but to see how each application components could feet in an event stage architecture.

The great thing I notice is how flexible the application become. it is extremely easy to map each stage with the others. One of the features of LARM is the ability to have different sources( db, web, filesystem, ... ), process them, and store them (lucene index, log, ...). This seems easily achieveable with SEDA. Also clustering the LARM could be done through a specific stage implementation.

This was for the pros. However, it will be great to get some feedback because I am really not sure on how to deal with SEDA. here is some problems I am facing.

- In order for the crawler to be efficient, I had to raise the number of threads, but from what I read in the past, only one or two threads should be used in a SEDA environment.

- Also, it looks to consume a lot of memory, which could be due to the number of messages put into the queue.

Clemens, and others, please have a look of it, and give me some feedback.

http://67.116.155.180/~wdavidw/stage.zip

David.


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]