Otis, Kelvin and me have been discussing how we could leverage Lucene on a next level. We have some components in the sandbox (LARM Webcrawler, Indyo indexing framework) that have to be weaved together with Lucene.
This could end up in a real search engine server. I call it "Lucene Advanced Retrieval Machine" for myself right now, 'cause then I can stick with the LARM acronym... :-) This will be a server that gets data from somewhere (i.e. the Web or the file system), does processing steps on it (i.e. extracting text from HTML or PDF) and puts the results into a Lucene index. The whole process forms an assembly line, with data sources at the beginning and a storage at the end. A source is an active component that decides when to put a URL or a document into the queue (i.e. because there's a notification if the doc has changed, if the stored doc is obsolete, or if a recrawl is said to be necessary). This assembly line can also be broken up (in serial or in parallel) such that different steps of it can be executed on more than one machine. As such, the storage could be a JMS queue (or even a topic) that is filled in batch mode. On the other side, the JMS then is the data source. There are lots of construction sites, though. The architecture must be solid and efficient. The crawler has to be made more scalable. We're also discussing on how we can save time implementing server related stuff and configuration - taking the Avalon framework is (probably a good) option, but no one of us has used it before. Clemens ----- Original Message ----- From: "Peter Mularien" <[EMAIL PROTECTED]> To: "Lucene Developers List" <[EMAIL PROTECTED]> Sent: Wednesday, October 30, 2002 6:10 PM Subject: Development plans for Lucene? > Hi - > > I was just wondering what is the current development plan status for > Lucene? I have been monitoring the developer's list for some time, and > have seen very little in the way of CVS commits. I know that may not be > an accurate barometer of development activity, but there it is. > > Is there any plan for continuing active development of Lucene? How many > people are working on it? Why aren't there more, and if so, how can we > recruit them? > > I am willing to help contribute, but without some other active > developers, I'm not sure how useful it will be. > > The frustrating thing is that from the users list, we all know that > there is a significant amount of interest in the product, and some > feature requests, etc. > > Thanks > Peter Mularien > Deploy Solutions > > > > -- > To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@;jakarta.apache.org> > For additional commands, e-mail: <mailto:lucene-dev-help@;jakarta.apache.org> > -- To unsubscribe, e-mail: <mailto:lucene-dev-unsubscribe@;jakarta.apache.org> For additional commands, e-mail: <mailto:lucene-dev-help@;jakarta.apache.org>
