I think the problem is Peter's bias towards Pico, and its IoC philosophy. This coupled with a J2EE Web Environment leads to massive object creation, something most highly scalable systems attempt to avoid.
Peter, am I safe to assume that you would ideally like to code something like (in response to each web request). IndexReader r = pico.getObjectInstance(IndexReader.class) If so, this is probably not going to work in the case of Lucene, or in ANY system that has objects that maintain a lot of state - the creation of the state consumes far too many resources, so the proper way is to create the state once, and share access to it. As Doug points out, it is the same thing with database connections... they are extremely expensive to create, so a pool of these connections is created, and each request contends for a connection. If there are more incoming requests than the pool has connections some requests will either block or fail. So create a pool of IndexReaders and have the requests contend for them... BUT, Lucene is more efficient than database connections, in that, once the IndexReader is created, then multiple request can share it - not block waiting on it. The only time you need to drop and recreate IndexReaders is if you want the IndexReader to be able to see the "latest" data, because otherwise they will only a snapshot of the data when they were created. Robert -----Original Message----- From: Doug Cutting [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 14, 2004 4:13 PM To: Lucene Developers List Subject: Re: Lock handling and Lucene 1.9 / 2.0 Pete Lewis wrote: > The only way to continually use the same IndexReader would be to use a > stateful session bean (frowned upon by J2EE Container writers) Can one implement DB connection pooling in J2EE? This is analogous. One may keep a pool of IndexReaders that are reused by subsequent queries. One difference is that the cache need only contain a single IndexReader per index, rather than a DB connection pool, which typically keeps multiple connections per DB. Also, at checkout, the cache code should check whether a newer version of the index is available, and, if it is, update the cache. If there are lots of different indexes, more than you'd like to keep open at once, then an LRU cache might work well, implemented e.g. with LinkedHashMap. Such a cache might be a useful contribution to Lucene. > I thought that it might be a good candidate for Lucene 2 as the FSDirectory > code is horrible and non-J2EE compliant. Your constructive criticism is greatly appreciated! Have a nice day, Doug --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
