Hello, i am ProjectManager from the columba.sourceforge.net java mailclient-project and we integrated Lucene as the search-backend half a year ago. It is now working for small scale mailtraffic but with increasing mailtraffic Lucene throws OutOfMemory and TooManyFilesOpen-Exceptions. I am now wondering if Lucene is capable of doing the job for us (like Otis Gospodnetic suggested) and would appreciate any help and knowledge you can share on this topic.
I think the problem arises from following issues: - Lucene is designed to create an index once in a while and not to update an index frequently. We need it to add and delete documents very often *and* search the index eventualy after every operation. Has anyone experiences running Lucene in such an environment or do you think it is impossible? - Do you have an suggestion on how to use Lucene in such an environment because it is not very nice code if you have to create a new IndexReader/Writer after every operation? - We introduced a RAMIndex that is merged to the FileIndex after N operations to reduce the load and to not merge documents that are removed directly after they are added (with filters on the mailboxes that is happening very often). Any ideas if that was wise or if there is a better solution? - Does Lucene have problems with many indices in the same virtual machine? We have an index for every mailfolder and get TooManyFilesOpen-Excpetions when having >10 indices open. Maybe we should try to have only a single index that holds all messages? If you like to look at sourcecode, how we implememted all this look at http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/columba/columba/src/mail/core/org/columba/mail/folder/search/LuceneSearchEngine.java?rev=1.7&content-type=text/vnd.viewcvs-markup Its not nice to just give you the plain code and not the relevant snippets, but these are more general design issues that i think are better explained in words than in code. I would really like to see Lucene integrated in Columba, but i had to learn that it is no easy task, maybe an impossible one. Based on the responses i willl decide if we continue to work with Lucene or sadly have to drop it. Thanks in advance Timo Stich [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
