After indexing is done, you can copy the index files and merge them to one large index. Or you can maintain several small indexes and search across indexes.
Aviran http://www.aviransplace.com -----Original Message----- From: Java Programmer [mailto:[EMAIL PROTECTED] Sent: Friday, February 17, 2006 5:18 AM To: java-user@lucene.apache.org Subject: Speedup indexing process Hi, Maybe this question is trivial but I need to ask it. I've some problem with indexing large number of documents, and I seek for better solution. Task is to index about 33GB text data CSV (each record about 30kB), it possible of course to index these data but I'm not very happy with timings (about 26 hours), so I want to know how can i speed up this process. First I think about splitting CVS file into smaller ones, eg 5GB and index them on 6 indexing computers, but now is my question - can I join such parts into one index after indexing jobs on each computer is finished? I saw example wit RAMDirectory which could be merged with FSDirectory, but this example was about same IndexWriter, in my case I need some separate IndexWriters on few computers. So does it possible with Lucene? Thx in advance for hints, Adrian --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
