And how long does it take just to read and parse the files, without indexing them? Often that is the problem - nothing to do with lucene.
There is plenty of good advice in http://wiki.apache.org/lucene-java/ImproveIndexingSpeed. A good match on the subject of your message! -- Ian. On Thu, Nov 10, 2011 at 7:22 PM, Simon Willnauer <simon.willna...@googlemail.com> wrote: > can you provide more information about your setup? things like how > much time does it take to index you documents, how many docs do you > index, what are your index writer settings, how many cores do you > have, where do you read from and write to (disks). oh and what version > of lucene are you using? > > thanks, > > simon > > On Thu, Nov 10, 2011 at 10:40 AM, antony jospeh > <antony.joseph.webm...@gmail.com> wrote: >> Hi all, >> >> I have a large number of files in a directory need to be index them. All >> the files are in specific format need to parse to extract information after >> that i had to index. >> Single thread process one file at a time then i decided to use multi >> threads when the main thread that loops the directory and pass the file >> into pool of worker threads using a queue >> all of the which share same index writer, How ever there is no any >> significant changes in indexing speed >> >> Any hints I am doing wrong or any suggestion >> >> >> Thanks >> Antony >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
