Hi the file to be indexed depends on the type of Document / data extractor ....
My Document types are usually XML type and every time 2+ Million XML's are indexed and time taken is less then 5 minuts. with regards karthik On Fri, Nov 11, 2011 at 1:17 AM, Ian Lea <ian....@gmail.com> wrote: > And how long does it take just to read and parse the files, without > indexing them? Often that is the problem - nothing to do with lucene. > > There is plenty of good advice in > http://wiki.apache.org/lucene-java/ImproveIndexingSpeed. A good match > on the subject of your message! > > -- > Ian. > > > On Thu, Nov 10, 2011 at 7:22 PM, Simon Willnauer > <simon.willna...@googlemail.com> wrote: > > can you provide more information about your setup? things like how > > much time does it take to index you documents, how many docs do you > > index, what are your index writer settings, how many cores do you > > have, where do you read from and write to (disks). oh and what version > > of lucene are you using? > > > > thanks, > > > > simon > > > > On Thu, Nov 10, 2011 at 10:40 AM, antony jospeh > > <antony.joseph.webm...@gmail.com> wrote: > >> Hi all, > >> > >> I have a large number of files in a directory need to be index them. All > >> the files are in specific format need to parse to extract information > after > >> that i had to index. > >> Single thread process one file at a time then i decided to use multi > >> threads when the main thread that loops the directory and pass the file > >> into pool of worker threads using a queue > >> all of the which share same index writer, How ever there is no any > >> significant changes in indexing speed > >> > >> Any hints I am doing wrong or any suggestion > >> > >> > >> Thanks > >> Antony > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > -- *N.S.KARTHIK R.M.S.COLONY BEHIND BANK OF INDIA R.M.V 2ND STAGE BANGALORE 560094*
