Thank you so much for this infor. it looks pretty complicated for me but I will try.
On Wed, Oct 20, 2010 at 1:18 AM, Johnbin Wang <johnbin.w...@gmail.com>wrote: > You can start a fixedThreadPool to index all these files in the multhread > way. Every thread execute an index task which could index a part of all the > files. In the index task, when indexing 10000 files, you need execute the > indexWrite.commit() method to flush all the index add operation to disk > file. > > If you need index all these files into only one index file, you need to > hold > only one indexWriter instance among all the index thread. > > Hope it's helpful. > > > > On Wed, Oct 20, 2010 at 1:05 PM, Sahin Buyrukbilen < > sahin.buyrukbi...@gmail.com> wrote: > > > Thank you Johnbin, > > do you know which parameter I have to play with? > > > > On Wed, Oct 20, 2010 at 12:59 AM, Johnbin Wang <johnbin.w...@gmail.com > > >wrote: > > > > > I think you can write index file once every 10,000 files or less have > > been > > > read. > > > > > > On Wed, Oct 20, 2010 at 12:11 PM, Sahin Buyrukbilen < > > > sahin.buyrukbi...@gmail.com> wrote: > > > > > > > Hi all, > > > > > > > > I have to index about 4.5Million txt files. When I run the my > indexing > > > > application through Eclipse, I get this error : "Exception in thread > > > "main" > > > > java.lang.OutOfMemoryError: Java heap space" > > > > > > > > eclipse -vmargs -Xmx2000m -Xss8192k > > > > > > > > eclipse -vmargs -Xms40M -Xmx2G > > > > > > > > I tried running Eclipse with above memory parameters, but still had > > the > > > > same error. The architecture of my computer is AMD x2 64bit 2GHz > > > processor, > > > > Ubuntu 10.04 LTS 64bit. java-6-openjdk. > > > > > > > > Anybody has a suggestion? > > > > > > > > thank you. > > > > Sahin. > > > > > > > > > > > > > > > > -- > > > cheers, > > > Johnbin Wang > > > > > > > > > -- > cheers, > Johnbin Wang >