Ok, I see. Seems most ppl think is the third possiblity
On Fri, 10 Dec 2004, Xiangyu Jin wrote:
>
> I am not sure. But guess there are three possilities,
>
> (1). see that you use
> Field.Text("contents", stringBuffer.toString())
> This will store all your string of text into document object.
> And it might be long ...
>
> I do not know the detail how Lucene implemented.
> I think you can try use unstored first to see
> if the same problem happen.
>
> BTW, how large is your document. Mine has 1M docs and
> max-length less than 1 M, usually has length about several k.
>
> (2) I guess another possiblilty is that record 898 is a very long
> document, maybe java' s string object has a maxlength?
> Just trace the code, see when the exception occur.
>
> (3) Moreover, if you run it on a java VM, it also has a setting of
> its "virtual" mem. It has nothing to do with the hardware
> you are running. I has met this before when I use the directory's
> ListOfFile function, where it easily exceed the max mem, if
> there are 1M docs under the same dir (a stupid mistake I made).
> But if I expand the VM's mem, it is then appears ok.
>
> :)
>
>
>
>
>
> On Fri, 10 Dec 2004, Jin, Ying wrote:
>
> > Hi, Everyone,
> >
> >
> >
> > We're trying to index ~1500 archives but get OutOfMemoryError about
> > halfway through the index process. I've tried to run program under two
> > different Redhat Linux servers: One with 256M memory and 365M swap
> > space. The other one with 512M memory and 1G swap space. However, both
> > got OutOfMemoryError at the same place (at record 898).
> >
> >
> >
> > Here is my code for indexing:
> >
> > ===============================================
> >
> > Document doc = new Document();
> >
> > doc.add(Field.UnIndexed("path", f.getPath()));
> >
> > doc.add(Field.Keyword("modified",
> >
> >
> > DateField.timeToString(f.lastModified())));
> >
> > doc.add(Field.UnIndexed("eprintid", id));
> >
> > doc.add(Field.Text("metadata", metadata));
> >
> >
> >
> > FileInputStream is = new FileInputStream(f); // the text file
> >
> > BufferedReader reader = new BufferedReader(new
> > InputStreamReader(is));
> >
> >
> >
> > StringBuffer stringBuffer = new StringBuffer();
> >
> > String line = "";
> >
> > try{
> >
> > while((line = reader.readLine()) != null){
> >
> > stringBuffer.append(line);
> >
> > }
> >
> > doc.add(Field.Text("contents", stringBuffer.toString()));
> >
> > // release the resources
> >
> > is.close();
> >
> > reader.close();
> >
> > }catch(java.io.IOException e){}
> >
> > =================================================
> >
> > Is there anything wrong with my code or I need more memory?
> >
> >
> >
> > Thanks for any help!
> >
> > Ying
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
