I finally rerun the program and it stops at exactly the sampe place.This time
the exception came out.Writer cant add the 10000th document to the index...
Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491886.xml
Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491887.xml
Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491891.xml
Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491893.xml
Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491896.xml
java.lang.Exception: cannot add document to index
And this is the code..
public static void addDocToIndex(Document doc) throws Exception
{
try
{
writer.addDocument(doc);
counter++;
}
catch (Exception e)
{
throw new Exception("cannot add document to index");
}
}
I already put -Xmx512m in Java VM argument,since previously it has exception
of Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
Maureen
Chris Hostetter <[EMAIL PROTECTED]> wrote:
did you try triggering a thread dump to see what it was doing at that
point?
depending on your merge factors and other IndexWriter settings it could
just be doing a relaly big merge.
: Date: Sat, 27 Jan 2007 09:40:47 -0800 (PST)
: From: maureen tanuwidjaja
: Reply-To: java-user@lucene.apache.org
: To: java-user@lucene.apache.org
: Subject: My program stops indexing after 10000th documents is indexed
:
: Hi all,
:
: Is there any limitation of number of file that lucene can handle?
: I indexed a total of 30000 XML Documents,however it stops at 10000th
documents.
: No warning,no error ,no exception as well.
: ....
: Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491876.xml
: Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491886.xml
: Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491887.xml
: Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491891.xml
: Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491893.xml
: Indexing C:\sweetpea\wikipedia_xmlfiles\part-180000\491896.xml <--10000th
doc
: --it idles here--
:
: At first I thought that it was the size of 10000th document is so big so
that it took quite a long time to put into the index.Then i found out that the
10000th document has the size of 6 KB only.Indexing process stops for about 1
hour,so that i decide to terminate the progress.
:
: Is there anything to do with smt like setCompoundFiles etc?cause I dont
include any in my program...
:
: Any suggestion pls?
:
: THanks and Best Regards,
:
: Maureen
:
:
: ---------------------------------
: Any questions? Get answers on any topic at Yahoo! Answers. Try it now.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------
Cheap Talk? Check out Yahoo! Messenger's low PC-to-Phone call rates.
---------------------------------
Expecting? Get great news right away with email Auto-Check.
Try the Yahoo! Mail Beta.
