1. Why do you use RamDirectory and not the hard disk?
2. as John said, you should reuse the index instead of creating it each time in the main function
if(!indexExists(File indexFile))
IndexWriter writer = new IndexWriter(directory, new StandardAnalyzer(), true);
else
IndexWriter writer = new IndexWriter(directory, new StandardAnalyzer(), false);
(in some cases indexExists can be as simple as verifying if the file exits on the hard disk)
3. you iterate in a loop over 10.000 times and you create a lot of objects
for (int i = 0; i < 365 * 30; i++) {
Document doc = new Document();doc.add(Field.Keyword("date", df.format(new Date(c.getTimeInMillis()))));
doc.add(Field.Keyword("id", "AB" + String.valueOf(i)));
doc.add(Field.Text("text", "Tohle je text " + i));
writer.addDocument(doc);
c.add(Calendar.DAY_OF_YEAR, 1);
}
all the underlined lines of code create new ojects, and all of them are kept in memory.
This is a lot of memory allocated only by this loop. I think that you create more than 100.000 object in this loop ...
What do you think?
And none of them cannot be realeased (collected by gc) untill you close the index writer.
None says that your code is complicated, but all programmers should understand that this is a poor design...
And ... more then that your information is kept in a RamDirectory ....when you will close the writer you will still keep the information in memory ...
Sory if I was too agressive with my comments .... but ... I cannot see what were you thinking when you wrote that code ...
If you are trying to make a test .... then I sugest you to replace the hard codded 365 value ... with a variable, to iterate over it and to test the power of your machine
(PC + JVM) :))
I wish you luck,
Sergiu
Jiří Kuhn wrote:
I disagree or I don't understand.
I can change the code as it is shown below. Now I must reopen the index to see the changes, but the memory problem remains. I realy don't know what I'm doing wrong, the code is so simple.
Jiri.
...
public static void main(String[] args) throws IOException { Directory directory = create_index();
for (int i = 1; i < 100; i++) { System.err.println("loop " + i + ", index version: " + IndexReader.getCurrentVersion(directory)); search_index(directory); add_to_index(directory, i); } }
private static void add_to_index(Directory directory, int i) throws IOException { IndexWriter writer = new IndexWriter(directory, new StandardAnalyzer(), false);
SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd"); Document doc = new Document();
doc.add(Field.Keyword("date", df.format(new Date(System.currentTimeMillis())))); doc.add(Field.Keyword("id", "CD" + String.valueOf(i))); doc.add(Field.Text("text", "Tohle neni text " + i)); writer.addDocument(doc);
System.err.println("index size: " + writer.docCount()); writer.close(); }
...
-----Original Message----- From: John Moylan [mailto:[EMAIL PROTECTED] Sent: Monday, September 13, 2004 3:25 PM To: Lucene Users List Subject: Re: OutOfMemory example
You should reuse your old index (as eg an application variable) unless it has changed - use getCurrentVersion to check the index for updates. This has come up before.
John
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
