how to free memory after index ist build.

[Jan Philipp Seng]

I am using the Lucene 1.4.3 API. After building the index over 150000
documents (~250 MB data), Lucene does not free the memory that is used
during indexing. The searcher runs as a servlet under Tomcat. Every time the
index is build new, the indexing process takes free memory, so after ten
runs the memory is completly full. I tried to call the garbage collector
explicitly, but that does not help.
I build the index to harddisc and load it into a RAMDirectory after
building. There is no reference to my indexer left after indexing and I
cannot find a reason why the garbage collector does not free the memory.
Here are some important points of my code reduced to the central
functionality. Do you have any ideas, what kind of problem this could be ?
An answer would help me a lot.


IndexTablesDaemon start the indexing process:
IndexTablesDaemon.run():
-----------------------------
while (true) {
  IndexTables indexer = new IndexTables();
  indexer.indexTables(); // indexing if necessary, writing the new index to
harddisc
  indexer = null;        // for freeing the memory, does not help
  FTS.initNewIndex();    // switching the old with the new index
  Runtime.getRuntime().gc();   // calling the garbage collector. Does not
help.
  Thread.sleep(lWait);         // waiting a fix space of time
} 

                
                
Building the index: sending queries to a mySQLDatabase, 
                                                                                
buidling a Lucene-document with the mySQL-data und indexing it:
IndexTables.indexTables():
--------------------------
  IndexWriter writer = new IndexWriter(PATHNAME_INDEX_NEW), 
    new TTAnalyser(), true);            

  writer.mergeFactor = 250;
  writer.minMergeDocs = 250;
 
  Document doc = null;

  String sQuery = "SELECT columns FROM table";

  Connection conn = DriverManager.getConnection("jdbc:mysql: ...");
  Statement stmt = conn.createStatement()
  ResultSet     rs = stmt.executeQuery(sQuery);

  while (m_rs.next()) {
    doc = getTTDocument();
    // fill doc with fields from the database query
    writer.addDocument(doc);
  }

  writer.optimize();
  writer.close();
  writer = null;              // for freeing memory. Does not help

  Runtime.getRuntime().gc();  // explicitely running the garbage collector.
Does not help
}
        
        
                
Exchanging the old an the new index for queries to the Lucene-index.
FTS.initNewIndex():
-------------------
  File oldIndex = new File(sPathIndex);
  File newIndex = new File(sPathIndexNew);
 
  // deleting all files in the old index from harddisc
  // let the new index become the operating index
  oldIndex.delete();
  newIndex.renameTo(oldIndex);
 
  // load the new index to RAMDirectory from harddisc
  SearchTables.reloadIndex();           
                                
                        
Thanks for your help,

Jan Philipp Seng, Germany, Aachen

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]