Hi All - What is the best way to load a RAM Directory from a FS Directory, and periodically reload the RAM Directory to pick up new documents?
The scenario I have is I create several large directories which I create to a file system, then load them into ram for faster searching. They takes several hours to create, so i want to retain a file copy so in the event of a service/server crash or reboot, they can load again in a few seconds. Once a day I append or recreate the FS Directories with new entries, and then reload the RAM Directories to pick up the new entries. Problem is, what I am doing seems to cause a memory leak. The directories take ~12 GB of RAM to load, but bloats to 24 GB (the -Xmx setting) after a few reloads and stays there. // Map to hold single instance of the IndexSearcher, 1 per RAM Directory, to be reused across requests. private static final Map<IndexName, IndexSearcher> searchers = new HashMap<IndexName, IndexSearcher>(); // Creates the IndexSearcher and stores it in the static map. public static void load(IndexName index, ...) { // sync... etc RAMDirectory dir = new RAMDirectory(FSDirectory.getDirectory(directoryPath); IndexReader reader = IndexReader.open(dir, true); IndexSearcher searcher = new IndexSearcher(reader); searchers.put(index, searcher); } public static IndexSearcher get(IndexName index, ...) { // sync... etc return searchers.get(index); } // After an indexing service appends the existing FS Directory, this reloads it. public static void reload(IndexName index, ...) { // sync... etc IndexSearcher searcher = searchers.get(index); FSDirectory fsDirectory = FSDirectory.getDirectory(directoryPath); Directory ramDirectory = searcher.getIndexReader().directory(); Directory.copy(fsDirectory, ramDirectory, false); } I've tries some theme and variations on this with the same issue. TIA! gd --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org