Hi Michael
Thank you. Your suggestions were great and they were implemented
(see attached source code), however, unfortunately, I am still
getting file not found errors on the automatic merging of indexes.
Regards,
Jamie
Michael McCandless wrote:
Jamie,
I'd love to get to the root cause of your exception.
Last time we talked (a few weeks back) I saw several possible
causes in the source you had posted:
http://markmail.org/message/dqovvcwgwof5f7wl
Did you test any of the ideas there? You are potentially manually
deleting files from the index, and also forcibly removing the
write.lock, which can easily cause this exception.
Mike
Jamie wrote:
Hi Paul,
I just noticed the discussion around this.
All most all of my customers have/are experiencing the
intermittant FileNotFound problem.
Our software uses Lucene 2.3.1. I have just upgraded to Lucene
2.3.2 in the hope that this was one of the bugs that was fixed.
I would be very interested in a resolution or known workaround.
Thanks!
Jamie
Paul J. Lucas wrote:
On May 29, 2008, at 6:35 PM, Michael McCandless wrote:
Can you use lsof (or something similar) to see how many files
you have?
FYI: I personally can't reproduce this; only a coworker can and
even then it's sporadic, so it could take a little while.
Merging, especially several running at once, can greatly
increase open file count, and especially if mergeFactor is
increased.
That raises a few questions:
1. Should I lower my mergeFactor?
2. Is there any way to insist that only one merger runs?
3. Is there any way to insist that all merges happen
synchronously, i.e., on IndexWriter.close() only and not to use
a separate merge thread?
- Paul
package com.stimulus.archiva.index;
/* Copyright (C) 2005-2007 Jamie Angus Band
* MailArchiva Open Source Edition Copyright (c) 2005-2007 Jamie
Angus Band
* This program is free software; you can redistribute it and/or
modify it under the terms of
* the GNU General Public License as published by the Free Software
Foundation; either version
* 2 of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY;
* without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.
* See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public
License along with this program;
* if not, see http://www.gnu.org/licenses or write to the Free
Software Foundation,Inc., 51
* Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
import javax.mail.*;
import javax.mail.internet.*;
import java.io.*;
import java.util.*;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.zip.*;
import com.ice.tar.*;
import org.apache.log4j.Logger;
import org.apache.lucene.document.*;
import org.apache.lucene.index.*;
import org.apache.lucene.store.*;
import com.stimulus.archiva.domain.*;
import com.stimulus.archiva.exception.*;
import com.stimulus.archiva.language.*;
import com.stimulus.archiva.extraction.*;
import com.stimulus.archiva.search.*;
import com.stimulus.util.TempFiles;
import com.stimulus.archiva.domain.Service.Status;
import com.stimulus.archiva.domain.fields.*;
import com.stimulus.util.*;
import java.nio.charset.Charset;
import org.apache.log4j.Level;
public class MessageIndex extends Indexer implements Serializable {
private static final long serialVersionUID = -17692874371162272L;
protected static final Logger logger = Logger.getLogger
(MessageIndex.class.getName());
protected static int INDEX_WAIT_PERIOD = 50;
protected static int DEAD_PERIOD = 300000000;
static Hashtable<Volume,VolumeIndex> volumeIndexes = new
Hashtable<Volume,VolumeIndex>();
static Object volumeIndexLock = new Object();
protected ServiceDelegate serviceDelegate;
public MessageIndex() {
serviceDelegate = new ServiceDelegate("message index", this,
logger);
}
public String getServiceName() {
return serviceDelegate.getServiceName();
}
public boolean isAlive() {
return serviceDelegate.isAlive(true);
}
public void startup() {
for (VolumeIndex volumeIndex : volumeIndexes.values()) {
volumeIndex.startup();
}
serviceDelegate.startup();
}
public void prepareShutdown() {
serviceDelegate.prepareShutdown();
}
public void shutdown() {
for (VolumeIndex volumeIndex : volumeIndexes.values()) {
volumeIndex.shutdown();
}
serviceDelegate.shutdown();
}
public void reloadConfig() {
serviceDelegate.reloadConfig();
}
public Status getStatus() {
return serviceDelegate.getStatus();
}
public void deleteIndex(Volume volume) throws
MessageSearchException {
VolumeIndex volumeIndex = getVolumeIndex(volume);
volumeIndex.deleteIndex();
}
public VolumeIndex getVolumeIndex(Volume volume) {
VolumeIndex volumeIndex = null;
synchronized (volumeIndexLock) {
volumeIndex = volumeIndexes.get(volume);
if (volumeIndex==null) {
volumeIndex = new VolumeIndex(this,volume);
volumeIndexes.put(volume,volumeIndex);
}
}
return volumeIndex;
}
public void indexMessage(Email email) throws MessageSearchException {
if (serviceDelegate.getStatus()!=Status.STARTED) {
return;
}
VolumeIndex volumeIndex = getVolumeIndex(email.getEmailId
().getVolume());
volumeIndex.indexMessage(email);
}
public void deleteMessage(EmailID emailID) throws
MessageSearchException {
VolumeIndex volumeIndex = getVolumeIndex(emailID.getVolume());
volumeIndex.deleteMessage(emailID);
}
public void prepareIndex(Volume volume) throws
MessageSearchException {
if (volume==null)
throw new MessageSearchException("assertion failure:
null volume",logger);
if (volume.getIndexPath().startsWith("rmi://"))
return;
File indexDir = new File(volume.getIndexPath());
if (!indexDir.exists()) {
logger.info("index directory does not exist. will proceed with
creation {location='" + volume.getIndexPath() + "'}");
boolean success = indexDir.mkdir();
if (!success)
throw new MessageSearchException("failed to create index
directory {location='" + volume.getIndexPath() + "'}",logger);
logger.info("index directory successfully created {location='" +
volume.getIndexPath() + "'}");
}
}
}
package com.stimulus.archiva.index;
import java.io.File;
import java.io.IOException;
import java.io.PrintStream;
import javax.mail.MessagingException;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.*;
import org.apache.lucene.store.FSDirectory;
import com.stimulus.archiva.domain.Config;
import com.stimulus.archiva.domain.Email;
import com.stimulus.archiva.domain.EmailID;
import com.stimulus.archiva.domain.Indexer;
import com.stimulus.archiva.domain.Volume;
import com.stimulus.archiva.exception.ExtractionException;
import com.stimulus.archiva.exception.MessageSearchException;
import com.stimulus.archiva.language.AnalyzerFactory;
import com.stimulus.archiva.search.*;
import java.util.*;
import org.apache.lucene.store.AlreadyClosedException;
public class VolumeIndex extends Thread {
protected static final Logger logger = Logger.getLogger
(VolumeIndex.class.getName());
public static final int indexOpenTime = 2000;
IndexWriter writer = null;
Volume volume;
Timer closeIndexTimer = new Timer();
Object indexLock = new Object();
ArchivaAnalyzer analyzer = new ArchivaAnalyzer();
Indexer indexer = null;
public VolumeIndex(Indexer indexer, Volume volume) {
this.volume = volume;
this.indexer = indexer;
startup();
}
public void deleteMessage(EmailID emailID) throws
MessageSearchException {
if (emailID == null)
throw new MessageSearchException("assertion failure:
null emailID",logger);
logger.debug("delete message {'"+emailID+"'}");
Volume volume = emailID.getVolume();
File indexDir = new File(volume.getIndexPath());
if (!indexDir.exists())
throw new MessageSearchException("could not delete email from
index. volume does not exist. {'"+emailID+"}",logger);
synchronized(indexLock) {
IndexReader indexReader = null;
try {
indexReader = IndexReader.open(indexDir);
} catch (IOException e ) {
throw new MessageSearchException("failed to open index to
delete email",e,logger);
}
try {
indexReader.deleteDocuments(new Term("uid",emailID.getUniqueID
()));
indexReader.close();
} catch (Exception e) {
throw new MessageSearchException("failed to delete email from
index.",e,logger);
}
}
}
protected void openIndex() throws MessageSearchException {
Exception lastError = null;
synchronized(indexLock) {
if (writer==null) {
logger.debug("openIndex() index will be opened. it is
currently closed.");
} else {
logger.debug("openIndex() did not bother opening index. it
is already open.");
return;
}
logger.debug("opening index for write
{"+volume+"}");
indexer.prepareIndex(volume);
logger.debug("opening search index for write
{indexpath='"+volume.getIndexPath()+"'}");
boolean error = false;
int attempt = 0;
do {
try {
writer = new IndexWriter(FSDirectory.getDirectory
(volume.getIndexPath()),false, analyzer);
writer.setMaxFieldLength(50000);
if
(logger.isDebugEnabled()) {
File file = new File(Config.getFileSystem().getLogPath()
+File.separator+"index.log");
PrintStream
debugout = new PrintStream(file);
writer.setInfoStream(debugout);
}
} catch (IOException io) {
lastError = io;
if
(Config.getConfig().getIndex().getMultipleIndexProcesses()) {
logger.debug("write lock on
index. will reopen in 50ms.");
try { Thread.sleep(50);
} catch (Exception e) {}
error = true;
attempt++;
} else {
throw new MessageSearchException("failed to open index
writer. you must delete the file write.lock in your the directory.
{location='"+volume.getIndexPath()+"'}",lastError,logger);
}
}
} while (error && attempt>1000);
if (attempt>=1000 || error)
throw new MessageSearchException("failed to open index writer
{location='"+volume.getIndexPath()+"'}",lastError,logger);
}
}
public void indexMessage(Email message) throws
MessageSearchException {
long s = (new Date()).getTime();
if (message == null)
throw new MessageSearchException("assertion failure: null
message",logger);
logger.debug("indexing message {"+message+"}");
Document doc = new Document();
try {
DocumentIndex docIndex = new DocumentIndex(indexer);
docIndex.write(message,doc);
String language = doc.get("lang");
if (language==null)
language = indexer.getIndexLanguage();
synchronized (indexLock) {
openIndex();
writer.addDocument(doc,AnalyzerFactory.getAnalyzer
(language,AnalyzerFactory.Operation.INDEX));
writer.flush();
}
doc = null;
logger.debug("message indexed successfully {"+message
+",language='"+language+"'}");
} catch (MessagingException me)
{
throw new MessageSearchException("failed to decode message
during indexing",me,logger, Level.DEBUG);
} catch (IOException me) {
throw new MessageSearchException("failed to index message
{"+message+"}",me,logger, Level.DEBUG);
} catch (ExtractionException ee)
{
// we will want to continue indexing
//throw new MessageSearchException("failed to decode
attachments in message {"+message+"}",ee,logger, Level.DEBUG);
} catch (AlreadyClosedException ace) {
indexMessage(message);
} catch (Exception e) {
throw new MessageSearchException("failed to index
message",e,logger, Level.DEBUG);
}
logger.debug("indexing message end {"+message+"}");
long e = (new Date()).getTime();
logger.debug("indexing time {time='"+(e-s)+"'}");
}
protected void closeIndex() {
synchronized(indexLock) {
if (writer==null)
return;
try {
writer.close();
logger.debug("writer closed");
} catch (Exception io) {
logger.error("failed to close index writer:"+io.getMessage
(),io);
}
writer = null;
}
}
public void deleteIndex() throws MessageSearchException {
logger.debug("delete index {indexpath='"+volume.getIndexPath()
+"'}");
try {
writer = new IndexWriter(FSDirectory.getDirectory
(volume.getIndexPath()),false, analyzer,true);
} catch (Exception cie) {
logger.error("failed to delete index
{index='"+volume.getIndexPath()+"'}",cie);
return;
}
try {
writer.close();
} catch (Exception e) {
logger.error("failed to delete index
{index='"+volume.getIndexPath()+"'}",e);
}
MessageIndex.volumeIndexes.remove(this);
}
public void startup() {
logger.debug("volumeindex is starting up");
File lockFile = new File(volume.getIndexPath()
+File.separatorChar + "write.lock");
if (lockFile.exists()) {
logger.warn("The server lock file already exists. Either
another indexer running or the server was not shutdown correctly.");
logger.warn("If it is the latter, the lock file must be
manually deleted at "+lockFile.getAbsolutePath());
if (indexer.getMultipleIndexProcesses()) {
logger.debug("index lock file detected on
volumeindex startup.");
} else {
logger.warn("index lock file detected. the server was shutdown
incorrectly. automatically deleting lock file.");
lockFile.delete();
}
}
closeIndexTimer.scheduleAtFixedRate(new TimerTask() {
public void run() {
closeIndex();
}
}, indexOpenTime, indexOpenTime);
Runtime.getRuntime().addShutdownHook(this);
/*
*
* io.printStackTrace();
if (attempt==9) {
// most obvious reason for error is that there is a lock on
the index, due hard shutdown
// resolution delete the
lock, and try again
logger.warn("failed to open search index for write.
possible write lock due to hard system shutdown.",io);
logger.info("attempting recovery. deleting index lock
file and retrying..");
File lockFile = new File(activeIndex.getPath()
+File.separatorChar + "write.lock");
lockFile.delete();
}
attempt++;
try { Thread.sleep(10); } catch
(Exception e) {}
lastError = io;
*/
}
public void shutdown() {
logger.debug("volumeindex is shutting down");
closeIndexTimer.cancel();
closeIndex();
}
public void run() {
closeIndex();
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
