Hi Michael
Thank you. Your suggestions were great and they were implemented (see
attached source code), however, unfortunately, I am still getting file
not found errors on the automatic merging of indexes.
Regards,
Jamie
Michael McCandless wrote:
Jamie,
I'd love to get to the root cause of your exception.
Last time we talked (a few weeks back) I saw several possible causes
in the source you had posted:
http://markmail.org/message/dqovvcwgwof5f7wl
Did you test any of the ideas there? You are potentially manually
deleting files from the index, and also forcibly removing the
write.lock, which can easily cause this exception.
Mike
Jamie wrote:
Hi Paul,
I just noticed the discussion around this.
All most all of my customers have/are experiencing the intermittant
FileNotFound problem.
Our software uses Lucene 2.3.1. I have just upgraded to Lucene 2.3.2
in the hope that this was one of the bugs that was fixed.
I would be very interested in a resolution or known workaround.
Thanks!
Jamie
Paul J. Lucas wrote:
On May 29, 2008, at 6:35 PM, Michael McCandless wrote:
Can you use lsof (or something similar) to see how many files you
have?
FYI: I personally can't reproduce this; only a coworker can and even
then it's sporadic, so it could take a little while.
Merging, especially several running at once, can greatly increase
open file count, and especially if mergeFactor is increased.
That raises a few questions:
1. Should I lower my mergeFactor?
2. Is there any way to insist that only one merger runs?
3. Is there any way to insist that all merges happen synchronously,
i.e., on IndexWriter.close() only and not to use a separate merge
thread?
- Paul
package com.stimulus.archiva.index;
/* Copyright (C) 2005-2007 Jamie Angus Band
* MailArchiva Open Source Edition Copyright (c) 2005-2007 Jamie Angus Band
* This program is free software; you can redistribute it and/or modify it
under the terms of
* the GNU General Public License as published by the Free Software Foundation;
either version
* 2 of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY;
* without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
* See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along with
this program;
* if not, see http://www.gnu.org/licenses or write to the Free Software
Foundation,Inc., 51
* Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
import javax.mail.*;
import javax.mail.internet.*;
import java.io.*;
import java.util.*;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.zip.*;
import com.ice.tar.*;
import org.apache.log4j.Logger;
import org.apache.lucene.document.*;
import org.apache.lucene.index.*;
import org.apache.lucene.store.*;
import com.stimulus.archiva.domain.*;
import com.stimulus.archiva.exception.*;
import com.stimulus.archiva.language.*;
import com.stimulus.archiva.extraction.*;
import com.stimulus.archiva.search.*;
import com.stimulus.util.TempFiles;
import com.stimulus.archiva.domain.Service.Status;
import com.stimulus.archiva.domain.fields.*;
import com.stimulus.util.*;
import java.nio.charset.Charset;
import org.apache.log4j.Level;
public class MessageIndex extends Indexer implements Serializable {
private static final long serialVersionUID = -17692874371162272L;
protected static final Logger logger =
Logger.getLogger(MessageIndex.class.getName());
protected static int INDEX_WAIT_PERIOD = 50;
protected static int DEAD_PERIOD = 300000000;
static Hashtable<Volume,VolumeIndex> volumeIndexes = new
Hashtable<Volume,VolumeIndex>();
static Object volumeIndexLock = new Object();
protected ServiceDelegate serviceDelegate;
public MessageIndex() {
serviceDelegate = new ServiceDelegate("message index", this,
logger);
}
public String getServiceName() {
return serviceDelegate.getServiceName();
}
public boolean isAlive() {
return serviceDelegate.isAlive(true);
}
public void startup() {
for (VolumeIndex volumeIndex : volumeIndexes.values()) {
volumeIndex.startup();
}
serviceDelegate.startup();
}
public void prepareShutdown() {
serviceDelegate.prepareShutdown();
}
public void shutdown() {
for (VolumeIndex volumeIndex : volumeIndexes.values()) {
volumeIndex.shutdown();
}
serviceDelegate.shutdown();
}
public void reloadConfig() {
serviceDelegate.reloadConfig();
}
public Status getStatus() {
return serviceDelegate.getStatus();
}
public void deleteIndex(Volume volume) throws MessageSearchException {
VolumeIndex volumeIndex = getVolumeIndex(volume);
volumeIndex.deleteIndex();
}
public VolumeIndex getVolumeIndex(Volume volume) {
VolumeIndex volumeIndex = null;
synchronized (volumeIndexLock) {
volumeIndex = volumeIndexes.get(volume);
if (volumeIndex==null) {
volumeIndex = new VolumeIndex(this,volume);
volumeIndexes.put(volume,volumeIndex);
}
}
return volumeIndex;
}
public void indexMessage(Email email) throws MessageSearchException {
if (serviceDelegate.getStatus()!=Status.STARTED) {
return;
}
VolumeIndex volumeIndex =
getVolumeIndex(email.getEmailId().getVolume());
volumeIndex.indexMessage(email);
}
public void deleteMessage(EmailID emailID) throws
MessageSearchException {
VolumeIndex volumeIndex = getVolumeIndex(emailID.getVolume());
volumeIndex.deleteMessage(emailID);
}
public void prepareIndex(Volume volume) throws MessageSearchException {
if (volume==null)
throw new MessageSearchException("assertion
failure: null volume",logger);
if (volume.getIndexPath().startsWith("rmi://"))
return;
File indexDir = new File(volume.getIndexPath());
if (!indexDir.exists()) {
logger.info("index directory does not exist. will proceed with
creation {location='" + volume.getIndexPath() + "'}");
boolean success = indexDir.mkdir();
if (!success)
throw new MessageSearchException("failed to
create index directory {location='" + volume.getIndexPath() + "'}",logger);
logger.info("index directory successfully created {location='"
+ volume.getIndexPath() + "'}");
}
}
}
package com.stimulus.archiva.index;
import java.io.File;
import java.io.IOException;
import java.io.PrintStream;
import javax.mail.MessagingException;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.*;
import org.apache.lucene.store.FSDirectory;
import com.stimulus.archiva.domain.Config;
import com.stimulus.archiva.domain.Email;
import com.stimulus.archiva.domain.EmailID;
import com.stimulus.archiva.domain.Indexer;
import com.stimulus.archiva.domain.Volume;
import com.stimulus.archiva.exception.ExtractionException;
import com.stimulus.archiva.exception.MessageSearchException;
import com.stimulus.archiva.language.AnalyzerFactory;
import com.stimulus.archiva.search.*;
import java.util.*;
import org.apache.lucene.store.AlreadyClosedException;
public class VolumeIndex extends Thread {
protected static final Logger logger =
Logger.getLogger(VolumeIndex.class.getName());
public static final int indexOpenTime = 2000;
IndexWriter writer = null;
Volume volume;
Timer closeIndexTimer = new Timer();
Object indexLock = new Object();
ArchivaAnalyzer analyzer = new ArchivaAnalyzer();
Indexer indexer = null;
public VolumeIndex(Indexer indexer, Volume volume) {
this.volume = volume;
this.indexer = indexer;
startup();
}
public void deleteMessage(EmailID emailID) throws
MessageSearchException {
if (emailID == null)
throw new MessageSearchException("assertion
failure: null emailID",logger);
logger.debug("delete message {'"+emailID+"'}");
Volume volume = emailID.getVolume();
File indexDir = new File(volume.getIndexPath());
if (!indexDir.exists())
throw new MessageSearchException("could not delete
email from index. volume does not exist. {'"+emailID+"}",logger);
synchronized(indexLock) {
IndexReader indexReader = null;
try {
indexReader = IndexReader.open(indexDir);
} catch (IOException e ) {
throw new MessageSearchException("failed to
open index to delete email",e,logger);
}
try {
indexReader.deleteDocuments(new
Term("uid",emailID.getUniqueID()));
indexReader.close();
} catch (Exception e) {
throw new MessageSearchException("failed to
delete email from index.",e,logger);
}
}
}
protected void openIndex() throws MessageSearchException {
Exception lastError = null;
synchronized(indexLock) {
if (writer==null) {
logger.debug("openIndex() index will be opened.
it is currently closed.");
} else {
logger.debug("openIndex() did not bother
opening index. it is already open.");
return;
}
logger.debug("opening index for write
{"+volume+"}");
indexer.prepareIndex(volume);
logger.debug("opening search index for write
{indexpath='"+volume.getIndexPath()+"'}");
boolean error = false;
int attempt = 0;
do {
try {
writer = new
IndexWriter(FSDirectory.getDirectory(volume.getIndexPath()),false, analyzer);
writer.setMaxFieldLength(50000);
if
(logger.isDebugEnabled()) {
File file = new
File(Config.getFileSystem().getLogPath()+File.separator+"index.log");
PrintStream
debugout = new PrintStream(file);
writer.setInfoStream(debugout);
}
} catch (IOException io) {
lastError = io;
if
(Config.getConfig().getIndex().getMultipleIndexProcesses()) {
logger.debug("write
lock on index. will reopen in 50ms.");
try { Thread.sleep(50);
} catch (Exception e) {}
error = true;
attempt++;
} else {
throw new
MessageSearchException("failed to open index writer. you must delete the file
write.lock in your the directory.
{location='"+volume.getIndexPath()+"'}",lastError,logger);
}
}
} while (error && attempt>1000);
if (attempt>=1000 || error)
throw new MessageSearchException("failed to
open index writer {location='"+volume.getIndexPath()+"'}",lastError,logger);
}
}
public void indexMessage(Email message) throws
MessageSearchException {
long s = (new Date()).getTime();
if (message == null)
throw new MessageSearchException("assertion
failure: null message",logger);
logger.debug("indexing message {"+message+"}");
Document doc = new Document();
try {
DocumentIndex docIndex = new DocumentIndex(indexer);
docIndex.write(message,doc);
String language = doc.get("lang");
if (language==null)
language = indexer.getIndexLanguage();
synchronized (indexLock) {
openIndex();
writer.addDocument(doc,AnalyzerFactory.getAnalyzer(language,AnalyzerFactory.Operation.INDEX));
writer.flush();
}
doc = null;
logger.debug("message indexed successfully
{"+message+",language='"+language+"'}");
} catch (MessagingException me)
{
throw new MessageSearchException("failed to decode
message during indexing",me,logger, Level.DEBUG);
} catch (IOException me) {
throw new MessageSearchException("failed to index
message {"+message+"}",me,logger, Level.DEBUG);
} catch (ExtractionException ee)
{
// we will want to continue indexing
//throw new MessageSearchException("failed to decode
attachments in message {"+message+"}",ee,logger, Level.DEBUG);
} catch (AlreadyClosedException ace) {
indexMessage(message);
} catch (Exception e) {
throw new MessageSearchException("failed to index
message",e,logger, Level.DEBUG);
}
logger.debug("indexing message end {"+message+"}");
long e = (new Date()).getTime();
logger.debug("indexing time {time='"+(e-s)+"'}");
}
protected void closeIndex() {
synchronized(indexLock) {
if (writer==null)
return;
try {
writer.close();
logger.debug("writer closed");
} catch (Exception io) {
logger.error("failed to close
index writer:"+io.getMessage(),io);
}
writer = null;
}
}
public void deleteIndex() throws MessageSearchException {
logger.debug("delete index
{indexpath='"+volume.getIndexPath()+"'}");
try {
writer = new
IndexWriter(FSDirectory.getDirectory(volume.getIndexPath()),false,
analyzer,true);
} catch (Exception cie) {
logger.error("failed to delete index
{index='"+volume.getIndexPath()+"'}",cie);
return;
}
try {
writer.close();
} catch (Exception e) {
logger.error("failed to delete index
{index='"+volume.getIndexPath()+"'}",e);
}
MessageIndex.volumeIndexes.remove(this);
}
public void startup() {
logger.debug("volumeindex is starting up");
File lockFile = new
File(volume.getIndexPath()+File.separatorChar + "write.lock");
if (lockFile.exists()) {
logger.warn("The server lock file already
exists. Either another indexer running or the server was not shutdown
correctly.");
logger.warn("If it is the latter, the lock file
must be manually deleted at "+lockFile.getAbsolutePath());
if (indexer.getMultipleIndexProcesses()) {
logger.debug("index lock file detected
on volumeindex startup.");
} else {
logger.warn("index lock file detected.
the server was shutdown incorrectly. automatically deleting lock file.");
lockFile.delete();
}
}
closeIndexTimer.scheduleAtFixedRate(new TimerTask() {
public void run() {
closeIndex();
}
}, indexOpenTime, indexOpenTime);
Runtime.getRuntime().addShutdownHook(this);
/*
*
* io.printStackTrace();
if (attempt==9) {
// most obvious reason
for error is that there is a lock on the index, due hard shutdown
// resolution delete the
lock, and try again
logger.warn("failed to open
search index for write. possible write lock due to hard system shutdown.",io);
logger.info("attempting
recovery. deleting index lock file and retrying..");
File lockFile = new
File(activeIndex.getPath()+File.separatorChar + "write.lock");
lockFile.delete();
}
attempt++;
try { Thread.sleep(10); } catch
(Exception e) {}
lastError = io;
*/
}
public void shutdown() {
logger.debug("volumeindex is shutting down");
closeIndexTimer.cancel();
closeIndex();
}
public void run() {
closeIndex();
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
