The ERROR log o the IOException immediately above my comment:) --tim
On Sunday, May 26, 2013, Aaron McCurry wrote: > I'm looking at the logging statements, and they look like they have shard > context in them. Can you point me to a line(s) that don't? Thanks! > > Aaron > > > On Sun, May 26, 2013 at 7:40 AM, Tim Williams > <[email protected]<javascript:;>> > wrote: > > > On Sun, May 26, 2013 at 1:17 AM, <[email protected] <javascript:;>> > wrote: > > > Updated Branches: > > > refs/heads/0.1.5 a18bce613 -> 91afa673f > > > > > > > > > BLUR:95 fixed. > > > > > > > > > Project: http://git-wip-us.apache.org/repos/asf/incubator-blur/repo > > > Commit: > > http://git-wip-us.apache.org/repos/asf/incubator-blur/commit/91afa673 > > > Tree: > > http://git-wip-us.apache.org/repos/asf/incubator-blur/tree/91afa673 > > > Diff: > > http://git-wip-us.apache.org/repos/asf/incubator-blur/diff/91afa673 > > > > > > Branch: refs/heads/0.1.5 > > > Commit: 91afa673f0e56d717590c0f4efe9afe906fa1c4d > > > Parents: a18bce6 > > > Author: Gagan <[email protected] <javascript:;>> > > > Authored: Sun May 26 10:46:57 2013 +0530 > > > Committer: Gagan <[email protected] <javascript:;>> > > > Committed: Sun May 26 10:46:57 2013 +0530 > > > > > > ---------------------------------------------------------------------- > > > .../apache/blur/manager/writer/IndexImporter.java | 54 +++++- > > > .../blur/manager/writer/IndexImporterTest.java | 141 > > +++++++++++++++ > > > 2 files changed, 185 insertions(+), 10 deletions(-) > > > ---------------------------------------------------------------------- > > > > > > > > > > > > http://git-wip-us.apache.org/repos/asf/incubator-blur/blob/91afa673/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java > > > ---------------------------------------------------------------------- > > > diff --git > > > a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java > > > b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java > > > index 5c09ab3..4efd908 100644 > > > --- > > > a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java > > > +++ > > > b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java > > > @@ -13,13 +13,16 @@ import java.util.concurrent.locks.ReadWriteLock; > > > > > > import org.apache.blur.log.Log; > > > import org.apache.blur.log.LogFactory; > > > +import org.apache.blur.manager.BlurPartitioner; > > > import org.apache.blur.server.ShardContext; > > > import org.apache.blur.store.hdfs.HdfsDirectory; > > > import org.apache.blur.utils.BlurConstants; > > > +import org.apache.blur.utils.BlurUtil; > > > import org.apache.hadoop.conf.Configuration; > > > import org.apache.hadoop.fs.FileStatus; > > > import org.apache.hadoop.fs.FileSystem; > > > import org.apache.hadoop.fs.Path; > > > +import org.apache.hadoop.io.Text; > > > import org.apache.lucene.index.AtomicReader; > > > import org.apache.lucene.index.AtomicReaderContext; > > > import org.apache.lucene.index.CompositeReaderContext; > > > @@ -92,15 +95,35 @@ public class IndexImporter extends TimerTask > > implements Closeable { > > > for (HdfsDirectory directory : indexesToImport) { > > > LOG.info("Starting import [{0}], commiting on [{1}/{2}]", > > directory, shard, table); > > > indexWriter.commit(); > > > - applyDeletes(directory, indexWriter); > > > - LOG.info("Add index [{0}] [{1}/{2}]", directory, shard, > > table); > > > - indexWriter.addIndexes(directory); > > > - LOG.info("Finishing import [{0}], commiting on [{1}/{2}]", > > directory, shard, table); > > > - indexWriter.commit(); > > > - Path dirPath = directory.getPath(); > > > - LOG.info("Cleaning up old directory [{0}] for [{1}/{2}]", > > dirPath, shard, table); > > > - fileSystem.delete(dirPath, true); > > > - LOG.info("Import complete on [{0}/{1}]", shard, table); > > > + boolean isSuccess = false; > > > + boolean isRollbackDueToException = false; > > > + try { > > > + isSuccess = applyDeletes(directory, indexWriter, shard); > > > + }catch(IOException e){ > > > + LOG.error("Some issue with deleting the old index", e); > > > > Shouldn't we add some context (e.g. shard) to the log so the problem's > > easier to hunt down? > > > > --tim > > >
