I'm looking at the logging statements, and they look like they have shard context in them. Can you point me to a line(s) that don't? Thanks!
Aaron On Sun, May 26, 2013 at 7:40 AM, Tim Williams <[email protected]> wrote: > On Sun, May 26, 2013 at 1:17 AM, <[email protected]> wrote: > > Updated Branches: > > refs/heads/0.1.5 a18bce613 -> 91afa673f > > > > > > BLUR:95 fixed. > > > > > > Project: http://git-wip-us.apache.org/repos/asf/incubator-blur/repo > > Commit: > http://git-wip-us.apache.org/repos/asf/incubator-blur/commit/91afa673 > > Tree: > http://git-wip-us.apache.org/repos/asf/incubator-blur/tree/91afa673 > > Diff: > http://git-wip-us.apache.org/repos/asf/incubator-blur/diff/91afa673 > > > > Branch: refs/heads/0.1.5 > > Commit: 91afa673f0e56d717590c0f4efe9afe906fa1c4d > > Parents: a18bce6 > > Author: Gagan <[email protected]> > > Authored: Sun May 26 10:46:57 2013 +0530 > > Committer: Gagan <[email protected]> > > Committed: Sun May 26 10:46:57 2013 +0530 > > > > ---------------------------------------------------------------------- > > .../apache/blur/manager/writer/IndexImporter.java | 54 +++++- > > .../blur/manager/writer/IndexImporterTest.java | 141 > +++++++++++++++ > > 2 files changed, 185 insertions(+), 10 deletions(-) > > ---------------------------------------------------------------------- > > > > > > > http://git-wip-us.apache.org/repos/asf/incubator-blur/blob/91afa673/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java > > ---------------------------------------------------------------------- > > diff --git > a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java > b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java > > index 5c09ab3..4efd908 100644 > > --- > a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java > > +++ > b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java > > @@ -13,13 +13,16 @@ import java.util.concurrent.locks.ReadWriteLock; > > > > import org.apache.blur.log.Log; > > import org.apache.blur.log.LogFactory; > > +import org.apache.blur.manager.BlurPartitioner; > > import org.apache.blur.server.ShardContext; > > import org.apache.blur.store.hdfs.HdfsDirectory; > > import org.apache.blur.utils.BlurConstants; > > +import org.apache.blur.utils.BlurUtil; > > import org.apache.hadoop.conf.Configuration; > > import org.apache.hadoop.fs.FileStatus; > > import org.apache.hadoop.fs.FileSystem; > > import org.apache.hadoop.fs.Path; > > +import org.apache.hadoop.io.Text; > > import org.apache.lucene.index.AtomicReader; > > import org.apache.lucene.index.AtomicReaderContext; > > import org.apache.lucene.index.CompositeReaderContext; > > @@ -92,15 +95,35 @@ public class IndexImporter extends TimerTask > implements Closeable { > > for (HdfsDirectory directory : indexesToImport) { > > LOG.info("Starting import [{0}], commiting on [{1}/{2}]", > directory, shard, table); > > indexWriter.commit(); > > - applyDeletes(directory, indexWriter); > > - LOG.info("Add index [{0}] [{1}/{2}]", directory, shard, > table); > > - indexWriter.addIndexes(directory); > > - LOG.info("Finishing import [{0}], commiting on [{1}/{2}]", > directory, shard, table); > > - indexWriter.commit(); > > - Path dirPath = directory.getPath(); > > - LOG.info("Cleaning up old directory [{0}] for [{1}/{2}]", > dirPath, shard, table); > > - fileSystem.delete(dirPath, true); > > - LOG.info("Import complete on [{0}/{1}]", shard, table); > > + boolean isSuccess = false; > > + boolean isRollbackDueToException = false; > > + try { > > + isSuccess = applyDeletes(directory, indexWriter, shard); > > + }catch(IOException e){ > > + LOG.error("Some issue with deleting the old index", e); > > Shouldn't we add some context (e.g. shard) to the log so the problem's > easier to hunt down? > > --tim >
