I see it. Yep we should. Missed it in the code. Sent from my iPhone
On May 26, 2013, at 11:31 AM, Tim Williams <[email protected]> wrote: > The ERROR log o the IOException immediately above my comment:) > > --tim > > On Sunday, May 26, 2013, Aaron McCurry wrote: > >> I'm looking at the logging statements, and they look like they have shard >> context in them. Can you point me to a line(s) that don't? Thanks! >> >> Aaron >> >> >> On Sun, May 26, 2013 at 7:40 AM, Tim Williams >> <[email protected]<javascript:;>> >> wrote: >> >>> On Sun, May 26, 2013 at 1:17 AM, <[email protected] <javascript:;>> >> wrote: >>>> Updated Branches: >>>> refs/heads/0.1.5 a18bce613 -> 91afa673f >>>> >>>> >>>> BLUR:95 fixed. >>>> >>>> >>>> Project: http://git-wip-us.apache.org/repos/asf/incubator-blur/repo >>>> Commit: >>> http://git-wip-us.apache.org/repos/asf/incubator-blur/commit/91afa673 >>>> Tree: >>> http://git-wip-us.apache.org/repos/asf/incubator-blur/tree/91afa673 >>>> Diff: >>> http://git-wip-us.apache.org/repos/asf/incubator-blur/diff/91afa673 >>>> >>>> Branch: refs/heads/0.1.5 >>>> Commit: 91afa673f0e56d717590c0f4efe9afe906fa1c4d >>>> Parents: a18bce6 >>>> Author: Gagan <[email protected] <javascript:;>> >>>> Authored: Sun May 26 10:46:57 2013 +0530 >>>> Committer: Gagan <[email protected] <javascript:;>> >>>> Committed: Sun May 26 10:46:57 2013 +0530 >>>> >>>> ---------------------------------------------------------------------- >>>> .../apache/blur/manager/writer/IndexImporter.java | 54 +++++- >>>> .../blur/manager/writer/IndexImporterTest.java | 141 >>> +++++++++++++++ >>>> 2 files changed, 185 insertions(+), 10 deletions(-) >>>> ---------------------------------------------------------------------- >>>> >>>> >>>> >>> >> http://git-wip-us.apache.org/repos/asf/incubator-blur/blob/91afa673/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java >>>> ---------------------------------------------------------------------- >>>> diff --git >>> >> a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java >>> >> b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java >>>> index 5c09ab3..4efd908 100644 >>>> --- >>> >> a/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java >>>> +++ >>> >> b/src/blur-core/src/main/java/org/apache/blur/manager/writer/IndexImporter.java >>>> @@ -13,13 +13,16 @@ import java.util.concurrent.locks.ReadWriteLock; >>>> >>>> import org.apache.blur.log.Log; >>>> import org.apache.blur.log.LogFactory; >>>> +import org.apache.blur.manager.BlurPartitioner; >>>> import org.apache.blur.server.ShardContext; >>>> import org.apache.blur.store.hdfs.HdfsDirectory; >>>> import org.apache.blur.utils.BlurConstants; >>>> +import org.apache.blur.utils.BlurUtil; >>>> import org.apache.hadoop.conf.Configuration; >>>> import org.apache.hadoop.fs.FileStatus; >>>> import org.apache.hadoop.fs.FileSystem; >>>> import org.apache.hadoop.fs.Path; >>>> +import org.apache.hadoop.io.Text; >>>> import org.apache.lucene.index.AtomicReader; >>>> import org.apache.lucene.index.AtomicReaderContext; >>>> import org.apache.lucene.index.CompositeReaderContext; >>>> @@ -92,15 +95,35 @@ public class IndexImporter extends TimerTask >>> implements Closeable { >>>> for (HdfsDirectory directory : indexesToImport) { >>>> LOG.info("Starting import [{0}], commiting on [{1}/{2}]", >>> directory, shard, table); >>>> indexWriter.commit(); >>>> - applyDeletes(directory, indexWriter); >>>> - LOG.info("Add index [{0}] [{1}/{2}]", directory, shard, >>> table); >>>> - indexWriter.addIndexes(directory); >>>> - LOG.info("Finishing import [{0}], commiting on [{1}/{2}]", >>> directory, shard, table); >>>> - indexWriter.commit(); >>>> - Path dirPath = directory.getPath(); >>>> - LOG.info("Cleaning up old directory [{0}] for [{1}/{2}]", >>> dirPath, shard, table); >>>> - fileSystem.delete(dirPath, true); >>>> - LOG.info("Import complete on [{0}/{1}]", shard, table); >>>> + boolean isSuccess = false; >>>> + boolean isRollbackDueToException = false; >>>> + try { >>>> + isSuccess = applyDeletes(directory, indexWriter, shard); >>>> + }catch(IOException e){ >>>> + LOG.error("Some issue with deleting the old index", e); >>> >>> Shouldn't we add some context (e.g. shard) to the log so the problem's >>> easier to hunt down? >>> >>> --tim >>> >>
