RE: GData, updateable IndexSearcher

[Robert Engels]

fyi, using my reopen(0 implementation (which rereads the deletions)

on a 135mb index, with 5000 iterations

open & close time using new reader = 585609
open & close time using reopen = 27422

Almost 20x faster. Important in a highly interactive/incremental updating
index.

-----Original Message-----
From: jason rutherglen [mailto:[EMAIL PROTECTED]
Sent: Monday, May 01, 2006 1:24 PM
To: java-dev@lucene.apache.org
Subject: Re: GData, updateable IndexSearcher


I wanted to post a quick hack to see if it is along the correct lines.  A
few of the questions regard whether to resuse existing MultiReaders or
simply strip out only the SegmentReaders.  I do a compare on the segment
name and made it public.  Thanks!


public static IndexReader reopen(IndexReader indexReader) throws IOException
{
    if (indexReader instanceof MultiReader) {
      MultiReader multiReader = (MultiReader)indexReader;

      SegmentInfos segmentInfos = new SegmentInfos();
      segmentInfos.read(indexReader.directory());
      if (segmentInfos.size() == 1) {          // index is optimized
        return SegmentReader.get(segmentInfos, segmentInfos.info(0), false);
      }

      IndexReader[] existingIndexReaders = multiReader.getSubReaders();
      // now go through and compare the segment readers
      Map<String,SegmentReader> existingSegmentMap = new
HashMap<String,SegmentReader>();
      getSegmentReaders(existingIndexReaders, existingSegmentMap);

      Map<String,SegmentInfo> newSegmentInfosMap = new
HashMap<String,SegmentInfo>();

      List<SegmentReader> newSegmentReaders = new
ArrayList<SegmentReader>();

      Iterator segmentInfosIterator = segmentInfos.iterator();
      while (segmentInfosIterator.hasNext()) {
        SegmentInfo segmentInfo = (SegmentInfo)segmentInfosIterator.next();

        if (!existingSegmentMap.containsKey(segmentInfo.name)) {
          // it's new
          SegmentReader newSegmentReader = SegmentReader.get(segmentInfo);
          newSegmentReaders.add(newSegmentReader);
        }
      }
      List<IndexReader> allSegmentReaders = new ArrayList<IndexReader>();
      allSegmentReaders.add(multiReader);
      allSegmentReaders.addAll(newSegmentReaders);

      return new MultiReader(indexReader.directory(), segmentInfos, false,
(IndexReader[])allSegmentReaders.toArray(new IndexReader[0]));
    }
    throw new RuntimeException("indexReader not supported at this time");
  }

  public static void getSegmentReaders(IndexReader[] indexReaders,
Map<String,SegmentReader> map) {
    for (int x=0; x < indexReaders.length; x++) {
      if (indexReaders[x] instanceof MultiReader) {
        MultiReader multiReader = (MultiReader)indexReaders[x];
        IndexReader[] subReaders = multiReader.getSubReaders();
        getSegmentReaders(subReaders, map);
      } else if (indexReaders[x] instanceof SegmentReader) {
        SegmentReader segmentReader = (SegmentReader)indexReaders[x];
        map.put(segmentReader.segment, segmentReader);
      }
    }
  }



---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]