Hi Uwe,
Below is the code that shows how we are opening writer and searchermanager:
LimitTokenCountAnalyzer limit_analyzer = new
LimitTokenCountAnalyzer(analyzer, 100000, true);
IndexWriterConfig writer_config = new IndexWriterConfig(limit_analyzer);
writer_config.setRAMBufferSizeMB(320);
writer_config.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
JobDivaIndexWarmer indexWarmer = new JobDivaIndexWarmer();
writer_config.setMergedSegmentWarmer(indexWarmer);
if (_useCompoundIndexFormat==1) writer_config.setUseCompoundFile(true);
else writer_config.setUseCompoundFile(false);
indexWriter = new IndexWriter(fsDir, writer_config);
searcherManager = new SearcherManager(indexWriter, new SearcherFactory());
And then we keep that indexwriter alive and after each batch of document we do
the following:
indexWriter.commit();
searcherManager.maybeRefresh();
Should these sequence of events cause the MergedSegmentWarmer to get called?
------
Regards
-Siraj Haider
(212) 306-0154
-----Original Message-----
From: Uwe Schindler [mailto:[email protected]]
Sent: Wednesday, December 21, 2016 11:37 AM
To: [email protected]
Subject: RE: Warming Indexes
Hi,
The warmup only happens if you reopen the searcher using the NRT APIs or
searcher manager. If you just index with a single IndexWriter that has no open
NRT readers, nothing will happen.
To warmup when the application starts, I'd suggest to use the first
IndexSearcher. The warmer in IndexWriter is just there to run queries on new
index segments that get live through SearcherManager or other NRT APIs.
Uwe
-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [email protected]
> -----Original Message-----
> From: Siraj Haider [mailto:[email protected]]
> Sent: Wednesday, December 21, 2016 3:58 PM
> To: [email protected]
> Subject: RE: Warming Indexes
>
> Thanks Uwe, I implemented the interface and am printing some lines to
> see if the warmup is happening, but I never see those prints in the
> log after some documents are indexed. Another question is how to
> warmup the index when I open the index first time, i.e. when the
> application starts. Should I simply run warmup queries on the acquired
> IndexSearcher from SearcherManager or is there a better way to accomplish
> that.
>
> ------
> Regards
> -Siraj Haider
> (212) 306-0154
>
> -----Original Message-----
> From: Uwe Schindler [mailto:[email protected]]
> Sent: Monday, December 19, 2016 1:29 PM
> To: [email protected]
> Subject: RE: Warming Indexes
>
> Hi,
>
> Just implement this interface and override the sole abstract method.
> The argument is a LeafReader that you can quickly wrap with an
> IndexSearcher (LeafReader is a subclass of IndexReader, so it's
> standard pattern) and execute one or multiple queries on it. This
> depends on your use case. Do whatever you want with the to-be warmed
> LeafReader.
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> http://www.thetaphi.de
> eMail: [email protected]
>
> > -----Original Message-----
> > From: Siraj Haider [mailto:[email protected]]
> > Sent: Monday, December 19, 2016 4:10 PM
> > To: [email protected]
> > Subject: RE: Warming Indexes
> >
> > Thanks for the information Uwe, it was very helpful. Do you have any
> > example code implementing IndexWriter.IndexReaderWarmer class? I am
> > having difficulty finding any examples on internet.
> >
> > ------
> > Regards
> > -Siraj Haider
> > (212) 306-0154
> >
> > -----Original Message-----
> > From: Uwe Schindler [mailto:[email protected]]
> > Sent: Wednesday, December 14, 2016 3:34 PM
> > To: [email protected]
> > Subject: RE: Warming Indexes
> >
> > Hi,
> >
> > How about subclassing SearcherFactory and include warming there?
> >
> https://lucene.apache.org/core/6_3_0/core/org/apache/lucene/search/Sea
> > r
> > cherFactory.html
> >
> > Refer especially to the last bullet point. As described there to do
> > warming in the background without blocking in the near realtime
> > case, use warmers on
> > IndexWriter.setMergedSegmentWarmer() instead. This has the effect
> > that only new index segments are warmed as needed instead of the
> > whole index on every reopen.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > http://www.thetaphi.de
> > eMail: [email protected]
> >
> > > -----Original Message-----
> > > From: Siraj Haider [mailto:[email protected]]
> > > Sent: Wednesday, December 14, 2016 9:15 PM
> > > To: [email protected]
> > > Subject: Warming Indexes
> > >
> > > Hi,
> > >
> > > We are in the process of moving from lucene 2.9 to 6.3. We are
> > > going to use SearcherManager and SearcherFactory classes for the
> > > first time and are wondering on how to warm our indexes the first time.
> > > Below are the snippets of code, where we create our IndexWriter
> > > and
> > SearcherManager.
> > > And then we use searcherManager.acquire() to get IndexSearcher.
> > > Can somebody please guide us on how to implement search warming?
> > > Would
> > we
> > > need to warm each IndexSearcher that we get from .acquire()?
> > >
> > > IndexWriter indexWriter = new IndexWriter(new
> > > NIOFSDirectory(new File(_indexLocation).toPath()), writer_config);
> > > SearcherManager searcherManager = new
> > SearcherManager(indexWriter,
> > > new SearcherFactory());
> > >
> > >
> > > ------
> > > Regards
> > > -Siraj Haider
> > > (212) 306-0154
> > >
> > >
> > > ________________________________
> > >
> > > This electronic mail message and any attachments may contain
> > > information which is privileged, sensitive and/or otherwise exempt
> > > from disclosure under applicable law. The information is intended
> > > only for the use of the individual or entity named as the addressee above.
> > > If you are not the intended recipient, you are hereby notified
> > > that any disclosure, copying, distribution (electronic or
> > > otherwise) or forwarding of, or the taking of any action in
> > > reliance on, the contents of this transmission is strictly
> > > prohibited. If you have received this electronic transmission in
> > > error, please notify us by telephone, facsimile, or e-mail as
> > > noted above to arrange for the return of
> > any electronic mail or attachments. Thank You.
> >
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: [email protected]
> > For additional commands, e-mail: [email protected]
> >
> >
> > ________________________________
> >
> > This electronic mail message and any attachments may contain
> > information which is privileged, sensitive and/or otherwise exempt
> > from disclosure under applicable law. The information is intended
> > only for the use of the individual or entity named as the addressee above.
> > If you are not the intended recipient, you are hereby notified that
> > any disclosure, copying, distribution (electronic or otherwise) or
> > forwarding of, or the taking of any action in reliance on, the
> > contents of this transmission is strictly prohibited. If you have
> > received this electronic transmission in error, please notify us by
> > telephone, facsimile, or e-mail as noted above to arrange for the
> > return of
> any electronic mail or attachments. Thank You.
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: [email protected]
> > For additional commands, e-mail: [email protected]
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
> ________________________________
>
> This electronic mail message and any attachments may contain
> information which is privileged, sensitive and/or otherwise exempt
> from disclosure under applicable law. The information is intended only
> for the use of the individual or entity named as the addressee above.
> If you are not the intended recipient, you are hereby notified that
> any disclosure, copying, distribution (electronic or otherwise) or
> forwarding of, or the taking of any action in reliance on, the
> contents of this transmission is strictly prohibited. If you have
> received this electronic transmission in error, please notify us by
> telephone, facsimile, or e-mail as noted above to arrange for the return of
> any electronic mail or attachments. Thank You.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
________________________________
This electronic mail message and any attachments may contain information which
is privileged, sensitive and/or otherwise exempt from disclosure under
applicable law. The information is intended only for the use of the individual
or entity named as the addressee above. If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution (electronic
or otherwise) or forwarding of, or the taking of any action in reliance on, the
contents of this transmission is strictly prohibited. If you have received this
electronic transmission in error, please notify us by telephone, facsimile, or
e-mail as noted above to arrange for the return of any electronic mail or
attachments. Thank You.
