Hi Nick, 

Thanks for your reply! The another constraint with the single index approach is 
our index locations are dynamic and search happened through APIs which 
construct the index location based on certain input parameters. 

However, I did modification in code to do fork process to complete the actions. 
In that process Searcher object gets created and destroyed and I get all search 
hits in an array that I process later. I did not use polysercher. This 
increases the speed. 

Rajiv Gupta

-----Original Message-----
From: Nick Wellnhofer [mailto:wellnho...@aevum.de] 
Sent: Friday, September 16, 2016 3:51 PM
To: user@lucy.apache.org
Subject: Re: [lucy-user] Speed up Search with Lucy::Search::IndexSearcher and 
Lucy::Search::PolySearcher from multiple index folders

On 14/09/2016 09:05, Gupta, Rajiv wrote:
> I'm creating indexes on multiple subfolders under one parent folder.
> Indexes are created on multiple folders since files are getting created in 
> parallel and I want to avoid segment locking between multiple indexers.

> I did profiling using Devel::NYTProf<https://metacpan.org/pod/Devel::NYTProf> 
> and found two places where the maximum time was taken:
> 1.    While scanning the directory. (This I will try to solve by generating a 
> list of directories while the application is generating the indexes).
> 2.    When creating the searchers using Lucy::Search::IndexSearcher. This 
> takes maximum time when running in loop for all indexed directories.

It sounds like you're working with an excessively large number of indices. 
Maybe you should simply rethink your approach and use a single index? If you're 
concerned about locking maybe a separate indexing process with some kind of 
notification mechanism would help?


Reply via email to