To 2nd Vitaly's suggestion. You should consider using Apache Solr instead - it handles such issues OOTB .
On Fri, May 23, 2014 at 7:52 PM, Vitaly Funstein <vfunst...@gmail.com> wrote: > At the risk of sounding overly critical here, I would say you need to scrap > your entire approach of building one small index per request, and just > build your entire searchable data store in Lucene/Solr. This is the > simplest and probably most maintainable and scalable solution. Even if your > index contains 10M+ documents, returning at most 500 search results should > be lightning fast compared to the latencies you're seeing right now. To > facilitate data export from the DB, take a look at this: > http://wiki.apache.org/solr/DataImportHandler > > > On Tue, May 20, 2014 at 7:36 AM, Shruthi <sse...@imedx.com> wrote: > >> >> >> >> >> -----Original Message----- >> From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] >> Sent: Tuesday, May 20, 2014 3:48 PM >> To: java-user@lucene.apache.org >> Subject: Re: NewBie To Lucene || Perfect configuration on a 64 bit server >> >> On Tue, 2014-05-20 at 11:56 +0200, Shruthi wrote: >> >> Toke: >> > Is 20 second an acceptable response time for your users? >> > >> > Shruthi: Its definitely not acceptable. PFA the piece of code that we >> > are using..Its taking 20seconds. That’s why I drafted this ticket to >> > see where I was going wrong. >> >> Indexing 1000 documents/sec in Lucene is quite common, so even taking >> into account large documents, 20 seconds sounds like quite a bit. >> Shruthi: I had attached the code snippet in previous mail. Do you suspect >> a foul play there? >> >> > Shruthi: Well, its two stage process: Client is looking at >> > historical data based on a parameters like names, dates,MRN, fields >> > etc.. SO the query actually gets the data set fulfilling the >> > requirements >> > >> > If client is interested in doing a text search then he would pass the >> > search phrase on the result set. >> >> So it is not possible for a client to perform a broad phrase search to >> start with. And it sounds like your DB-queries are all simple matching? >> No complex joins and such? If so, this calls even more for a full >> Lucene-index solution, which handles all aspect of the search process. >> Shruthi: We call a DB stored procedure to get us the result set for >> working with.. >> We will be using highlighter API and I don’t think Memory index can be >> used with highlighter. >> >> > >> - Toke Eskildsen, State and University Library, Denmark >> >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> -- sk...@alum.mit.edu (617) 595-5946 --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
