Retrieving large num of docs

2009-11-26 Thread Raghuveer Kancherla
Hi, I am using Solr1.4 for searching through half a million documents. The problem is, I want to retrieve nearly 200 documents for each search query. The query time in Solr logs is showing 0.02 seconds and I am fairly happy with that. However Solr is taking a long time (4 to 5 secs) to return the

Re: Retrieving large num of docs

2009-11-27 Thread Raghuveer Kancherla
on how different Solr components perform we injected perf4j statements into key points in the code. And a profiler was helpful too. Hope it helps somehow. On Thu, Nov 26, 2009 at 8:48 PM, Raghuveer Kancherla raghuveer.kanche...@aplopio.com wrote: Hi, I am using Solr1.4 for searching through

Re: Retrieving large num of docs

2009-11-28 Thread Raghuveer Kancherla
Hi Andrew, I applied the patch you suggested. I am not finding any significant changes in the response times. I am wondering if I forgot some important configuration setting etc. Here is what I did: 1. Wrote a small program using solrj to use EmbeddedSolrServer (most of the code is from the

Re: Retrieving large num of docs

2009-11-29 Thread Raghuveer Kancherla
Thanks Hoss, In my previous mail, I was measuring the system time difference between sending a (http) request and receiving a response. This was being run on a (different) client machine Like you suggested, I tried to time the response on the server itself as follows: $ /usr/bin/time -p curl -sS

Re: Retrieving large num of docs

2009-12-01 Thread Raghuveer Kancherla
(time for Solr to return the http response) is very close to the QTime Solr is showing in the logs. Thanks for all the help, Raghu On Mon, Nov 30, 2009 at 11:37 AM, Raghuveer Kancherla raghuveer.kanche...@aplopio.com wrote: Thanks Hoss, In my previous mail, I was measuring the system time

Re: Retrieving large num of docs

2009-12-03 Thread Raghuveer Kancherla
Hi Hoss, I was experimenting with various queries to solve this problem and in one such test I remember that requesting only the ID did not change the retrieval time. To be sure, I tested it again using the curl command today and it confirms my previous observation. Also, enableLazyFieldLoading

Re: Retrieving large num of docs

2009-12-05 Thread Raghuveer Kancherla
... .and that was the only change you made and got cca x10 search performance improvement? Sorry for repeating your words, just trying to confirm and understand. Thanks, Otis -- Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch - Original Message From: Raghuveer Kancherla

Re: WELCOME to solr-user@lucene.apache.org

2009-12-05 Thread Raghuveer Kancherla
2 ways I can think of ... - ExtractingRequestHandler (this is what I am guessing you are using now) Set extractOnly=true while making a request to the extractingRequestHandler and get the parsed content back. Now make a post request on update request handler with what ever fields and field

Payloads with Phrase queries

2009-12-10 Thread Raghuveer Kancherla
Hi, I am looking for a way to use payloads in my search application. Indexing data with payloads into Solr is pretty straightforward. However using the payloads during search time is a bit confusing. Can anyone point me in the right direction to enable payloads on a *PhraseQuery*. I looked at the

Re: Payloads with Phrase queries

2009-12-10 Thread Raghuveer Kancherla
I was looking through some lucene source codes and found the following class org.apache.lucene.search.payloads.PayloadSpanUtil There is a function named queryToSpanQuery in this class. Is this the preferred way to convert a PhraseQuery to PayloadNearQuery? Also, are there any performance

Re: Payloads with Phrase queries

2009-12-14 Thread Raghuveer Kancherla
Hi, Thanks everyone for the responses, I am now able to get both phrase queries and term queries to use payloads. However the the score value for each document (and consequently, the ordering of documents) are coming out wrong. In the solr output appended below, document 4 has a score higher

Re: Payloads with Phrase queries

2009-12-15 Thread Raghuveer Kancherla
somewhere that this is deprecated .. but i was just experimenting) and the scoring seems to work as expected now for a single term. Now, the important question is what is the Payload version of a TermQuery? Regards Raghu On Tue, Dec 15, 2009 at 12:45 PM, Raghuveer Kancherla raghuveer.kanche

Re: parsedquery becomes PhraseQuery

2009-12-16 Thread Raghuveer Kancherla
Its likely that your analyzer has WordDelimiterFilterFactory (look at your schema for the field in question). If a single token is split into more tokens during the analysis phase, solr will do a phrase query instead of a term query. In your case disk/1.0 is being analyzed into disk 1 0 (three

payload queries running slow

2009-12-17 Thread Raghuveer Kancherla
Hi, With help from the group here, I have been able to set up a search application with payloads enabled. However, there is a noticeable increase in query response times with payloads as compared to the same queries without payloads. I am also seeing a lot more disk IO (I have a 7200 rpm disk) and

Re: payload queries running slow

2009-12-20 Thread Raghuveer Kancherla
if a ramDirectory will help instead of the default Directory implementation for the indexReader? Thanks, Raghu On Thu, Dec 17, 2009 at 6:58 PM, Grant Ingersoll gsing...@apache.orgwrote: On Dec 17, 2009, at 4:52 AM, Raghuveer Kancherla wrote: Hi, With help from the group here, I have been able

Re: Multi Solr

2009-12-21 Thread Raghuveer Kancherla
Based on your need you can choose one of the options listed at http://wiki.apache.org/solr/MultipleIndexes - Raghu On Tue, Dec 22, 2009 at 10:46 AM, Olala hthie...@gmail.com wrote: Hi all! I have developed Solr on Tomcat, but now I want to building many Solr on only one Tomcat server.Is

Re: Configuring Solr to use RAMDirectory

2010-01-02 Thread Raghuveer Kancherla
Hi Dipti, Just out of curiosity, are you trying to use RAMDirectory for improvement in speed? I tried doing that and did not see any significant improvement. Would be nice to know what your experiment shows. - Raghu On Thu, Dec 31, 2009 at 4:17 PM, Erik Hatcher erik.hatc...@gmail.comwrote:

Updating a single field in a Solr document

2010-01-18 Thread Raghuveer Kancherla
Hi, I have 2 fields one with captures the category of the documents and an other which is a pre processed text of the document. Text of the document is fairly large. The category of the document changes often while the text remains the same. Search happens on both fields. The problem is, I have

Re: Updating a single field in a Solr document

2010-01-19 Thread Raghuveer Kancherla
Is this feature planned in any of the future releases. I ask because it will help me plan my system architecture accordingly. Thanks, Raghu On Tue, Jan 19, 2010 at 7:28 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Mon, Jan 18, 2010 at 5:11 PM, Raghuveer Kancherla