Re: search is slow for URL fields of type String.
Srini, Whats the size of your index? You are saying that searching on 'string' fieldType takes 400 milli seconds but did you try searching on any other fieldType other than string? If so, how much time did it take? On Wed, Aug 22, 2012 at 10:35 AM, srinalluri nallurisr...@yahoo.com wrote: This is string fieldType: fieldType name=string class=solr.StrField sortMissingLast=true / These are the filelds using 'string' fieldType: field name=image_url type=string indexed=true stored=true multiValued=true / field name=url type=string indexed=true stored=true multiValued=true / And this the sample query: /select/?q=url:http\:// www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/ AND image_url:* Each query like this taking around 400 milli seconds. What are the change I can do to the fieldType to improve query performance? thanks Srini -- View this message in context: http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: search is slow for URL fields of type String.
There was just a thread on this, it's may be your image_url:* try removing this clause just to test response time. If that shows a vast improvement, try adding a boolean field has_image_url, and then add a fq clause like fq=has_image_url:true Best Erick On Wed, Aug 22, 2012 at 10:35 AM, srinalluri nallurisr...@yahoo.com wrote: This is string fieldType: fieldType name=string class=solr.StrField sortMissingLast=true / These are the filelds using 'string' fieldType: field name=image_url type=string indexed=true stored=true multiValued=true / field name=url type=string indexed=true stored=true multiValued=true / And this the sample query: /select/?q=url:http\://www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/ AND image_url:* Each query like this taking around 400 milli seconds. What are the change I can do to the fieldType to improve query performance? thanks Srini -- View this message in context: http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: search is slow for URL fields of type String.
And we should probably add a doc note with this same advice since it is an easy mistake to make. -- Jack Krupansky -Original Message- From: Erick Erickson Sent: Thursday, August 23, 2012 8:44 AM To: solr-user@lucene.apache.org Subject: Re: search is slow for URL fields of type String. There was just a thread on this, it's may be your image_url:* try removing this clause just to test response time. If that shows a vast improvement, try adding a boolean field has_image_url, and then add a fq clause like fq=has_image_url:true Best Erick On Wed, Aug 22, 2012 at 10:35 AM, srinalluri nallurisr...@yahoo.com wrote: This is string fieldType: fieldType name=string class=solr.StrField sortMissingLast=true / These are the filelds using 'string' fieldType: field name=image_url type=string indexed=true stored=true multiValued=true / field name=url type=string indexed=true stored=true multiValued=true / And this the sample query: /select/?q=url:http\://www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/ AND image_url:* Each query like this taking around 400 milli seconds. What are the change I can do to the fieldType to improve query performance? thanks Srini -- View this message in context: http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: search is slow for URL fields of type String.
Also note, emphasizing what Erick said below, that with this type of has a value in field type clause, it works fine as an fq as that gets cached so you only take the performance hit once on it. Generally this is a clause that is reused so be sure to peel it off as an fq rather than AND'ing it to a q(uery) parameter. The advice to make a separate has_fieldname field (or field_size) is the best advice, but when dealing with low cardinality fields it's not really an issue to use something like category:* where there are only a handful of category values in use. Erik On Aug 23, 2012, at 08:51 , Jack Krupansky wrote: And we should probably add a doc note with this same advice since it is an easy mistake to make. -- Jack Krupansky -Original Message- From: Erick Erickson Sent: Thursday, August 23, 2012 8:44 AM To: solr-user@lucene.apache.org Subject: Re: search is slow for URL fields of type String. There was just a thread on this, it's may be your image_url:* try removing this clause just to test response time. If that shows a vast improvement, try adding a boolean field has_image_url, and then add a fq clause like fq=has_image_url:true Best Erick On Wed, Aug 22, 2012 at 10:35 AM, srinalluri nallurisr...@yahoo.com wrote: This is string fieldType: fieldType name=string class=solr.StrField sortMissingLast=true / These are the filelds using 'string' fieldType: field name=image_url type=string indexed=true stored=true multiValued=true / field name=url type=string indexed=true stored=true multiValued=true / And this the sample query: /select/?q=url:http\://www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/ AND image_url:* Each query like this taking around 400 milli seconds. What are the change I can do to the fieldType to improve query performance? thanks Srini -- View this message in context: http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html Sent from the Solr - User mailing list archive at Nabble.com.
search is slow for URL fields of type String.
This is string fieldType: fieldType name=string class=solr.StrField sortMissingLast=true / These are the filelds using 'string' fieldType: field name=image_url type=string indexed=true stored=true multiValued=true / field name=url type=string indexed=true stored=true multiValued=true / And this the sample query: /select/?q=url:http\://www.foxbusiness.com/personal-finance/2012/08/10/social-change-coming-from-gas-prices-to-rent-prices-and-beyond/ AND image_url:* Each query like this taking around 400 milli seconds. What are the change I can do to the fieldType to improve query performance? thanks Srini -- View this message in context: http://lucene.472066.n3.nabble.com/search-is-slow-for-URL-fields-of-type-String-tp4002662.html Sent from the Solr - User mailing list archive at Nabble.com.
