Re: improving search response time

2010-12-21 Thread Anurag
I am using spellchecker in the query part. Now my search time has become more. say initiallly it was 1000ms now its 3000ms.I have data index of size 9GB. My query http://localhost:8983/solr/spellCheckCompRH/?q=

Explanation of the different caches.

2010-12-21 Thread Stijn Vanhoorelbeke
Hi, I want to do a quickdirt load testing - but all my results are cached. I commented out all the Solr caches - but still everything is cached. * Can the caching come from the 'Field Collapsing Cache'. -- although I don't see this element in my config file. ( As the system now jumps from 1GB

Re: Dismax score - maximu of any one field?

2010-12-21 Thread Erick Erickson
Also take a look at debugQuery=on output. It takes a while to decipher what this is telling you, but it'll let you know exactly. Best Erick On Mon, Dec 20, 2010 at 5:37 AM, Jason Brown jason.br...@sjp.co.uk wrote: Can anyone tell me hoe the dismax score is computed? Is it the maximum score

Consequences for using multivalued on all fields

2010-12-21 Thread Tim TerlegÄrd
In our application we use dynamic fields and there can be about 50 of them and there can be up to 100 million documents. Are there any disadvantages having multivalued=true on all fields in the schema? An admin of the application can specify dynamic fields and if they should be indexed or stored.

Re: Consequences for using multivalued on all fields

2010-12-21 Thread kenf_nc
I have about 30 million documents and with the exception of the Unique ID, Type and a couple of date fields, every document is made of dynamic fields. Now, I only have maybe 1 in 5 being multi-value, but search and facet performance doesn't look appreciably different from a fixed schema solution.

Re: Consequences for using multivalued on all fields

2010-12-21 Thread J.J. Larrea
Someone please correct me if I am wrong, but as far as I am aware index format is identical in either case. One benefit of allowing one to specify a field as single-valued is similar to specifying that a field is required: Providing a safeguard that index data conforms to requirements. So

RE: Explanation of the different caches.

2010-12-21 Thread Toke Eskildsen
Stijn Vanhoorelbeke [stijn.vanhoorelb...@gmail.com] wrote: I want to do a quickdirt load testing - but all my results are cached. I commented out all the Solr caches - but still everything is cached. * Can the caching come from the 'Field Collapsing Cache'. -- although I don't see this

Re: Explanation of the different caches.

2010-12-21 Thread Stijn Vanhoorelbeke
I am aware of the power of the caches. I do not want to completely remove the caches - I want them to be small. - So I can launch a stress test with small amount of data. ( Some items may come from cache - some need to be searched up - right now everything comes from the cache... ) 2010/12/21

backup of Index or Snapshoot ?

2010-12-21 Thread stockii
Hello. Iam working with the shell-scripts for solr for performing a snapshot of the index. to do a snapshot is really easy and works fine. but how can i install a snaposhot for multi-cores. i wrote a little script wich install each snapshot for each core: cd $HOME_DIR/solr/bin ./snapinstaller

Re: improving search response time

2010-12-21 Thread Shawn Heisey
On 12/21/2010 3:02 AM, Anurag wrote: I am using spellchecker in the query part. Now my search time has become more. say initiallly it was 1000ms now its 3000ms.I have data index of size 9GB. My query http://localhost:8983/solr/spellCheckCompRH/?q=

Re: Consequences for using multivalued on all fields

2010-12-21 Thread Geert-Jan Brits
You should be aware that the behavior of sorting on a multi-valued field is undefined. After all, which of the multiple values should be used for sorting? So if you need sorting on the field, you shouldn't make it multi-valued. Geert-Jan 2010/12/21 J.J. Larrea j...@panix.com Someone please

Re: Consequences for using multivalued on all fields

2010-12-21 Thread Dennis Gearon
Thanks you for the input. You might have seen my posts about doing a flexible schema for derived objects. Sounds like dynamic fields might be the ticket. We'll be ready to test the idea in about a month, mabye 3 weeks. I'll post a comment about it whn it gets there. I don't know if I would

Re: improving search response time

2010-12-21 Thread Anurag
Thanks a lot! you mean i have to increase the resources. 1.Can the distributed search improve the speed.? 2.I have read from some thread that spellchecker takes time.Is spellchecker is one of the curlprit for more response time? On Tue, Dec 21, 2010 at 10:20 PM, Shawn Heisey-4 [via Lucene]

Re: Case Insensitive sorting while preserving case during faceted search

2010-12-21 Thread Chris Hostetter
: I am trying to do a facet search and sort the facet values too. ... : Then I followed the sample example schema.xml, created a copyField of type ... : fieldType name=alphaOnlySort class=solr.TextField : sortMissingLast=true omitNorms=true ... : But the

Faceting memory requirements

2010-12-21 Thread Rok Rejc
Dear all, I have created an index with aprox. 1.1 billion of documents (around 500GB) running on Solr 1.4.1. (64 bit JVM). I want to enable faceted navigation on am int field, which contains around 250 unique values. According to the wiki there are two methods: facet.method=fc which uses field

Re: Case Insensitive sorting while preserving case during faceted search

2010-12-21 Thread Jonathan Rochkind
Hoss, I think the use case being asked about is specifically doing a facet.sort though, for cases where you actually do want to sort facet values with facet.sort, not sort records -- while still presenting the facet values with original case, but sorting them case insensitively. The solutions

Re: Faceting memory requirements

2010-12-21 Thread Yonik Seeley
On Tue, Dec 21, 2010 at 4:02 PM, Rok Rejc rokrej...@gmail.com wrote: Dear all, I have created an index with aprox. 1.1 billion of documents (around 500GB) running on Solr 1.4.1. (64 bit JVM). I want to enable faceted navigation on am int field, which contains around 250 unique values.

Re: [Reload-Config] not working

2010-12-21 Thread Adam Estrada
I also noticed that when I run the config-reload command, the following warning is thrown. I changed all my PK=id to see if that changed anything. Anyone have any ideas why this is not working for me? INFO: id is a required field in SolrSchema . But not found in DataConfig. Regards, Adm On Mon,

[Import Timeout] using /dataimport

2010-12-21 Thread Adam Estrada
All, I've noticed that there are some RSS feeds that are slow to respond, especially during high usage times throughout the day. Is there a way to set the timeout to something really high or have it just wait until the feed is returned? The entire thing stops working when the feed doesn't

Re: [Import Timeout] using /dataimport

2010-12-21 Thread Koji Sekiguchi
(10/12/22 9:35), Adam Estrada wrote: All, I've noticed that there are some RSS feeds that are slow to respond, especially during high usage times throughout the day. Is there a way to set the timeout to something really high or have it just wait until the feed is returned? The entire thing

Solr branch_3x problems

2010-12-21 Thread Alexey Kovyrin
Hello guys, We at scribd.com have recently deployed our new search cluster based on Dec 1st, 2010 branch_3x solr code and we're very happy about the new features in brings. Though looks like we have a weird problem here: once a day our servers handling sharded search queries (frontend servers

White space in facet values

2010-12-21 Thread Andy
How do I handle facet values that contain whitespace? Say I have a field Product that I want to facet on. A value for Product could be Electric Guitar. How should I handle the white space in Electric Guitar during indexing? What about when I apply the constraint fq=Product:Electric Guitar?

Duplicate values in multiValued field

2010-12-21 Thread Andy
If I put duplicate values into a multiValued field, would that cause any issues? For example I have a multiValued field Color. Some of my documents have duplicate values for that field, such as: Green, Red, Blue, Green, Green. Would the above (having 3 duplicate Green) be the same as having