merge index files without restart solr

2008-09-12 Thread Mice
Hi guys I tried to merge some index files to solr's index file while solr is online. The merge succeeds, but solr can not query the new merged data unless restart solr's web container. Is there some configuration that i should set to make it work? Thanks

Re: What's the bottleneck?

2008-09-12 Thread Jeryl Cook
I think you should justs break up your index across boxes and do a federated search across them... since you mentioned you have a single machine.. Jeryl Cook /^\ Pharaoh /^\ http://pharaohofkush.blogspot.com/ Whether we bring our enemies to justice, or bring justice to our enemies, justice will

Re: Polish/Turkish stemming schema.xml Click to flag this post

2008-09-12 Thread Jarek Zgoda
Stempel (http://www.getopt.org/stempel/) provides Lucene implementation of algorythmic stemmer for Polish language. All you have to do is to implement appropriate factory for Solr, as described in http://www.ibm.com/developerworks/library/j-solr2/index.html#analyzers . Wiadomość napisana

update solr

2008-09-12 Thread sunnyfr
Hi - I am a newbie to Solr and would like to know how to update solr version properly. I saw a lot of patch everywhere and I don't want to mess up everything. My environment is Linux. Thanks a lot, Sunny -- View this message in context:

Re: Polish/Turkish stemming schema.xml Click to flag this post

2008-09-12 Thread sunnyfr
THANKS AGAIN !! Somebody knows about Turkish language? Thanks, sunnyfr wrote: Hi everybody, I'm working now on solr implementation for a multi-language website. I've found a lot of language managed by solr like, japon, greek . spanish .. But I didn't found anything about

Re: Search 'proxy' when using multiple 'shards'

2008-09-12 Thread Lars Kotthoff
Does anything like this exist, or do I have to write it? It doesn't come with Solr, but it should be quite easy to implement a proxy e.g. with Apache httpd mod_rewrite [1]. Lars [1] http://httpd.apache.org/docs/2.2/rewrite/

RE: What's the bottleneck?

2008-09-12 Thread r.prieto
Ok, have you a average size about the memory ocupation, by Solr ? You must to have a look about the really memory usage from cached fields, and try to set java memory to upper value Are you evaluate the performance factors: http://wiki.apache.org/solr/SolrPerformanceFactors I think that is a

recip(myfield,m,a,b)

2008-09-12 Thread sunnyfr
Hi, Is there a way to convert to integer to check if a = b ... like recip(myfield,m,language,lang) But I would like to boost(scoring) field which have the same user language and book language ... But for that I need to know convert.int(language) Thanks, Johanna -- View this message in

Search 'proxy' when using multiple 'shards'

2008-09-12 Thread Upayavira
The http://wiki.apache.org/solr/DistributedSearch page implies that you must know what shards exist when doing a search across multiple shards. A colleague tells me that there is a feature that makes it possible to hide this knowledge. That is, a something that you query that then forwards your

Re: Search 'proxy' when using multiple 'shards'

2008-09-12 Thread Erik Hatcher
Even in the example in that page, the client _is_ just querying a single Solr instance - it is that Solr instance that is then querying the shards. Is your interest in moving the shards parameter to the server-side instead? You can do that with the request handler mapping configuration

Re: What's the bottleneck?

2008-09-12 Thread Grant Ingersoll
The bottleneck may simply be there are a lot of docs to score since you are using fairly common terms. Also, what file format (compound, non-compound) are you using? Is it optimized? Have you profiled your app for these queries? When you say the query is longer, define longer... 5

Re: Search 'proxy' when using multiple 'shards'

2008-09-12 Thread Upayavira
On Fri, 2008-09-12 at 06:05 -0400, Erik Hatcher wrote: Even in the example in that page, the client _is_ just querying a single Solr instance - it is that Solr instance that is then querying the shards. Is your interest in moving the shards parameter to the server-side instead? You

Re: Extending Solr with custom filter

2008-09-12 Thread sunnyfr
Hi guys, Thanks for this clear explanation about how to implement Polish stemming. Can you tell me how do you explain to point on this Stempel file in the schema.xml ? Thanks guys, Johanna Jarek Zgoda-7 wrote: Rafał Kuć pisze: I usually put my own jar files in Solr webapp in lib

Re: Extending Solr with custom filter

2008-09-12 Thread Jarek Zgoda
Use full class access path when declaring factory class. Mine is com.redefine.solr.analysis.StempelTokenFilterFactory. Wiadomość napisana w dniu 2008-09-12, o godz. 16:33, przez sunnyfr: Hi guys, Thanks for this clear explanation about how to implement Polish stemming. Can you tell me

No server response code on insert: how do I avoid this at high speed?

2008-09-12 Thread Paleo Tek
I have a largish index with a lot of churn, and inserts that come in large bursts. My server is a multiprocessor with plenty of memory, so I can multi-thread and stuff in about 1.6 million records per hour, going full speed. I use a dozen or so threads to post curl inserts, and monitor the

Re: Extending Solr with custom filter

2008-09-12 Thread sunnyfr
ok .. that? fieldType name=text_po class=solr.TextField analyzer class=com.redefine.solr.analysis.StempelTokenFilterFactoryr/ /fieldType thanks Jarek Zgoda-7 wrote: Use full class access path when declaring factory class. Mine is

Re: Extending Solr with custom filter

2008-09-12 Thread Jarek Zgoda
Exactly like that. Wiadomość napisana w dniu 2008-09-12, o godz. 17:27, przez sunnyfr: ok .. that? fieldType name=text_po class=solr.TextField analyzer class=com.redefine.solr.analysis.StempelTokenFilterFactoryr/ /fieldType thanks Jarek Zgoda-7 wrote: Use full class access

Re: Turkish stemming ??? stemming?

2008-09-12 Thread Grant Ingersoll
Snowball has a Turkish stemmer. It is available in the trunk version of Solr. On Sep 12, 2008, at 11:29 AM, sunnyfr wrote: Hi everybody, Does somebody found a way to manage Turkish's language? Thanks, Sunny -- View this message in context:

Re: Extending Solr with custom filter

2008-09-12 Thread Andrzej Bialecki
Jarek Zgoda wrote: Exactly like that. Wiadomość napisana w dniu 2008-09-12, o godz. 17:27, przez sunnyfr: ok .. that? fieldType name=text_po class=solr.TextField analyzer class=com.redefine.solr.analysis.StempelTokenFilterFactoryr/ /fieldType I recommend using Stempelator (or

Re: What's the bottleneck?

2008-09-12 Thread Jason Rennie
Thanks for all the replies! Mike: we're not using pf. Our qf is always status:0. The status field is 0 for all good docs (90%+) and some other integer for any docs we don't want returned. Jeyrl: federated search is definitely something we'll consider. On Fri, Sep 12, 2008 at 8:39 AM, Grant

Re: What's the bottleneck?

2008-09-12 Thread Ken Krugler
Thanks for all the replies! Mike: we're not using pf. Our qf is always status:0. The status field is 0 for all good docs (90%+) and some other integer for any docs we don't want returned. Jeyrl: federated search is definitely something we'll consider. On Fri, Sep 12, 2008 at 8:39 AM, Grant

Re: Search 'proxy' when using multiple 'shards'

2008-09-12 Thread Yonik Seeley
On Fri, Sep 12, 2008 at 9:02 AM, Upayavira [EMAIL PROTECTED] wrote: On Fri, 2008-09-12 at 06:05 -0400, Erik Hatcher wrote: Is your interest in moving the shards parameter to the server-side instead? You can do that with the request handler mapping configuration in solrconfig.xml, and specify

Re: Search 'proxy' when using multiple 'shards'

2008-09-12 Thread Upayavira
On Fri, 2008-09-12 at 14:02 +0100, Upayavira wrote: On Fri, 2008-09-12 at 06:05 -0400, Erik Hatcher wrote: Even in the example in that page, the client _is_ just querying a single Solr instance - it is that Solr instance that is then querying the shards. Is your interest in moving the

Re: Search 'proxy' when using multiple 'shards'

2008-09-12 Thread Upayavira
On Fri, 2008-09-12 at 16:44 -0400, Yonik Seeley wrote: On Fri, Sep 12, 2008 at 9:02 AM, Upayavira [EMAIL PROTECTED] wrote: On Fri, 2008-09-12 at 06:05 -0400, Erik Hatcher wrote: Is your interest in moving the shards parameter to the server-side instead? You can do that with the request

Re: merge index files without restart solr

2008-09-12 Thread Otis Gospodnetic
Issue a commit/ to your Solr to make it recreate the searcher after the merge. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Mice [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Friday, September 12, 2008 2:25:29 AM Subject:

Re: index update and re-building

2008-09-12 Thread Otis Gospodnetic
Roger, yes, it does sound like the DIH is the most straight forward approach for you. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Roger Kjensrud [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Thursday, September 11, 2008

Re: No server response code on insert: how do I avoid this at high speed?

2008-09-12 Thread Otis Gospodnetic
Perhaps the container logs explain what happened? How about just throttling to the point where the failure rate is 0%? Too slow? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Paleo Tek [EMAIL PROTECTED] To: solr-user@lucene.apache.org

Re: What's the bottleneck?

2008-09-12 Thread Otis Gospodnetic
Jason, you could also post what the final query looks like (after dismax chews on it) - use debugQuery=true and let's see if there is anything strange there. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Jason Rennie [EMAIL PROTECTED]

Re: No server response code on insert: how do I avoid this at high speed?

2008-09-12 Thread Yonik Seeley
On Fri, Sep 12, 2008 at 11:19 AM, Paleo Tek [EMAIL PROTECTED] wrote: Using jetty, there is ~10% failure rate with no server response code received. What happened then? Did the network connection just drop, or did the server or client time it out? How can you tell it failed? -Yonik

Re: What's the bottleneck?

2008-09-12 Thread Grant Ingersoll
See also https://issues.apache.org/jira/browse/SOLR-502 (timeout searches) and https://issues.apache.org/jira/browse/LUCENE-997 This is committed on trunk and will be in 1.3. Don't ask me how it works, b/c I haven't tried it yet, but maybe Sean Timm or someone can help out. I'm not sure