Re: Solr 1.3 - response time very long

2008-12-04 Thread sunnyfr
Thanks a lot guys for your time, I appreciate it. I will follow all your advice. Yonik Seeley wrote: On Wed, Dec 3, 2008 at 11:49 AM, sunnyfr [EMAIL PROTECTED] wrote: Sorry the request is more :

changing schema is dynamic or not

2008-12-04 Thread Neha Bhardwaj
Hi, Every time I make any change in schema , I have to restart the server. Is this because I have made some mistake or It is like this only I mean, I have this doubt that if we make any kind of changes to schema.xml , do we need to restart the server or we can continue without restarting the

Re: changing schema is dynamic or not

2008-12-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
you have to restart the server You may also need to re-index the data if the changes are incompatible On Thu, Dec 4, 2008 at 3:09 PM, Neha Bhardwaj [EMAIL PROTECTED] wrote: Hi, Every time I make any change in schema , I have to restart the server. Is this because I have made some mistake or

RE: changing schema is dynamic or not

2008-12-04 Thread Neha Bhardwaj
Is there any way by which this can be avoided. -Original Message- From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Thursday, December 04, 2008 3:12 PM To: solr-user@lucene.apache.org Subject: Re: changing schema is dynamic or not you have to restart the server You may

Re: changing schema is dynamic or not

2008-12-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
It is possible you can reload a core through the http API but if the changes are incompatible you will have to re-index the data On Thu, Dec 4, 2008 at 3:16 PM, Neha Bhardwaj [EMAIL PROTECTED] wrote: Is there any way by which this can be avoided. -Original Message- From: Noble

RE: changing schema is dynamic or not

2008-12-04 Thread Neha Bhardwaj
Could you brief ,What exactly I need to do? -Original Message- From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Thursday, December 04, 2008 3:47 PM To: solr-user@lucene.apache.org Subject: Re: changing schema is dynamic or not It is possible you can reload a core through

Re: changing schema is dynamic or not

2008-12-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
http://wiki.apache.org/solr/CoreAdmin#head-3f125034c6a64611779442539812067b8b430930 On Thu, Dec 4, 2008 at 4:06 PM, Neha Bhardwaj [EMAIL PROTECTED] wrote: Could you brief ,What exactly I need to do? -Original Message- From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED] Sent:

Re: Multi Language Search

2008-12-04 Thread Grant Ingersoll
On Dec 2, 2008, at 4:52 AM, tushar kapoor wrote: 1. Russian Word 1 AND Russian Word 2 This is the way the query should look, but there's no reason why you can't let you're users input AND in Russian and then you substitute it when you create the query. or rather, 2 . Russian Word 1

Re: Newbie question - using existing Lucene Index

2008-12-04 Thread Grant Ingersoll
On Dec 3, 2008, at 11:53 AM, Sudarsan, Sithu D. wrote: Hi All, Using Lucene, index has been created. It has five different fields. How to just use those index from SOLR for searching? I tried changing the schema as in tutorial, and copied the index to the data directory, but all searches

Re: Solr 1.3 - response time very long

2008-12-04 Thread sunnyfr
Hi Yonik, I've tried everything but it's doesn't change anything, I tried as well the last trunk version but nothing changed. There is nothings that I can do about the indexation ...maybe I can optimize something before searching ??? I'm using linux system, apache 5.5, last solr version updated.

Re: Re[4]: solr performance

2008-12-04 Thread sunnyfr
Hi, I was reading this post and I wondering how can I parallelize document processing??? Thanks Erik Erik Hatcher wrote: On Feb 21, 2007, at 4:25 PM, Jack L wrote: couple of times today at around 158 documents / sec. This is not bad at all. How about search performance? How many

Re: Resolr performance

2008-12-04 Thread sunnyfr
Hi, When I check my CPU, all my CPU are not full, how can I change this ? Do I have to change a parameter ?? Thanks a lot , Johanna Walter Underwood wrote: Try running your submits while watching a CPU load meter. Do this on a multi-CPU machine. If all CPUs are busy, you are running

Re: solr performance

2008-12-04 Thread Mark Miller
Kick off some indexing more than once - eg, post a folder of docs, and while thats working, post another. I've been thinking about a multi threaded UpdateProcessor as well - that could be interesting. - Mark sunnyfr wrote: Hi, I was reading this post and I wondering how can I parallelize

Re: solr performance

2008-12-04 Thread sunnyfr
Ok ... Actually my problem is more multi thread which take long time ... like 3sec when 100 threads/sec. I thought that could have helped me .. but no link actually :s sorry markrmiller wrote: Kick off some indexing more than once - eg, post a folder of docs, and while thats working,

Re: solr performance

2008-12-04 Thread Yonik Seeley
On Thu, Dec 4, 2008 at 8:39 AM, Mark Miller [EMAIL PROTECTED] wrote: Kick off some indexing more than once - eg, post a folder of docs, and while thats working, post another. I've been thinking about a multi threaded UpdateProcessor as well - that could be interesting. Not sure how that

Re: Resolr performance

2008-12-04 Thread Yonik Seeley
On Thu, Dec 4, 2008 at 8:36 AM, sunnyfr [EMAIL PROTECTED] wrote: When I check my CPU, all my CPU are not full, how can I change this ? If this is while you are indexing, then it simply means that you are not feeding documents to Solr fast enough (use multiple threads to send to Solr, and send

Re: Resolr performance

2008-12-04 Thread sunnyfr
When I run my stress test ..sending multi thread ... around 100/sec I don't start indexation at all ... ? maybe my cache ??? will check that Yonik Seeley wrote: On Thu, Dec 4, 2008 at 8:36 AM, sunnyfr [EMAIL PROTECTED] wrote: When I check my CPU, all my CPU are not full, how can I change

Re: Resolr performance

2008-12-04 Thread Yonik Seeley
On Thu, Dec 4, 2008 at 8:52 AM, sunnyfr [EMAIL PROTECTED] wrote: When I run my stress test ..sending multi thread ... around 100/sec I don't start indexation at all ... If you can't go higher than 100 requests / sec and the CPUs arent at 100% then the possibilities are: - If the index is

Re: Solr 1.3 - response time very long

2008-12-04 Thread Yonik Seeley
On Thu, Dec 4, 2008 at 8:13 AM, sunnyfr [EMAIL PROTECTED] wrote: Hi Yonik, I've tried everything but it's doesn't change anything, I tried as well the last trunk version but nothing changed. There is nothings that I can do about the indexation ...maybe I can optimize something before

Response status

2008-12-04 Thread Robert Young
In the standard response format, what does the status mean? It always seems to be 0. Thanks Rob

Re: Solr 1.3 - response time very long

2008-12-04 Thread sunnyfr
Huge thanks for your help Yonik, I optimized the index so I will try to reduce the size ... like I explained you I stored all language text ... So I will reduce my stored data. Cheers... I will let you know :) Yonik Seeley wrote: On Thu, Dec 4, 2008 at 8:13 AM, sunnyfr [EMAIL PROTECTED]

Re: Response status

2008-12-04 Thread Erik Hatcher
It means the request was successful. If the status is non-zero (err, 1) then there was an error of some sort. Erik On Dec 4, 2008, at 9:32 AM, Robert Young wrote: In the standard response format, what does the status mean? It always seems to be 0. Thanks Rob

Re: Response status

2008-12-04 Thread Robert Young
Thanks On Thu, Dec 4, 2008 at 2:53 PM, Erik Hatcher [EMAIL PROTECTED]wrote: It means the request was successful. If the status is non-zero (err, 1) then there was an error of some sort. Erik On Dec 4, 2008, at 9:32 AM, Robert Young wrote: In the standard response format, what

Re: Solr 1.3 - response time very long

2008-12-04 Thread sunnyfr
Hi Yonik, I will index my data again Can you advice me to optimize a lot my data and tell me if you see something very wrong or bad for the memory, according to the fact that I just need to show back the ID, that's it. But I need to boost some field ... like description ..

Re: Solr 1.3 - response time very long

2008-12-04 Thread Yonik Seeley
remove this entry from the example schema unless you need the timestamp when it was indexed: field name=timestamp type=date indexed=true stored=true default=NOW multiValued=false/ Also, only index fields you really need to search separately. For example, if the description field is also

Re: solr performance

2008-12-04 Thread Mark Miller
Yonik Seeley wrote: Not sure what would be the best for error handling though - perhaps just polling (allow user to ask for failed or successful operations). Thats how I've handled similar situations in the past. Your submitting a batch of data to be processed, and if your so inclined to

Re: Solr 1.3 - response time very long

2008-12-04 Thread sunnyfr
Ok thanks a lot, so I can remove all this part field name=title type=text indexed=true stored=false / field name=description type=textindexed=true stored=false / field name=tag1 type=string indexed=true stored=false

Re: Solr 1.3 - response time very long

2008-12-04 Thread Yonik Seeley
On Thu, Dec 4, 2008 at 11:41 AM, sunnyfr [EMAIL PROTECTED] wrote: Ok thanks a lot, so I can remove all this part I wouldn't remove them if they are the source of a copyField (with the destination being text). Simply change to indexed=false stored=false otherwise you may get an undefined field

Re: Solr 1.3 - response time very long

2008-12-04 Thread sunnyfr
right !!! Yonik Seeley wrote: On Thu, Dec 4, 2008 at 11:41 AM, sunnyfr [EMAIL PROTECTED] wrote: Ok thanks a lot, so I can remove all this part I wouldn't remove them if they are the source of a copyField (with the destination being text). Simply change to indexed=false stored=false

Re: Throughput Optimization

2008-12-04 Thread wojtekpia
It looks like file locking was the bottleneck - CPU usage is up to ~98% (from the previous peak of ~50%). I'm running the trunk code from Dec 2 with the faceting improvement (SOLR-475) turned off. Thanks for all the help! Yonik Seeley wrote: FYI, SOLR-465 has been committed. Let us know if

Re: Throughput Optimization

2008-12-04 Thread Yonik Seeley
On Thu, Dec 4, 2008 at 1:54 PM, wojtekpia [EMAIL PROTECTED] wrote: It looks like file locking was the bottleneck - CPU usage is up to ~98% (from the previous peak of ~50%). Great to hear it! I'm running the trunk code from Dec 2 with the faceting improvement (SOLR-475) turned off. Thanks for

Re: Ordering updates

2008-12-04 Thread Shalin Shekhar Mangar
It is not clear how you are using Solr i.e. distributed vs single index. Summarily, Solr does not update documents. It overwrites the old document with the new one if an old document with the same uniqueKey exists in the index. Does that answer your question? On Thu, Dec 4, 2008 at 1:46 AM,

Re: new faceting algorithm

2008-12-04 Thread wojtekpia
I'm seeing some strange behavior with my garbage collector that disappears when I turn off this optimization. I'm running load tests on my deployment. For the first few minutes, everything is fine (and this patch does make things faster - I haven't quantified the improvement yet). After that, the

Re: Throughput Optimization

2008-12-04 Thread wojtekpia
New faceting stuff off because I'm encountering some problems when I turn it on, I posted the details: http://www.nabble.com/new-faceting-algorithm-td20674902.html#a20840622 Yonik Seeley wrote: On Thu, Dec 4, 2008 at 1:54 PM, wojtekpia [EMAIL PROTECTED] wrote: It looks like file locking was

Re: Throughput Optimization

2008-12-04 Thread Yonik Seeley
On Thu, Dec 4, 2008 at 2:30 PM, wojtekpia [EMAIL PROTECTED] wrote: New faceting stuff off because I'm encountering some problems when I turn it on, I posted the details: http://www.nabble.com/new-faceting-algorithm-td20674902.html#a20840622 Missed that, thanks... will respond there. -Yonik

Re: new faceting algorithm

2008-12-04 Thread Yonik Seeley
On Thu, Dec 4, 2008 at 2:28 PM, wojtekpia [EMAIL PROTECTED] wrote: I'm seeing some strange behavior with my garbage collector that disappears when I turn off this optimization. I'm running load tests on my deployment. For the first few minutes, everything is fine (and this patch does make

Re: new faceting algorithm

2008-12-04 Thread wojtekpia
Yonik Seeley wrote: Are you doing commits at any time? One possibility is the caching mechanism (weak-ref on the IndexReader)... that's going to be changing soon hopefully. -Yonik No commits during this test. Should I start looking into my heap size distribution and garbage

Re: new faceting algorithm

2008-12-04 Thread Yonik Seeley
On Thu, Dec 4, 2008 at 2:57 PM, wojtekpia [EMAIL PROTECTED] wrote: Yonik Seeley wrote: Are you doing commits at any time? One possibility is the caching mechanism (weak-ref on the IndexReader)... that's going to be changing soon hopefully. -Yonik No commits during this test. Should I

Is there a clean way to determine whether a core exists?

2008-12-04 Thread Dean Thompson
The ping command gives me a 500 status if the core exists, or a 404 if it doesn't. For example, when I hit http://doom:8983/solr/content_item_representations_20081201/admin/ping I see HTTP ERROR: 500 INTERNAL_SERVER_ERROR RequestURI=/solr/admin/ping Powered by

Re: Is there a clean way to determine whether a core exists?

2008-12-04 Thread Ryan McKinley
what about just calling: http://doom:8983/solr/content_item_representations_20081201/select That should give you a 404 if it does not exist. the admin stuff will behave funny if the core does not exist (perhaps you can file a JIRA issue for that) ryan On Dec 4, 2008, at 3:38 PM, Dean

Re: Is there a clean way to determine whether a core exists?

2008-12-04 Thread Dean Thompson
Thanks for the quick response, Ryan! Actually, my admin/ping call gives me a 404 if the core doesn't exist, which seemed reasonable. I get the 500 if the core *did* exist. Thanks for the suggestion of using the select URL, but that gives me: HTTP ERROR: 500 null

Re: Ordering updates

2008-12-04 Thread Laurence Rowe
Hi, We currently have a single Solr server, with a single index. There are a number of CMS processes distributed over a number of servers, with each CMS process sending an update to the Solr index when changes are made to a content object. My concern is that a scenario is possible where a

Re: Ordering updates

2008-12-04 Thread Shalin Shekhar Mangar
On Fri, Dec 5, 2008 at 2:42 AM, Laurence Rowe [EMAIL PROTECTED] wrote: We currently have a single Solr server, with a single index. There are a number of CMS processes distributed over a number of servers, with each CMS process sending an update to the Solr index when changes are made to a

Boost a query by field at query time - Standard Request Handler

2008-12-04 Thread ashokc
Here is the problem I am trying to solve. I have to use the Standard Request Handler. Query (can be quite complex, as it gets built from an advanced search form): term1^2.0 OR term2 OR term3 term4 I have 3 fields - content (the default search field), title and url. Any matches in the title or

Standard request with functional query

2008-12-04 Thread Sammy Yu
Hi guys, I have a standard query that searches across multiple text fields such as q=title:iphone OR bodytext:iphone OR title:firmware OR bodytext:firmware This comes back with documents that have iphone and firmware (I know I can use dismax handler but it seems to be really slow), which is

Re: Standard request with functional query

2008-12-04 Thread Yonik Seeley
On Thu, Dec 4, 2008 at 4:35 PM, Sammy Yu [EMAIL PROTECTED] wrote: bodytext:firmware) AND _val_:ord(dateCreated)^0.1': Expected ',' at position 16 in 'ord(dateCreated)^0.1' ^0.1 is not function query syntax, it's Lucene/Solr QueryParser syntax. Try _val_:ord(dateCreated)^0.1 -Yonik

Merging Indices

2008-12-04 Thread ashokc
The SOLR wiki says 3. Make sure both indexes you want to merge are closed. What exactly does 'closed' mean? 1. Do I need to stop SOLR search on both indexes before running the merge command? So a brief downtime is required? Or do I simply prevent any 'updates/deletes' to these indices during

Re: Solrj queries

2008-12-04 Thread lupiss
Hola! soy nueva en solr, yo uso solrj, y tu pregunta es como consultar con solrj mira, ahí te va una pequeña descripción de lo que puedes hacer: 1. hacer tu jsp o php, etc. donde el usuario introducira el nombre y apellido, después lees esos valores de tal vez con algun getText. 2. entonces

Re: Solrj queries

2008-12-04 Thread lupiss
me faltó ; jejeje -- View this message in context: http://www.nabble.com/Solrj-queries-tp20494859p20845222.html Sent from the Solr - User mailing list archive at Nabble.com.

Quick thanks for all the assistance getting me up to speed

2008-12-04 Thread Ian Connor
Hi Yonik Seeley, Erik Hatcher and others. Thanks for all the help fixing the bugs I ran into using the new 1.3 distributed features with rails (shards). I now have medline fully indexed in 7 solr shards (with 2 spare). Each server has 8GB RAM and a Quad Core 2.4GHz. As a test, I ran about 2

Re: NIO not working yet

2008-12-04 Thread wojtekpia
I've updated my deployment to use NIOFSDirectory. Now I'd like to confirm some previous results with the original FSDirectory. Can I turn it off with a parameter? I tried: java -Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.FSDirectory ... but that didn't work. -- View this

Solr on Solaris

2008-12-04 Thread Kashyap, Raghu
We are running solr on a solaris box with 4 CPU's(8 cores) and 3GB Ram. When we try to index sometimes the HTTP Connection just hangs and the client which is posting documents to solr doesn't get any response back. We since then have added timeouts to our http requests from the clients. I

Re: Solr on Solaris

2008-12-04 Thread Jon Baer
Just curious, is this off a zone by any chance? - Jon On Dec 4, 2008, at 10:40 PM, Kashyap, Raghu wrote: We are running solr on a solaris box with 4 CPU's(8 cores) and 3GB Ram. When we try to index sometimes the HTTP Connection just hangs and the client which is posting documents to solr

Re: Is there a clean way to determine whether a core exists?

2008-12-04 Thread Chris Hostetter
: Subject: Is there a clean way to determine whether a core exists? doesn't the CoreAdminHandler's STATUS feature make this easy? -Hoss

Re: Is there a clean way to determine whether a core exists?

2008-12-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
SOLR-880 is an issue raised for the same On Fri, Dec 5, 2008 at 10:16 AM, Chris Hostetter [EMAIL PROTECTED] wrote: : Subject: Is there a clean way to determine whether a core exists? doesn't the CoreAdminHandler's STATUS feature make this easy? -Hoss -- --Noble Paul

Re: Is there a clean way to determine whether a core exists?

2008-12-04 Thread Ryan McKinley
On Dec 4, 2008, at 3:57 PM, Dean Thompson wrote: Thanks for the quick response, Ryan! Actually, my admin/ping call gives me a 404 if the core doesn't exist, which seemed reasonable. I get the 500 if the core *did* exist. aaah -- check what ping query you have configured and make sure

Re: Is there a clean way to determine whether a core exists?

2008-12-04 Thread Ryan McKinley
yes: http://localhost:8983/solr/admin/cores?action=STATUS will give you a list of running cores. However that is not easy to check with a simple status != 404 see: http://wiki.apache.org/solr/CoreAdmin On Dec 4, 2008, at 11:46 PM, Chris Hostetter wrote: : Subject: Is there a clean way