RE: Availability Issues

2007-10-08 Thread Chris Hostetter
: My logs don't look anything like that. They look like HTTP : requests. Am I looking in the wrong place? what servlet container are you using? every servlet container handles applications logs differently -- it's especially tricky becuse even the format can be changed, the examples i gave

Re: Availability Issues

2007-10-08 Thread James liu
* * *i think text not need "stored='true'" unless u will show it.(it will help u decrease index size and not affect search )* *index and search use same box? if it is true, u should moniter search response time when indexing.(include CPU, RAM change)* *i have similar problem and i increase JVM s

Solr deployment in tomcat

2007-10-08 Thread Cool Coder
Hello Group, Does anyone able to deploy solr.war @ tomcat. I just tried to deploy it as per wiki and it gives bunch of exceptions and I dont think those exceptions have any relevance with the actual cause. I was wondering if there is any speciaf configuration needed? The exce

Re: Spell Check Handler

2007-10-08 Thread Pieter Berkel
I started to look at this back in August and decided to wait for climbingrose's implementation, however since then my priorities changed and I hadn't had a chance to re-visit it. Sounds like there is quite a bit of interest in this feature, so it would be great if those who have make progress on t

RE: Spell Check Handler

2007-10-08 Thread Lance Norskog
Great! One comment: if I type a word that happens to be real, it may not be what I actually want. A spell checker should still recommend similar words. Computer programmers are all perfect spellers, and this can blind us to what matters to ordinary people :) Lance Norskog -Original Messa

Re: Set of docid's

2007-10-08 Thread Ryan McKinley
how about: /select?q=*:*&fl=id (where "id" is your unique id) you may need to do paging with: &start=2000&rows=1000 if you have a lot of documents Jay Booth wrote: Hey all, sorry for the elementary question but I was poking around and couldn't find an easy answer. Is there an easy way to

Set of docid's

2007-10-08 Thread Jay Booth
Hey all, sorry for the elementary question but I was poking around and couldn't find an easy answer. Is there an easy way to get the current set of all unique docid's from a running instance of solr? Thanks, Jay

RE: Availability Issues

2007-10-08 Thread David Whalen
Thanks for letting me know that. Okay, here they are: BEGIN SCHEMA.XML ===

RE: Availability Issues

2007-10-08 Thread David Whalen
Hi Chris. My logs don't look anything like that. They look like HTTP requests. Am I looking in the wrong place? Dave > -Original Message- > From: Chris Hostetter [mailto:[EMAIL PROTECTED] > Sent: Monday, October 08, 2007 5:02 PM > To: solr-user > Subject: RE: Availability Issues > >

RE: Availability Issues

2007-10-08 Thread Chris Hostetter
: > Do the slow requests start after a commit? : : Based on the way the logs read, you could argue that point. : The stream of POSTs end in the logs and then subsequent queries : take longer to run, but it's hard to be sure there's a direct : correlation. you would know based on the INFO level m

RE: Availability Issues

2007-10-08 Thread Chris Hostetter
: I've attached our schema/config files. They are pretty much : out-of-the-box values, except for our index. FYI: the mailing list strips most attachemnts ... the best thing to do is just inline them in your mail. Quick question: do you have autoCommit turned on in your solrconfig.xml? Second

RE: Availability Issues

2007-10-08 Thread David Whalen
> Oh, so you are using the same boxes for updating and querying? Yep. We have a MySQL database on the box and we query it and POST directly into SOLR via wget in PERL. We then also hit the box for queries. [We'd be very interested in hearing about best practices on how to seperate-out the data

Re: Availability Issues

2007-10-08 Thread Yonik Seeley
On 10/8/07, David Whalen <[EMAIL PROTECTED]> wrote: > > Do you see any requests that took a really long time to finish? > > The requests that take a long time to finish are just simple > queries. And the same queries run at a later time come back > much faster. > > Our logs contain 99% inserts and

RE: Availability Issues

2007-10-08 Thread David Whalen
Hi Yonik. > Do you see any requests that took a really long time to finish? The requests that take a long time to finish are just simple queries. And the same queries run at a later time come back much faster. Our logs contain 99% inserts and 1% queries. We are constantly adding documents to t

Re: Availability Issues

2007-10-08 Thread Yonik Seeley
On 10/8/07, David Whalen <[EMAIL PROTECTED]> wrote: > The logs show nothing but regular activity. We do a "tail -f" > on the logfile and we can read it during the unresponsive period > and we don't see any errors. You don't see log entries for requests until after they complete. When a server bec

Re: Availability Issues

2007-10-08 Thread Yonik Seeley
On 10/8/07, David Whalen <[EMAIL PROTECTED]> wrote: > > Have you taken a thread dump to see what is going on? > > We can't do it b/c during the unresponsive time we can't access > the admin site (/solr/admin) at all. I don't know how to do a > thread dump via the command line kill -3 Start

RE: Availability Issues

2007-10-08 Thread David Whalen
Hi Yonik. > What version of Solr are you running? We're running: Solr Specification Version: 1.2.2007.08.24.08.06.00 Solr Implementation Version: nightly ${svnversion} - yonik - 2007-08-24 08:06:00 Lucene Specification Version: 2.2.0 Lucene Implementation Version: 2.2.0 548010 - buschmi - 200

RE: Availability Issues

2007-10-08 Thread David Whalen
Hi Tom. The logs show nothing but regular activity. We do a "tail -f" on the logfile and we can read it during the unresponsive period and we don't see any errors. I've attached our schema/config files. They are pretty much out-of-the-box values, except for our index. Dave > -Original Me

Re: Availability Issues

2007-10-08 Thread Yonik Seeley
On 10/8/07, David Whalen <[EMAIL PROTECTED]> wrote: > We're running SOLR 1.2 with a 2.5G heap size. On any > given day, the system becomes completely unresponsive. > We can't even get /solr/admin/ to come up, much less > any select queries. What version of Solr are you running? The first step to

Re: Availability Issues

2007-10-08 Thread Tom Hill
Hi - We're definitely not seeing that. What do your logs show? What do your schema/solrconfig look like? Tom On 10/8/07, David Whalen <[EMAIL PROTECTED]> wrote: > > Hi All. > > I'm seeing all these threads about availability and I'm > wondering why my situation is so different than others'. > >

Availability Issues

2007-10-08 Thread David Whalen
Hi All. I'm seeing all these threads about availability and I'm wondering why my situation is so different than others'. We're running SOLR 1.2 with a 2.5G heap size. On any given day, the system becomes completely unresponsive. We can't even get /solr/admin/ to come up, much less any select que

Re: High-Availability deployment

2007-10-08 Thread Chris Hostetter
: I'm setting up a backup task to keep a copy of my master index, just to : avoid having to re-build my index from scratch. And other important issue is every slave is a backup of the master, so you don't usually need a seperate backup mechanism. re-building hte index is more about peace of mind

Re: High-Availability deployment

2007-10-08 Thread Daniel Alheiros
OK, I'll define it as a procedure in my disaster recovery plan. That would be great. I'm looking forward to it. Thanks, Daniel On 8/10/07 18:07, "Yonik Seeley" <[EMAIL PROTECTED]> wrote: > On 10/8/07, Daniel Alheiros <[EMAIL PROTECTED]> wrote: >> Hmm, is there any exception thrown in case the i

Re: Merging multiple Solr Indexes

2007-10-08 Thread Ycrux
Seems good. Thanks cheers Y. J.J. Larrea a écrit : At 9:51 PM -0700 10/7/07, Chris Hostetter wrote: : Thanks for the pointer. After two silent days waiting for reply, : I decided to implement a command line for that. Works like a charm !!! well, sometimes people just don't post because th

Re: High-Availability deployment

2007-10-08 Thread Yonik Seeley
On 10/8/07, Daniel Alheiros <[EMAIL PROTECTED]> wrote: > Hmm, is there any exception thrown in case the index get corrupted (if it's > not caused by OOM and the JVM crashes)? The document uniqueness SOLR offers > is one of the many reasons I'm using it and should be excellent to know when > it's go

Re: High-Availability deployment

2007-10-08 Thread Daniel Alheiros
Hi Yonik. It looks pretty good. I hope I'm not the one who will post a very odd crash after a while. :) OK, so is very unlikely that a OOM it's going to happen, as I've set my JVM heap size to 1.5G. Hmm, is there any exception thrown in case the index get corrupted (if it's not caused by OOM and

Re: Spell Check Handler

2007-10-08 Thread Chris Hostetter
: If you like, I can post the source code changes that I made to the : SpellCheckerRequestHandler, but at this time I am not ready to open a : JIRA issue and submit the changes back through the subversion. I will : need to do a little more testing, documentation, and create some unit : tests

Re: High-Availability deployment

2007-10-08 Thread Yonik Seeley
On 10/8/07, Daniel Alheiros <[EMAIL PROTECTED]> wrote: > Well I believe I can live with some staleness at certain moments, but it's > not good as users are supposed to need it 24x7. So the common practice is to > make one of the slaves as the new master and switch things over to it and > after the

Re: High-Availability deployment

2007-10-08 Thread Daniel Alheiros
Hi YoniK. I'll check if I comment about it at this level and if it's OK I'll bring other details. Sorry if I can't do it right now, but I don't want to brake my company's policies. Well I believe I can live with some staleness at certain moments, but it's not good as users are supposed to need it

Re: High-Availability deployment

2007-10-08 Thread Walter Underwood
We run multiple, identical, independent copies. No master/slave dependencies. Yes, we run indexing N times for N servers, but that's what CPU is for and I sleep better at night. It makes testing and deployment trivial, too. wunder == Walter Underwood Search Guy, Netflix On 10/8/07 4:05 AM, "Dani

Re: Merging multiple Solr Indexes

2007-10-08 Thread J.J. Larrea
At 9:51 PM -0700 10/7/07, Chris Hostetter wrote: >: Thanks for the pointer. After two silent days waiting for reply, >: I decided to implement a command line for that. Works like a charm !!! > >well, sometimes people just don't post because they don't know the >answer to something (better then 50

Re: Spell Check Handler

2007-10-08 Thread scott.tabar
Greetings, I too have a strong need to handle multiple words. Also I have run in to a limitation within the current SpellCheckerRequestHandler where it does not identify if a word is spelled correctly or not. This is a problem, for if there are no suggestions, one cannot currently tell if the

Re: High-Availability deployment

2007-10-08 Thread Yonik Seeley
On 10/8/07, Daniel Alheiros <[EMAIL PROTECTED]> wrote: > I'm about to deploy SOLR in a production environment Cool, can you share exactly what it will be used for? > and so far I'm a bit > concerned about availability. > > I have a system that is responsible for fetching data from a database and

Re: Urldecode Problem

2007-10-08 Thread Frederik M. Kraus
This was a good hint. Apparently php's ext/filter does some fancy recoding of some sort for the ' which then later is producing this encoded url. Now that I am not using ext/filter everything is fine ;) Thanks a lot! Fred. Am 07.10.2007 16:13 Uhr schrieb "Yonik Seeley" unter <[EMAIL PROTECTED]

problems with arabic search

2007-10-08 Thread Heba Farouk
Hello I’m a newbie to solr and I need ur help in developing an Arabic search engine using solr. I succeeded to build the index but failed searching it. I got that error when I submit a query like “محمد”. XML Parsing Error: mismatched tag. Expected: . Location: http://localhost:8080/solrSe

High-Availability deployment

2007-10-08 Thread Daniel Alheiros
Hi I'm about to deploy SOLR in a production environment and so far I'm a bit concerned about availability. I have a system that is responsible for fetching data from a database and then pushing it to SOLR using its XML/HTTP interface. So I'm going to deploy N instances of my application so it's