Re: solr hangs

2017-08-07 Thread Erick Erickson
I've seen this error happen when there are, well, too many threads simultaneously active and the stack space exhausts available memory, here's some discussion: https://stackoverflow.com/questions/36898701/how-does-java-jvm-allocate-stack-for-each-thread With your settings, you can _create_ 65K th

Re: solr hangs

2017-08-07 Thread hawk....@139.com
Hi Shawn, The software config of the server is: OS: CentOS 7 Solr Version: 6.5 The system sends delete requests to the Solr server, the requests are around 5k to 6k.The exception occurred from time to time, and we have to restart the node to continue the service. The max threads setting os t

Re: solr hangs

2017-08-07 Thread Shawn Heisey
On 8/7/2017 12:35 AM, hawk@139.com wrote: > Below is the OOM exception. > > 2017-08-07 12:45:48.446 WARN (qtp33524623-4275) [c:taoke s:shard2 > r:core_node4 x:taoke_shard2_replica1] o.e.j.u.t.QueuedThreadPool > java.lang.OutOfMemoryError: unable to create new native thread The last line that

Re: solr hangs

2017-08-06 Thread hawk....@139.com
Below is the OOM exception. 2017-08-07 12:45:48.446 WARN (qtp33524623-4275) [c:taoke s:shard2 r:core_node4 x:taoke_shard2_replica1] o.e.j.u.t.QueuedThreadPool java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thr

Re: solr hangs

2017-08-06 Thread Shawn Heisey
On 8/6/2017 10:29 PM, hawk@139.com wrote: > We found the problem is caused by the delete command. The request is used to > delete document by id. > > url --> http://10.91.1.120:8900/solr/taoke/update?&commit=true&wt=json > body --> {"delete":["20ec36ade0ca4da3bcd78269e2300f6f"]} > > When

Re: solr hangs

2017-08-06 Thread hawk....@139.com
n Sat, Aug 5, 2017 at 6:19 PM, hawk wrote: >>>> Hi All, >>>> >>>> I have encountered one problem of Solr. In our environment, we setup 2 >>>> Solr nodes, every hour we will update request to Solr to update the >>>> docume

Re: solr hangs

2017-08-06 Thread Erick Erickson
we setup 2 Solr >>> nodes, every hour we will update request to Solr to update the documents, >>> the total documents are around 50k. From time to time, the Solr hangs and >>> the client encounters the timeout issue. >>> >>> bel

Re: solr hangs

2017-08-06 Thread hawk....@139.com
gt;> nodes, every hour we will update request to Solr to update the documents, >> the total documents are around 50k. From time to time, the Solr hangs and >> the client encounters the timeout issue. >> >> below is the exception in Solr log. >> >>

Re: solr hangs

2017-08-05 Thread Erick Erickson
very hour we will update request to Solr to update the documents, the > total documents are around 50k. From time to time, the Solr hangs and the > client encounters the timeout issue. > > below is the exception in Solr log. > > 2017-08-06 07:28:03.682 ERROR (qtp401424608-31250) [c:

solr hangs

2017-08-05 Thread hawk
Hi All, I have encountered one problem of Solr. In our environment, we setup 2 Solr nodes, every hour we will update request to Solr to update the documents, the total documents are around 50k. From time to time, the Solr hangs and the client encounters the timeout issue. below is the

Re: Solr hangs / LRU operations are heavy on cpu

2015-03-22 Thread Umesh Prasad
We use filter very heavily because we run an e-commerce site which has a lot of faceting and drill downs configured at different paths on the store .. We are using master slave replication and we use slaves to support higher qps. filterCache : Concurrent LFU Cache(maxSize=1, initialSize

Re: Solr hangs / LRU operations are heavy on cpu

2015-03-20 Thread Erick Erickson
Are you faceting? That can sometimes use one of the caches (just glanced at stack trace...) as entries are pushed into and removed from the cache during the same request. Shot in the dark. Best, Erick On Fri, Mar 20, 2015 at 12:17 PM, Yonik Seeley wrote: > The document cache is not really going

Re: Solr hangs / LRU operations are heavy on cpu

2015-03-20 Thread Yonik Seeley
The document cache is not really going to be taking up time here. How many concurrent requests (threads) are you testing with here? One thing I've seen over the years is a false sense of what is taking up time when benchmarks with a lot of threads are used. The reason is that when there are a lot

Re: Solr hangs / LRU operations are heavy on cpu

2015-03-20 Thread Chris Hostetter
: we have quite a problem with Solr. We are running it in a config 6x3, and : suddenly solr started to hang, taking all the available cpu on the nodes. : : In the threads dump noticed things like this can eat lot of CPU time : : :- org.apache.solr.search.LRUCache.put​(LRUCache.java:116) :

Re: Solr hangs / LRU operations are heavy on cpu

2015-03-20 Thread Sergey Shvets
Hello Shawn, In that case it makes it a bit strange the behavior as it was noticed. LRU was heavy on the CPU in threads dump, and I don't have any reasonable explanation for that. However switch to LFU seemingly solved the case. -- Best regards, Sergeymail

Re: Solr hangs / LRU operations are heavy on cpu

2015-03-20 Thread Shawn Heisey
On 3/19/2015 8:49 PM, Umesh Prasad wrote: > It might be because LRUCache by default will try to evict its entries on > each call to put and putAll. LRUCache is built on top of java's > LinkedHashMap. Check the javadoc of removeEldestEntry >

Re: Solr hangs / LRU operations are heavy on cpu

2015-03-20 Thread Sergey Shvets
Hello Umesh, Thank you, indeed that gave positive results so far. we changed completely to LFU. Today it went quite okay. We wait till it shows more stability and then work out the optimal cache size. Below is a summary of the changes. - - - - + + + + + -- Best regards, Sergey

Re: Solr hangs / LRU operations are heavy on cpu

2015-03-19 Thread Umesh Prasad
It might be because LRUCache by default will try to evict its entries on each call to put and putAll. LRUCache is built on top of java's LinkedHashMap. Check the javadoc of removeEldestEntry

Solr hangs / LRU operations are heavy on cpu

2015-03-19 Thread Sergey Shvets
Hi, we have quite a problem with Solr. We are running it in a config 6x3, and suddenly solr started to hang, taking all the available cpu on the nodes. In the threads dump noticed things like this can eat lot of CPU time - org.apache.solr.search.LRUCache.put​(LRUCache.java:116) - org.a

Re: Solr hangs on distributed updates

2014-12-16 Thread Peter Keegan
> As of 4.10, commits/optimize etc are executed in parallel. Excellent - thanks. On Tue, Dec 16, 2014 at 6:51 AM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > > On Tue, Dec 16, 2014 at 11:34 AM, Peter Keegan > wrote: > > > > > A distributed update is streamed to all available replicas

Re: Solr hangs on distributed updates

2014-12-16 Thread Shalin Shekhar Mangar
On Tue, Dec 16, 2014 at 11:34 AM, Peter Keegan wrote: > > > A distributed update is streamed to all available replicas in parallel. > > Hmm, that's not what I'm seeing with 4.6.1, as I tail the logs on leader > and replicas. Mark Miller comments on this last May: > > > http://mail-archives.apache.

Re: Solr hangs on distributed updates

2014-12-16 Thread Peter Keegan
> A distributed update is streamed to all available replicas in parallel. Hmm, that's not what I'm seeing with 4.6.1, as I tail the logs on leader and replicas. Mark Miller comments on this last May: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201404.mbox/%3CetPan.534d8d6d.74b0dc51.

Re: Solr hangs on distributed updates

2014-12-15 Thread Shalin Shekhar Mangar
On Mon, Dec 15, 2014 at 8:41 PM, Peter Keegan wrote: > > If a timeout occurs, does the distributed update then go to the next > replica? > A distributed update is streamed to all available replicas in parallel. > > On Fri, Dec 12, 2014 at 3:42 PM, Shalin Shekhar Mangar < > shalinman...@gmail.co

Re: Solr hangs on distributed updates

2014-12-15 Thread Peter Keegan
If a timeout occurs, does the distributed update then go to the next replica? On Fri, Dec 12, 2014 at 3:42 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > > Sorry I should have specified. These timeouts go inside the > section and apply for inter-shard update requests only. The socke

Re: Solr hangs on distributed updates

2014-12-15 Thread Peter Keegan
I added distribUpdateConnTimeout and distribUpdateSoTimeout to solr.xml and the commit did timeout.(btw, is there any way to view solr.xml in the admin console?). Also, although we do have an init.d start/stop script for Solr, the 'stop' command was not executed during shutdown because there was n

Re: Solr hangs on distributed updates

2014-12-12 Thread Peter Keegan
The AMIs are Red Hat (not Amazon's) and the instances are properly sized for the environment (t1.micro for ZK, m3.xlarge for Solr). I do plan to add hooks for a clean shutdown of Solr when the VM is shut down, but if Solr takes too long, AWS may clobber it anyway. One frustrating part of auto scali

Re: Solr hangs on distributed updates

2014-12-12 Thread Peter Keegan
> The Solr leader should stop sending requests to the stopped replica once > that replica's live node is removed from ZK (after session expiry). Fwiw, here's the Zookeeper log entry for a graceful shutdown of the Solr replica: 2014-12-12 15:04:21,304 [myid:2] - INFO [ProcessThread(sid:2 cport:81

Re: Solr hangs on distributed updates

2014-12-12 Thread Chris Hostetter
: No, I wasn't aware of these. I will give that a try. If I stop the Solr : jetty service manually, things recover fine, but the hang occurs when I : 'stop' or 'terminate' the EC2 instance. The Zookeeper leader reports a I don't know squat about AWS Auto-Scaling, (and barely anything about AWS)

Re: Solr hangs on distributed updates

2014-12-12 Thread Shalin Shekhar Mangar
Sorry I should have specified. These timeouts go inside the section and apply for inter-shard update requests only. The socket and connection timeout inside the shardHandlerFactory section apply for inter-shard search requests. On Fri, Dec 12, 2014 at 8:38 PM, Peter Keegan wrote: > Btw, are the

Re: Solr hangs on distributed updates

2014-12-12 Thread Shalin Shekhar Mangar
Okay, that should solve the hung threads on the leader. When you stop the jetty service then it is a graceful shutdown where existing requests finish before the searcher thread pool is shutdown completely. A EC2 terminate probably just kills the processes and leader threads just wait due to a lack

Re: Solr hangs on distributed updates

2014-12-12 Thread Peter Keegan
Btw, are the following timeouts still supported in solr.xml, and do they only apply to distributed search? ${socketTimeout:0} ${connTimeout:0} Thanks, Peter On Fri, Dec 12, 2014 at 3:14 PM, Peter Keegan wrote: > No, I wasn't aware of these. I will give that a try. If I stop the S

Re: Solr hangs on distributed updates

2014-12-12 Thread Peter Keegan
No, I wasn't aware of these. I will give that a try. If I stop the Solr jetty service manually, things recover fine, but the hang occurs when I 'stop' or 'terminate' the EC2 instance. The Zookeeper leader reports a 15-sec timeout from the stopped node, and expires the session, but the Solr leader n

Re: Solr hangs on distributed updates

2014-12-12 Thread Shalin Shekhar Mangar
Do you have distribUpdateConnTimeout and distribUpdateSoTimeout set to reasonable values in your solr.xml? These are the timeouts used for inter-shard update requests. On Fri, Dec 12, 2014 at 2:20 PM, Peter Keegan wrote: > We are running SolrCloud in AWS and using their auto scaling groups to sp

Solr hangs on distributed updates

2014-12-12 Thread Peter Keegan
We are running SolrCloud in AWS and using their auto scaling groups to spin up new Solr replicas when CPU utilization exceeds a threshold for a period of time. All is well until the replicas are terminated when CPU utilization falls below another threshold. What happens is that index updates sent t

Re: Solr Hangs During Updates for over 10 minutes

2013-07-10 Thread Otis Gospodnetic
d >> >>8 replicas with each shard at 100gb and more load and more indexing >> >>requests without this problem but we send docs in batches here and all >> >>fields are stored. Where as the trouble index has only 1 or 2 stored >> >>fields and only send docs 1 at a tim

Re: Solr Hangs During Updates for over 10 minutes

2013-07-10 Thread Shawn Heisey
On 7/10/2013 6:57 AM, Jed Glazner wrote: So, while it's 'just as risky' as you say, it's 'less risky' than a new version of java and is possible to implement without downtime. I believe that if you update one node at a time, there should be no downtime. I've not actually tried this, so it wou

AW: Solr Hangs During Updates for over 10 minutes

2013-07-10 Thread Jed Glazner
2013 17:35 (GMT+01:00) An: solr-user@lucene.apache.org Betreff: Re: Solr Hangs During Updates for over 10 minutes On 7/10/2013 6:57 AM, Jed Glazner wrote: > So we'll do what we can quickly to see if we can 'band-aid' the problem > until we can upgrade to solr 4.4 Speaking of

Re: Solr Hangs During Updates for over 10 minutes

2013-07-10 Thread Shawn Heisey
On 7/10/2013 6:57 AM, Jed Glazner wrote: So we'll do what we can quickly to see if we can 'band-aid' the problem until we can upgrade to solr 4.4 Speaking of band-aids - does anyone know of a way to change the socket timeout/connection timeout for distributed updates? If you need to change Htt

Re: Solr Hangs During Updates for over 10 minutes

2013-07-10 Thread Jed Glazner
013 at 4:44 PM, Jed Glazner >>>>wrote: >>>> >> Hi Shawn, >>>> >> >>>> >> I have been trying to duplicate this problem without success for >>>>the >>>> >>last 2 weeks which is one reason I'm getting flustered

Re: Solr Hangs During Updates for over 10 minutes

2013-07-10 Thread Erick Erickson
gt;> >> >>> >> We do have a story to upgrade but that is still weeks if not months >>> >>before that gets rolled out to production. >>> >> >>> >> We have another cluster running the same version but with 8 shards >>>and >>>

Re: Solr Hangs During Updates for over 10 minutes

2013-07-10 Thread Jed Glazner
ade but that is still weeks if not months >> >>before that gets rolled out to production. >> >> >> >> We have another cluster running the same version but with 8 shards >>and >> >>8 replicas with each shard at 100gb and more load and more indexing &

Re: Solr Hangs During Updates for over 10 minutes

2013-07-10 Thread Daniel Collins
s and > >>8 replicas with each shard at 100gb and more load and more indexing > >>requests without this problem but we send docs in batches here and all > >>fields are stored. Where as the trouble index has only 1 or 2 stored > >>fields and only send docs 1 at a

Re: Solr Hangs During Updates for over 10 minutes

2013-07-10 Thread Jed Glazner
as the trouble index has only 1 or 2 stored >>fields and only send docs 1 at a time. >> >> Could that have anything to do with it? >> >> Jed >> >> >> Von Samsung Mobile gesendet >> >> >> >> Ursprüngliche Nachricht ---

Re: Solr Hangs During Updates for over 10 minutes

2013-07-09 Thread Otis Gospodnetic
thing to do with it? > > Jed > > > Von Samsung Mobile gesendet > > > > Ursprüngliche Nachricht > Von: Shawn Heisey > Datum: 07.09.2013 18:33 (GMT+01:00) > An: solr-user@lucene.apache.org > Betreff: Re: Solr Hangs During Updates for over 10 minu

AW: Solr Hangs During Updates for over 10 minutes

2013-07-09 Thread Jed Glazner
r 2 stored fields and only send docs 1 at a time. Could that have anything to do with it? Jed Von Samsung Mobile gesendet Ursprüngliche Nachricht Von: Shawn Heisey Datum: 07.09.2013 18:33 (GMT+01:00) An: solr-user@lucene.apache.org Betreff: Re: Solr Hangs During Updates f

Re: Solr Hangs During Updates for over 10 minutes

2013-07-09 Thread Shawn Heisey
On 7/9/2013 9:50 AM, Jed Glazner wrote: I'll give you the high level before delving deep into setup etc. I have been struggeling at work with a seemingly random problem when solr will hang for 10-15 minutes during updates. This outage always seems to immediately be proceeded by an EOF excepti

Solr Hangs During Updates for over 10 minutes

2013-07-09 Thread Jed Glazner
I'll give you the high level before delving deep into setup etc. I have been struggeling at work with a seemingly random problem when solr will hang for 10-15 minutes during updates. This outage always seems to immediately be proceeded by an EOF exception on the replica. Then 10-15 minutes la

Re: [solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-26 Thread Vinay Pothnis
Thank you Erick! Will look at all these suggestions. -Vinay On Wed, Jun 26, 2013 at 6:37 AM, Erick Erickson wrote: > Right, unfortunately this is a gremlin lurking in the weeds, see: > http://wiki.apache.org/solr/DistributedSearch#Distributed_Deadlock > > There are a couple of ways to deal wit

Re: [solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-26 Thread Erick Erickson
Right, unfortunately this is a gremlin lurking in the weeds, see: http://wiki.apache.org/solr/DistributedSearch#Distributed_Deadlock There are a couple of ways to deal with this: 1> go ahead and up the limit and re-compile, if you look at SolrCmdDistributor the semaphore is defined there. 2> http

Re: [solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-25 Thread Vinay Pothnis
Jason and Scott, Thanks for the replies and pointers! Yes, I will consider the 'maxDocs' value as well. How do i monitor the transaction logs during the interval between commits? Thanks Vinay On Mon, Jun 24, 2013 at 8:48 PM, Jason Hellman < jhell...@innoventsolutions.com> wrote: > Scott, > > M

Re: [solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-24 Thread Jason Hellman
Scott, My comment was meant to be a bit tongue-in-cheek, but my intent in the statement was to represent hard failure along the lines Vinay is seeing. We're talking about OutOfMemoryException conditions, total cluster paralysis requiring restart, or other similar and disastrous conditions. Wh

Re: [solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-24 Thread Scott Lundgren
Jason, Regarding your statement "push you over the edge"- what does that mean? Does it mean "uncharted territory with unknown ramifications" or something more like specific, known symptoms? I ask because our use is similar to Vinay's in some respects, and we want to be able to push the capabilit

Re: [solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-24 Thread Jason Hellman
Vinay, You may wish to pay attention to how many transaction logs are being created along the way to your hard autoCommit, which should truncate the open handles for those files. I might suggest setting a maxDocs value in parallel with your maxTime value (you can use both) to ensure the commit

Re: [solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-24 Thread Vinay Pothnis
I have 'softAutoCommit' at 1 second and 'hardAutoCommit' at 30 seconds. On Mon, Jun 24, 2013 at 1:54 PM, Jason Hellman < jhell...@innoventsolutions.com> wrote: > Vinay, > > What autoCommit settings do you have for your indexing process? > > Jason > > On Jun 24, 2013, at 1:28 PM, Vinay Pothnis wr

Re: [solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-24 Thread Jason Hellman
Vinay, What autoCommit settings do you have for your indexing process? Jason On Jun 24, 2013, at 1:28 PM, Vinay Pothnis wrote: > Here is the ulimit -a output: > > core file size (blocks, -c) 0 data seg size(kbytes, > -d) unlimited scheduling priority (-

Re: [solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-24 Thread Vinay Pothnis
Here is the ulimit -a output: core file size (blocks, -c) 0 data seg size(kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 179963 max locked memory(kbytes, -l) 64 max memo

Re: [solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-24 Thread Yago Riveiro
Hi, I have the same issue too, and the deploy is quasi exact like than mine, http://lucene.472066.n3.nabble.com/updating-docs-in-solr-cloud-hangs-td4067388.html#a4067862 With some concurrence and batches of 10 solr apparently have some deadlock distributing updates Can you dump the configurat

[solr cloud] solr hangs when indexing large number of documents from multiple threads

2013-06-24 Thread Vinay Pothnis
Hello All, I have the following set up of solr cloud. * solr version 4.3.1 * 3 node solr cloud + replciation factor 2 * 3 zoo keepers * load balancer in front of the 3 solr nodes I am seeing this strange behavior when I am indexing a large number of documents (10 mil). When I have more than 3-5

Re: Solr Hangs on startup

2013-06-14 Thread Jack Krupansky
What are the last few lines of the Solr log? No errors, exceptions, or warnings? -- Jack Krupansky -Original Message- From: Cool Techi Sent: Friday, June 14, 2013 4:49 AM To: solr-user@lucene.apache.org Subject: Solr Hangs on startup Hi, We are using solr4.3 cloud setup, but for

Solr Hangs on startup

2013-06-14 Thread Cool Techi
Hi, We are using solr4.3 cloud setup, but for some reason solr fails to startup, I see the following in the log file, post this there are no logs., org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@17586ed7 main 15293 [coreLoadExecutor-3-thread-1] INFO org.apache.solr.core.SolrCore

Solr hangs after core reload

2012-11-29 Thread O. Klein
Every time I try to do something with the cores from the admin UI, Solr hangs with no exceptions. Anyone else experiencing this? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-hangs-after-core-reload-tp4023206.html Sent from the Solr - User mailing list archive at

Re: solr hangs

2012-04-12 Thread Peter Markey
Thanks for the response. I have given a size of 8gb for the instance and has only around few thousands of documents (with 15 fields each having small amount of data)..apparently the problem is the process (solr jetty instance) is consuming lots of threads...one time it consumed around 50k threads a

Re: solr hangs

2012-04-11 Thread Pawel Rog
You wrote that you can see such error "OutOfMemoryError". I had such problems when my caches were to big. It means that there is no more free memory in JVM and probably full gc starts running. How big is your Java heap? Maybe cache sizes in yout solr are to big according to your JVM settings. -- R

Re: SOLR hangs - update timeout - please help

2012-04-10 Thread rafal.gwizd...@gmail.com
Working for a week now, no signs of fatigue. Many thanks for all the hints R -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-hangs-update-timeout-please-help-tp3863851p3899004.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SOLR hangs - update timeout - please help

2012-04-03 Thread rafal.gwizd...@gmail.com
nabble.com/SOLR-hangs-update-timeout-please-help-tp3863851p3881674.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Yonik Seeley
So maybe this is the missing piece. > I didn't enable nio with Jetty, probably it's there by default. Not with the jetty that comes with Solr. bq. If solr hangs next time I'll try to make a thread dump when the update request is waiting for completion. Great! We need to see where it

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Rafal Gwizdala
ng to check. If solr hangs next time I'll try to make a thread dump when the update request is waiting for completion. Best regards RG On Thu, Mar 29, 2012 at 8:19 PM, Yonik Seeley wrote: > On Thu, Mar 29, 2012 at 1:50 PM, Rafal Gwizdala > wrote: > > Below i'm pasting the

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Yonik Seeley
On Thu, Mar 29, 2012 at 1:50 PM, Rafal Gwizdala wrote: > Below i'm pasting the thread dump taken when the update was hung (it's also > attached to the first message of this topic) Interesting... It looks like there's only one thread in solr code (the one generating the thread dump). The stack tr

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Erick Erickson
More memory is not necessarily better, it can lead to longer, more intense garbage collections that cause things to stop. You might also consider lowering your memory allocation, but 2G is really not all that much so I somewhat doubt it's a problem but thought I'd mention it. Best Erick On Thu, M

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Rafal Gwizdala
Guys, thanks for all the suggestions I will be trying them, one at a time. Imho it's too early to give up and look for another tool, I'll try to work on configuration and see what happens. The NRT looks quite promising, there are also tons of config options to change. As for now, I have made the up

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Walter Underwood
If you must have real-time search, you might look at systems that are designed to do that. MarkLogic isn't free, but it is fast and real-time. You can use their no-charge Express license for development and prototyping: http://developer.marklogic.com/express OK, back to Solr. wunder Search Guy

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Yonik Seeley
On Thu, Mar 29, 2012 at 4:24 AM, Lance Norskog wrote: > 5-7 seconds- there's the problem. If you want to have documents > visible for search within that time, you want to use the trunk and > "near-real-time" search. A hard commit does several hard writes to the > disk (with the fsync() system call

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Nagendra Nagarajayya
Have you tried using Solr 3.5 with RankingAlgorithm 1.4.1 ? Has NRT support and is very fast, updates about 5000 documents in about 490 ms (while updating 1m docs in batches of 5k). You can get more info from here: http://solr-ra.tgels.com/wiki/en/Near_Real_Time_Search_ver_3.x Regards, Nagen

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Shawn Heisey
On 3/29/2012 2:49 AM, Rafal Gwizdala wrote: That's bad news. If 5-7 seconds is not safe then what is the safe interval for updates? Near real-time is not for me as it works only when querying by document Id - this doesn't solve anything in my case. I just want the index to be updated in real-time

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Erick Erickson
Could be garbage collection. Could be larger and larger merges. At some point your commit will cause all segments to be merged. It's likely that what's happening is you need to hit the "magic combination" of events, particularly the problem of too many warming searchers. So, look at your log files

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Rafal Gwizdala
That's bad news. If 5-7 seconds is not safe then what is the safe interval for updates? Near real-time is not for me as it works only when querying by document Id - this doesn't solve anything in my case. I just want the index to be updated in real-time, 30-40 seconds delay is acceptable but not mu

Re: SOLR hangs - update timeout - please help

2012-03-29 Thread Lance Norskog
5-7 seconds- there's the problem. If you want to have documents visible for search within that time, you want to use the trunk and "near-real-time" search. A hard commit does several hard writes to the disk (with the fsync() system call). It does not run smoothly at that rate. It is no surprise tha

Re: SOLR hangs - update timeout - please help

2012-03-28 Thread Rafal Gwizdala
Lance, I know there are many variables that's why I'm asking where to start and what to check. Updates are sent every 5-7 seconds, each update contains between 1 and 50 docs. Commit is done every time (on each update). Currently queries aren't very frequent - about 1 query every 3-5 seconds, but th

Re: SOLR hangs - update timeout - please help

2012-03-28 Thread Lance Norskog
few million document updates/inserts every month. > The problem is that after running for some time (about a day) SOLR hangs on > update and stops processing further update requests. At the same time > searching works normally. I'm updating SOLR by http requests and when the > server

Re: Solr Hangs

2011-09-02 Thread Govind @ Gmail
as 24G of memory > and I am try to start solr with "-Xms2g -Xmx16g -XX:MaxPermSize=3072m -D64" > option. But, this is not my main concern, how do I find out why solr hangs > and is there a way to automatically kill and restart. > > > Regards, > Rohit > Mobile:

RE: Solr Hangs

2011-09-02 Thread Rohit
t why solr hangs and is there a way to automatically kill and restart. Regards, Rohit Mobile: +91-9901768202 About Me: http://about.me/rohitg -Original Message- From: simon [mailto:mtnes...@gmail.com] Sent: 02 September 2011 14:03 To: solr-user@lucene.apache.org Subject: Re: Solr Hangs

Re: Solr Hangs

2011-09-02 Thread simon
> The number of updates are very high into 2 of the cores, solr is running in > tomcat with the following JAVA_OPTS values "-Xms2g -Xmx16g > -XX:MaxPermSize=3072m -D64" . > > > > When Solr hangs and I try to restart it I am getting the following error, > wh

Solr Hangs

2011-09-01 Thread Rohit
The number of updates are very high into 2 of the cores, solr is running in tomcat with the following JAVA_OPTS values "-Xms2g -Xmx16g -XX:MaxPermSize=3072m -D64" . When Solr hangs and I try to restart it I am getting the following error, which does indicate that it's a memory p

Re: Solr Hangs up after couple of hours

2010-08-23 Thread Bill Au
It would be very useful if you can take a threads dump while Solr is hanging. That will give indication where/why Solr is hanging. Bill On Mon, Aug 23, 2010 at 9:32 PM, Manepalli, Kalyan < kalyan.manepa...@orbitz.com> wrote: > Hi all, > I am facing a peculiar problem with Solr querying. D

Solr Hangs up after couple of hours

2010-08-23 Thread Manepalli, Kalyan
Hi all, I am facing a peculiar problem with Solr querying. During our indexing process we analyze the existing index. For this we query the index. We found that the solr server just hangs on a arbitrary query. If we access the admin/stats.jsp, it again resumes executing the queries. The t

Re: starting solr hangs

2008-07-10 Thread Umar Shah
On Fri, Jul 11, 2008 at 3:11 AM, Chris Hostetter <[EMAIL PROTECTED]> wrote: > > : I am using the same folder to start solr/ jetty on many local machines > : running same version of linux. > > if the same solr + jetty + solr.solr.home combination is working for you > on other machines, then i don'

Re: starting solr hangs

2008-07-10 Thread Chris Hostetter
: I am using the same folder to start solr/ jetty on many local machines : running same version of linux. if the same solr + jetty + solr.solr.home combination is working for you on other machines, then i don't think anyone here can really help you -- sounds like the problem isn't related to S

starting solr hangs

2008-07-01 Thread Umar Shah
Hi, I am experiencing a strange behavior while using the default example solr/ jetty container on a hosted ubuntu 7.10 linux machine after running java -jar start.jar from the example folder, it just outputs 2 lines and hangs 2008-07-01 09:46:35.299::INFO: Logging to STDERR via org.mortbay.log.