Re: SOLR 4 Alpha Out Of Mem Err
Hello Mark, Has this issue been fixed in the BETA release? - Sauvik -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-4-Alpha-Out-Of-Mem-Err-tp3995033p4001266.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR 4 Alpha Out Of Mem Err
Hi Mark, I am am also facing the same issue when trying to index in SolrCloud using DIH running on a non-leader server. The DIH server is creating around 10k threads and then OOM cannot create thread error. Do you know when or which version this issue will be solved. I think a workaround for this issue is to find the leader from zookeeper and run the DIH on the leader. Sauvik -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-4-Alpha-Out-Of-Mem-Err-tp3995033p3996378.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR 4 Alpha Out Of Mem Err
Hi Mark, I am am also facing the same issue when trying to index in SolrCloud using DIH running on a non-leader server. The DIH server is creating around 10k threads and then OOM cannot create thread error. Do you know when or which version this issue will be solved. I think a workaround for this issue is to find the leader from zookeeper and run the DIH on the leader. Sauvik Oddly, I didn't get the above email that I can find - just found it on markmail...odd. I've made a JIRA issue for this: https://issues.apache.org/jira/browse/SOLR-3658 I'm working on / testing a fix. Yes, it affects both httpconcurrent server and dih. Anything that adds docs with a single UpdateProcessor chain rather than a new one for each doc - as long as it's adding fast enough.
Re: SOLR 4 Alpha Out Of Mem Err
I think what makes the most sense is to limit the number of connections to another host. A host only has so many CPU resources, and beyond a certain point throughput would start to suffer anyway (and then only make the problem worse). It also makes sense in that a client could generate documents faster than we can index them (either for a short period of time, or on average) and having flow control to prevent unlimited buffering (which is essentially what this is) makes sense. Nick - when you switched to HttpSolrServer, things worked because this added an explicit flow control mechanism. A single request (i.e. an add with one or more documents) is fully indexed to all endpoints before the response is returned. Hence if you have 10 indexing threads and are adding documents in batches of 100, there can be only 1000 documents buffered in the system at any one time. -Yonik http://lucidimagination.com
Re: SOLR 4 Alpha Out Of Mem Err
Nick, to solve out of memory issue, i think you can make below change: 1) in solrsconfig.xml, reduce ramBufferSizeMB (there are two, change both) 2) in solrsconfig.xml, reduce documentCache value to solve call commit slow down index issue, i think you can change new search default queyr: in solrsconfig.xml, search for listener event=newSearcher class=solr.QuerySenderListener change str name=qcontent:*/str str name=start0/str str name=rows10/str to str name=qcontent:notexist/str str name=start0/str str name=rows10/str -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-4-Alpha-Out-Of-Mem-Err-tp3995033p3995695.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: SOLR 4 Alpha Out Of Mem Err
After trying a number of things, I am successful in allowing the server to auto commit and without having it hit thread/memory errors. I have isolated the required client change to replacing ConcurrentUpdateSolrServer with HttpSolrServer. I am able to maintain index rates of 3,000 documents/sec with 6 shards and two servers per shard. The servers receiving the index requests hit steady state with approximately 800 threads per server. So could there be something amiss in the server side implementation of ConcurrentUpdateSolrServer? Best regards, Nick -Original Message- From: Nick Koton [mailto:nick.ko...@gmail.com] Sent: Monday, July 16, 2012 5:53 PM To: 'solr-user@lucene.apache.org' Subject: RE: SOLR 4 Alpha Out Of Mem Err That suggests you're running out of threads Michael, Thanks for this useful observation. What I found just prior to the problem situation was literally thousands of threads in the server JVM. I have pasted a few samples below obtained from the admin GUI. I spent some time today using this barometer, but I don't have enough to share right now. I'm looking at the difference between ConcurrentUpdateSolrServer and HttpSolrServer and how my client may be misusing them. I'll assume my client is misbehaving and driving the server crazy for now. If I figure out how, I will share it so perhaps a safe guard can be put in place. Nick Server threads - very roughly 0.1 %: cmdDistribExecutor-9-thread-7161 (10096) java.util.concurrent.SynchronousQueue$TransferStack@17b90c55 . sun.misc.Unsafe.park(Native Method) . java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) . java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Synchronous Queue.java:424) . java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueu e.java:323) . java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874) . java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945) . java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9 07) . java.lang.Thread.run(Thread.java:662) -0.ms -0.ms cmdDistribExecutor-9-thread-7160 (10086) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5509b5 6 . sun.misc.Unsafe.park(Native Method) . java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) . java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await( AbstractQueuedSynchronizer.java:1987) . org.apache.http.impl.conn.tsccm.WaitingThread.await(WaitingThread.java:158) . org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(ConnPoolByR oute.java:403) . org.apache.http.impl.conn.tsccm.ConnPoolByRoute$1.getPoolEntry(ConnPoolByRou te.java:300) . org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConnection( ThreadSafeClientConnManager.java:224) . org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDir ector.java:401) . org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.ja va:820) . org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.ja va:754) . org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.ja va:732) . org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java :351) . org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java :182) . org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:325 ) . org.apache.solr.update.SolrCmdDistributor$1.call(SolrCmdDistributor.java:306 ) . java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) . java.util.concurrent.FutureTask.run(FutureTask.java:138) . java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) . java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) . java.util.concurrent.FutureTask.run(FutureTask.java:138) . java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja va:886) . java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9 08) . java.lang.Thread.run(Thread.java:662) 20.ms 20.ms cmdDistribExecutor-9-thread-7159 (10085) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6f062d d3 . sun.misc.Unsafe.park(Native Method) . java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) . java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await( AbstractQueuedSynchronizer.java:1987) . org.apache.http.impl.conn.tsccm.WaitingThread.await(WaitingThread.java:158) . org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(ConnPoolByR oute.java:403) . org.apache.http.impl.conn.tsccm.ConnPoolByRoute$1.getPoolEntry(ConnPoolByRou te.java:300) . org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConnection( ThreadSafeClientConnManager.java:224) . org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDir ector.java:401
Re: SOLR 4 Alpha Out Of Mem Err
On Jul 17, 2012, at 8:08 PM, Nick Koton wrote: So could there be something amiss in the server side implementation of ConcurrentUpdateSolrServer? See my earlier email. Once we decide on the appropriate change, we will address it. - Mark Miller lucidimagination.com
Re: SOLR 4 Alpha Out Of Mem Err
On Jul 15, 2012, at 2:45 PM, Nick Koton wrote: I converted my program to use the SolrServer::add(CollectionSolrInputDocument docs) method with 100 documents in each add batch. Unfortunately, the out of memory errors still occur without client side commits. This won't change much unfortunately - currently, each host has 10 add and 10 deletes buffered for it before it will flush. There are some recovery implications that have kept that buffer size low so far - but what it ends up meaning is that when you stream docs, every 10 docs is sent off on a thread. Generally, you might be able to keep up with this - but the commit cost appears to perhaps cause a small resource drop that backs things up a bit - and some of those threads take a little longer to finish while new threads fire off to keep servicing the constantly coming new documents. What appears will happen is large momentary spikes in the number of threads. Each thread needs a bit of space on the heap, and it would seem with a high enough spike you could get an OOM. In my testing, I have not triggered that yet, but I have seen large thread count spikes. Raising the add doc buffer to 100 docs makes those thread bursts much, much less severe. I can't remember all of the implications of that buffer size though - need to talk to Yonik about it. We could limit the number of threads for that executor, but I think that comes with some negatives as well. You could try lowering -Xss so that each thread uses less RAM (if possible) as a shorter term (possible) workaround. You could also use multiple threads with the std HttpSolrServer - it won't be quite as fast probably, but it can get close(ish). My guess is that your client commits help because a commit will cause a wait on all outstanding requests - so that the commit is in logical order - this probably is like releasing a pressure valve - the system has a chance to catch up and reclaim lots of threads. We will keep looking into what the best improvement is. - Mark Miller lucidimagination.com
RE: SOLR 4 Alpha Out Of Mem Err
$RunnableAdapter.call(Executors.java:441) . java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) . java.util.concurrent.FutureTask.run(FutureTask.java:138) . java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja va:886) . java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9 08) . java.lang.Thread.run(Thread.java:662) -Original Message- From: Michael Della Bitta [mailto:michael.della.bi...@appinions.com] Sent: Sunday, July 15, 2012 4:46 PM To: solr-user@lucene.apache.org Subject: Re: SOLR 4 Alpha Out Of Mem Err unable to create new native thread That suggests you're running out of threads, not RAM. Possibly you're using a multithreaded collector, and it's pushing you over the top of how many threads your OS lets a single process allocate? Or somehow the thread stack size is set too high? More here: http://stackoverflow.com/questions/763579/how-many-threads-can-a-java-vm-sup port Michael Della Bitta Appinions, Inc. -- Where Influence Isn't a Game. http://www.appinions.com On Sun, Jul 15, 2012 at 2:45 PM, Nick Koton nick.ko...@gmail.com wrote: Solrj multi-threaded client sends several 1,000 docs/sec Can you expand on that? How many threads at once are sending docs to solr? Is each request a single doc or multiple? I realize, after the fact, that my solrj client is much like org.apache.solr.client.solrj.LargeVolumeTestBase. The number of threads is configurable at run time as are the various commit parameters. Most of the test have been in the 4-16 threads range. Most of my testing has been with the single document SolrServer::add(SolrInputDocument doc )method. When I realized what LargeVolumeTestBase is doing, I converted my program to use the SolrServer::add(CollectionSolrInputDocument docs) method with 100 documents in each add batch. Unfortunately, the out of memory errors still occur without client side commits. If you agree my three approaches to committing are logical, would it be useful for me to try to reproduce this with example schema in a small cloud configuration using LargeVolumeTestBase or the like? It will take me a couple days to work it in. Or perhaps this sort of test is already run? Best Nick -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Sunday, July 15, 2012 11:05 AM To: Nick Koton Cc: solr-user@lucene.apache.org Subject: Re: SOLR 4 Alpha Out Of Mem Err On Sun, Jul 15, 2012 at 11:52 AM, Nick Koton nick.ko...@gmail.com wrote: Do you have the following hard autoCommit in your config (as the stock server does)? autoCommit maxTime15000/maxTime openSearcherfalse/openSearcher /autoCommit I have tried with and without that setting. When I described running with auto commit, that setting is what I mean. OK cool. You should be able to run the stock server (i.e. with this autocommit) and blast in updates all day long - it looks like you have more than enough memory. If you can't, we need to fix something. You shouldn't need explicit commits unless you want the docs to be searchable at that point. Solrj multi-threaded client sends several 1,000 docs/sec Can you expand on that? How many threads at once are sending docs to solr? Is each request a single doc or multiple? -Yonik http://lucidimagination.com
Re: SOLR 4 Alpha Out Of Mem Err
Do you have the following hard autoCommit in your config (as the stock server does)? autoCommit maxTime15000/maxTime openSearcherfalse/openSearcher /autoCommit This is now fairly important since Solr now tracks information on every uncommitted document added. At some point we should probably hardcode some mechanism based on number of documents or time. -Yonik http://lucidimagination.com
RE: SOLR 4 Alpha Out Of Mem Err
Do you have the following hard autoCommit in your config (as the stock server does)? autoCommit maxTime15000/maxTime openSearcherfalse/openSearcher /autoCommit I have tried with and without that setting. When I described running with auto commit, that setting is what I mean. I have varied the time in the range 10,000-60,000 msec. I have tried this setting with and without soft commit in the server config file. I have tried without this setting, but specifying the commit within time in the solrj client in the add method. In both these cases, the client seems to overrun the server and out of memory in the server results. One clarification I should make is that after the server gets out of memory, the solrj client does NOT receive an error. However, the documents indexed do not reliably appear to queries. Approach #3 is to remove the autocommit in the server config, issue the add method without commit within, but issue commits in the solrj client with wait for sync and searcher set to true. In case #3, I do not see the out of memory in the server. However, document index rates are restricted to about 1,000 per second. Nick -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Sunday, July 15, 2012 5:15 AM To: solr-user@lucene.apache.org Subject: Re: SOLR 4 Alpha Out Of Mem Err Do you have the following hard autoCommit in your config (as the stock server does)? autoCommit maxTime15000/maxTime openSearcherfalse/openSearcher /autoCommit This is now fairly important since Solr now tracks information on every uncommitted document added. At some point we should probably hardcode some mechanism based on number of documents or time. -Yonik http://lucidimagination.com
Re: SOLR 4 Alpha Out Of Mem Err
Maybe your rate of update is so high that the commit never gets a chance to run. So, maybe all these uncommitted updates are buffered up and using excess memory. Try explicit commits from SolrJ, but less frequently. Or maybe if you just pause your updates periodically (every 30 seconds or so) the auto-commit would get a chance to occur. Although I have no idea how long a pause might be needed. -- Jack Krupansky -Original Message- From: Nick Koton Sent: Sunday, July 15, 2012 10:52 AM To: solr-user@lucene.apache.org ; yo...@lucidimagination.com Subject: RE: SOLR 4 Alpha Out Of Mem Err Do you have the following hard autoCommit in your config (as the stock server does)? autoCommit maxTime15000/maxTime openSearcherfalse/openSearcher /autoCommit I have tried with and without that setting. When I described running with auto commit, that setting is what I mean. I have varied the time in the range 10,000-60,000 msec. I have tried this setting with and without soft commit in the server config file. I have tried without this setting, but specifying the commit within time in the solrj client in the add method. In both these cases, the client seems to overrun the server and out of memory in the server results. One clarification I should make is that after the server gets out of memory, the solrj client does NOT receive an error. However, the documents indexed do not reliably appear to queries. Approach #3 is to remove the autocommit in the server config, issue the add method without commit within, but issue commits in the solrj client with wait for sync and searcher set to true. In case #3, I do not see the out of memory in the server. However, document index rates are restricted to about 1,000 per second. Nick -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Sunday, July 15, 2012 5:15 AM To: solr-user@lucene.apache.org Subject: Re: SOLR 4 Alpha Out Of Mem Err Do you have the following hard autoCommit in your config (as the stock server does)? autoCommit maxTime15000/maxTime openSearcherfalse/openSearcher /autoCommit This is now fairly important since Solr now tracks information on every uncommitted document added. At some point we should probably hardcode some mechanism based on number of documents or time. -Yonik http://lucidimagination.com
Re: SOLR 4 Alpha Out Of Mem Err
On Sun, Jul 15, 2012 at 11:52 AM, Nick Koton nick.ko...@gmail.com wrote: Do you have the following hard autoCommit in your config (as the stock server does)? autoCommit maxTime15000/maxTime openSearcherfalse/openSearcher /autoCommit I have tried with and without that setting. When I described running with auto commit, that setting is what I mean. OK cool. You should be able to run the stock server (i.e. with this autocommit) and blast in updates all day long - it looks like you have more than enough memory. If you can't, we need to fix something. You shouldn't need explicit commits unless you want the docs to be searchable at that point. Solrj multi-threaded client sends several 1,000 docs/sec Can you expand on that? How many threads at once are sending docs to solr? Is each request a single doc or multiple? -Yonik http://lucidimagination.com
Re: SOLR 4 Alpha Out Of Mem Err
On Sun, Jul 15, 2012 at 12:52 PM, Jack Krupansky j...@basetechnology.com wrote: Maybe your rate of update is so high that the commit never gets a chance to run. I don't believe that is possible. If it is, it should be fixed. -Yonik http://lucidimagination.com
Re: SOLR 4 Alpha Out Of Mem Err
Agreed. That's why I say maybe. Clearly something sounds amiss here. -- Jack Krupansky -Original Message- From: Yonik Seeley Sent: Sunday, July 15, 2012 12:06 PM To: solr-user@lucene.apache.org Subject: Re: SOLR 4 Alpha Out Of Mem Err On Sun, Jul 15, 2012 at 12:52 PM, Jack Krupansky j...@basetechnology.com wrote: Maybe your rate of update is so high that the commit never gets a chance to run. I don't believe that is possible. If it is, it should be fixed. -Yonik http://lucidimagination.com
RE: SOLR 4 Alpha Out Of Mem Err
Solrj multi-threaded client sends several 1,000 docs/sec Can you expand on that? How many threads at once are sending docs to solr? Is each request a single doc or multiple? I realize, after the fact, that my solrj client is much like org.apache.solr.client.solrj.LargeVolumeTestBase. The number of threads is configurable at run time as are the various commit parameters. Most of the test have been in the 4-16 threads range. Most of my testing has been with the single document SolrServer::add(SolrInputDocument doc )method. When I realized what LargeVolumeTestBase is doing, I converted my program to use the SolrServer::add(CollectionSolrInputDocument docs) method with 100 documents in each add batch. Unfortunately, the out of memory errors still occur without client side commits. If you agree my three approaches to committing are logical, would it be useful for me to try to reproduce this with example schema in a small cloud configuration using LargeVolumeTestBase or the like? It will take me a couple days to work it in. Or perhaps this sort of test is already run? Best Nick -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Sunday, July 15, 2012 11:05 AM To: Nick Koton Cc: solr-user@lucene.apache.org Subject: Re: SOLR 4 Alpha Out Of Mem Err On Sun, Jul 15, 2012 at 11:52 AM, Nick Koton nick.ko...@gmail.com wrote: Do you have the following hard autoCommit in your config (as the stock server does)? autoCommit maxTime15000/maxTime openSearcherfalse/openSearcher /autoCommit I have tried with and without that setting. When I described running with auto commit, that setting is what I mean. OK cool. You should be able to run the stock server (i.e. with this autocommit) and blast in updates all day long - it looks like you have more than enough memory. If you can't, we need to fix something. You shouldn't need explicit commits unless you want the docs to be searchable at that point. Solrj multi-threaded client sends several 1,000 docs/sec Can you expand on that? How many threads at once are sending docs to solr? Is each request a single doc or multiple? -Yonik http://lucidimagination.com
Re: SOLR 4 Alpha Out Of Mem Err
unable to create new native thread That suggests you're running out of threads, not RAM. Possibly you're using a multithreaded collector, and it's pushing you over the top of how many threads your OS lets a single process allocate? Or somehow the thread stack size is set too high? More here: http://stackoverflow.com/questions/763579/how-many-threads-can-a-java-vm-support Michael Della Bitta Appinions, Inc. -- Where Influence Isn’t a Game. http://www.appinions.com On Sun, Jul 15, 2012 at 2:45 PM, Nick Koton nick.ko...@gmail.com wrote: Solrj multi-threaded client sends several 1,000 docs/sec Can you expand on that? How many threads at once are sending docs to solr? Is each request a single doc or multiple? I realize, after the fact, that my solrj client is much like org.apache.solr.client.solrj.LargeVolumeTestBase. The number of threads is configurable at run time as are the various commit parameters. Most of the test have been in the 4-16 threads range. Most of my testing has been with the single document SolrServer::add(SolrInputDocument doc )method. When I realized what LargeVolumeTestBase is doing, I converted my program to use the SolrServer::add(CollectionSolrInputDocument docs) method with 100 documents in each add batch. Unfortunately, the out of memory errors still occur without client side commits. If you agree my three approaches to committing are logical, would it be useful for me to try to reproduce this with example schema in a small cloud configuration using LargeVolumeTestBase or the like? It will take me a couple days to work it in. Or perhaps this sort of test is already run? Best Nick -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Sunday, July 15, 2012 11:05 AM To: Nick Koton Cc: solr-user@lucene.apache.org Subject: Re: SOLR 4 Alpha Out Of Mem Err On Sun, Jul 15, 2012 at 11:52 AM, Nick Koton nick.ko...@gmail.com wrote: Do you have the following hard autoCommit in your config (as the stock server does)? autoCommit maxTime15000/maxTime openSearcherfalse/openSearcher /autoCommit I have tried with and without that setting. When I described running with auto commit, that setting is what I mean. OK cool. You should be able to run the stock server (i.e. with this autocommit) and blast in updates all day long - it looks like you have more than enough memory. If you can't, we need to fix something. You shouldn't need explicit commits unless you want the docs to be searchable at that point. Solrj multi-threaded client sends several 1,000 docs/sec Can you expand on that? How many threads at once are sending docs to solr? Is each request a single doc or multiple? -Yonik http://lucidimagination.com
SOLR 4 Alpha Out Of Mem Err
I have been experiencing out of memory errors when indexing via solrj into a 4 alpha cluster. It seems when I delegate commits to the server (either auto commit or commit within) there is nothing to throttle the solrj clients and the server struggles to fan out the work. However, when I handle commits entirely within the client, the indexing rate is very restricted. Any suggestions would be appreciated Nick Cotton nick.ko...@gmail.com
Re: SOLR 4 Alpha Out Of Mem Err
Can you give more info? How much RAM are you giving Solr with Xmx? Can you be more specific about the behavior you are seeing with auto commit vs client commit? How often are you trying to commit? With the client? With auto commit? Are you doing soft commits? Std commits? A mix? What's the stack trace for the OOM? What OS are you using? Anything else you can add? -- Mark Miller On Saturday, July 14, 2012 at 4:21 PM, Nick Koton wrote: I have been experiencing out of memory errors when indexing via solrj into a 4 alpha cluster. It seems when I delegate commits to the server (either auto commit or commit within) there is nothing to throttle the solrj clients and the server struggles to fan out the work. However, when I handle commits entirely within the client, the indexing rate is very restricted. Any suggestions would be appreciated Nick Cotton nick.ko...@gmail.com
RE: SOLR 4 Alpha Out Of Mem Err
) at java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:727) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:657) at java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:152) at org.apache.solr.update.SolrCmdDistributor.submit(SolrCmdDistributor.java:340) at org.apache.solr.update.SolrCmdDistributor.submit(SolrCmdDistributor.java:296) at org.apache.solr.update.SolrCmdDistributor.flushAdds(SolrCmdDistributor.java:228) at org.apache.solr.update.SolrCmdDistributor.finish(SolrCmdDistributor.java:101) at org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:329) at org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:952) at org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:176) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:83) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1561) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263) ... 25 more Jul 14, 2012 9:20:57 PM org.apache.solr.common.SolrException log -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Saturday, July 14, 2012 2:44 PM To: solr-user@lucene.apache.org Subject: Re: SOLR 4 Alpha Out Of Mem Err Can you give more info? How much RAM are you giving Solr with Xmx? Can you be more specific about the behavior you are seeing with auto commit vs client commit? How often are you trying to commit? With the client? With auto commit? Are you doing soft commits? Std commits? A mix? What's the stack trace for the OOM? What OS are you using? Anything else you can add? -- Mark Miller On Saturday, July 14, 2012 at 4:21 PM, Nick Koton wrote: I have been experiencing out of memory errors when indexing via solrj into a 4 alpha cluster. It seems when I delegate commits to the server (either auto commit or commit within) there is nothing to throttle the solrj clients and the server struggles to fan out the work. However, when I handle commits entirely within the client, the indexing rate is very restricted. Any suggestions would be appreciated Nick Cotton nick.ko...@gmail.com
