Actually when I look at the errormessage, this has nothing to do with memory. The error message: java.lang.OutOfMemoryError: unable to create new native thread
means that the OS can not create any new native threads for this JVM. So the limit you are running into is not the JVM Memory. I guess you should rather look at a bottleneck inside your application that prevents your serverthreads from being reused when you fire concurrent batches to your sever. Do you do all that in paralell? In the stacktrace below your connector can not get any new threads from the pool which has nothing to do with memory. Try to figure out what is taking so much time during the batch process on the server. simon On Nov 20, 2007 5:16 PM, Brian Carmalt <[EMAIL PROTECTED]> wrote: > Hello all, > > I started looking into the scalability of solr, and have started getting > weird results. > I am getting the following error: > > Exception in thread "btpool0-3" java.lang.OutOfMemoryError: unable to > create new native thread > at java.lang.Thread.start0(Native Method) > at java.lang.Thread.start(Thread.java:574) > at > org.mortbay.thread.BoundedThreadPool.newThread(BoundedThreadPool.java:377) > at > org.mortbay.thread.BoundedThreadPool.dispatch(BoundedThreadPool.java:94) > at > org.mortbay.jetty.bio.SocketConnector$Connection.dispatch( > SocketConnector.java:187) > at > org.mortbay.jetty.bio.SocketConnector.accept(SocketConnector.java:101) > at > org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java > :516) > at > org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java > :442) > > This only occurs when I send docs to the server in batches of around 10 > as separate processes. > If I send the serially, the heap grows up to 1200M and with no errors. > > When I observe the VM during it's operation, It doesn't seem to run out > of memory. The VM starts > with 1024M and can allocate up to 1800M. I start getting the error > listed above when the memory > usage is right around 1 G. I have been using the Jconsole program on > windows to observe the > jetty server by using the com.sun.management.jmxremote* functions on the > server side. The number of threads > is always around 30, and jetty can create up 250, so I don't think > that's the problem. I can't really image that > the monitoring process is using the other 800M of the allowable heap > memory, but it could be. > But the problem occurs without monitoring, even when the VM heap is set > to 1500M. > > Does anyone have an idea as to why this error is occurring? > > Thanks, > Brian >
