More memory is not necessarily better, it can lead to longer, more
intense garbage collections that cause things to stop. You might
also consider lowering your memory allocation, but 2G is really not
all that much so I somewhat doubt it's a problem but thought I'd
mention it.

Best
Erick

On Thu, Mar 29, 2012 at 1:50 PM, Rafal Gwizdala
<rafal.gwizd...@gmail.com> wrote:
> Guys, thanks for all the suggestions
> I will be trying them, one at a time. Imho it's too early to give up and
> look for another tool, I'll try to work on configuration and see what
> happens.
> The NRT looks quite promising, there are also tons of config options to
> change.
> As for now, I have made the updates less frequent - about once every 30
> seconds (but now the batches are bigger, about 150-200 documents per
> update). I'll see if this makes SOLR more stable or users more aggressive.
> Unfortunately I have no resources for experimenting so I'll keep making
> small changes to production system and observing the effects.
> Shawn, I have given the JVM about 2 GB of memory but it's only using 300 MB
> so I don't think there's memory shortage now. The whole index is about 2 GB
> in size but I think there aren't enough queries to fill up the cache and
> make SOLR load everything in memory.
>
> Below i'm pasting the thread dump taken when the update was hung (it's also
> attached to the first message of this topic)
>
> Best regards, RG
>
> <solr>
>  <core>example</core>
>  <system>
>  <jvm>
>    <version>20.5-b03</version>
>    <name>Java HotSpot(TM) 64-Bit Server VM</name>
>  </jvm>
>  <threadCount>
>    <current>31</current>
>    <peak>32</peak>
>    <daemon>8</daemon>
>  </threadCount>
>
>  <threadDump>
>    <thread>
>      <id>39</id>
>      <name>pool-4-thread-1</name>
>      <state>WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@765bc9b8
> </lock>
>      <cpuTime>312,5000ms</cpuTime>
>      <userTime>265,6250ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.park(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.DelayQueue.take(Unknown Source)
>  </line>
>        <line>at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)        </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>38</id>
>      <name>pool-2-thread-1</name>
>      <state>WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4188bbd
> </lock>
>      <cpuTime>6484,3750ms</cpuTime>
>      <userTime>5546,8750ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.park(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.DelayQueue.take(Unknown Source)
>  </line>
>        <line>at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)        </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>37</id>
>      <name>DestroyJavaVM</name>
>      <state>RUNNABLE</state>
>      <cpuTime>4906,2500ms</cpuTime>
>      <userTime>4484,3750ms</userTime>
>      <stackTrace>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>36</id>
>      <name>qtp1033068770-36</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@677e2764
> </lock>
>      <cpuTime>134968,7500ms</cpuTime>
>      <userTime>114984,3750ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>35</id>
>      <name>qtp1033068770-35</name>
>      <state>RUNNABLE</state>
>      <cpuTime>147390,6250ms</cpuTime>
>      <userTime>126593,7500ms</userTime>
>      <stackTrace>
>        <line>at sun.management.ThreadImpl.getThreadInfo1(Native Method)
>     </line>
>        <line>at sun.management.ThreadImpl.getThreadInfo(Unknown Source)
>     </line>
>        <line>at
> org.apache.jsp.admin.threaddump_jsp._jspService(org.apache.jsp.admin.threaddump_jsp:264)
>       </line>
>        <line>at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:109)
>  </line>
>        <line>at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:820)        </line>
>        <line>at
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:389)
>       </line>
>        <line>at
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:486)
>     </line>
>        <line>at
> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:380)
>  </line>
>        <line>at
> javax.servlet.http.HttpServlet.service(HttpServlet.java:820)        </line>
>        <line>at
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:534)
>   </line>
>        <line>at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:475)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>       </line>
>        <line>at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:516)
>       </line>
>        <line>at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:929)
>       </line>
>        <line>at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:403)
>     </line>
>        <line>at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:864)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>       </line>
>        <line>at
> org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:285)
>  </line>
>        <line>at
> org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:116)
>  </line>
>        <line>at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275)
>       </line>
>        <line>at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322)
>       </line>
>        <line>at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:473)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
>       </line>
>        <line>at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:479)
>       </line>
>        <line>at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:929)
>       </line>
>        <line>at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:403)
>     </line>
>        <line>at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:864)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:247)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:151)
>       </line>
>        <line>at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114)
>       </line>
>        <line>at org.eclipse.jetty.server.Server.handle(Server.java:352)
>     </line>
>        <line>at
> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596)
>       </line>
>        <line>at
> org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1051)
>       </line>
>        <line>at
> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:590)
>  </line>
>        <line>at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:212)
>  </line>
>        <line>at
> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:426)
>   </line>
>        <line>at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:508)
>       </line>
>        <line>at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34)
>       </line>
>        <line>at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>34</id>
>      <name>qtp1033068770-34</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@677e2764
> </lock>
>      <cpuTime>131859,3750ms</cpuTime>
>      <userTime>112515,6250ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>32</id>
>      <name>qtp1033068770-32</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@677e2764
> </lock>
>      <cpuTime>143156,2500ms</cpuTime>
>      <userTime>121687,5000ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>31</id>
>      <name>qtp1033068770-31</name>
>      <state>RUNNABLE</state>
>      <inNative/>
>      <cpuTime>703,1250ms</cpuTime>
>      <userTime>390,6250ms</userTime>
>      <stackTrace>
>        <line>at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
>       </line>
>        <line>at sun.nio.ch.ServerSocketChannelImpl.accept(Unknown Source)
>       </line>
>        <line>at
> org.eclipse.jetty.server.nio.SelectChannelConnector$2.run(SelectChannelConnector.java:287)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>30</id>
>      <name>qtp1033068770-30</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@677e2764
> </lock>
>      <cpuTime>129718,7500ms</cpuTime>
>      <userTime>110812,5000ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>29</id>
>      <name>qtp1033068770-29 - Acceptor1 SelectChannelConnector@0.0.0.0:9070
> </name>
>      <state>RUNNABLE</state>
>      <inNative/>
>      <cpuTime>5734,3750ms</cpuTime>
>      <userTime>2875,0000ms</userTime>
>      <stackTrace>
>        <line>at sun.nio.ch.WindowsSelectorImpl.discardUrgentData(Native
> Method)        </line>
>        <line>at sun.nio.ch.WindowsSelectorImpl.access$2200(Unknown Source)
>       </line>
>        <line>at
> sun.nio.ch.WindowsSelectorImpl$SubSelector.processFDSet(Unknown Source)
>   </line>
>        <line>at
> sun.nio.ch.WindowsSelectorImpl$SubSelector.processSelectedKeys(Unknown
> Source)        </line>
>        <line>at
> sun.nio.ch.WindowsSelectorImpl$SubSelector.access$2900(Unknown Source)
>   </line>
>        <line>at sun.nio.ch.WindowsSelectorImpl.updateSelectedKeys(Unknown
> Source)        </line>
>        <line>at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source)
>     </line>
>        <line>at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)
>     </line>
>        <line>at sun.nio.ch.SelectorImpl.select(Unknown Source)
>  </line>
>        <line>at
> org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:481)
>       </line>
>        <line>at
> org.eclipse.jetty.io.nio.SelectorManager.doSelect(SelectorManager.java:219)
>       </line>
>        <line>at
> org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:122)
>       </line>
>        <line>at
> org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:837)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>28</id>
>      <name>qtp1033068770-28</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@677e2764
> </lock>
>      <cpuTime>137921,8750ms</cpuTime>
>      <userTime>118078,1250ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>27</id>
>      <name>qtp1033068770-27 - Acceptor0 SelectChannelConnector@0.0.0.0:9070
> </name>
>      <state>RUNNABLE</state>
>      <inNative/>
>      <cpuTime>6859,3750ms</cpuTime>
>      <userTime>3890,6250ms</userTime>
>      <stackTrace>
>        <line>at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native
> Method)        </line>
>        <line>at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown
> Source)        </line>
>        <line>at
> sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown Source)
>  </line>
>        <line>at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source)
>     </line>
>        <line>at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)
>     </line>
>        <line>at sun.nio.ch.SelectorImpl.select(Unknown Source)
>  </line>
>        <line>at
> org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:481)
>       </line>
>        <line>at
> org.eclipse.jetty.io.nio.SelectorManager.doSelect(SelectorManager.java:219)
>       </line>
>        <line>at
> org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:122)
>       </line>
>        <line>at
> org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:837)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>26</id>
>      <name>qtp1033068770-26</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@677e2764
> </lock>
>      <cpuTime>132546,8750ms</cpuTime>
>      <userTime>112984,3750ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>25</id>
>      <name>Scanner-1</name>
>      <state>TIMED_WAITING</state>
>      <lock>java.util.TaskQueue@b4c4b51</lock>
>      <cpuTime>46,8750ms</cpuTime>
>      <userTime>46,8750ms</userTime>
>      <stackTrace>
>        <line>at java.lang.Object.wait(Native Method)        </line>
>        <line>at java.util.TimerThread.mainLoop(Unknown Source)
>  </line>
>        <line>at java.util.TimerThread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>24</id>
>      <name>TransparentProxy-24</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3de4b8a2
> </lock>
>      <cpuTime>0,0000ms</cpuTime>
>      <userTime>0,0000ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>23</id>
>      <name>TransparentProxy-23</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3de4b8a2
> </lock>
>      <cpuTime>0,0000ms</cpuTime>
>      <userTime>0,0000ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>22</id>
>      <name>TransparentProxy-22</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3de4b8a2
> </lock>
>      <cpuTime>0,0000ms</cpuTime>
>      <userTime>0,0000ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>21</id>
>      <name>TransparentProxy-21</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3de4b8a2
> </lock>
>      <cpuTime>15,6250ms</cpuTime>
>      <userTime>15,6250ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>20</id>
>      <name>TransparentProxy-20</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3de4b8a2
> </lock>
>      <cpuTime>0,0000ms</cpuTime>
>      <userTime>0,0000ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>19</id>
>      <name>TransparentProxy-19</name>
>      <state>TIMED_WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3de4b8a2
> </lock>
>      <cpuTime>0,0000ms</cpuTime>
>      <userTime>0,0000ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.parkNanos(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown
> Source)        </line>
>        <line>at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:320)
>       </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:480)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>18</id>
>      <name>TransparentProxy-18</name>
>      <state>TIMED_WAITING</state>
>      <cpuTime>15,6250ms</cpuTime>
>      <userTime>15,6250ms</userTime>
>      <stackTrace>
>        <line>at java.lang.Thread.sleep(Native Method)        </line>
>        <line>at
> org.eclipse.jetty.client.HttpClient$1.run(HttpClient.java:483)
>  </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>17</id>
>      <name>TransparentProxy-17</name>
>      <state>RUNNABLE</state>
>      <inNative/>
>      <cpuTime>171,8750ms</cpuTime>
>      <userTime>140,6250ms</userTime>
>      <stackTrace>
>        <line>at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native
> Method)        </line>
>        <line>at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown
> Source)        </line>
>        <line>at
> sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown Source)
>  </line>
>        <line>at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source)
>     </line>
>        <line>at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)
>     </line>
>        <line>at sun.nio.ch.SelectorImpl.select(Unknown Source)
>  </line>
>        <line>at
> org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:481)
>       </line>
>        <line>at
> org.eclipse.jetty.io.nio.SelectorManager.doSelect(SelectorManager.java:219)
>       </line>
>        <line>at
> org.eclipse.jetty.client.SelectConnector.run(SelectConnector.java:163)
>   </line>
>        <line>at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451)
>       </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>16</id>
>      <name>HashSessionScavenger-1</name>
>      <state>TIMED_WAITING</state>
>      <lock>java.util.TaskQueue@7315b36c</lock>
>      <cpuTime>0,0000ms</cpuTime>
>      <userTime>0,0000ms</userTime>
>      <stackTrace>
>        <line>at java.lang.Object.wait(Native Method)        </line>
>        <line>at java.util.TimerThread.mainLoop(Unknown Source)
>  </line>
>        <line>at java.util.TimerThread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>15</id>
>      <name>Scanner-0</name>
>      <state>TIMED_WAITING</state>
>      <lock>java.util.TaskQueue@15b6b27b</lock>
>      <cpuTime>93,7500ms</cpuTime>
>      <userTime>15,6250ms</userTime>
>      <stackTrace>
>        <line>at java.lang.Object.wait(Native Method)        </line>
>        <line>at java.util.TimerThread.mainLoop(Unknown Source)
>  </line>
>        <line>at java.util.TimerThread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>14</id>
>      <name>pool-3-thread-1</name>
>      <state>WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@d7fb460
> </lock>
>      <cpuTime>1046,8750ms</cpuTime>
>      <userTime>1000,0000ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.park(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.LinkedBlockingQueue.take(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)        </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>13</id>
>      <name>pool-1-thread-1</name>
>      <state>WAITING</state>
>
> <lock>java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3b1b118c
> </lock>
>      <cpuTime>4796,8750ms</cpuTime>
>      <userTime>4578,1250ms</userTime>
>      <stackTrace>
>        <line>at sun.misc.Unsafe.park(Native Method)        </line>
>        <line>at java.util.concurrent.locks.LockSupport.park(Unknown
> Source)        </line>
>        <line>at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.LinkedBlockingQueue.take(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown
> Source)        </line>
>        <line>at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)        </line>
>        <line>at java.lang.Thread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>11</id>
>      <name>HashSessionScavenger-0</name>
>      <state>TIMED_WAITING</state>
>      <lock>java.util.TaskQueue@6e1d1ca4</lock>
>      <cpuTime>0,0000ms</cpuTime>
>      <userTime>0,0000ms</userTime>
>      <stackTrace>
>        <line>at java.lang.Object.wait(Native Method)        </line>
>        <line>at java.util.TimerThread.mainLoop(Unknown Source)
>  </line>
>        <line>at java.util.TimerThread.run(Unknown Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>5</id>
>      <name>Attach Listener</name>
>      <state>RUNNABLE</state>
>      <cpuTime>0,0000ms</cpuTime>
>      <userTime>0,0000ms</userTime>
>      <stackTrace>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>4</id>
>      <name>Signal Dispatcher</name>
>      <state>RUNNABLE</state>
>      <cpuTime>0,0000ms</cpuTime>
>      <userTime>0,0000ms</userTime>
>      <stackTrace>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>3</id>
>      <name>Finalizer</name>
>      <state>WAITING</state>
>      <lock>java.lang.ref.ReferenceQueue$Lock@39438b76</lock>
>      <cpuTime>78,1250ms</cpuTime>
>      <userTime>62,5000ms</userTime>
>      <stackTrace>
>        <line>at java.lang.Object.wait(Native Method)        </line>
>        <line>at java.lang.ref.ReferenceQueue.remove(Unknown Source)
>  </line>
>        <line>at java.lang.ref.ReferenceQueue.remove(Unknown Source)
>  </line>
>        <line>at java.lang.ref.Finalizer$FinalizerThread.run(Unknown
> Source)        </line>
>      </stackTrace>
>    </thread>
>    <thread>
>      <id>2</id>
>      <name>Reference Handler</name>
>      <state>WAITING</state>
>      <lock>java.lang.ref.Reference$Lock@1790d59</lock>
>      <cpuTime>78,1250ms</cpuTime>
>      <userTime>62,5000ms</userTime>
>      <stackTrace>
>        <line>at java.lang.Object.wait(Native Method)        </line>
>        <line>at java.lang.Object.wait(Object.java:485)        </line>
>        <line>at java.lang.ref.Reference$ReferenceHandler.run(Unknown
> Source)        </line>
>      </stackTrace>
>    </thread>
>  </threadDump>
>
>  </system>
> </solr>

Reply via email to