Rolling back to 1.554.3 <http://jenkins-ci.org/> remove the log. I don't understand which change in core can produce this side-effect on crowd plugin / httpclient
On Fri, Aug 1, 2014 at 10:48 AM, Arnaud Héritier <[email protected]> wrote: > ok sadly this morning I discover that it wasn't a good idea to hide this > problem. My instance is really slow with ~10 threads blocked like this : > > RequestHandlerThread[#700] > > "RequestHandlerThread[#700]" Id=63426 Group=main BLOCKED on > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool@748b8ee1 > owned by "RequestHandlerThread[#701]" Id=63427 > at > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool.freeConnection(MultiThreadedHttpConnectionManager.java:984) > - blocked on > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool@748b8ee1 > at > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.releaseConnection(MultiThreadedHttpConnectionManager.java:643) > at > org.apache.commons.httpclient.HttpConnection.releaseConnection(HttpConnection.java:1179) > at > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.releaseConnection(MultiThreadedHttpConnectionManager.java:1423) > at > org.apache.commons.httpclient.HttpMethodBase.ensureConnectionRelease(HttpMethodBase.java:2430) > at > org.apache.commons.httpclient.HttpMethodBase.responseBodyConsumed(HttpMethodBase.java:2422) > at > org.apache.commons.httpclient.HttpMethodBase$1.responseConsumed(HttpMethodBase.java:1892) > at > org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoCloseInputStream.java:198) > at > org.apache.commons.httpclient.AutoCloseInputStream.checkClose(AutoCloseInputStream.java:170) > at > org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:109) > at > com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:2900) > at > com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.read(UTF8Reader.java:302) > at > com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(XMLEntityScanner.java:1753) > at > com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipSpaces(XMLEntityScanner.java:1502) > at > com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$TrailingMiscDriver.next(XMLDocumentScannerImpl.java:1372) > at > com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606) > at > com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117) > at > com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510) > at > com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848) > at > com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777) > at > com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) > at > com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) > at > com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:649) > at > com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:203) > at > com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:181) > at > com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:235) > at javax.xml.bind.JAXB.unmarshal(JAXB.java:223) > at > com.atlassian.crowd.integration.rest.service.RestExecutor$MethodExecutor.andReceive(RestExecutor.java:349) > at > com.atlassian.crowd.integration.rest.service.RestCrowdClient.getGroupsForUser(RestCrowdClient.java:641) > at > de.theit.jenkins.crowd.CrowdConfigurationService.getAuthoritiesForUser(CrowdConfigurationService.java:280) > at > de.theit.jenkins.crowd.CrowdRememberMeServices.autoLogin(CrowdRememberMeServices.java:136) > at > de.theit.jenkins.crowd.CrowdServletFilter.doFilter(CrowdServletFilter.java:181) > at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) > at > org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:46) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) > at > hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) > at org.eclipse.jetty.server.Server.handle(Server.java:370) > at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) > at > org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:949) > at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1011) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644) > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) > at > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668) > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) > at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > > Number of locked synchronizers = 1 > - java.util.concurrent.ThreadPoolExecutor$Worker@4aa15e5e > > > because of > > RequestHandlerThread[#701] > > > "RequestHandlerThread[#701]" Id=63427 Group=main RUNNABLE > at java.lang.String.regionMatches(String.java:1340) > at java.lang.String.equalsIgnoreCase(String.java:1090) > at org.apache.commons.httpclient.HttpHost.equals(HttpHost.java:199) > at > org.apache.commons.httpclient.util.LangUtils.equals(LangUtils.java:63) > at > org.apache.commons.httpclient.HostConfiguration.equals(HostConfiguration.java:506) > - locked org.apache.commons.httpclient.HostConfiguration@28793572 > at java.util.HashMap.getEntry(HashMap.java:471) > at java.util.HashMap.get(HashMap.java:421) > at > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool.getHostPool(MultiThreadedHttpConnectionManager.java:808) > - locked > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool@748b8ee1 > at > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool.freeConnection(MultiThreadedHttpConnectionManager.java:991) > - locked > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ConnectionPool@748b8ee1 > at > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager.releaseConnection(MultiThreadedHttpConnectionManager.java:643) > at > org.apache.commons.httpclient.HttpConnection.releaseConnection(HttpConnection.java:1179) > at > org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.releaseConnection(MultiThreadedHttpConnectionManager.java:1423) > at > org.apache.commons.httpclient.HttpMethodBase.ensureConnectionRelease(HttpMethodBase.java:2430) > at > org.apache.commons.httpclient.HttpMethodBase.responseBodyConsumed(HttpMethodBase.java:2422) > at > org.apache.commons.httpclient.HttpMethodBase$1.responseConsumed(HttpMethodBase.java:1892) > at > org.apache.commons.httpclient.AutoCloseInputStream.notifyWatcher(AutoCloseInputStream.java:198) > at > org.apache.commons.httpclient.AutoCloseInputStream.checkClose(AutoCloseInputStream.java:170) > at > org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:109) > at > com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:2900) > at > com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.read(UTF8Reader.java:302) > at > com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(XMLEntityScanner.java:1753) > at > com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipSpaces(XMLEntityScanner.java:1502) > at > com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$TrailingMiscDriver.next(XMLDocumentScannerImpl.java:1372) > at > com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606) > at > com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117) > at > com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510) > at > com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848) > at > com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777) > at > com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) > at > com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) > at > com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:649) > at > com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:203) > at > com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:181) > at > com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:235) > at javax.xml.bind.JAXB.unmarshal(JAXB.java:223) > at > com.atlassian.crowd.integration.rest.service.RestExecutor$MethodExecutor.andReceive(RestExecutor.java:349) > at > com.atlassian.crowd.integration.rest.service.RestCrowdClient.getGroupsForUser(RestCrowdClient.java:641) > at > de.theit.jenkins.crowd.CrowdConfigurationService.getAuthoritiesForUser(CrowdConfigurationService.java:280) > at > de.theit.jenkins.crowd.CrowdRememberMeServices.autoLogin(CrowdRememberMeServices.java:136) > at > de.theit.jenkins.crowd.CrowdServletFilter.doFilter(CrowdServletFilter.java:181) > at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) > at > org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:46) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) > at > hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) > at org.eclipse.jetty.server.Server.handle(Server.java:370) > at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) > at > org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:949) > at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1011) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644) > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) > at > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668) > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) > at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > > Number of locked synchronizers = 1 > - java.util.concurrent.ThreadPoolExecutor$Worker@5fb81882 > > > > I don't yet see the relation with the jenkins core upgrade (the crowd > plugin didn't change). > > I will see on crowd plugin side if there is something strange. > > cheers > > > > > On Thu, Jul 31, 2014 at 3:22 PM, Arnaud Héritier <[email protected]> > wrote: > >> Hi, >> >> I don't know if it is due to the upgrade to Jenkins ver. 1.565.1 >> <http://jenkins-ci.org/> but I'm noticing in my logs a lot of SEVERE >> errors I didn't see before : >> >> Jul 31, 2014 3:14:32 PM >> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager >> freeConnection >> >> SEVERE: Host connection pool not found, hostConfig=HostConfiguration[host= >> https://identity.exoplatform.org] >> >> >> It seems they are useless ( >> https://issues.apache.org/jira/browse/HTTPCLIENT-799) and due to the >> crowd plugin which uses commons http client 3.x >> >> If in jenkins Logs Admin UI I set the logger >> org.apache.commons.httpclient.MultiThreadedHttpConnectionManager to OFF >> it solves my issue but I loose it when I restart jenkins. >> >> I tried to use an init.groovy.d script but it doesn't work (I'm not >> sure why, I see it is executed but logs are here and I'm not a java logging >> fan/expert) >> >> Logger multiThreadedHttpConnectionManagerLoggerStopped = Logger.getLogger( >> "org.apache.commons.httpclient.MultiThreadedHttpConnectionManager") >> >> multiThreadedHttpConnectionManagerLoggerStopped.setLevel(Level.OFF) >> >> >> LogManager.getLogManager().addLogger(multiThreadedHttpConnectionManagerLoggerStopped) >> >> Is there another solution to set the logger level permanently ? >> >> Thx. >> -- >> ----- >> Arnaud Héritier >> http://aheritier.net >> Mail/GTalk: aheritier AT gmail DOT com >> Twitter/Skype : aheritier >> > > > > -- > ----- > Arnaud Héritier > http://aheritier.net > Mail/GTalk: aheritier AT gmail DOT com > Twitter/Skype : aheritier > -- ----- Arnaud Héritier http://aheritier.net Mail/GTalk: aheritier AT gmail DOT com Twitter/Skype : aheritier -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
