reboot the server (that host Jenkins), also docker rm exited containers, resolve the issue.
On Tuesday, March 29, 2016 at 5:08:30 PM UTC-7, Gang cheng wrote: > > In short, I think the container pool is full which cause this, but I don't > know how to resolve this. (I tried to restart Jenkins server, it doesn't > help...) > Any advices are appreciated! > > Please see details below. > ============================================ > I have a jenkins server setup to run build jobs on docker. Everything > works fine until today I see a new job is pending for next available > executor. > When I looked into the Jenkins log, I see below logs: > > INFO: Will provision 'xxxxxx', for label: 'xxxxxx', in cloud: 'Docker > Build Cloud' > Mar 29, 2016 4:29:59 PM com.nirima.jenkins.plugins.docker.DockerCloud > addProvisionedSlave > INFO: Not Provisioning 'xxxxxx'; Server 'Docker Build Cloud' full with > '10' container(s) > Mar 29, 2016 4:30:09 PM com.nirima.jenkins.plugins.docker.DockerCloud > provision > INFO: Asked to provision 1 slave(s) for: xxxxxx > ...... > > Also in Manage Jenkins->Dockers->Docker Build Cloud, when I tried to stop > one of the docker container, I got an error like below: > > javax.servlet.ServletException: javax.ws.rs.ProcessingException: > java.net.SocketTimeoutException: Read timed out > at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:796) > at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) > at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:233) > at > org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) > at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) > at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) > at org.kohsuke.stapler.MetaClass$11.dispatch(MetaClass.java:380) > at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) > at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) > at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649) > at org.kohsuke.stapler.Stapler.service(Stapler.java:238) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) > at > org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494) > at > hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:134) > at > hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:59) > at > hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:131) > at > hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:125) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) > at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:49) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) > at > hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) > at > hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) > at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) > at > org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) > at > hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482) > at > org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) > at org.eclipse.jetty.server.Server.handle(Server.java:370) > at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) > at > org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960) > at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) > at > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668) > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) > at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: javax.ws.rs.ProcessingException: java.net.SocketTimeoutException: > Read timed out > at > com.github.dockerjava.jaxrs.connector.ApacheConnector.apply(ApacheConnector.java:490) > at > org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:246) > at > org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:667) > at > org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:664) > at org.glassfish.jersey.internal.Errors.process(Errors.java:315) > at org.glassfish.jersey.internal.Errors.process(Errors.java:297) > at org.glassfish.jersey.internal.Errors.process(Errors.java:228) > at > org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:424) > at > org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:664) > at > org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:424) > at > org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:333) > at > com.github.dockerjava.jaxrs.StopContainerCmdExec.execute(StopContainerCmdExec.java:26) > at > com.github.dockerjava.jaxrs.StopContainerCmdExec.execute(StopContainerCmdExec.java:11) > at > com.github.dockerjava.jaxrs.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:24) > at > com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:33) > at > com.github.dockerjava.core.command.StopContainerCmdImpl.exec(StopContainerCmdImpl.java:67) > at > com.nirima.jenkins.plugins.docker.DockerManagementServer.doControlSubmit(DockerManagementServer.java:65) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:324) > at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:167) > at > org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:100) > at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:124) > at > org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) > at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) > ... 53 more > Caused by: java.net.SocketTimeoutException: Read timed out > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:152) > at java.net.SocketInputStream.read(SocketInputStream.java:122) > at > shaded.org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:136) > at > shaded.org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:152) > at > shaded.org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:270) > at > shaded.org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140) > at > shaded.org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) > at > shaded.org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260) > at > shaded.org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:161) > at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > shaded.org.apache.http.impl.conn.CPoolProxy.invoke(CPoolProxy.java:138) > at com.sun.proxy.$Proxy55.receiveResponseHeader(Unknown Source) > at > shaded.org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271) > at > shaded.org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123) > at > shaded.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:253) > at > shaded.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:194) > at > shaded.org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:85) > at > shaded.org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108) > at > shaded.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186) > at > shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72) > at > com.github.dockerjava.jaxrs.connector.ApacheConnector.apply(ApacheConnector.java:443) > ... 79 more > > > > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/9e5d3749-151d-4ff8-9faa-dadf2d974a96%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
