We don't expose the idle timeout parameter for users to modify. I am curious though, what happens when you increase max-threads in your jetty.ini to something like 200?
ken. On Tue, May 20, 2014 at 12:41 PM, Sergey Arlashin <sergeyarl.maill...@gmail.com> wrote: > Here is described > http://www.eclipse.org/jetty/documentation/current/configuring-connectors.html > how idle timeout for jetty cat be configured . But I don't know how to apply > this for puppetdb :( > > Does anybody have any ideas ? > > > > On May 20, 2014, at 9:43 AM, Sergey Arlashin <sergeyarl.maill...@gmail.com> > wrote: > > 2014-05-20 05:34:36,684 INFO [c.p.p.command] > [e806b0a2-7703-4a4a-8107-65cd2c0db9a8] [replace facts] prod1.site > 2014-05-20 05:34:44,112 INFO [c.p.p.command] > [e8cb3511-9734-4ba7-b040-96f663404c00] [replace facts] redmine.site > 2014-05-20 05:34:45,387 INFO [c.p.p.command] > [9026d779-3a43-4c06-a23d-aeefd4eb6e1f] [replace catalog] redmine.site > 2014-05-20 05:34:47,487 INFO [c.p.p.command] > [1234e95c-e254-4b1d-b46f-8ef5b07e9158] [replace facts] www-node1.site > 2014-05-20 05:34:48,350 INFO [c.p.p.command] > [56be0dc9-03b9-45b8-8b8a-527f1c25e057] [replace facts] dev01.site > 2014-05-20 05:34:48,404 INFO [c.p.p.command] > [17474841-6043-46cf-98a1-122f116d5cd6] [replace facts] parser1.site > 2014-05-20 05:34:48,634 INFO [c.p.p.command] > [44da2889-811f-4e80-8190-ad7313f0ca81] [replace facts] dev02.site > 2014-05-20 05:34:48,942 INFO [c.p.p.command] > [4080aa24-23c7-4f16-9011-2104a457e201] [replace catalog] www-node1.site > 2014-05-20 05:34:51,429 INFO [c.p.p.command] > [6b62c283-f306-44e7-8101-4948110e3d78] [replace catalog] dev01.site > 2014-05-20 05:34:51,437 INFO [c.p.p.command] > [55fb7348-3b36-4c5a-9265-61ca3c1f55c3] [replace catalog] dev02.site > 2014-05-20 05:34:52,925 INFO [c.p.p.command] > [9ace137e-f708-4656-829a-92de7722b388] [replace facts] lb-node1.site > 2014-05-20 05:34:55,017 INFO [c.p.p.command] > [0b6e6592-97f2-4b8a-8cc7-6d321793d2ed] [replace catalog] lb-node1.site > 2014-05-20 05:34:58,285 INFO [c.p.p.command] > [971285dd-b141-4f44-a6d0-375aa3af830a] [replace facts] lb-node2.site > 2014-05-20 05:34:59,333 INFO [c.p.p.command] > [3977fc34-aa41-43e7-8eb9-ef5398dc3567] [replace facts] www-node2.site > 2014-05-20 05:35:01,625 INFO [c.p.p.command] > [f095ef50-4b71-41d6-ba83-6dd4a0030fe2] [replace facts] parser2.site > 2014-05-20 05:35:02,107 INFO [c.p.p.command] > [539a78de-1743-4135-adeb-c5ed3b43cc24] [replace catalog] lb-node2.site > 2014-05-20 05:35:02,379 INFO [c.p.p.command] > [f4b7fc4d-cccd-4fb9-881e-da4007d57f6e] [replace facts] monitor.site > 2014-05-20 05:35:03,945 INFO [c.p.p.command] > [f49a6fcd-ca63-49eb-bbec-7712ba590675] [replace catalog] www-node2.site > 2014-05-20 05:35:05,350 INFO [c.p.p.command] > [6913aceb-0d60-4701-9d31-d84af8f92374] [replace facts] db-node1.site > 2014-05-20 05:35:05,587 INFO [c.p.p.command] > [bd9796df-e989-4788-addc-d31f2d2dda3a] [replace catalog] monitor.site > 2014-05-20 05:35:05,848 INFO [c.p.p.command] > [f9215b15-f646-403a-b4d3-21fd532e1c06] [replace facts] puppet.site > 2014-05-20 05:35:05,871 INFO [c.p.p.command] > [7820d545-1c5f-4b0f-a2e8-0dcbc12d4571] [replace facts] db-node2.site > 2014-05-20 05:35:07,642 WARN [o.e.j.s.HttpChannel] > /v3/commands?checksum=e51aa526e2236306cff22ef33c8cb4467166f9c0 > java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout > expired: 30002/30000 ms > at org.eclipse.jetty.util.BlockingCallback.block(BlockingCallback.java:101) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.HttpInputOverHTTP.blockForContent(HttpInputOverHTTP.java:62) > ~[puppetdb.jar:na] > at org.eclipse.jetty.server.HttpInput$1.waitForContent(HttpInput.java:392) > ~[puppetdb.jar:na] > at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:161) > ~[puppetdb.jar:na] > at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) ~[na:1.7.0_55] > at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) ~[na:1.7.0_55] > at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) ~[na:1.7.0_55] > at java.io.InputStreamReader.read(InputStreamReader.java:184) ~[na:1.7.0_55] > at java.io.BufferedReader.fill(BufferedReader.java:154) ~[na:1.7.0_55] > at java.io.BufferedReader.read(BufferedReader.java:175) ~[na:1.7.0_55] > at clojure.core$slurp.doInvoke(core.clj:6396) ~[puppetdb.jar:na] > at clojure.lang.RestFn.invoke(RestFn.java:410) ~[puppetdb.jar:na] > at ring.util.request$eval10113$fn__10114.invoke(request.clj:30) ~[na:na] > at clojure.lang.MultiFn.invoke(MultiFn.java:227) ~[puppetdb.jar:na] > at > com.puppetlabs.middleware$payload_to_body_string$fn__10429.invoke(middleware.clj:273) > ~[na:na] > at > com.puppetlabs.middleware$verify_content_type$fn__10379.invoke(middleware.clj:128) > ~[na:na] > at com.puppetlabs.puppetdb.http.v3$v3_app$fn__11827.invoke(v3.clj:23) > ~[na:na] > at com.puppetlabs.puppetdb.http.v3$v3_app.invoke(v3.clj:21) ~[na:na] > at > com.puppetlabs.puppetdb.http.server$routes$fn__12370.invoke(server.clj:47) > ~[na:na] > at net.cgrand.moustache$alter_request$fn__10771.invoke(moustache.clj:54) > ~[na:na] > at com.puppetlabs.puppetdb.http.server$routes.invoke(server.clj:47) ~[na:na] > at ring.middleware.resource$wrap_resource$fn__12339.invoke(resource.clj:24) > ~[na:na] > at ring.middleware.params$wrap_params$fn__10714.invoke(params.clj:58) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_authorization$fn__10346.invoke(middleware.clj:40) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_certificate_cn$fn__10350.invoke(middleware.clj:55) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_default_body$fn__10354.invoke(middleware.clj:62) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__10401$fn__10410.invoke(middleware.clj:207) > ~[na:na] > at > com.puppetlabs.utils.metrics$multitime_BANG__STAR_$fn__10068$fn__10069$fn__10070.invoke(metrics.clj:14) > ~[na:na] > at > com.puppetlabs.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown > Source) ~[na:na] > at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] > at > com.puppetlabs.utils.metrics$multitime_BANG__STAR_$fn__10068$fn__10069.invoke(metrics.clj:14) > ~[na:na] > at > com.puppetlabs.utils.metrics$multitime_BANG__STAR_$fn__10068$fn__10069$fn__10070.invoke(metrics.clj:14) > ~[na:na] > at > com.puppetlabs.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown > Source) ~[na:na] > at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] > at > com.puppetlabs.utils.metrics$multitime_BANG__STAR_$fn__10068$fn__10069.invoke(metrics.clj:14) > ~[na:na] > at com.puppetlabs.utils.metrics$multitime_BANG__STAR_.invoke(metrics.clj:17) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__10401.invoke(middleware.clj:206) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_globals$fn__10358.invoke(middleware.clj:73) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_debug_logging$fn__10342.invoke(middleware.clj:27) > ~[na:na] > at > puppetlabs.trapperkeeper.services.webserver.jetty9_core$proxy_handler$fn__16765.invoke(jetty9_core.clj:66) > ~[na:na] > at > puppetlabs.trapperkeeper.services.webserver.jetty9_core.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown > Source) ~[na:na] > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1112) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1048) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:199) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109) > ~[puppetdb.jar:na] > at org.eclipse.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:325) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) > ~[puppetdb.jar:na] > at org.eclipse.jetty.server.Server.handle(Server.java:459) > ~[puppetdb.jar:na] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:280) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:229) > [puppetdb.jar:na] > at > org.eclipse.jetty.io.AbstractConnection$1.run(AbstractConnection.java:505) > [puppetdb.jar:na] > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607) > [puppetdb.jar:na] > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536) > [puppetdb.jar:na] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55] > Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: > 30002/30000 ms > at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:153) > ~[puppetdb.jar:na] > at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50) > ~[puppetdb.jar:na] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > ~[na:1.7.0_55] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_55] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) > ~[na:1.7.0_55] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) > ~[na:1.7.0_55] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_55] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ~[na:1.7.0_55] > ... 1 common frames omitted > > > > On May 20, 2014, at 9:38 AM, Spencer Krum <krum.spen...@gmail.com> wrote: > > This looks like a puppetdb error. Can you check the PuppetDB logs and post > any relevant errors? > > Thanks, > Spencer > > > On Mon, May 19, 2014 at 10:02 PM, Sergey Arlashin > <sergeyarl.maill...@gmail.com> wrote: >> >> Hi! >> >> Every now and then I get the following error while running puppet agent. >> >> Error: Could not retrieve catalog from remote server: Error 400 on SERVER: >> Failed to submit 'replace catalog' command for prod2.site to PuppetDB at >> puppet.site:8081: [500 java.util.concurrent.TimeoutException: Idle timeout >> expired: 30000/30000 ms] <html><head><meta http-equiv="Content-Type" >> content="text/html;charset=ISO-8859-1"/><title>Error 500 >> </title></head><body><h2>HTTP ERROR: 500</h2><p>Problem accessing >> /v3/commands. Reason:<pre> java.util.concurrent.TimeoutException: Idle >> timeout expired: 30000/30000 ms</pre></p><hr /><i><small>Powered by >> Jetty://</small></i></body></html> >> >> Does anybody know how I can get rid of this error? >> >> Thanks in advance! >> >> >> -- >> Best regards, >> Sergey Arlashin >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to puppet-users+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-users/8A2FC518-6B42-42D5-8524-12D39B94DBD4%40gmail.com. >> For more options, visit https://groups.google.com/d/optout. > > > > > -- > Spencer Krum > (619)-980-7820 > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-users/CADt6FWNbXASsOh6_hzcSQAeJjzu8cZYTPG4qkYxTBoe%2B6heLtw%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. > > > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-users/FFF9F6EE-8B3A-48E4-A78B-42932EF41A3B%40gmail.com. > > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNTk07HmJ_M8g1ESWwVkoxQcE-qPfD%3D%2B3wUBKVGr3HTzhrw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
