Here is described http://www.eclipse.org/jetty/documentation/current/configuring-connectors.html how idle timeout for jetty cat be configured . But I don't know how to apply this for puppetdb :(
Does anybody have any ideas ? On May 20, 2014, at 9:43 AM, Sergey Arlashin <[email protected]> wrote: > 2014-05-20 05:34:36,684 INFO [c.p.p.command] > [e806b0a2-7703-4a4a-8107-65cd2c0db9a8] [replace facts] prod1.site > 2014-05-20 05:34:44,112 INFO [c.p.p.command] > [e8cb3511-9734-4ba7-b040-96f663404c00] [replace facts] redmine.site > 2014-05-20 05:34:45,387 INFO [c.p.p.command] > [9026d779-3a43-4c06-a23d-aeefd4eb6e1f] [replace catalog] redmine.site > 2014-05-20 05:34:47,487 INFO [c.p.p.command] > [1234e95c-e254-4b1d-b46f-8ef5b07e9158] [replace facts] www-node1.site > 2014-05-20 05:34:48,350 INFO [c.p.p.command] > [56be0dc9-03b9-45b8-8b8a-527f1c25e057] [replace facts] dev01.site > 2014-05-20 05:34:48,404 INFO [c.p.p.command] > [17474841-6043-46cf-98a1-122f116d5cd6] [replace facts] parser1.site > 2014-05-20 05:34:48,634 INFO [c.p.p.command] > [44da2889-811f-4e80-8190-ad7313f0ca81] [replace facts] dev02.site > 2014-05-20 05:34:48,942 INFO [c.p.p.command] > [4080aa24-23c7-4f16-9011-2104a457e201] [replace catalog] www-node1.site > 2014-05-20 05:34:51,429 INFO [c.p.p.command] > [6b62c283-f306-44e7-8101-4948110e3d78] [replace catalog] dev01.site > 2014-05-20 05:34:51,437 INFO [c.p.p.command] > [55fb7348-3b36-4c5a-9265-61ca3c1f55c3] [replace catalog] dev02.site > 2014-05-20 05:34:52,925 INFO [c.p.p.command] > [9ace137e-f708-4656-829a-92de7722b388] [replace facts] lb-node1.site > 2014-05-20 05:34:55,017 INFO [c.p.p.command] > [0b6e6592-97f2-4b8a-8cc7-6d321793d2ed] [replace catalog] lb-node1.site > 2014-05-20 05:34:58,285 INFO [c.p.p.command] > [971285dd-b141-4f44-a6d0-375aa3af830a] [replace facts] lb-node2.site > 2014-05-20 05:34:59,333 INFO [c.p.p.command] > [3977fc34-aa41-43e7-8eb9-ef5398dc3567] [replace facts] www-node2.site > 2014-05-20 05:35:01,625 INFO [c.p.p.command] > [f095ef50-4b71-41d6-ba83-6dd4a0030fe2] [replace facts] parser2.site > 2014-05-20 05:35:02,107 INFO [c.p.p.command] > [539a78de-1743-4135-adeb-c5ed3b43cc24] [replace catalog] lb-node2.site > 2014-05-20 05:35:02,379 INFO [c.p.p.command] > [f4b7fc4d-cccd-4fb9-881e-da4007d57f6e] [replace facts] monitor.site > 2014-05-20 05:35:03,945 INFO [c.p.p.command] > [f49a6fcd-ca63-49eb-bbec-7712ba590675] [replace catalog] www-node2.site > 2014-05-20 05:35:05,350 INFO [c.p.p.command] > [6913aceb-0d60-4701-9d31-d84af8f92374] [replace facts] db-node1.site > 2014-05-20 05:35:05,587 INFO [c.p.p.command] > [bd9796df-e989-4788-addc-d31f2d2dda3a] [replace catalog] monitor.site > 2014-05-20 05:35:05,848 INFO [c.p.p.command] > [f9215b15-f646-403a-b4d3-21fd532e1c06] [replace facts] puppet.site > 2014-05-20 05:35:05,871 INFO [c.p.p.command] > [7820d545-1c5f-4b0f-a2e8-0dcbc12d4571] [replace facts] db-node2.site > 2014-05-20 05:35:07,642 WARN [o.e.j.s.HttpChannel] > /v3/commands?checksum=e51aa526e2236306cff22ef33c8cb4467166f9c0 > java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout > expired: 30002/30000 ms > at > org.eclipse.jetty.util.BlockingCallback.block(BlockingCallback.java:101) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.HttpInputOverHTTP.blockForContent(HttpInputOverHTTP.java:62) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.HttpInput$1.waitForContent(HttpInput.java:392) > ~[puppetdb.jar:na] > at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:161) > ~[puppetdb.jar:na] > at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) > ~[na:1.7.0_55] > at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) > ~[na:1.7.0_55] > at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) ~[na:1.7.0_55] > at java.io.InputStreamReader.read(InputStreamReader.java:184) > ~[na:1.7.0_55] > at java.io.BufferedReader.fill(BufferedReader.java:154) ~[na:1.7.0_55] > at java.io.BufferedReader.read(BufferedReader.java:175) ~[na:1.7.0_55] > at clojure.core$slurp.doInvoke(core.clj:6396) ~[puppetdb.jar:na] > at clojure.lang.RestFn.invoke(RestFn.java:410) ~[puppetdb.jar:na] > at ring.util.request$eval10113$fn__10114.invoke(request.clj:30) ~[na:na] > at clojure.lang.MultiFn.invoke(MultiFn.java:227) ~[puppetdb.jar:na] > at > com.puppetlabs.middleware$payload_to_body_string$fn__10429.invoke(middleware.clj:273) > ~[na:na] > at > com.puppetlabs.middleware$verify_content_type$fn__10379.invoke(middleware.clj:128) > ~[na:na] > at com.puppetlabs.puppetdb.http.v3$v3_app$fn__11827.invoke(v3.clj:23) > ~[na:na] > at com.puppetlabs.puppetdb.http.v3$v3_app.invoke(v3.clj:21) ~[na:na] > at > com.puppetlabs.puppetdb.http.server$routes$fn__12370.invoke(server.clj:47) > ~[na:na] > at > net.cgrand.moustache$alter_request$fn__10771.invoke(moustache.clj:54) ~[na:na] > at com.puppetlabs.puppetdb.http.server$routes.invoke(server.clj:47) > ~[na:na] > at > ring.middleware.resource$wrap_resource$fn__12339.invoke(resource.clj:24) > ~[na:na] > at ring.middleware.params$wrap_params$fn__10714.invoke(params.clj:58) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_authorization$fn__10346.invoke(middleware.clj:40) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_certificate_cn$fn__10350.invoke(middleware.clj:55) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_default_body$fn__10354.invoke(middleware.clj:62) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__10401$fn__10410.invoke(middleware.clj:207) > ~[na:na] > at > com.puppetlabs.utils.metrics$multitime_BANG__STAR_$fn__10068$fn__10069$fn__10070.invoke(metrics.clj:14) > ~[na:na] > at > com.puppetlabs.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown > Source) ~[na:na] > at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] > at > com.puppetlabs.utils.metrics$multitime_BANG__STAR_$fn__10068$fn__10069.invoke(metrics.clj:14) > ~[na:na] > at > com.puppetlabs.utils.metrics$multitime_BANG__STAR_$fn__10068$fn__10069$fn__10070.invoke(metrics.clj:14) > ~[na:na] > at > com.puppetlabs.utils.metrics.proxy$java.lang.Object$Callable$7da976d4.call(Unknown > Source) ~[na:na] > at com.yammer.metrics.core.Timer.time(Timer.java:91) ~[puppetdb.jar:na] > at > com.puppetlabs.utils.metrics$multitime_BANG__STAR_$fn__10068$fn__10069.invoke(metrics.clj:14) > ~[na:na] > at > com.puppetlabs.utils.metrics$multitime_BANG__STAR_.invoke(metrics.clj:17) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_metrics_STAR_$fn__10401.invoke(middleware.clj:206) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_globals$fn__10358.invoke(middleware.clj:73) > ~[na:na] > at > com.puppetlabs.middleware$wrap_with_debug_logging$fn__10342.invoke(middleware.clj:27) > ~[na:na] > at > puppetlabs.trapperkeeper.services.webserver.jetty9_core$proxy_handler$fn__16765.invoke(jetty9_core.clj:66) > ~[na:na] > at > puppetlabs.trapperkeeper.services.webserver.jetty9_core.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown > Source) ~[na:na] > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1112) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1048) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:199) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:325) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) > ~[puppetdb.jar:na] > at org.eclipse.jetty.server.Server.handle(Server.java:459) > ~[puppetdb.jar:na] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:280) > ~[puppetdb.jar:na] > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:229) > [puppetdb.jar:na] > at > org.eclipse.jetty.io.AbstractConnection$1.run(AbstractConnection.java:505) > [puppetdb.jar:na] > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607) > [puppetdb.jar:na] > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536) > [puppetdb.jar:na] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55] > Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: > 30002/30000 ms > at > org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:153) > ~[puppetdb.jar:na] > at org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50) > ~[puppetdb.jar:na] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > ~[na:1.7.0_55] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > ~[na:1.7.0_55] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) > ~[na:1.7.0_55] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) > ~[na:1.7.0_55] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > ~[na:1.7.0_55] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > ~[na:1.7.0_55] > ... 1 common frames omitted > > > > On May 20, 2014, at 9:38 AM, Spencer Krum <[email protected]> wrote: > >> This looks like a puppetdb error. Can you check the PuppetDB logs and post >> any relevant errors? >> >> Thanks, >> Spencer >> >> >> On Mon, May 19, 2014 at 10:02 PM, Sergey Arlashin >> <[email protected]> wrote: >> Hi! >> >> Every now and then I get the following error while running puppet agent. >> >> Error: Could not retrieve catalog from remote server: Error 400 on SERVER: >> Failed to submit 'replace catalog' command for prod2.site to PuppetDB at >> puppet.site:8081: [500 java.util.concurrent.TimeoutException: Idle timeout >> expired: 30000/30000 ms] <html><head><meta http-equiv="Content-Type" >> content="text/html;charset=ISO-8859-1"/><title>Error 500 >> </title></head><body><h2>HTTP ERROR: 500</h2><p>Problem accessing >> /v3/commands. Reason:<pre> java.util.concurrent.TimeoutException: Idle >> timeout expired: 30000/30000 ms</pre></p><hr /><i><small>Powered by >> Jetty://</small></i></body></html> >> >> Does anybody know how I can get rid of this error? >> >> Thanks in advance! >> >> >> -- >> Best regards, >> Sergey Arlashin >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-users/8A2FC518-6B42-42D5-8524-12D39B94DBD4%40gmail.com. >> For more options, visit https://groups.google.com/d/optout. >> >> >> >> -- >> Spencer Krum >> (619)-980-7820 >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-users/CADt6FWNbXASsOh6_hzcSQAeJjzu8cZYTPG4qkYxTBoe%2B6heLtw%40mail.gmail.com. >> For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/FFF9F6EE-8B3A-48E4-A78B-42932EF41A3B%40gmail.com. For more options, visit https://groups.google.com/d/optout.
