Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-23 Thread Ken Barber
So this problem is tracked here: https://tickets.puppetlabs.com/browse/PDB-686. I've been unable to find a work-around beyond 'switch ruby' or completely change the way we submit JSON (which would be a shame, the new method is more efficient). Some people have had success using the newer brightbox

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-22 Thread Ken Barber
Sergey, Are you by any chance running Ubuntu 12.04 with the 3 year old Ruby 1.9.3-p0? If not what distro and exact ruby version are you running? ken. On Thu, May 22, 2014 at 5:06 AM, Sergey Arlashin sergeyarl.maill...@gmail.com wrote: Yesterday I downgraded both puppet to 3.5.1 and pupetdb to

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-22 Thread Ken Barber
I've been able to replicate this now for those following along at home. It seems to be that Ubuntu 12.04 and ruby-1.9.3-p0 exhibits this bug, which on the surface seems to be a bug in calculating the Content-Length for the POST submission header, but I haven't completely confirmed this. Updating

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-22 Thread Sergey Arlashin
On May 22, 2014, at 4:54 PM, Ken Barber k...@puppetlabs.com wrote: Sergey, Are you by any chance running Ubuntu 12.04 with the 3 year old Ruby 1.9.3-p0? If not what distro and exact ruby version are you running? Problem client: # cat /etc/issue Ubuntu 12.04.4 LTS \n \l # dpkg -l |grep

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-21 Thread Ken Barber
Catalogue size might be a factor, as templates get stored in the catalogue. On Wed, May 21, 2014 at 2:45 PM, Phil Fenstermacher phillip.fenstermac...@gmail.com wrote: I encountered this same issue yesterday, and unfortunately haven't found a fix either. The error seems to be happening

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-21 Thread Ken Barber
So one thing I'm sure about - is that there was no idle timeout defined in 1.6.3 at all. I can connect idle for quite some time for example. Its quite possible that there is another delay/lag somewhere causing the connection to be idle in the first place - but with 1.6.3 this would not have

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-21 Thread Ken Barber
Phil, Do you happen to have a tcpdump I can look at, one where this problem is replicated? I know its SSL - but I'm primarily interested in seeing when (and _IF_) the network connection goes idle - so packet timing is more important to me then content. I'm trying to prove/disprove that

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-21 Thread Ken Barber
Phil and I have been conversing offline ... we found that downgrading the terminus to 1.6.3 (not PuppetDB) fixes the issue. So far we can see that a submission occurs but the server does not respond. This is synonymous with a bad Content-Length, and certainly I'm able to replicate this with some

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-21 Thread Sergey Arlashin
Yesterday I downgraded both puppet to 3.5.1 and pupetdb to 1.6.3, and the problem disappeared. Unfortunately this is production infrastructure and I don't have an ability to test it further :( On May 21, 2014, at 11:54 PM, Ken Barber k...@puppetlabs.com wrote: Phil and I have been

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-20 Thread Sergey Arlashin
Here is described http://www.eclipse.org/jetty/documentation/current/configuring-connectors.html how idle timeout for jetty cat be configured . But I don't know how to apply this for puppetdb :( Does anybody have any ideas ? On May 20, 2014, at 9:43 AM, Sergey Arlashin

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-20 Thread Ken Barber
We don't expose the idle timeout parameter for users to modify. I am curious though, what happens when you increase max-threads in your jetty.ini to something like 200? ken. On Tue, May 20, 2014 at 12:41 PM, Sergey Arlashin sergeyarl.maill...@gmail.com wrote: Here is described

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-20 Thread Ken Barber
Also ... I'd be curious to know what exact version of the JVM you are running. I'm cautious this is a jetty bug of some kind, perhaps we can ship a new Jar with a later version of Jetty to you and see if that helps. ken. On Tue, May 20, 2014 at 2:03 PM, Ken Barber k...@puppetlabs.com wrote: We

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-20 Thread Sergey Arlashin
Unfortunately setting max-threads to 200 did not help. # java -version java version 1.7.0_55 OpenJDK Runtime Environment (IcedTea 2.4.7) (7u55-2.4.7-1ubuntu1~0.12.04.2) OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode) # dpkg -l |grep -i java ii ca-certificates-java

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-19 Thread Spencer Krum
This looks like a puppetdb error. Can you check the PuppetDB logs and post any relevant errors? Thanks, Spencer On Mon, May 19, 2014 at 10:02 PM, Sergey Arlashin sergeyarl.maill...@gmail.com wrote: Hi! Every now and then I get the following error while running puppet agent. Error: Could

Re: [Puppet Users] puppetdb 2 : Idle timeout expired: 30000/30000 ms

2014-05-19 Thread Sergey Arlashin
2014-05-20 05:34:36,684 INFO [c.p.p.command] [e806b0a2-7703-4a4a-8107-65cd2c0db9a8] [replace facts] prod1.site 2014-05-20 05:34:44,112 INFO [c.p.p.command] [e8cb3511-9734-4ba7-b040-96f663404c00] [replace facts] redmine.site 2014-05-20 05:34:45,387 INFO [c.p.p.command]