Did puppet hit a ulimit?
On Wed, Apr 30, 2014 at 8:44 AM, jcbollinger <[email protected]>wrote: > > > On Tuesday, April 29, 2014 2:53:18 PM UTC-5, Ádám Sándor wrote: >> >> I managed to solve the problem :) First we added more memory to the >> machine. With 1GB of memory the script ran further but still failed. In >> this case it became apparent that it's doing something with file >> permissions. It was setting group 70 (don't even know what kind of group >> that is) to some scripts it was supposed to copy for execution. It was >> doing this very slowly - tens of seconds for each file - and eating more >> and more memory in the process, until it reached 1GB and the vserver killed >> it. >> >> > > When you say you fixed the problem, do you mean that the new failure > occurs while applying a different resource? > > > >> The fix was very simple, but I'm still intrigued as to what went wrong, >> especially since it was such a hard-to-debug issue. I changed this resource: >> file { 'migration-scripts': >> path => $scriptPath, >> ensure => directory, >> source => 'puppet:///modules/ks/migration', >> recurse => true, >> owner => 'root' >> } >> >> to this: >> file { 'migration-scripts': >> path => $scriptPath, >> ensure => directory, >> source => 'puppet:///modules/ks/migration', >> recurse => true >> } >> >> If someone can give an explanation I would be very happy to read it. Also >> because I only got 1GB of memory on the server temporarily and I'm >> suspecting another similar problem in my manifest since with 512GB memory >> the execution didn't get to the part where I now spotted the group >> permission setting. >> >> > > As Felix said, this is not generally expected or documented behavior. The > best we can do is guess, and we're working with incomplete information when > it comes to that. I observe, however, that recursive File resources can be > a source of trouble. They do work, but they require memory on the agent > proportional to the total number of files, plus a bit. Also, if your > recursive File is following links ('links' parameter) then it is possible > to give it a logically infinite directory tree to attempt to sync. You > don't specify 'links' in your example, and I don't recall offhand what the > default behavior is in that regard, but it's something to watch out for. > > In addition, File resources in general are expensive when the managed > content is large and you use the default checksum algorithm. The choice of > algorithm is a trade-off between expense of the test and likelihood of a > wrong result, and Puppet defaults to the least likelihood of a wrong result. > > If you are managing large collections of files or file collections having > large aggregate size, then you would be better off avoiding recursive File > resources. The usual recommendation is to bundle up the files into a > package (e.g. an RPM or DEB), put that in a local repository, and manage it > via a Package resource. > > As for the group assigned to newly-created files, it is probably the > (numeric) group the source files have on the master. If you don't specify > a group for a source'd file, and that file does not initially exist on the > target node, then the group assigned to it at creation time is the one the > source file has. Similarly for the file owner. If that's indeed what you > are seeing then that aspect of the observed behavior is as expected. (In > contrast, if you don't specify the group (or owner), and the file does > already exist, then Puppet does not modify its group (or owner).) > > I cannot explain why declining to manage the file owner makes a > significant difference in memory consumption, but it's reasonable to > suppose that the large memory consumption is a factor in the poor > performance. The 'physical' memory assigned to your virtual machine is not > necessarily matched with real physical memory on the underlying vserver > host. Therefore, processes in the VM that consume a lot of memory may > cause a lot of page faults on the host, with the attending performance > impact. The negative performance effects of such page faulting might even > be magnified in the VM. > > > John > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-users/569c0062-4372-4428-a865-51e0d587b127%40googlegroups.com<https://groups.google.com/d/msgid/puppet-users/569c0062-4372-4428-a865-51e0d587b127%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAEx7SAyjnwC1wVZ%3DuGAf%3DB5jkz8SN9Eu4AED5iBGQ73NeS%3Df_A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
