For packing files (or applications, or anything) as John suggested (#2), I recommend FPM. It's really easy to create an rpm, deb, or whatever quickly without having to learn the arcane options for each package builder. It's not considered suitable for distribution-quality packages but I do not think that is a problem here. https://github.com/jordansissel/fpm
Rob Nelson rnels...@gmail.com On Wed, Dec 13, 2017 at 11:57 AM, jcbollinger <john.bollin...@stjude.org> wrote: > > > On Tuesday, December 12, 2017 at 11:17:20 AM UTC-6, n.be...@gaijin.ru > wrote: >> >> >> We have a problem with very slow work puppetserver. >> >> >> We have ~300 nodes, master runs on a server with 24 cores and 20 GB of >> memory. >> >> Pappet agent log: >> [...] >> Time: >> [...] >> File: 4466.81 >> Total: 4810.97 >> Config retrieval: 75.19 >> Service: 8.01 >> Version: >> Config: 1513029567 >> Puppet: 4.4.1 >> >> > Your log shows very long runtime (80+ minutes) for the Puppet *agent*. > That's neither puppetserver nor the master. > > The vast majority of the agent runtime is consumed in syncing File > resources. That tells me that you are syncing an enormous volume of files, > and possibly also a great number of them. You will find numerous previous > discussions of such problems in this group; here are the usual > recommendations: > > 1. Prefer to reserve File resources for smallish numbers of smallish > files. Config files are the sweet spot for this resource type. > 2. Prefer to package files and manage them via Package resources, as > opposed to recursively syncing directories full of files, or otherwise > syncing large numbers of related files. > 3. Avoid syncing temporary files. If you use a File resource to > manage a file, then it should be one that will remain on the system, > because if you remove it (or modify it) then Puppet will just sync it again > on the next run. This can mean leaving a file in place that otherwise you > would remove. > 4. If you must manage a large file via a File resource then consider > specifying a different `checksum` attribute for it. The default is md5, > but 'md5lite' will be faster and still give an ok test for modification. > Or you can even go with 'mtime', which is very fast, but is susceptible to > both false positives and false negatives. > > > Your log also shows a longish runtime (75 seconds) for catalog retrieval. > The `stat()` calls in your strace suggest that you're using the 'eyaml' > Hiera back end, and this does add overhead. If you're storing all your > Hiera data in the eyaml back end, then it may add a lot of overhead. I > suggest using eyaml only for those data that actually need to be kept in > confidence, and among them, only those for which the general access > controls provided by the master's system are insufficient. Configure the > standard back end too, and use it for the rest of your data. There may be > other issues, too, such as the system load from all those File transfers, > but I see no details that point me to specific server-side problems. > > > John > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/puppet-users/4f0e88f7-2f2a-4c86-9a22-e1e072879c95%40googlegroups.com > <https://groups.google.com/d/msgid/puppet-users/4f0e88f7-2f2a-4c86-9a22-e1e072879c95%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAC76iT_XrMwZWLRrNCVmKrJ4rux8%3DTE1e4aAQed%3DL%3DVRJ6P8dw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
