For packing files (or applications, or anything) as John suggested (#2), I recommend FPM. It's really easy to create an rpm, deb, or whatever quickly without having to learn the arcane options for each package builder. It's not considered suitable for distribution-quality packages but I do not think that is a problem here. https://github.com/jordansissel/fpm
Rob Nelson [email protected] On Wed, Dec 13, 2017 at 11:57 AM, jcbollinger <[email protected]> wrote: > > > On Tuesday, December 12, 2017 at 11:17:20 AM UTC-6, [email protected] > wrote: >> >> >> We have a problem with very slow work puppetserver. >> >> >> We have ~300 nodes, master runs on a server with 24 cores and 20 GB of >> memory. >> >> Pappet agent log: >> [...] >> Time: >> [...] >> File: 4466.81 >> Total: 4810.97 >> Config retrieval: 75.19 >> Service: 8.01 >> Version: >> Config: 1513029567 >> Puppet: 4.4.1 >> >> > Your log shows very long runtime (80+ minutes) for the Puppet *agent*. > That's neither puppetserver nor the master. > > The vast majority of the agent runtime is consumed in syncing File > resources. That tells me that you are syncing an enormous volume of files, > and possibly also a great number of them. You will find numerous previous > discussions of such problems in this group; here are the usual > recommendations: > > 1. Prefer to reserve File resources for smallish numbers of smallish > files. Config files are the sweet spot for this resource type. > 2. Prefer to package files and manage them via Package resources, as > opposed to recursively syncing directories full of files, or otherwise > syncing large numbers of related files. > 3. Avoid syncing temporary files. If you use a File resource to > manage a file, then it should be one that will remain on the system, > because if you remove it (or modify it) then Puppet will just sync it again > on the next run. This can mean leaving a file in place that otherwise you > would remove. > 4. If you must manage a large file via a File resource then consider > specifying a different `checksum` attribute for it. The default is md5, > but 'md5lite' will be faster and still give an ok test for modification. > Or you can even go with 'mtime', which is very fast, but is susceptible to > both false positives and false negatives. > > > Your log also shows a longish runtime (75 seconds) for catalog retrieval. > The `stat()` calls in your strace suggest that you're using the 'eyaml' > Hiera back end, and this does add overhead. If you're storing all your > Hiera data in the eyaml back end, then it may add a lot of overhead. I > suggest using eyaml only for those data that actually need to be kept in > confidence, and among them, only those for which the general access > controls provided by the master's system are insufficient. Configure the > standard back end too, and use it for the rest of your data. There may be > other issues, too, such as the system load from all those File transfers, > but I see no details that point me to specific server-side problems. > > > John > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/puppet-users/4f0e88f7-2f2a-4c86-9a22-e1e072879c95%40googlegroups.com > <https://groups.google.com/d/msgid/puppet-users/4f0e88f7-2f2a-4c86-9a22-e1e072879c95%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAC76iT_XrMwZWLRrNCVmKrJ4rux8%3DTE1e4aAQed%3DL%3DVRJ6P8dw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
