Re: [Puppet Users] Puppet Packet Rate?

[Ramin K]

On 7/7/17 9:52 AM, Peter Berghold wrote:
Has anybody out there done any sort of study on what Puppet produces in 
terms of I/O packet rate?  I'm being asked to fill in a spreadsheet with 
that information


I can share some rough numbers. We do a lot of files so the numbers are 
weird and inefficient. Puppet 3.8.7/Passenger 5.1.5/Ruby 2.0/Apache 2.4 
on the server side.


avg file metadata per puppetserver: 25k/s
avg hosts per puppetserver: 1200

avg catalog size: 15MB (lots and lots of files)

Peak TX rate: 4MB/s
Peak RX rate: 1.5MB/s

I'd suspect the average Puppetserver to be a fraction of that bandwidth 
for the same number of hosts.


Ramin

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/5ca353eb-2548-c4c8-89fc-d4011b2847cd%40badapple.net.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: Puppet Rolling Repos Available

[Melissa Stone]

Hi all!

If you haven’t heard of our new Puppet 5 repos, check out the announcement
eric0 sent out here

or read more about the Puppet 5 Platform on our website
.

I’m excited to announce a new and easy way for you to stay on the cutting
edge of all our major releases. These are the new rolling Puppet repos.
These rolling repos are links to the latest versioned Puppet repo we have
available. Right now, that is the Puppet 5 repos. We’ll update these links
for each major release moving forward. This option is for those of you who
want to be at the forefront of all our releases. If you want to lock into a
specific major version series, install the release package that corresponds
to the series you want. However, we want to encourage everyone to use the
rolling repos. We want them to be the default way people find and download
Puppet. This is the best way to stay up to date on all security updates,
new features, and performance improvements.

Please note that you cannot have both the puppet5-release and
puppet-release packages installed at the same time. The same will hold true
for all future versioned Puppet release packages. If you would like to
enable the rolling repos but you already have the puppet5 repos enabled,
you will have to remove the puppet5-release packages.


Apt-Based Systems

# If you already have the puppet5 repos enabled, remove that package
sudo dpkg -P puppet5-release

# Install the puppet repos
wget http://apt.puppetlabs.com/puppet-release-xenial.deb
sudo dpkg -i puppet-release-xenial.deb
sudo apt-get update


Yum-Based Systems
-
# If you already have the puppet5 repos enabled, remove that package
sudo rpm -ev puppet5-release

# Install the puppet repos
sudo rpm -Uvh
http://yum.puppetlabs.com/puppet/puppet-release-el-7.noarch.rpm
-- 
*Melissa Stone*
release engineering
meli...@puppet.com | @melissaapiedra

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAHEe_kqXvdt%2B1GVHWGpBAMBhSDkEYJBY3sgyGfZxq%3Dx6ptDBrw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Puppet Packet Rate?

[Christopher Wood]

The answer is that it depends on your catalogs and what you're ramming through 
them. You might get a squinty rule of thumb measurement from figuring out the 
size of uploaded facts/report, downloaded catalog/files, add some more for 
random discussion traffic, divide by the number of seconds an agent run 
allegedly takes.

That said, you sound like you're living an even more corporate life than I am, 
and they might not take an answer of "it depends what you make puppet do".

You might also do a packet capture or some hypervisor i/o monitoring during an 
agent run, see how much that is and divide by the agent run time.

And then your numbers may turn out fictional when somebody decides they're 
going to ram tarballs in via file resource.

On Fri, Jul 07, 2017 at 04:52:02PM +, Peter Berghold wrote:
>Has anybody out there done any sort of study on what Puppet produces in
>terms of I/O packet rate?  I'm being asked to fill in a spreadsheet with
>that information
> 
>--
>You received this message because you are subscribed to the Google Groups
>"Puppet Users" group.
>To unsubscribe from this group and stop receiving emails from it, send an
>email to [1]puppet-users+unsubscr...@googlegroups.com.
>To view this discussion on the web visit
>
> [2]https://groups.google.com/d/msgid/puppet-users/CAArvnv3G8Lbe8GL11SGC251YYn-SjUs-KEPR2NfDjKGqzgZhEw%40mail.gmail.com.
>For more options, visit [3]https://groups.google.com/d/optout.
> 
> References
> 
>Visible links
>1. mailto:puppet-users+unsubscr...@googlegroups.com
>2. 
> https://groups.google.com/d/msgid/puppet-users/CAArvnv3G8Lbe8GL11SGC251YYn-SjUs-KEPR2NfDjKGqzgZhEw%40mail.gmail.com?utm_medium=email_source=footer
>3. https://groups.google.com/d/optout

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/20170707194535.5vx25lcl7qxu2uun%40iniquitous.heresiarch.ca.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Puppet Packet Rate?

[Peter Berghold]

Has anybody out there done any sort of study on what Puppet produces in
terms of I/O packet rate?  I'm being asked to fill in a spreadsheet with
that information

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CAArvnv3G8Lbe8GL11SGC251YYn-SjUs-KEPR2NfDjKGqzgZhEw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Trigger apt-get update if packages are to be installed

[Christopher Wood]

Not sure what their solution was, but mine was (back when puppeting 
Debian/Ubuntu) to run apt-get update on every run, but make sure to also run 
apt-cacher-ng to be an apt proxy on the local network. That sped things up.

On Fri, Jul 07, 2017 at 07:23:17AM -0700, Klavs Klavsen wrote:
>Did you ever find a good solution to this? found stuff like this:
>
> https://blog.bluemalkin.net/puppet-trick-running-apt-get-update-only-when-needed/
>- but thats really ugly
> 
>--
>You received this message because you are subscribed to the Google Groups
>"Puppet Users" group.
>To unsubscribe from this group and stop receiving emails from it, send an
>email to [1]puppet-users+unsubscr...@googlegroups.com.
>To view this discussion on the web visit
>
> [2]https://groups.google.com/d/msgid/puppet-users/52bf65e9-5458-4053-95e5-842eaebe7203%40googlegroups.com.
>For more options, visit [3]https://groups.google.com/d/optout.
> 
> References
> 
>Visible links
>1. mailto:puppet-users+unsubscr...@googlegroups.com
>2. 
> https://groups.google.com/d/msgid/puppet-users/52bf65e9-5458-4053-95e5-842eaebe7203%40googlegroups.com?utm_medium=email_source=footer
>3. https://groups.google.com/d/optout

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/20170707153129.ujpuob5dqxzhxejk%40iniquitous.heresiarch.ca.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Puppet Minor Upgrade

[Peter Krawetzky]

I need a clarification on a comment in the puppet upgrade doci.  Does this 
mean (last sentence below) I can upgrade the puppetdb servers before the 
puppetservers and puppet agent?  It's the "nodes" comment that has me 
confused.  I take that as it can go before anything.

A minor upgrade is an upgrade from one Puppet 4 release to another. The 
order in which you upgrade packages is important. Always upgrade 
puppetserver on your masters *before* you upgrade agents. *You can upgrade 
PuppetDB before or after you upgrade other nodes.*

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/074f9f65-d10b-4ab6-b544-07717b7b438e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Trigger apt-get update if packages are to be installed

[Klavs Klavsen]

Did you ever find a good solution to this? found stuff like this: 
https://blog.bluemalkin.net/puppet-trick-running-apt-get-update-only-when-needed/
 
- but thats really ugly

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/52bf65e9-5458-4053-95e5-842eaebe7203%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Re: Avoiding duplicate declaration in a loop?

[jcbollinger]

On Thursday, July 6, 2017 at 4:45:53 PM UTC-5, buoyant_puppy wrote:
>
> Excuse my newbieness, but I'm having a basic misunderstanding regarding 
> loops.
>
> Say I have: joesfriends = jack, sam, sally
>
> I need to add each entry into a file - one per line.
>
> $joesfriends.each | String $joesfriends| {# loop
>
>   file { "/etc/list_of_joes_friends":
>
> line => "${joesfriends}"
>
>   }
>
> }
>
>
>
> This fails on a duplicate declaration errorbut how can there not be a 
> duplicate, it's a loop? The examples in the docs avoid this particular 
> situation by using the var in the filename, but that's not usually going to 
> be an option.
>


You have it backwards.  If you're trying to declare a single resource then 
it's the (Puppet) loop that's not an option.

No resource can be declared more than once.  You can, however, declare 
*distinct* resources in various iterations of a loop -- for example, 
resources representing different parts of the same file -- or you can 
construct resource properties via an iterative process in a template or 
custom function.

It's not clear exactly what you mean to do because the File resource type 
does not have a "line" property.  If you mean to describe the complete 
contents of a single file then you are looking for a way to construct a 
value for the 'content' property of one File resource.  I'd recommend a 
template  for that 
purpose.  Puppet 4 has two flavors to choose from, ERB and EPP, either one 
of which would support building the file content by iterating over the 
elements of an array.


John

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/16b621db-af5e-47fb-aaba-b19e7d7a4f60%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Hiring Puppet Developers

[Michael Catuara]

Hi gang, I work for TransUnion and we are looking to hire two Puppet 
Developers/Linux Engineers. Here's a few bullets to describe the job...
- Member of the Engineering team that staffs projects, maintains the Puppet 
infra, writes Puppet code, and BAU time consists of exploring new technology. 
- Not a member of the Admin team that does patching and tickets and provides 
primary on-call duties. 
- We're looking to implement Open Stack and partner with App Dev to 
containerize apps this year
- AWS skills a definite plus

Please apply at our career website... 
https://transunion.taleo.net/careersection/tuext/jobdetail.ftl?job=17000947=GMT-05%3A00

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/85c194f8-37a5-47d9-846d-d33c05db7413%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Re: PuppetDB low catalog-duplication rate Puppet DB 4.3.0

[Peter Krawetzky]

So I went to run the curl command listed below and it came back with 
nothing.  So I used pgadmin to look at the catalogs table and it's 
completely empty.  The system has been running for almost 24 hours after 
dropping/creating the postgresql database.  Any idea why the catalog table 
would be empty?

On Wednesday, June 28, 2017 at 2:11:17 PM UTC-4, Mike Sharpton wrote:
>
> Hey all,
>
> I am hoping there is someone else in the same boat as I am.  We are 
> running Puppet 4.2.2, along with PuppetDB 4.3.0.  I am seeing low 
> duplication rate which I think is contributing to our queuing problems in 
> PuppetDB.  The queue will fluctuate from 0-100 queued, to up to 2000.  We 
> have around 4500 nodes, and we are using 8 threads on our PuppetDB server. 
>  I am seeing that the low duplication rate is caused by hashes not matching 
> and a full insert running which is expensive on the DB instead of just 
> updating the time stamp.  I don't know why these would not be matching, and 
> may need help as far as how to find something like this.  I see items in 
> PuppetDB3 for this, but not 4.  I see that using timestamp and other items 
> which change each time will cause the catalog to never be the same, but I 
> would think we would have 0% duplication if this was the case.  I am also 
> seeing that things are improved in 4.4.0 as far as performance and a 
> missing index is corrected that may speed things.  I am wondering what 
> others have done/seen with this and whether upgrading to 4.4.0 would do me 
> good.  I am thinking it would as many things appear to fixed around the 
> issues I am seeing.  Thanks in advance,
>
> Mike
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/d2aad3a9-40d2-4d79-bb46-32e2ffe357e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: [Puppet Users] Avoiding duplicate declaration in a loop?

[Bart-Jan Vrielink]

Hello,



They are duplicate resources because Puppet (on the server) parses the 
manifests and adds all resources found to a catalog that is sent to the client, 
which uses this catalog to update the configuration.

The catalog cannot have duplicate resources.



I know of 3 basic methods to alter the contents of a file with Puppet: augeas, 
file_line and concat. Augeas is quite powerful, and maybe not best suited for 
this specific task. file_line is part of the puppetlabs-stdlibs module you most 
likely already have installed and concat is in a separate puppetlabs-concat 
module.



Untested, but with file_line I would do something like this:



joesfriends = jack, sam, sally

$joesfriends.each | String $joesfriend | {

  file_line { $joesfriend:

    ensure => present,

    path   => '/etc/list_of_joes_friends',

    line    => 'whatever you want',

  }

}



This assumes the file /etc/list_of_joes_friends is already present.



-Original message-
From: buoyant_puppy 
Sent: Thursday 6th July 2017 23:45
To: Puppet Users 
Subject: [Puppet Users] Avoiding duplicate declaration in a loop?

Excuse my newbieness, but I'm having a basic misunderstanding regarding loops.

Say I have: joesfriends = jack, sam, sally

I need to add each entry into a file - one per line.

$joesfriends.each | String $joesfriends| {    # loop



  file { "/etc/list_of_joes_friends":    

    line => "${joesfriends}"

  }

}





This fails on a duplicate declaration errorbut how can there not be a 
duplicate, it's a loop? The examples in the docs avoid this particular 
situation by using the var in the filename, but that's not usually going to be 
an option.



-- 
 You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
email to puppet-users+unsubscr...@googlegroups.com 
 .
 To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/220aaba3-2cc4-4338-b575-349b8397f694%40googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.
 

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/zarafa.595f4f95.0ca2.541e22d26cae771f%40anjie.dontpanic.nl.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Multiple vhosts, same document root

['Bas van de Wiel' via Puppet Users]

So I managed to hack a fix for this:

|# Hackish pair of execs to allow multiple sites with the same docroot|
|# See: https:||//confluence.maxserv.com/display/OPS/MaxServ+Apache|
|exec| |{ ||"${name}_${webroot}"||:|
|||cwd => ||'/'||,|
|||user=> ||'root'||,|
|||path=> 
||'/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/home/beheer/bin'||,|

|||command => ||"`which mkdir` -p ${webroot}"||,|
|||creates => ||$webroot||,|
|||notify  => ||Exec||[||"${name}_webroot_permissions"||],|
|}|
|exec| |{ ||"${name}_webroot_permissions"||:|
|||cwd => ||'/'||,|
|||user=> ||'root'||,|
|||path=> 
||'/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/home/beheer/bin'||,|
|||command => ||"`which chown` ${apache::user}:${apache::group} 
${webroot}"||,|

|||refreshonly => true,|
|}

Any comments?

Bas
|

On 06-07-17 21:32, bas via Puppet Users wrote:

Hello Puppet users,

I'm working on a webserver module that needs to ensure directories for 
document roots. I tried doing this by using file resources like so:


file { 'vhost-A':
  ensure => 'directory',
  path   => '/var/www/sharedvhost',
  ...clipped for brevity...
}

file { 'vhost-B':
  ensure => 'directory',
  path   => '/var/www/sharedvhost',
  ...clipped for brevity...
}

This, sadly, does not work. Even though I avoid the pitfall of 
declaring two resources by the same name Puppet is clever enough to 
see that I'm ensuring the same path in the end.


What I'd like the result to be, is the existence of 
/var/www/sharedvhost with the ability for my vhost resource type to 
know about it but without throwing errors.


Any help would be much appreciated!

Bas



--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/fbdc265c-4fe1-59b3-fb07-c951abe957d6%40area536.com.
For more options, visit https://groups.google.com/d/optout.