Re: [Puppet Users] Issue with module from forge - Duplicate declaration Class[Rsyslog]

[re-glaue]

On Thursday, October 20, 2016 at 10:31:59 AM UTC-5, jcbollinger wrote:
>
>
>
> On Wednesday, October 19, 2016 at 4:51:20 PM UTC-5, re-g...@wiu.edu wrote:
>>
>>
>>
>> On Tuesday, October 18, 2016 at 9:49:23 AM UTC-5, jcbollinger wrote:
>>>
>>>
>>>
>>> On Monday, October 17, 2016 at 3:15:51 PM UTC-5, re-g...@wiu.edu wrote:



 On Friday, October 14, 2016 at 5:08:31 PM UTC-5, re-g...@wiu.edu wrote:
>
>
>
> On Thursday, October 13, 2016 at 9:15:58 AM UTC-5, Rob Nelson wrote:
>>
>> I've worked with saz in the past and believe he would be very 
>> receptive to PRs that address this issue, as well, if RG or anyone else 
>> wanted to contribute them. That would be one of the better solutions to 
>> the 
>> problem.
>>
>> Rob Nelson
>> rnel...@gmail.com
>>
>>
>> I think I've given you a pretty good handle on the nature of the 
>>> problem, and I'm inclined to think that a solution that addresses the 
>>> problem at its root will take care of the whole suite of issues.
>>>
>>
> Lots and lots of thanks to jcbollinger.
> Here is what I have done as a result of this thread:
>
> Submitted an issue with saz-rsyslog:
> Duplicate Declaration Class[Rsyslog] with The Foreman as ENC during 
> catalog compilation #237
> https://github.com/saz/puppet-rsyslog/issues/237
>
> Posted a follow-up question to a related thread on The Foreman mail 
> list to determine if The Foreman also may be exhibiting a bug, or if the 
> issue may be my configuration:
> Duplicate declaration error.
> https://groups.google.com/d/topic/foreman-users/k0JgABtszdU/discussion 
>   (my post is waiting for approval as of 22:00 GMT 10/14/2016)
>
> My current thought at this time is two points:
> 1. saz-rsyslog is written in such a way that declaring a subclass 
> before the parent class causes duplicate declaration errors - but I am 
> told 
> the module inheritance should be addressed, and would resolve this issue.
> 2. The Foreman, under undetermined circumstances (maybe my 
> misconfiguration), will randomly declare parent classes and subclass out 
> of 
> order - but I am told this is not an issue with puppet modules that use 
> inheritance correctly - thus there might be an underlying bug with The 
> Foreman because this error will not appear under modules' expected 
> inheritance.
>
>
 This is what was posted on The Foreman mail list group:
 https://groups.google.com/d/topic/foreman-users/k0JgABtszdU/discussion

> This is an issue with the format of the ENC YAML used between Foreman 
> and Puppet, and is best fixed in the module. 
> The list of classes is given as a hash/dictionary and so has no 
> particular order defined - it's down to the Puppet master/server to 
> iterate over it, and it probably does so in no particular order. It 
> isn't under Foreman's control.


 I now believe this is truly the root of my problem. 

 The evidence is looking at the ENC output and the Puppet-generated 
 Catalog file on each of the master puppet servers.
 As you can see below, the puppet-generated catalog on the 
 secondary-master puppet server has put rsyslog::client before rsyslog - 
 this causes the failure.
 However, The Foreman ENC output is the same on both primary and 
 secondary master puppet servers.


>>>
>>> You have taken a step backward there.  I remind you that duplicate 
>>> declaration errors occur *during catalog building*, and that catalog 
>>> building fails if one occurs.  It is therefore impossible to compare a 
>>> catalog from the failure case to a catalog from the success case, because 
>>> no catalog is ever created in the failure case.
>>>
>>>
>> Then I was mistaken in thinking that the node file 
>> (/var/lib/puppet/yaml/node/server1.mydomain.example.com.yaml) was the data 
>> (catalog pre-data) written to a file before the catalog compilation 
>> procedure.
>>
>
>
> But you said you were looking at the *catalog* not "catalog pre-data". 
> I'm sure you appreciate the tremendous difference between those.
>
>  
>
>> I assumed this because both the foreman and node files are created new 
>> every time server1 performs a puppet agent update, even when the result is 
>> a 400 error for the catalog compilation.
>> If I am mistaken in this, then I need someone to tell me how to get the 
>> data of the catalog that is failing to be compiled, so that I can compare 
>> it with other sources.
>>
>> Here is the procedure I have been assuming (I am not an expert on puppet 
>> internals):
>> 1. (server1)puppet-agent -> puppet-master
>> 2. puppet-master -> (exec)puppet[external_nodes] 
>> server1.mydomain.example.com -> foreman ENC
>> 3. foreman ENC -> 
>> /var/lib/puppet/yaml/foreman/server1.mydomain.example.com.yaml -> 
>> puppet-master
>> 4. 

[Puppet Users] Announce: Puppet Agent 1.7.1 is available (with a critical security fix)

[Geoff Nichols]

Puppet Agent 1.7.1 is available (with a critical security fix).

This is a security release of Puppet Agent that addresses a critical
vulnerability in PXP Agent as well as several other vulnerabilities.

Please see our security page 
and these security notices regarding the vulnerabilities fixed in this
release:


   -

   Puppet Execution Protocol (PXP) Command Whitelist Validation
   Vulnerability 
   -

   Unprivileged Access to Environment Catalogs
   


Puppet Agent 1.7.1 also contains updated versions of OpenSSL and Curl to
address vulnerabilities recently announced by those projects.

This is the second release of the puppet-agent with .deb and .rpm packages
signed with our new GPG signing key.

Please see our recent announcement
 for more
information about the new GPG signing key.

Full release notes are available here:
https://docs.puppet.com/puppet/latest/reference/release_notes_agent.html

To install or upgrade puppet-agent, follow the getting started directions:

http://docs.puppetlabs.com/puppet/latest/reference/index.html


-- 
Geoff Nichols
Puppet Ecosystem - Agent and Platform Team

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CADjnYBy%3D4OiSo9ORRDW67SU5y72JTGW0zHQrF6jnfyB4ZXzEUA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Announce: Puppet Enterprise 2016.4.0 is available

[Morgan Rhodes]

Hello -

Today we announced the major updates to Puppet Enterprise
,
which give you the power to use automation as the bridge to your future,
whether that's moving to the cloud or adopting containers in production.

With our latest release, Puppet is laying the foundation for DevOps
practices by making it easier than ever to gain situational awareness and
orchestrate software change confidently, no matter what your infrastructure
runs on. Updates include:


   -

   Visualize intentional vs. corrective changes - Get detailed visibility
   into why changes occur and whether those changes are intentional or due to
   unexpected configuration drift with new reporting in the Puppet Enterprise
   web UI.
   -

   Orchestrate phased deployments - Significant enhancements to Puppet’s
   orchestration capabilities enable you to easily segment your infrastructure
   and run phased, canary deployments to as many servers at a time as you
   define. Using the Puppet orchestrator, it is now possible to segment
   infrastructure based on characteristics such as location, operating system,
   specific configurations applied, or any additional facts, and run ordered
   deployments to targeted portions of your infrastructure.
   -

   Puppet & Docker - Automate the container build process using Puppet as
   you define, build, and deploy containers into production environments.
   -

   VMware vRealize integration - A new Integration with VMware's vRealize
   Suite (vRA/vRO) significantly reduces lead times by enabling fully
   automated, self-service provisioning workflows.
   -

   Jenkins integration - A new integration with Jenkins lets you scale your
   DevOps practice by building continuous delivery pipelines and orchestrating
   infrastructure deployment with an integrated toolchain.


Puppet Enterprise 2016.4 has been designated as our long-term support (LTS)
release, meaning you can expect full support, security updates and fixes
for 24-months. Make sure to upgrade to Puppet Enterprise 2016.4, and check
out the new features in our release notes
 to see how we use
data analytics for continuous product improvement. You can also register
for our webinar, What’s New in Puppet Enterprise on 3 November 2016 to
learn more and see the product in action:
http://info.puppet.com/2365-Whats-New-PE-2016.4-Register.html

-Puppet
-- 
Morgan Rhodes
mor...@puppet.com
Release Engineer
-- 
PuppetConf 2016 , 19 - 21 October, San
Diego, California
*Register to attend or sign up to view the Live Stream
*

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/CA%2BFnDv0FYno9DQq88pPEtrs93-F0nugUGmu8boztwEkJuVRA4Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet Users] Issue with module from forge - Duplicate declaration Class[Rsyslog]

[jcbollinger]

On Wednesday, October 19, 2016 at 4:51:20 PM UTC-5, re-g...@wiu.edu wrote:
>
>
>
> On Tuesday, October 18, 2016 at 9:49:23 AM UTC-5, jcbollinger wrote:
>>
>>
>>
>> On Monday, October 17, 2016 at 3:15:51 PM UTC-5, re-g...@wiu.edu wrote:
>>>
>>>
>>>
>>> On Friday, October 14, 2016 at 5:08:31 PM UTC-5, re-g...@wiu.edu wrote:



 On Thursday, October 13, 2016 at 9:15:58 AM UTC-5, Rob Nelson wrote:
>
> I've worked with saz in the past and believe he would be very 
> receptive to PRs that address this issue, as well, if RG or anyone else 
> wanted to contribute them. That would be one of the better solutions to 
> the 
> problem.
>
> Rob Nelson
> rnel...@gmail.com
>
>
> I think I've given you a pretty good handle on the nature of the 
>> problem, and I'm inclined to think that a solution that addresses the 
>> problem at its root will take care of the whole suite of issues.
>>
>
 Lots and lots of thanks to jcbollinger.
 Here is what I have done as a result of this thread:

 Submitted an issue with saz-rsyslog:
 Duplicate Declaration Class[Rsyslog] with The Foreman as ENC during 
 catalog compilation #237
 https://github.com/saz/puppet-rsyslog/issues/237

 Posted a follow-up question to a related thread on The Foreman mail 
 list to determine if The Foreman also may be exhibiting a bug, or if the 
 issue may be my configuration:
 Duplicate declaration error.
 https://groups.google.com/d/topic/foreman-users/k0JgABtszdU/discussion 
   (my post is waiting for approval as of 22:00 GMT 10/14/2016)

 My current thought at this time is two points:
 1. saz-rsyslog is written in such a way that declaring a subclass 
 before the parent class causes duplicate declaration errors - but I am 
 told 
 the module inheritance should be addressed, and would resolve this issue.
 2. The Foreman, under undetermined circumstances (maybe my 
 misconfiguration), will randomly declare parent classes and subclass out 
 of 
 order - but I am told this is not an issue with puppet modules that use 
 inheritance correctly - thus there might be an underlying bug with The 
 Foreman because this error will not appear under modules' expected 
 inheritance.


>>> This is what was posted on The Foreman mail list group:
>>> https://groups.google.com/d/topic/foreman-users/k0JgABtszdU/discussion
>>>
 This is an issue with the format of the ENC YAML used between Foreman 
 and Puppet, and is best fixed in the module. 
 The list of classes is given as a hash/dictionary and so has no 
 particular order defined - it's down to the Puppet master/server to 
 iterate over it, and it probably does so in no particular order. It 
 isn't under Foreman's control.
>>>
>>>
>>> I now believe this is truly the root of my problem. 
>>>
>>> The evidence is looking at the ENC output and the Puppet-generated 
>>> Catalog file on each of the master puppet servers.
>>> As you can see below, the puppet-generated catalog on the 
>>> secondary-master puppet server has put rsyslog::client before rsyslog - 
>>> this causes the failure.
>>> However, The Foreman ENC output is the same on both primary and 
>>> secondary master puppet servers.
>>>
>>>
>>
>> You have taken a step backward there.  I remind you that duplicate 
>> declaration errors occur *during catalog building*, and that catalog 
>> building fails if one occurs.  It is therefore impossible to compare a 
>> catalog from the failure case to a catalog from the success case, because 
>> no catalog is ever created in the failure case.
>>
>>
> Then I was mistaken in thinking that the node file 
> (/var/lib/puppet/yaml/node/server1.mydomain.example.com.yaml) was the data 
> (catalog pre-data) written to a file before the catalog compilation 
> procedure.
>


But you said you were looking at the *catalog* not "catalog pre-data". I'm 
sure you appreciate the tremendous difference between those.

 

> I assumed this because both the foreman and node files are created new 
> every time server1 performs a puppet agent update, even when the result is 
> a 400 error for the catalog compilation.
> If I am mistaken in this, then I need someone to tell me how to get the 
> data of the catalog that is failing to be compiled, so that I can compare 
> it with other sources.
>
> Here is the procedure I have been assuming (I am not an expert on puppet 
> internals):
> 1. (server1)puppet-agent -> puppet-master
> 2. puppet-master -> (exec)puppet[external_nodes] 
> server1.mydomain.example.com -> foreman ENC
> 3. foreman ENC -> 
> /var/lib/puppet/yaml/foreman/server1.mydomain.example.com.yaml -> 
> puppet-master
> 4. puppet-master 
> -> /var/lib/puppet/yaml/node/server1.mydomain.example.com.yaml -> 
> puppet-master(compile-catalog)
> 5. puppet-master(compile-catalog) -> Error -> 400 Server Error -> 
> (server1)puppet-agent

[Puppet Users] Re: Providing SELECT rights to a PG DB

[Russell Mull]

On Wednesday, October 19, 2016 at 6:11:33 PM UTC-7, Werner van der Merwe 
wrote:
>
> I am having difficulty creating users with read-only access to all tables 
> in a schema.
>
>
> I am using the following in the manifest
>
>   postgresql::server::grant {:
> db  => ,
> object_name => ,
> object_type => 'ALL TABLES IN SCHEMA',
> privilege   => 'SELECT',
> role=> ,
>   }
>
>
> In the puppet run, I get the following error:
> Could not evaluate: Error evaluating 'unless' clause, returned pid 6933 
> exit 1: 'ERROR: unrecognized privilege type: "SELECT ON ALL TABLES"
>
>
> Yet looking at the manifest:
>
> postgresql::server::grant
> Manages grant-based access privileges for roles. See PostgreSQL documentation 
> for grant for more information.
>
> db - Specifies the database to which you are granting access.
> object_type - Specifies the type of object to which you are granting 
> privileges. Valid options: DATABASE, SCHEMA, SEQUENCE, ALL SEQUENCES IN 
> SCHEMA, TABLE or ALL TABLES IN SCHEMA.
> object_name - Specifies name of object_type to which to grant access.
> port - Port to use when connecting. Default: undef, which generally defaults 
> to port 5432 depending on your PostgreSQL packaging.
> privilege - Specifies the privilege to grant. Valid options: ALL, ALL 
> PRIVILEGES or object_type dependent string.
> psql_db - Specifies the database to execute the grant against. This should 
> not ordinarily be changed from the default, which is postgres.
> psql_user - Sets the OS user to run psql. Default: the default user for the 
> module, usually postgres.
> role - Specifies the role or user whom you are granting access to.
>
>
> Hunting down the 'privilege type':
> privilege_type='${custom_privilege}'
>
>
> $custom_privilege = $_privilege ? {
> 'ALL'=> 'INSERT',
> 'ALL PRIVILEGES' => 'INSERT',
> default  => $_privilege,
>   }
>
>
> And finally 
> validate_string($_privilege,'SELECT','INSERT','UPDATE','DELETE','TRUNCATE','REFERENCES','TRIGGER','ALL','ALL
>  
> PRIVILEGES')
>
>
> Any ideas?
>


Hi Werner, 

Unfortunately, the world of PostgreSQL permissions is rather more complex 
than the model exposed by the module. I recommend dropping down to psql to 
make sure you get exactly what you want, when doing any non-trivial 
PostgreSQL permissions management. It may look something like this (hacked 
up, untested code):

  psql {"${unique_name}/tables":
db  => $database,
command => "GRANT SELECT
ON ALL TABLES IN SCHEMA \"${schema}\"
TO \"${table_reader}\"",
onlyif  => "SELECT *
FROM pg_tables
WHERE schemaname='${schema}'
  AND has_table_privilege('${table_reader}', schemaname || 
'.' || tablename, 'SELECT')=false
)",
  }

You'll probably need to do the same thing for sequences, and maybe for 
functions if you're using them. 

The other important part is to make sure permissions are set correctly for 
newly created schema objects. Puppet will of course catch anything set 
incorrectly and fix it, but it's nice to have the database configuration 
completely correct so you don't have to rely on that. Here's a (hacked up) 
define type I've used for that:

# Grant read permissions to table_reader by default, for new tables created 
by
# table_creator.
define default_read_grant(
  String $database,
  String $schema,
  String $table_creator,
  String $table_reader,
) {
  psql {"${title}/sql":
db  => $database,
command => "ALTER DEFAULT PRIVILEGES
  FOR USER \"${table_creator}\"
  IN SCHEMA \"${schema}\"
GRANT SELECT ON TABLES
  TO \"${table_reader}\"",
unless  => "SELECT *
FROM pg_default_acl acl
JOIN pg_namespace ns ON acl.defaclnamespace=ns.oid
WHERE acl.defaclacl::text ~ 
'.*\"${table_reader}\"=r/\"${table_creator}\".*'
AND ns.nspname = '${schema}'",
  }
}

I definitely recommend reading up on the details of the default permissions 
and poking around in the various acl tables if you haven't already. The 
format of the pg_default_acl table in particular is a little idiosyncratic. 
The above is what worked for my application, but it may not be appropriate 
for yours. 

Hope this helps!

 - Russell Mull

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/3c0aacf1-9e90-4cd1-8e0e-7cb14e73af95%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet Users] Re: Using a module that is not 100% hiera-compliant

[Chadwick Banning]

create_resources 
 
is a function that just loops through a hash of resources and adds them to 
the catalog. You can set any key name you want in Hiera and look it up 
directly with the hiera* functions. You can do something like this:

class profiles::postfix {
  include ::postfix
  $configs = hiera('profiles::postfix::configs')
  create_resources('postfix::config', $configs)
}

Then just set the 'profiles::postfix::config' key containing the settings 
you want anywhere in your Hiera hierarchy. The hiera function lookup will 
retrieve the value and this value gets passed to the create_resources 
function that will auto-generate the postfix::config resources.

On Wednesday, October 19, 2016 at 10:16:26 AM UTC-4, Ugo Bellavance wrote:
>
> Hi Chadwick,
>
> I'll definitely look a it soon, but can you explain what the lines do?
>
> What would key_name be?
>
> Thanks!
>
> On Wednesday, October 19, 2016 at 8:01:34 AM UTC-4, Chadwick Banning wrote:
>>
>> I looked at the PR (https://github.com/camptocamp/puppet-postfix/pull/50) 
>> to add "Hiera support" and it appears that it just add some parameters that 
>> take hashes that are used to auto-generate instances of defined types. I 
>> wouldn't call this "Hiera support", I'd just call it "convenience 
>> parameters".
>>
>> There's no reason you couldn't just do this on your own in a profile (or 
>> anywhere else):
>>
>> $configs = hiera('')
>> create_resources('postfix::config', $configs)
>>
>> On Tuesday, October 18, 2016 at 1:56:59 PM UTC-4, Ugo Bellavance wrote:
>>>
>>> Hi,
>>>
>>> I am using camptocamp/postfix for my postfix configuration.  I 
>>> originally defined all my configs manifests but now I would like to change 
>>> to using hiera.  Unfortunately, this module doesn't support hiera for some 
>>> of the configs, so I must define many parameters in the manifests.  I 
>>> wanted to use hiera for simplicity, but also because I have a very nice use 
>>> case:  I have one SMTP front-end with its own specific configs 
>>> (anti-spam/virus), and a series of regular hosts. Traditionally, all hosts 
>>> that are in the same subnet as the Exchange server would use it as 
>>> relayhost and all the other hosts use the smtp front-ends.  Therefore, 
>>> here's what I did:
>>>
>>> hiera.yaml:
>>>
>>> ---
>>> :backends:
>>> #  - regex
>>>   - yaml
>>> :yaml:
>>>   :datadir: /etc/puppet/hiera
>>> #:regex:
>>> #  :datadir: /var/lib/hiera
>>> :hierarchy:
>>>   - "host/%{fqdn}"
>>>   - "domain/%{domain}"
>>>   - "env/%{::environment}"
>>>   - "os/%{operatingsystem}"
>>>   - "osfamily/%{osfamily}"
>>>   - "networks/%{network_ens192}"
>>>   - "virtual/%{::virtual}"
>>>   - common
>>>
>>> This way, I define the exchange server as relayhost for the exchange 
>>> network in /etc/puppet/hiera/networks/192.168.155.0.yaml, and set the smtp 
>>> frontend as relayhost in /etc/puppet/hiera/common.yaml.
>>>
>>> However, since I can't put all the settings in hiera, I must put some in 
>>> the class declaration for the smtp frontends.  When I declare the postfix 
>>> class in both my default profile and in the smtp frontend profile, I get an 
>>> error saying that the class cannot be declared twice (Class[Postfix] is 
>>> already declared; cannot redeclare at 
>>> /etc/puppet/manifests/nodes/smtp_postfix_servers.pp:19)
>>>
>>> Another solution would be to declare the profile in all my roles, but 
>>> it's far from perfect.
>>>
>>> Is there a simple solution?
>>>
>>> I guess that I could do an if based on ipaddress in my default profile, 
>>> but I wanted to use hiera as much as possible. Yes I created an issue to 
>>> ask for full hiera support.
>>>
>>> Thanks,
>>>
>>> Ugo
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/711dca3a-0a18-4dfe-8928-9ddc05d05f83%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.