(sorry if I have a lot of configs, just wanted to make sure all the
relevant bits were included)
tldr; The problem is some of the nodes will not communicate with one of the
smart-proxy puppet masters (non-alpha ones) because of cert errors.
This might be a puppet issue but I'm not entirely sure. First let me
explain the setup. I have a Foreman server setup as the "alpha" puppet
master. Then I have 2 other puppet masters that use the alpha as the
ca_server and run smart-proxy-puppet. I've several "nodes". Some under
alpha-master and some under each of the other proxy puppet masters.
* george-dev = alpha-puppet master (foreman and ca_server)
* construction-dev = puppet master smart-proxy
* grill-dev = puppet master smart-proxy
* h8-dev = node calling grill-dev as it's puppet master
For each puppet master smart-proxy I created certs on the alpha-master
(george) and located them on the host before foreman-installer was run.
Here are the files/names placed onto grill-dev.
/etc/puppetlabs/puppet/ssl/certs/ca.pem
/etc/puppetlabs/puppet/ssl/certs/grill-dev.ourdomain.com.pem
/etc/puppetlabs/puppet/ssl/private_keys/grill-dev.ourdomain.com.pem
/etc/puppetlabs/puppet/ssl/public_keys/grill-dev.ourdomain.com.pem
/etc/puppetlabs/puppet/ssl/certs/george-dev.ourdomain.com.pem
/etc/puppetlabs/puppet/ssl/private_keys/george-dev.ourdomain.com.pem
/etc/puppetlabs/puppet/ssl/public_keys/george-dev.ourdomain.com.pem
Here is a sample puppet.conf on one of the smart-proxy puppet masters
(grill-dev)
## Module: 'puppet'
[main]
# Where Puppet's general dynamic and/or growing data is kept
vardir = /opt/puppetlabs/puppet/cache
# The Puppet log directory.
# The default value is '$vardir/log'.
logdir = /var/log/puppetlabs/puppet
# Where Puppet PID files are kept.
# The default value is '$vardir/run'.
rundir = /var/run/puppetlabs
# Where SSL certificates are kept.
# The default value is '$confdir/ssl'.
ssldir = /etc/puppetlabs/puppet/ssl
# Allow services in the 'puppet' group to access key (Foreman + proxy)
privatekeydir = $ssldir/private_keys { group = service }
hostprivkey = $privatekeydir/$certname.pem { mode = 640 }
show_diff = false
## Server config
reports = foreman
environmentpath = /etc/puppetlabs/code/environments
basemodulepath =
/etc/puppetlabs/code/environments/common:/etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules
hiera_config = $confdir/hiera.yaml
### Next part of the file is managed by a different template ###
## Module: 'puppet'
[agent]
# The file in which puppetd stores a list of the classes
# associated with the retrieved configuration. Can be loaded in
# the separate ``puppet`` executable using the ``--loadclasses``
# option.
# The default value is '$statedir/classes.txt'.
classfile = $statedir/classes.txt
# Where puppetd caches the local configuration. An
# extension indicating the cache format is added automatically.
# The default value is '$confdir/localconfig'.
localconfig = $vardir/localconfig
# Disable the default schedules as they cause continual skipped
# resources to be displayed in Foreman - only for Puppet >= 3.4
default_schedules = false
report = true
pluginsync = true
masterport = 8140
environment = production
certname = grill-dev.ourdomain.com
server = george-dev.ourdomain.com
listen = false
splay = false
splaylimit = 1800
runinterval = 1800
noop = false
usecacheonfailure = true
### Next part of the file is managed by a different template ###
## Module: 'puppet'
[master]
autosign = /etc/puppetlabs/puppet/autosign.conf { mode = 0664 }
external_nodes = /etc/puppetlabs/puppet/node.rb
node_terminus = exec
ca = false
ssldir = /etc/puppetlabs/puppet/ssl
certname = grill-dev.ourdomain.com
parser = current
strict_variables = false
The problem is some of the nodes will not communicate with one of the
smart-proxy puppet masters (non-alpha ones) because of cert errors. Here is
a example error.
[root@h8-dev puppet]# puppet agent --test
Warning: Unable to fetch my node definition, but the agent run will
continue:
Warning: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate
B: certificate verify failed: [certificate revoked for
/CN=grill-dev.ourdomain.com]
Info: Retrieving pluginfacts
Error: /File[/var/lib/puppet/facts.d]: Failed to generate additional
resources using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3
read server certificate B: certificate verify failed: [certificate revoked
for /CN=grill-dev.ourdomain.com]
Error: /File[/var/lib/puppet/facts.d]: Could not evaluate: Could not
retrieve file metadata for puppet://grill-dev.ourdomain.com/pluginfacts:
SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B:
certificate verify failed: [certificate revoked for
/CN=grill-dev.ourdomain.com]
Wrapped exception:
SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B:
certificate verify failed: [certificate revoked for
/CN=grill-dev.ourdomain.com]
Info: Retrieving plugin
Error: /File[/var/lib/puppet/lib]: Failed to generate additional resources
using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3 read
server certificate B: certificate verify failed: [certificate revoked for
/CN=grill-dev.ourdomain.com]
Error: /File[/var/lib/puppet/lib]: Could not evaluate: Could not retrieve
file metadata for puppet://grill-dev.ourdomain.com/plugins: SSL_connect
returned=1 errno=0 state=SSLv3 read server certificate B: certificate
verify failed: [certificate revoked for /CN=grill-dev.ourdomain.com]
Wrapped exception:
SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B:
certificate verify failed: [certificate revoked for
/CN=grill-dev.ourdomain.com]
Error: Could not retrieve catalog from remote server: SSL_connect
returned=1 errno=0 state=SSLv3 read server certificate B: certificate
verify failed: [certificate revoked for /CN=grill-dev.ourdomain.com]
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3
read server certificate B: certificate verify failed: [certificate revoked
for /CN=grill-dev.ourdomain.com]
The puppet master is a smart-proxy, has 3 nodes, 1 works, 2 do not. When I
point the 2 that do NOT work at the alpha-master instead of the proxy
puppet agent --test works. I can also point them to the other smart-proxy
and they work.This seems to imply that the proxy master is having problems.
I've tried to find any indications on the proxy host in logs but cannot
find any logs for these connects at all. tcpdump shows traffic on port 8140
however.
These devices have firewalld disabled to avoid firewall issues for this
test.
I've "puppet cert clean'd" the keys on the ca_server and remade thinking
there was some issue... didn't help. I checked --fingerprint on them and
they match what the ca_server has. I also copied the crl.pem from one of
the masters that worked to the master that isn't thinking it was a bad
revocation entry... no joy.
I've entirely rebuilt these 2 nodes (and the puppet master proxy). I've
compared the puppet.conf on the one node that is working with the 2 that
are not and outside of their own name they are exactly the same (example):
[main]
vardir = /var/lib/puppet
logdir = /var/log/puppet
rundir = /var/run/puppet
ssldir = $vardir/ssl
[agent]
pluginsync = true
report = true
ignoreschedules = true
ca_server = george-dev.ourdomain.com
certname = h8-dev.ourdomain.com
environment = production
server = grill-dev.ourdomain.com
Any theory or tips where to look for the cause?
--
You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.
