Ah nice, thanks for pointing that out.
On Wed, Apr 3, 2013 at 3:13 PM, Ken Barber <[email protected]> wrote: > Yes, in this way it behaves a lot like the puppet masters. Don't > forget to check out certificate-whitelist > ( > http://docs.puppetlabs.com/puppetdb/1.1/configure.html#certificate-whitelist > ) > this gives you the ability to only allow certain hosts to connect if > you desire it. > > On Wed, Apr 3, 2013 at 7:41 AM, Mohit Chawla > <[email protected]> wrote: > > Hey Ken, that definitely cleared some misconceptions, thanks ! > > > > I now know what the problem was. First, I assumed that the client must've > > the same certificate as in puppetdb's truststore. I didn't realize that > any > > cert signed by that CA should be able to make calls. In our scenario, > where > > we have separate certs for the agent and master component on the same > server > > (under /etc/puppet/ssl & /var/lib/puppet/ssl respectively), invoking > > --compile without giving the certname meant no certificate at all ! And > so, > > passing --ssldir to use the agent's certs (signed by the same CA) worked. > > > > On Tue, Apr 2, 2013 at 3:40 PM, Ken Barber <[email protected]> wrote: > >> > >> If you specify 'certname' it will use the local certificate with that > >> name as apposed to using the certificate with the same name as the > >> boxes fqdn (ie. facter fqdn). > >> > >> For example, if I use 'puppet master --compile puppetdb1.vm' on its > >> own, it uses my local fqdn (determined with `facter fqdn`) as an > >> indicator of where the certificate is: > >> /var/lib/puppet/ssl/certs/puppetdb1.vm.pem. If I specify --certname > >> foo it uses /var/lib/puppet/ssl/certs/foo.pem. > >> > >> I presume that the certificate that matches the fqdn on the host you > >> are testing is probably _not_ signed by the CA that is loaded into > >> PuppetDB. Its just a guess, but we should confirm this. > >> > >> Lets take a step back a bit and do an exercise. On your PuppetDB node, > >> in /etc/puppetdb/ssl/ you have three files: > >> > >> * truststore.jks > >> * keystore.jks > >> * puppetdb_keystore_pw.txt > >> > >> If you look inside truststore.jks, you'll see that it contains the CA > >> certificate: > >> > >> # keytool -list -keystore /etc/puppetdb/ssl/truststore.jks > >> Enter keystore password: (this is contained inside > >> puppetdb_keystore_pw.txt) > >> > >> Keystore type: JKS > >> Keystore provider: SUN > >> > >> Your keystore contains 1 entry > >> > >> puppetdb ca, 28-Mar-2013, trustedCertEntry, > >> Certificate fingerprint (SHA1): > >> B1:5E:8C:2C:BE:BB:97:5C:09:42:34:7A:F2:02:3F:7B:06:C6:A2:C1 > >> > >> This CA certificate is the same as what matches my CA file here, we > >> can test this by matching the fingerprint: > >> > >> # openssl x509 -noout -in /var/lib/puppet/ssl/ca/ca_crt.pem -fingerprint > >> SHA1 > >> Fingerprint=B1:5E:8C:2C:BE:BB:97:5C:09:42:34:7A:F2:02:3F:7B:06:C6:A2:C1 > >> > >> For client validation, this is really what matters - the client > >> certificate must be signed by this CA certificate to be accepted by > >> PuppetDB. It doesn't matter what the client thinks its CA is in Puppet > >> configuration or whatever (so setting --ca_name is not enough here), > >> all that matters is the client certificate must have been signed by > >> that CA, and it is verified using OpenSSL techniques on the PuppetDB > >> server during SSL negotiation. > >> > >> You can do this manually for example. Lets confirm this on my master - > >> we know that my local CA is confirmed as the thing loaded into the > >> PuppetDB trust store, so now we just run the verify routine against my > >> certificate: > >> > >> # openssl verify -CAfile /var/lib/puppet/ssl/ca/ca_crt.pem `puppet > >> master --configprint hostcert` > >> /var/lib/puppet/ssl/certs/puppetdb1.vm.pem: OK > >> > >> You must be sure to test this host certificate against the exact same > >> CA loaded into PuppetDB ... not just the local CA (in case they differ > >> - you can check this with the -fingerprint command). > >> > >> If the certificate on the other hand is NOT signed by the CA loaded > >> into PuppetDB you get something like: > >> > >> # openssl verify -CAfile /var/lib/puppet/ssl/ca/ca_crt.pem > >> /var/lib/puppet/ssl/certs/puppetdb2.vm.pem > >> /var/lib/puppet/ssl/certs/puppetdb2.vm.pem: CN = puppetdb2.vm > >> error 20 at 0 depth lookup:unable to get local issuer certificate > >> > >> Which is by design for this example - that certificate 'puppetdb2.vm' > >> was a certificate I copied from another Puppetmaster which has a > >> separate CA, and thus it fails verification. In the real world, > >> puppetdb2.vm would get reject by PuppetDB - which is the correct > >> behaviour. > >> > >> ken. > >> > >> On Mon, Apr 1, 2013 at 9:57 AM, Mohit Chawla > >> <[email protected]> wrote: > >> > Slightly OT here - shouldn't setting ca_name be enough for --compile > to > >> > not > >> > fail ? Why does it need the --certname param ? > >> > > >> > > >> > On Mon, Apr 1, 2013 at 2:03 PM, Mohit Chawla > >> > <[email protected]> > >> > wrote: > >> >> > >> >> Hi, > >> >> > >> >> Got it working by passing --certname param (the ca server hostname, > >> >> instead of this master). > >> >> > >> >> > >> >> > >> >> On Mon, Apr 1, 2013 at 1:19 PM, Mohit Chawla > >> >> <[email protected]> wrote: > >> >>> > >> >>> Hi Ken, > >> >>> > >> >>> Got a trace from running puppetdb-foreground --debug - > >> >>> https://gist.github.com/alcy/5283661. Weird that this doesn't > happen > >> >>> during > >> >>> standard puppet runs as opposed to doing a --compile. Here's the > >> >>> puppet > >> >>> trace when doing --compile - https://gist.github.com/alcy/5283712. > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> On Fri, Mar 29, 2013 at 3:14 AM, Ken Barber <[email protected]> > >> >>> wrote: > >> >>>> > >> >>>> Yeah, it does seem very odd though ... if agent works - and the > >> >>>> master > >> >>>> is able to talk to PuppetDB no problem, then its weird that running > >> >>>> puppet master on the command line doesn't seem to work. > >> >>>> > >> >>>> What is strange is that the SSL error is very very unspecific: > >> >>>> > >> >>>> Failed to submit 'replace catalog' command for foo.com to > PuppetDB at > >> >>>> puppetdb:8081: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 > >> >>>> read > >> >>>> finished A > >> >>>> > >> >>>> It doesn't speak of a particular problem, ie. remote host doesn't > >> >>>> match certificate, CA not verified etc. etc. it just says 'read > >> >>>> finished A'. I think this is significant, the last time I've seen > it > >> >>>> is when there was an SSL bug, and the transport is cut short - > which > >> >>>> is why I wanted to see what you had in your logs. > >> >>>> > >> >>>> If you try running the command with --debug --trace do you get any > >> >>>> more information? If you tcpdump the connection to the puppetdb > >> >>>> server > >> >>>> do you see the TCP packets hit the server on port 8081? Try running > >> >>>> puppetdb-foreground --debug on your puppetdb server and see what > >> >>>> happens when you attempt to compile like this. You'd obviously want > >> >>>> to > >> >>>> try these things when the other activity on the server is disabled > >> >>>> :-). > >> >>>> > >> >>>> ken. > >> >>>> > >> >>>> On Thu, Mar 28, 2013 at 8:54 PM, Mohit Chawla > >> >>>> <[email protected]> wrote: > >> >>>> > Hi, > >> >>>> > > >> >>>> > Not at the workstation right now, but regarding puppet.conf I > cant > >> >>>> > think of any peculiar settings apart from this being one of the > two > >> >>>> > puppet masters apart from a separate ca server (we took care of > >> >>>> > having > >> >>>> > the ca server's certs being available at these masters). And > afaik > >> >>>> > right now, there wasn't any ~/.puppet dir for root, however I > need > >> >>>> > to > >> >>>> > confirm this. > >> >>>> > > >> >>>> > On Thu, Mar 28, 2013 at 7:07 PM, Ken Barber <[email protected]> > >> >>>> > wrote: > >> >>>> >> I'm just trying to run up the same environment so I can try to > >> >>>> >> replicate it, as yet I can't replicate it on the newer > >> >>>> >> environment. > >> >>>> >> > >> >>>> >> What does your puppet.conf look like on the host you are trying > to > >> >>>> >> run > >> >>>> >> puppet master --compile btw? I presume you are trying to run the > >> >>>> >> command as root, is there a ~/.puppet directory for that user at > >> >>>> >> all? > >> >>>> >> > >> >>>> >> ken. > >> >>>> >> > >> >>>> >> On Thu, Mar 28, 2013 at 1:17 PM, Mohit Chawla > >> >>>> >> <[email protected]> wrote: > >> >>>> >>> Hello Ken, > >> >>>> >>> > >> >>>> >>> Thanks for the response. > >> >>>> >>> > >> >>>> >>> On Thu, Mar 28, 2013 at 6:42 PM, Ken Barber < > [email protected]> > >> >>>> >>> wrote: > >> >>>> >>>> So I have some questions, as the error could mean a number of > >> >>>> >>>> things: > >> >>>> >>>> > >> >>>> >>>> What version of PuppetDB are you running? And what exact > version > >> >>>> >>>> of > >> >>>> >>>> Java is it using? > >> >>>> >>> > >> >>>> >>> puppetdb version is puppetdb-1.0.4-1.el6.noarch. > >> >>>> >>> > >> >>>> >>> $ java -version > >> >>>> >>> java version "1.6.0_24" > >> >>>> >>> OpenJDK Runtime Environment (IcedTea6 1.11.5) > >> >>>> >>> (rhel-1.50.1.11.5.el6_3-x86_64) > >> >>>> >>> OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode) > >> >>>> >>> > >> >>>> >>>> > >> >>>> >>>> Can you take a look at puppetdb.log and tell me if you see any > >> >>>> >>>> meaningful error messages? > >> >>>> >>> > >> >>>> >>> I took a look, no error messages, only info messages. > >> >>>> >>>> > >> >>>> >>>> Without trying to compile a catalog in this manner - are you > >> >>>> >>>> getting > >> >>>> >>>> any trouble with replace facts/replace catalogs generally? ie. > >> >>>> >>>> Just > >> >>>> >>>> trying to running puppet agent -t for example? > >> >>>> >>> > >> >>>> >>> Nope, those runs are working fine. > >> >>>> >>> > >> >>>> >>>> > >> >>>> >>>> Have you tried re-initialisation your SSL setup with > >> >>>> >>>> puppetdb-ssl-setup at all yet? > >> >>>> >>> > >> >>>> >>> Yes, and confirmed the validity of certificates and their > >> >>>> >>> fingerprints > >> >>>> >>> in the truststore and master ca. > >> >>>> >>> > >> >>>> >>>> > >> >>>> >>>> Does the hostname you have specified in > >> >>>> >>>> /etc/puppet/puppetdb.conf > >> >>>> >>>> to > >> >>>> >>>> talk to your puppetdb server match the SSL certificate > assigned > >> >>>> >>>> to > >> >>>> >>>> the > >> >>>> >>>> puppetdb host itself? > >> >>>> >>> > >> >>>> >>> Yes. > >> >>>> >>> > >> >>>> >>>> > >> >>>> >>>> On Thu, Mar 28, 2013 at 12:55 PM, Mohit Chawla > >> >>>> >>>> <[email protected]> wrote: > >> >>>> >>>>> Forgot mentioning the env details: > >> >>>> >>>>> > >> >>>> >>>>> [user@puppetmaster ~]# rpm -qa | grep puppet > >> >>>> >>>>> puppetlabs-release-6-6.noarch > >> >>>> >>>>> puppetdb-terminus-1.0.4-1.el6.noarch > >> >>>> >>>>> mcollective-puppet-agent-1.4.1-1.noarch > >> >>>> >>>>> puppet-2.7.20-1.el6.noarch > >> >>>> >>>>> hiera-puppet-1.0.0-1.el6.noarch > >> >>>> >>>>> puppet-server-2.7.20-1.el6.noarch > >> >>>> >>>>> mcollective-puppet-common-1.4.1-1.noarch > >> >>>> >>>>> > >> >>>> >>>>> [user@puppetmaster ~]# cat /etc/centos-release > >> >>>> >>>>> CentOS release 6.3 (Final) > >> >>>> >>>>> > >> >>>> >>>>> On Thu, Mar 28, 2013 at 6:23 PM, Mohit Chawla > >> >>>> >>>>> <[email protected]> wrote: > >> >>>> >>>>>> Hello, > >> >>>> >>>>>> > >> >>>> >>>>>> Stuck in a weird place here. I am trying to do 'puppet > master > >> >>>> >>>>>> --compile foo.com', however I am not getting the catalog > json. > >> >>>> >>>>>> So > >> >>>> >>>>>> far, > >> >>>> >>>>>> I have noticed two sort of outputs: > >> >>>> >>>>>> > >> >>>> >>>>>> 1) The above command results in : > >> >>>> >>>>>> notice: Compiled catalog for foo.com in environment > production > >> >>>> >>>>>> in > >> >>>> >>>>>> 10.60 seconds > >> >>>> >>>>>> Failed to submit 'replace catalog' command for foo.com to > >> >>>> >>>>>> PuppetDB at > >> >>>> >>>>>> puppetdb:8081: SSL_connect SYSCALL returned=5 errno=0 > >> >>>> >>>>>> state=SSLv3 > >> >>>> >>>>>> read > >> >>>> >>>>>> finished A > >> >>>> >>>>>> > >> >>>> >>>>>> Notice that I don't actually get any json & of course, the > ssl > >> >>>> >>>>>> error. > >> >>>> >>>>>> > >> >>>> >>>>>> 2) Other output: > >> >>>> >>>>>> Failed when searching for node bar.com: Could not retrieve > >> >>>> >>>>>> facts > >> >>>> >>>>>> for > >> >>>> >>>>>> bar.com: Failed to find facts from PuppetDB at > puppetdb:8081: > >> >>>> >>>>>> SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read > >> >>>> >>>>>> finished > >> >>>> >>>>>> A > >> >>>> >>>>>> > >> >>>> >>>>>> Disabling storeconfig allows me to get a proper catalog > json. > >> >>>> >>>>>> > >> >>>> >>>>>> Regarding the ssl errors, we've established the correctness > of > >> >>>> >>>>>> certificates in the relevant places - and done tests using > >> >>>> >>>>>> curl > >> >>>> >>>>>> and > >> >>>> >>>>>> using ruby net/http. Here's a gist of for the curl and ruby > >> >>>> >>>>>> snippet. > >> >>>> >>>>>> https://gist.github.com/alcy/5262866 > >> >>>> >>>>>> > >> >>>> >>>>>> Any suggestions what could be happening here ? Perhaps in > >> >>>> >>>>>> indirector/facts/puppetdb.rb, http_get doesn't get the > >> >>>> >>>>>> correct > >> >>>> >>>>>> ssl > >> >>>> >>>>>> params or something ? > >> >>>> >>>>> > >> >>>> >>>>> -- > >> >>>> >>>>> You received this message because you are subscribed to the > >> >>>> >>>>> Google > >> >>>> >>>>> Groups "Puppet Users" group. > >> >>>> >>>>> To unsubscribe from this group and stop receiving emails from > >> >>>> >>>>> it, > >> >>>> >>>>> send an email to [email protected]. > >> >>>> >>>>> To post to this group, send email to > >> >>>> >>>>> [email protected]. > >> >>>> >>>>> Visit this group at > >> >>>> >>>>> http://groups.google.com/group/puppet-users?hl=en. > >> >>>> >>>>> For more options, visit > >> >>>> >>>>> https://groups.google.com/groups/opt_out. > >> >>>> >>>>> > >> >>>> >>>>> > >> >>>> >>>> > >> >>>> >>>> -- > >> >>>> >>>> You received this message because you are subscribed to the > >> >>>> >>>> Google > >> >>>> >>>> Groups "Puppet Users" group. > >> >>>> >>>> To unsubscribe from this group and stop receiving emails from > >> >>>> >>>> it, > >> >>>> >>>> send an email to [email protected]. > >> >>>> >>>> To post to this group, send email to > >> >>>> >>>> [email protected]. > >> >>>> >>>> Visit this group at > >> >>>> >>>> http://groups.google.com/group/puppet-users?hl=en. > >> >>>> >>>> For more options, visit > >> >>>> >>>> https://groups.google.com/groups/opt_out. > >> >>>> >>>> > >> >>>> >>>> > >> >>>> >>> > >> >>>> >>> -- > >> >>>> >>> You received this message because you are subscribed to the > >> >>>> >>> Google > >> >>>> >>> Groups "Puppet Users" group. > >> >>>> >>> To unsubscribe from this group and stop receiving emails from > it, > >> >>>> >>> send an email to [email protected]. > >> >>>> >>> To post to this group, send email to > >> >>>> >>> [email protected]. > >> >>>> >>> Visit this group at > >> >>>> >>> http://groups.google.com/group/puppet-users?hl=en. > >> >>>> >>> For more options, visit > https://groups.google.com/groups/opt_out. > >> >>>> >>> > >> >>>> >>> > >> >>>> >> > >> >>>> >> -- > >> >>>> >> You received this message because you are subscribed to the > Google > >> >>>> >> Groups "Puppet Users" group. > >> >>>> >> To unsubscribe from this group and stop receiving emails from > it, > >> >>>> >> send an email to [email protected]. > >> >>>> >> To post to this group, send email to > >> >>>> >> [email protected]. > >> >>>> >> Visit this group at > >> >>>> >> http://groups.google.com/group/puppet-users?hl=en. > >> >>>> >> For more options, visit > https://groups.google.com/groups/opt_out. > >> >>>> >> > >> >>>> >> > >> >>>> > > >> >>>> > -- > >> >>>> > You received this message because you are subscribed to the > Google > >> >>>> > Groups "Puppet Users" group. > >> >>>> > To unsubscribe from this group and stop receiving emails from it, > >> >>>> > send > >> >>>> > an email to [email protected]. > >> >>>> > To post to this group, send email to > [email protected]. > >> >>>> > Visit this group at > >> >>>> > http://groups.google.com/group/puppet-users?hl=en. > >> >>>> > For more options, visit https://groups.google.com/groups/opt_out > . > >> >>>> > > >> >>>> > > >> >>>> > >> >>>> -- > >> >>>> You received this message because you are subscribed to the Google > >> >>>> Groups "Puppet Users" group. > >> >>>> To unsubscribe from this group and stop receiving emails from it, > >> >>>> send > >> >>>> an email to [email protected]. > >> >>>> To post to this group, send email to [email protected] > . > >> >>>> Visit this group at > >> >>>> http://groups.google.com/group/puppet-users?hl=en. > >> >>>> For more options, visit https://groups.google.com/groups/opt_out. > >> >>>> > >> >>>> > >> >>> > >> >> > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "Puppet Users" group. > >> > To unsubscribe from this group and stop receiving emails from it, send > >> > an > >> > email to [email protected]. > >> > To post to this group, send email to [email protected]. > >> > Visit this group at http://groups.google.com/group/puppet-users?hl=en > . > >> > For more options, visit https://groups.google.com/groups/opt_out. > >> > > >> > > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Puppet Users" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to [email protected]. > >> To post to this group, send email to [email protected]. > >> Visit this group at http://groups.google.com/group/puppet-users?hl=en. > >> For more options, visit https://groups.google.com/groups/opt_out. > >> > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Users" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > To post to this group, send email to [email protected]. > > Visit this group at http://groups.google.com/group/puppet-users?hl=en. > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/puppet-users?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
