Presuming that this is all because of ssl (various log files and web searches later), I decided to do:
foreman-installer --scenario katello --certs-update-all as per https://github.com/Katello/katello-installer#updating-certificates At first the errors lead me to this https://bugzilla.redhat.com/show_bug.cgi?id=1218251#c0 Which I implemented, which lead me to this: https://bugzilla.redhat.com/show_bug.cgi?id=1356955 But the solutions in the second bug ("delete /etc/pki/katello/nssdb") doesn't work for me. I still get the error listed in that bug, but a more verbose version: [ERROR 2017-02-15 14:59:26 main] qpid-config --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key -b 'amqps:// vmpr-res-utils.unix.petermac.org.au:5671' add exchange topic event --durable returned 1 instead of one of [0] [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/errors.rb:106:in `fail' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/exec.rb:160:in `sync' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:236:in `sync' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:134:in `sync_if_needed' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:88:in `block in perform_changes' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `each' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `perform_changes' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:21:in `evaluate' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:230:in `apply' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:246:in `eval_resource' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `call' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:386:in `block in thinmark' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:385:in `thinmark' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate' [ERROR 2017-02-15 14:59:26 main] /usr/share/gems/gems/kafo-0.9.8/modules/kafo_configure/lib/puppet/parser/functions/add_progress.rb:31:in `evaluate_with_trigger' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:142:in `as_logging_destination' [ERROR 2017-02-15 14:59:26 main] /usr/share/gems/gems/kafo-0.9.8/modules/kafo_configure/lib/kafo/puppet/report_wrapper.rb:34:in `method_missing' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `block in benchmark' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:293:in `override' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:195:in `run' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:350:in `apply_catalog' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:293:in `override' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:344:in `block in run' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:541:in `exit_on_fail' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:344:in `run' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute' [ERROR 2017-02-15 14:59:26 main] /opt/puppetlabs/puppet/bin/puppet:5:in `<main>' [ERROR 2017-02-15 14:59:26 main] /Stage[main]/Certs::Candlepin/Exec[create candlepin qpid exchange]/returns: change from notrun to 0 failed: qpid-config --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key -b 'amqps:// vmpr-res-utils.unix.petermac.org.au:5671' add exchange topic event --durable returned 1 instead of one of [0] [ERROR 2017-02-15 14:59:39 main] Errors encountered during run: [ERROR 2017-02-15 14:59:39 main] qpid-config --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key -b 'amqps:// vmpr-res-utils.unix.petermac.org.au:5671' add exchange topic event --durable returned 1 instead of one of [0] [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/errors.rb:106:in `fail' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/exec.rb:160:in `sync' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:236:in `sync' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:134:in `sync_if_needed' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:88:in `block in perform_changes' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `each' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:87:in `perform_changes' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:21:in `evaluate' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:230:in `apply' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:246:in `eval_resource' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `call' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block (2 levels) in evaluate' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:386:in `block in thinmark' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:385:in `thinmark' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:163:in `block in evaluate' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:154:in `evaluate' [ERROR 2017-02-15 14:59:39 main] /usr/share/gems/gems/kafo-0.9.8/modules/kafo_configure/lib/puppet/parser/functions/add_progress.rb:31:in `evaluate_with_trigger' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:222:in `block in apply' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:155:in `with_destination' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:142:in `as_logging_destination' [ERROR 2017-02-15 14:59:39 main] /usr/share/gems/gems/kafo-0.9.8/modules/kafo_configure/lib/kafo/puppet/report_wrapper.rb:34:in `method_missing' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:221:in `apply' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:171:in `block in apply_catalog' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:223:in `block in benchmark' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/2.1.0/benchmark.rb:294:in `realtime' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:222:in `benchmark' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:170:in `apply_catalog' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:343:in `run_internal' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:221:in `block in run' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:293:in `override' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:195:in `run' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:350:in `apply_catalog' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:274:in `block in main' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:293:in `override' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:225:in `main' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/apply.rb:170:in `run_command' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:344:in `block in run' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:541:in `exit_on_fail' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:344:in `run' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute' [ERROR 2017-02-15 14:59:39 main] /opt/puppetlabs/puppet/bin/puppet:5:in `<main>' [ERROR 2017-02-15 14:59:39 main] /Stage[main]/Certs::Candlepin/Exec[create candlepin qpid exchange]/returns: change from notrun to 0 failed: qpid-config --ssl-certificate /etc/pki/katello/certs/java-client.crt --ssl-key /etc/pki/katello/private/java-client.key -b 'amqps:// vmpr-res-utils.unix.petermac.org.au:5671' add exchange topic event --durable returned 1 instead of one of [0] Any ideas? cheers L. ------ The most dangerous phrase in the language is, "We've always done it this way." - Grace Hopper On 15 February 2017 at 11:24, Lachlan Musicman <[email protected]> wrote: > After a seemingly successful - if traumatic - upgrading of puppet from 3.x > to 4.x as per https://www.theforeman.org/plugins/katello/3.2/upgrade/puppe > t.html I find that neither Candlepin nor pulp are starting. > >> >> CentOS 7.3, Katello 3.2, Foreman 1.13 >> >> [root@vmpr-res-utils etc]# hammer ping >> candlepin: >> Status: FAIL >> Server Response: >> candlepin_auth: >> Status: FAIL >> Server Response: >> pulp: >> Status: FAIL >> Server Response: >> foreman_tasks: >> Status: ok >> Server Response: Duration: 12ms >> >> Well, actually, systemctl has pulp-* as working (status = active >> (running) except the normal active(exited) for pulp_workers.service ) >> >> The Katello Administration/About page show me >> >> Backend System Status Component Status Message >> > > >> candlepin FAIL Connection refused - connect(2) for " >> vmpr-res-utils.unix.petermac.org.au" port 8443 >> > > >> candlepin_auth FAIL A backend service [ Candlepin ] is unreachable >> > > >> foreman_tasks OK >> > > >> pulp FAIL 404 Resource Not Found pulp_auth FAIL Skipped pulp_auth check >> after failed pulp check >> >> > > I'm looking in /var/log/ everything to see what I can see and not coming > up with much tbh, apart from the previously mentioned katelloplunin:208 > error message in /var/log/messages and journalctl > > > All of this points to a CA/Cert error somewhere. I didn't set this system > up, so I'm not 100% sure which certs are where - there seems to be a lot, > in a number of places. > > Since Candlepin isn't working, I thought I'd start there. While searching > I found this (very old) wiki page on Certs https://fedorahosted.org/ > katello/wiki/CertificatesDeployed > > which suggested that > > /etc/gopher/plugins/katelloplugin.conf should look like > > [messaging] > uuid= > url=ssl://$(host):5674 > cacert=/etc/pki/katello/KATELLO-TRUSTED-SSL-CERT > clientcert=/etc/pki/consumer/qpid_client.crt > > > but we have: > > [main] >> enabled=1 >> latency=1 >> >> [messaging] >> url= >> uuid= >> cacert=/etc/rhsm/ca/candlepin-local.pem >> clientcert=/etc/pki/consumer/bundle.pem >> >> > > So, while they are different, one thing is noticable - on my system > /etc/rhsm/ca/candlepin-local.pem doesn't exist, although candlepin seems > to be conf'd to it. Do I need to create this or should I point it to one of > the other files in /etc/rhsm/ca/ - katello-default-ca.pem > katello-server-ca.pem > redhat-uep.pem > > ? > > L. > > > -- You received this message because you are subscribed to the Google Groups "Foreman users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/foreman-users. For more options, visit https://groups.google.com/d/optout.
