You'll need to regenerate a certificate tarball for each proxy and deploy it with the installer. This is due to the hostname change on the main server and how the certificates are used to sync from the server to the proxy.
On Mar 14, 2017 10:12 PM, "jpavel" <[email protected]> wrote: > I'm running foreman 1.14.2, and Katello 3.3.0. > > On the foreman server, I'm seeing these messages: > [Wed Mar 15 01:57:02.739257 2017] [ssl:error] [pid 18720] [client > 10.9.0.1:42382] AH02039: Certificate Verification: Error (20): unable to > get local issuer certificate > ... > eventually followed by a burst of something like this: > 2017-03-15 01:57:02 [foreman-tasks/action] [E] RPM1004: Error retrieving > metadata: Not found (Katello::Errors::PulpError) > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/pulp/abstract_async_task.rb:121:in `block in > external_task=' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/pulp/abstract_async_task.rb:119:in `each' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/pulp/abstract_async_task.rb:119:in `external_task=' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action/polling.rb:98:in `poll_external_task_with_rescue' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action/polling.rb:21:in `run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action/cancellable.rb:9:in `run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/pulp/abstract_async_task.rb:45:in `run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action.rb:506:in `block (3 levels) in execute_run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:26:in `call' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:26:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware.rb:17:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware.rb:30:in `run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:22:in `call' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:26:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware.rb:17:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/middleware/remote_action.rb:16:in `block in run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/middleware/remote_action.rb:40:in `block in > as_remote_user' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/models/katello/concerns/user_extensions.rb:21:in `cp_config' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/middleware/remote_action.rb:16:in `run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:22:in `call' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:26:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware.rb:17:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action/progress.rb:30:in `with_progress_calculation' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action/progress.rb:16:in `run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:22:in `call' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:26:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware.rb:17:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/middleware/keep_locale.rb:11:in `block in run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale' > | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.3.0. > 1/app/lib/actions/middleware/keep_locale.rb:11:in `run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:22:in `call' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:26:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware.rb:17:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware.rb:30:in `run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/stack.rb:22:in `call' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/middleware/world.rb:30:in `execute' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action.rb:505:in `block (2 levels) in execute_run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action.rb:504:in `catch' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action.rb:504:in `block in execute_run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action.rb:419:in `call' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action.rb:419:in `block in with_error_handling' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action.rb:419:in `catch' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action.rb:419:in `with_error_handling' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action.rb:499:in `execute_run' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/action.rb:260:in `execute' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 > levels) in execute' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/execution_plan/steps/abstract.rb:155:in `call' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/execution_plan/steps/abstract.rb:155:in > `with_meta_calculation' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in > execute' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in > `open_action' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/director.rb:55:in `execute' > | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8. > 17/lib/dynflow/executors/parallel/worker.rb:11:in `on_message' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in `on_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in > `on_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' > | > /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.17/lib/dynflow/actor.rb:26:in > `on_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/awaits.rb:15:in > `on_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in > `on_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:38:in > `process_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:31:in > `process_envelopes?' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:20:in > `on_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/termination.rb:55:in > `on_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/removes_child.rb:10:in > `on_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in > `on_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/core.rb:161:in `process_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/core.rb:95:in `block in on_envelope' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/core.rb:118:in `block (2 levels) in > schedule_execution' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in > `block in synchronize' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in > `synchronize' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in > `synchronize' > | /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent- > ruby-edge-0.2.0/lib/concurrent/actor/core.rb:115:in `block in > schedule_execution' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in `call' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:96:in `work' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:77:in `block > in call_job' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in > `call' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in > `run_task' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:322:in > `block (3 levels) in create_worker' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in > `loop' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in > `block (2 levels) in create_worker' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in > `catch' > | /opt/rh/sclo-ror42/root/usr/share/gems/gems/concurrent- > ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in > `block in create_worker' > | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/ > lib/logging/diagnostic_context.rb:323:in `call' > | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/ > lib/logging/diagnostic_context.rb:323:in `block in > create_with_logging_context' > > > On the proxy side, I see this: > Mar 15 01:56:58 smart-proxy-02 pulp: nectar.downloaders.threaded:ERROR: > Skipping requests to <foreman server> due to repeated connection failures: > [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:579) > ... > eventually followed by this: > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31246-98880) Exception while retrieving metadata for repository <blah blah> > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31246-98880) Traceback (most recent call last): > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31246-98880) File "/usr/lib/python2.7/site- > packages/pulp_puppet/plugins/importers/forge.py", line 113, in > _parse_metadata > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31246-98880) metadata_json_docs = downloader.retrieve_metadata( > self.progress_report) > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31246-98880) File "/usr/lib/python2.7/site- > packages/pulp_puppet/plugins/importers/downloaders/web.py", line 57, in > retrieve_metadata > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31246-98880) raise exceptions.FileRetrievalException(report. > error_msg) > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31246-98880) FileRetrievalException: FileRetrievalException: A connection > error occurred > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31234-04096) Exception while retrieving metadata for repository > <nuance_mobility-Production-Smart-Proxy> > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31234-04096) Traceback (most recent call last): > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31234-04096) File "/usr/lib/python2.7/site- > packages/pulp_puppet/plugins/importers/forge.py", line 113, in > _parse_metadata > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31234-04096) metadata_json_docs = downloader.retrieve_metadata( > self.progress_report) > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31234-04096) File "/usr/lib/python2.7/site- > packages/pulp_puppet/plugins/importers/downloaders/web.py", line 57, in > retrieve_metadata > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31234-04096) raise exceptions.FileRetrievalException(report. > error_msg) > Mar 15 01:57:04 smart-proxy-02 pulp: > pulp_puppet.plugins.importers.forge:ERROR: > (31234-04096) FileRetrievalException: FileRetrievalException: A connection > error occurred > > On my foreman server, pulp is configured with this: > [security] > cacert: /etc/pki/pulp/ca.crt > cakey: /etc/pki/pulp/ca.key > > And the proxy is configured with this: > [security] > cacert: /etc/pki/katello/certs/katello-default-ca.crt > cakey: /etc/pki/pulp/ca.key > > *Every* single proxy is experiencing the same error. I installed a new > proxy to test it, and it fails to sync with the same error as well. > > It's probably worth noting that I did change the name of the foreman > server about a week ago (this was succeeding prior to that), and I used a > new script: > https://github.com/Katello/katello-packaging/pull/323/commits > > Everything seemed to be fine after that, but I only recently got around to > checking out the proxies. > > From the proxy, I could run: > openssl s_client -connect foreman-01.prod.mcs.som.mob.nuance.com:443 > -CAfile /etc/pki/katello/certs/katello-default-ca.crt > ...and it completes successfully. > > Thanks for any help pointing me in the right direction! > > -- > You received this message because you are subscribed to the Google Groups > "Foreman users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/foreman-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Foreman users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/foreman-users. For more options, visit https://groups.google.com/d/optout.
