Hi,
Using CentOS 7.3 Katello 3.2 and Foreman 1.13.4
I'm getting an error while promoting a LCE. It is stuck on Paused. I can
skip the erroneous step, but then the repos are broken.
I'm confused about how to cancel, kill and/or unwind the job once it gets
into this state. I have been skipping, deleting and trying again, but that
seems like an awfully long way around?
When I look in dynaflow it's always this step that fails
85: Actions::Pulp::Repository::CopyYumMetadataFile (skipping) [ 57540.42s /
1.59s ]
I note that the error below has references to Katello 3.2.1.1:
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/middleware/remote_action.rb:16:in
`block in run'
But I thought I was using Katello 3.2.2 - and, infact, if I look at the
about page I see the inconsistency there too - the plugins screen and the
"installed packages" screen disagree (see attached screenshots attached).
Could this be the issue?
Here is the error:
Action:
Actions::Pulp::Repository::CopyYumMetadataFile
Input:
{"source_pulp_id"=>
"Peter_MacCallum_Cancer_Centre-GAFFA-3_0-_sssd-Copr_repository_for_sssd_1_15",
"target_pulp_id"=>
"Peter_MacCallum_Cancer_Centre-Gaffa-dev-GAFFA-_sssd-Copr_repository_for_sssd_1_15",
"clauses"=>nil,
"remote_user"=>"admin",
"remote_cp_user"=>"admin"}
Output:
{"pulp_tasks"=>
[{"exception"=>nil,
"task_type"=>
"pulp.server.managers.repo.unit_association.associate_from_repo",
"_href"=>"/pulp/api/v2/tasks/2a32001e-ed4d-4c83-9a02-cffd045b8f30/",
"task_id"=>"2a32001e-ed4d-4c83-9a02-cffd045b8f30",
"tags"=>
["pulp:repository:Peter_MacCallum_Cancer_Centre-Gaffa-dev-GAFFA-_sssd-Copr_repository_for_sssd_1_15",
"pulp:repository:Peter_MacCallum_Cancer_Centre-GAFFA-3_0-_sssd-Copr_repository_for_sssd_1_15",
"pulp:action:associate"],
"finish_time"=>"2017-02-20T21:40:42Z",
"_ns"=>"task_status",
"start_time"=>"2017-02-20T21:40:41Z",
"traceback"=>
"Traceback (most recent call last):\n File
\"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 240, in
trace_task\n R = retval = fun(*args, **kwargs)\n File
\"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line
484, in __call__\n return super(Task, self).__call__(*args,
**kwargs)\n File
\"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line
103, in __call__\n return super(PulpTask, self).__call__(*args,
**kwargs)\n File
\"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 437, in
__protected_call__\n return self.run(*args, **kwargs)\n File
\"/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py\",
line 273, in associate_from_repo\n units=transfer_units)\n File
\"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py\",
line 58, in import_units\n return associate.associate(source_repo,
dest_repo, import_conduit, config, units)\n File
\"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py\",
line 53, in associate\n associated_units =
set([_associate_unit(dest_repo, unit) for unit in units])\n File
\"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py\",
line 339, in _associate_unit\n return associate_copy_for_repo(unit,
dest_repo, True)\n File
\"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py\",
line 379, in associate_copy_for_repo\n
new_unit.safe_import_content(unit._storage_path)\n File
\"/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py\",
line 905, in safe_import_content\n self.import_content(path,
location)\n File
\"/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py\",
line 879, in import_content\n storage.put(self, path, location)\n
File \"/usr/lib/python2.7/site-packages/pulp/server/content/storage.py\",
line 123, in put\n mkdir(os.path.dirname(destination))\n File
\"/usr/lib/python2.7/site-packages/pulp/server/content/storage.py\",
line 20, in mkdir\n os.makedirs(path)\n File
\"/usr/lib64/python2.7/os.py\", line 157, in makedirs\n mkdir(name,
mode)\nPulpExecutionException: Pulp exception occurred:
PulpExecutionException\n",
"spawned_tasks"=>[],
"progress_report"=>{},
"queue"=>
"reserved_resource_worke...@vmpr-res-utils.unix.petermac.org.au.dq",
"state"=>"error",
"worker_name"=>
"reserved_resource_worke...@vmpr-res-utils.unix.petermac.org.au",
"error"=>
{"code"=>"PLP0000",
"data"=>{},
"description"=>"Pulp exception occurred: PulpExecutionException",
"sub_errors"=>[]},
"_id"=>{"$oid"=>"58ab62595ddf3db34f386add"},
"id"=>"58ab62595ddf3db34f386add"}],
"poll_attempts"=>{"total"=>3, "failed"=>3}}
Exception:
Katello::Errors::PulpError: PLP0000: Pulp exception occurred:
PulpExecutionException
Backtrace:
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/pulp/abstract_async_task.rb:121:in
`block in external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/pulp/abstract_async_task.rb:119:in
`each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/pulp/abstract_async_task.rb:119:in
`external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/pulp/repository/abstract_copy_content.rb:40:in
`external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action/polling.rb:98:in
`poll_external_task_with_rescue'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action/polling.rb:21:in
`run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action/cancellable.rb:9:in
`run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/pulp/abstract_async_task.rb:45:in
`run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action.rb:506:in
`block (3 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:26:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:26:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware.rb:17:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware.rb:30:in
`run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:22:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:26:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware.rb:17:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/middleware/remote_action.rb:16:in
`block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/middleware/remote_action.rb:40:in
`block in as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/models/katello/concerns/user_extensions.rb:21:in
`cp_config'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/middleware/remote_action.rb:27:in
`as_cp_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/middleware/remote_action.rb:39:in
`as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/middleware/remote_action.rb:16:in
`run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:22:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:26:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware.rb:17:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action/progress.rb:30:in
`with_progress_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action/progress.rb:16:in
`run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:22:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:26:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware.rb:17:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/middleware/keep_locale.rb:11:in
`block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/middleware/keep_locale.rb:22:in
`with_locale'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.2.1.1/app/lib/actions/middleware/keep_locale.rb:11:in
`run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:22:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:26:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware.rb:17:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware.rb:30:in
`run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/stack.rb:22:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/middleware/world.rb:30:in
`execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action.rb:505:in
`block (2 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action.rb:504:in
`catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action.rb:504:in
`block in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action.rb:419:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action.rb:419:in
`block in with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action.rb:419:in
`catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action.rb:419:in
`with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action.rb:499:in
`execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/action.rb:260:in
`execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in
`block (2 levels) in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/execution_plan/steps/abstract.rb:155:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/execution_plan/steps/abstract.rb:155:in
`with_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in
`block in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in
`open_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in
`execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/executors/parallel/worker.rb:15:in
`block in on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in
`block in assigns'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in
`tap'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in
`assigns'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:in
`match_value'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in
`block in match?'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in
`each'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in
`match?'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:in
`match'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/executors/parallel/worker.rb:12:in
`on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/context.rb:46:in
`on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/executes_context.rb:7:in
`on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.15/lib/dynflow/actor.rb:26:in
`on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/awaits.rb:15:in
`on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in
`on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:38:in
`process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:31:in
`process_envelopes?'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/buffer.rb:20:in
`on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/termination.rb:55:in
`on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/removes_child.rb:10:in
`on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/abstract.rb:25:in
`pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/behaviour/sets_results.rb:14:in
`on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:161:in
`process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:95:in
`block in on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:118:in
`block (2 levels) in schedule_execution'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
`block in synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
`synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/synchronization/mri_lockable_object.rb:38:in
`synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.0/lib/concurrent/actor/core.rb:115:in
`block in schedule_execution'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:18:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:96:in
`work'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/serialized_execution.rb:77:in
`block in call_job'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:333:in
`run_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:322:in
`block (3 levels) in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in
`loop'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:305:in
`block (2 levels) in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in
`catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.0/lib/concurrent/executor/ruby_thread_pool_executor.rb:304:in
`block in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in
`call'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in
`block in create_with_logging_context'
------
The most dangerous phrase in the language is, "We've always done it this
way."
- Grace Hopper
--
You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to foreman-users+unsubscr...@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.
