Reviewed: https://review.openstack.org/628302 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a371f11835e422c06be038d8ce79f38e8f64ab0d Submitter: Zuul Branch: master
commit a371f11835e422c06be038d8ce79f38e8f64ab0d Author: Matt Riedemann <[email protected]> Date: Thu Jan 3 16:12:56 2019 -0500 Fix circular import in nova.privsep.utils Commit 26521718bdba3bccbf6270e26b76754c26304658 imported nova.utils into nova.privsep.utils which can cause an ImportError due to an ArgsAlreadyParseError because of nova.utils importing nova.conf which registers config options. For some obscure reason, this is only being noticed when using [libvirt]/image_type=lvm, so something in the libvirt lvm image backend using privsep is tickling this import error, but regardless the nova.privsep code should avoid importing stuff from the rest of nova, so this change simply adds a simple "generate_random_string" utility to nova.privsep.utils to avoid the import. Change-Id: I3799869fd4217d12b92d79e27484043ef5b8dc13 Closes-Bug: #1808247 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1808247 Title: nova-lvm job failing with ArgsAlreadyParsedError (privsep) Status in OpenStack Compute (nova): Fix Released Bug description: Seen here: http://logs.openstack.org/43/619143/12/check/nova- lvm/786180c/logs/screen-n-cpu.txt.gz?level=TRACE#_Dec_12_12_35_39_607002 Dec 12 12:35:39.607002 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [None req-107a6ea1-f573-4c73-b191-2498e9d6d5ea tempest-AutoAllocateNetworkTest-1112737108 tempest-AutoAllocateNetworkTest-1112737108] [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] Instance failed to spawn: ArgsAlreadyParsedError: arguments already parsed Dec 12 12:35:39.607240 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] Traceback (most recent call last): Dec 12 12:35:39.607458 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/compute/manager.py", line 2361, in _build_resources Dec 12 12:35:39.607676 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] yield resources Dec 12 12:35:39.607886 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/compute/manager.py", line 2125, in _build_and_run_instance Dec 12 12:35:39.608118 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] block_device_info=block_device_info) Dec 12 12:35:39.608334 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3063, in spawn Dec 12 12:35:39.608549 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] block_device_info=block_device_info) Dec 12 12:35:39.608761 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3441, in _create_image Dec 12 12:35:39.608975 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] fallback_from_host) Dec 12 12:35:39.609190 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3532, in _create_and_inject_local_root Dec 12 12:35:39.609415 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] instance, size, fallback_from_host) Dec 12 12:35:39.609654 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 7830, in _try_fetch_image_cache Dec 12 12:35:39.609878 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] trusted_certs=instance.trusted_certs) Dec 12 12:35:39.610094 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 243, in cache Dec 12 12:35:39.610310 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] *args, **kwargs) Dec 12 12:35:39.610524 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 763, in create_image Dec 12 12:35:39.610758 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] create_lvm_image(base, size) Dec 12 12:35:39.610973 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 328, in inner Dec 12 12:35:39.611198 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] return f(*args, **kwargs) Dec 12 12:35:39.611423 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 731, in create_lvm_image Dec 12 12:35:39.611639 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] run_as_root=True) Dec 12 12:35:39.611852 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/virt/images.py", line 117, in convert_image_unsafe Dec 12 12:35:39.612060 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] _convert_image(source, dest, None, out_format, run_as_root) Dec 12 12:35:39.612272 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/opt/stack/new/nova/nova/virt/images.py", line 128, in _convert_image Dec 12 12:35:39.612506 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] source, dest, in_format, out_format, CONF.instances_path) Dec 12 12:35:39.612745 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/priv_context.py", line 207, in _wrap Dec 12 12:35:39.612960 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] return self.channel.remote_call(name, args, kwargs) Dec 12 12:35:39.613174 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] File "/usr/local/lib/python2.7/dist-packages/oslo_privsep/daemon.py", line 202, in remote_call Dec 12 12:35:39.613393 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] raise exc_type(*result[2]) Dec 12 12:35:39.613616 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] ArgsAlreadyParsedError: arguments already parsed Dec 12 12:35:39.613830 ubuntu-xenial-rax-iad-0001148680 nova-compute[29772]: ERROR nova.compute.manager [instance: d0a6774a-7b73-431b-8768-706e4e4fcc68] The nova-lvm job is currently non-voting which is why no one probably noticed the regression. https://github.com/openstack/nova/blob/c72dafad8011198b78fd906f2fc7d1126e3f79f3/.zuul.yaml#L115 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1808247/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
