[Yahoo-eng-team] [Bug 1857114] Re: zuul job dsvm-grenade error

[Takashi Kajinami]

This does not look like an actual problem in oslo.cache, but it's very
likely a deployment problem

** Changed in: oslo.cache
       Status: New => Incomplete

** Changed in: oslo.cache
       Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1857114

Title:
  zuul job dsvm-grenade error

Status in networking-ovn:
  Incomplete
Status in OpenStack Compute (nova):
  Incomplete
Status in oslo.cache:
  Invalid

Bug description:
  Many tasks have the following errors

  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage [None 
req-12e00068-1f43-43ea-9608-98b5c83b16bc None None] 
Error attempting to run <function fill_virtual_interface_list 
at 0x7f67744092f0>: ModuleNotFoundError: No module named 'memcache'
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage Traceback (most 
recent call last):
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/cmd/manage.py", line 504, in _run_migration
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     found, done 
= migration_meth(ctxt, count)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/usr/local/lib/python3.6/dist-packages/oslo_db/sqlalchemy/enginefacade.py", 
line 1015, in wrapper
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     return 
fn(*args, **kwargs)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/objects/virtual_interface.py", line 279, in 
fill_virtual_interface_list
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     
_set_or_delete_marker_for_migrate_instances(cctxt, marker)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/db/sqlalchemy/api.py", line 223, in wrapped
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     return 
f(context, *args, **kwargs)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/objects/virtual_interface.py", line 305, in 
_set_or_delete_marker_for_migrate_instances
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     
instance.create()
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 
226, in wrapper
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     return 
fn(self, *args, **kwargs)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/objects/instance.py", line 629, in create
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     
self._load_ec2_ids()
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/objects/instance.py", line 949, in _load_ec2_ids
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     self.ec2_ids 
= objects.EC2Ids.get_by_instance(self._context, self)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 
184, in wrapper
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     result = 
fn(cls, context, *args, **kwargs)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/objects/ec2.py", line 231, in get_by_instance
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     ec2_ids = 
cls._get_ec2_ids(context, instance)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/objects/ec2.py", line 217, in _get_ec2_ids
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     
ec2_ids['instance_id'] = id_to_ec2_inst_id(context, instance.uuid)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/objects/ec2.py", line 57, in id_to_ec2_inst_id
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     int_id = 
get_int_id_from_instance_uuid(context, instance_id)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/objects/ec2.py", line 36, in memoizer
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     _CACHE = 
cache_utils.get_client(expiration_time=_CACHE_TIME)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/cache_utils.py", line 54, in get_client
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     
_get_default_cache_region(expiration_time=expiration_time))
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/opt/stack/new/nova/nova/cache_utils.py", line 65, in _get_default_cache_region
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     
cache.configure_cache_region(CONF, region)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/usr/local/lib/python3.6/dist-packages/oslo_cache/core.py", line 235, in 
configure_cache_region
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     '%s.' % 
conf.cache.config_prefix)
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/usr/local/lib/python3.6/dist-packages/dogpile/cache/region.py", line 608, in 
configure_from_config
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     
"%sreplace_existing_backend" % prefix, False
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/usr/local/lib/python3.6/dist-packages/dogpile/cache/region.py", line 436, in 
configure
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage     
_config_argument_dict, _config_prefix
  2019-12-19 05:32:35.240 | ERROR nova.cmd.manage   File 
"/usr/local/lib/python3.6/dist-packages/dogpile/cache/api.py", line 95, in 
from_config_dict
  2019-12-19 05:32:35.241 | ERROR nova.cmd.manage     for key in 
config_dict
  2019-12-19 05:32:35.241 | ERROR nova.cmd.manage   File 
"/usr/local/lib/python3.6/dist-packages/dogpile/cache/backends/memcached.py", 
line 207, in __init__
  2019-12-19 05:32:35.241 | ERROR nova.cmd.manage     
super(MemcacheArgs, self).__init__(arguments)
  2019-12-19 05:32:35.241 | ERROR nova.cmd.manage   File 
"/usr/local/lib/python3.6/dist-packages/dogpile/cache/backends/memcached.py", 
line 115, in __init__
  2019-12-19 05:32:35.241 | ERROR nova.cmd.manage     
self._imports()
  2019-12-19 05:32:35.241 | ERROR nova.cmd.manage   File 
"/usr/local/lib/python3.6/dist-packages/dogpile/cache/backends/memcached.py", 
line 287, in _imports
  2019-12-19 05:32:35.241 | ERROR nova.cmd.manage     import 
memcache  # noqa
  2019-12-19 05:32:35.241 | ERROR nova.cmd.manage 
ModuleNotFoundError: No module named 'memcache'


  full log:
  
https://6081989ad82b8be7d120-53e9ef0ccb08d29c43f34421d87aa388.ssl.cf1.rackcdn.com/698632/3/check/networking-ovn-dsvm-grenade/72e94e8/logs/grenade.sh.txt.gz

To manage notifications about this bug go to:
https://bugs.launchpad.net/networking-ovn/+bug/1857114/+subscriptions


-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to     : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp