This started spiking in failures in the last 24 hours once oslo.log 3.26.0 was used in upper-constraints on master, and it's failing in the journal handler:
https://github.com/openstack/oslo.log/blob/3.26.0/oslo_log/handlers.py#L120 The switch in devstack to use systemd by default happened yesterday too: https://github.com/openstack- dev/devstack/commit/7f8df450dbdfe754eb3f7311de56327878a3d1c3 So everything is running with that handler now and we're seeing the spike in failures: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Setting%20instance%20vm_state%20to%20ERROR%5C%22%20AND%20message%3A%5C%22_unplug_vifs%5C%22%20AND%20message%3A%5C%22UnicodeDecodeError%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d It's failing specifically hard in the cells v1 job, I'm not really sure why. ** Also affects: oslo.log Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1580728 Title: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 386: ordinal not in range(128) in nova.virt.libvirt.vif:unplug with unicode instance.display_name Status in devstack: Confirmed Status in OpenStack Compute (nova): Triaged Status in oslo.log: New Status in oslo.versionedobjects: New Bug description: I saw this in the n-cpu logs for a xenproject CI run: http://logs.openstack.xenproject.org/00/315100/1/check/dsvm-tempest- xen/9649dc5/logs/screen-n-cpu.txt.gz 2016-05-11 16:19:09.457 27252 INFO nova.virt.libvirt.driver [-] [instance: 76c4ad96-87dd-4300-acdc-cbe65d3aa0a6] Instance destroyed successfully. Traceback (most recent call last): File "/usr/lib/python2.7/logging/__init__.py", line 851, in emit msg = self.format(record) File "/usr/local/lib/python2.7/dist-packages/oslo_log/handlers.py", line 73, in format return logging.StreamHandler.format(self, record) File "/usr/lib/python2.7/logging/__init__.py", line 724, in format return fmt.format(record) File "/usr/local/lib/python2.7/dist-packages/oslo_log/formatters.py", line 265, in format return logging.Formatter.format(self, record) File "/usr/lib/python2.7/logging/__init__.py", line 464, in format record.message = record.getMessage() File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage msg = msg % self.args UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 386: ordinal not in range(128) Logged from file vif.py, line 966 That would be logging the vif object in unplug: https://github.com/openstack/nova/blob/15abb39ef20ae76d602d50e67e43c3500a00cd3e/nova/virt/libvirt/vif.py#L966 To manage notifications about this bug go to: https://bugs.launchpad.net/devstack/+bug/1580728/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
