[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Also affects: neutron-dynamic-routing (Ubuntu) Importance: Undecided Status: New ** Changed in: neutron-dynamic-routing (Ubuntu Focal) Importance: Undecided => High ** Changed in: neutron-dynamic-routing (Ubuntu Focal) Status: New => Triaged ** Changed in: neutron-dynamic-routing (Ubuntu Groovy) Importance: Undecided => High ** Changed in: neutron-dynamic-routing (Ubuntu Groovy) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in OpenStack Shared File Systems Service (Manila): In Progress Status in masakari: In Progress Status in Mistral: New Status in BaGPipe: In Progress Status in networking-hyperv: In Progress Status in networking-l2gw: In Progress Status in Mellanox backend integration with Neutron (networking-mlnx): In Progress Status in networking-sfc: In Progress Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in oslo.service: In Progress Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in ironic-inspector package in Ubuntu: Triaged Status in magnum package in Ubuntu: Triaged Status in manila package in Ubuntu: Triaged Status in masakari package in Ubuntu: Triaged Status in mistral package in Ubuntu: Triaged Status in networking-bagpipe package in Ubuntu: Triaged Status in networking-hyperv package in Ubuntu: Triaged Status in networking-l2gw package in Ubuntu: Triaged Status in networking-mlnx package in Ubuntu: Triaged Status in networking-sfc package in Ubuntu: New Status in neutron package in Ubuntu: Fix Released Status in neutron-dynamic-routing package in Ubuntu: Triaged Status in nova package in Ubuntu: Fix Released Status in python-oslo.service package in Ubuntu: Triaged Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in magnum source package in Focal: Triaged Status in manila source package in Focal: Triaged Status in masakari source package in Focal: Triaged Status in mistral source package in Focal: Triaged Status in networking-bagpipe source package in Focal: Triaged Status in networking-hyperv source package in Focal: Triaged Status in networking-l2gw source package in Focal: Triaged Status in networking-mlnx source package in Focal: Triaged Status in networking-sfc source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in neutron-dynamic-routing source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in python-oslo.service source package in Focal: Triaged Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in ironic-inspector source package in Groovy: Triaged Status in magnum source package in Groovy: Triaged Status in manila source package in Groovy: Triaged Status in masakari source package in Groovy: Triaged Status in mistral source package in Groovy: Triaged Status in networking-bagpipe source package in Groovy: Triaged Status in networking-hyperv source package in Groovy: Triaged Status in networking-l2gw source package in Groovy: Triaged Status in networking-mlnx source package in Groovy: Triaged Status in networking-sfc source package in Groovy: New Status in neutron source package in Groovy: Fix Released Status in neutron-dynamic-routing source package in Groovy: Triaged Status in nova source package in Groovy: Fix Released Status in python-oslo.service source package in Groovy: Triaged Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Changed in: masakari (Ubuntu Groovy) Importance: Undecided => High ** Changed in: masakari (Ubuntu Groovy) Status: New => Triaged ** Also affects: mistral Importance: Undecided Status: New ** Also affects: mistral (Ubuntu) Importance: Undecided Status: New ** Changed in: mistral (Ubuntu Focal) Importance: Undecided => High ** Changed in: mistral (Ubuntu Focal) Status: New => Triaged ** Changed in: mistral (Ubuntu Groovy) Importance: Undecided => High ** Changed in: mistral (Ubuntu Groovy) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in OpenStack Shared File Systems Service (Manila): In Progress Status in masakari: In Progress Status in Mistral: New Status in BaGPipe: In Progress Status in networking-hyperv: In Progress Status in networking-l2gw: In Progress Status in Mellanox backend integration with Neutron (networking-mlnx): In Progress Status in networking-sfc: In Progress Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in oslo.service: In Progress Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in ironic-inspector package in Ubuntu: Triaged Status in magnum package in Ubuntu: Triaged Status in manila package in Ubuntu: Triaged Status in masakari package in Ubuntu: Triaged Status in mistral package in Ubuntu: Triaged Status in networking-bagpipe package in Ubuntu: Triaged Status in networking-hyperv package in Ubuntu: Triaged Status in networking-l2gw package in Ubuntu: Triaged Status in networking-mlnx package in Ubuntu: Triaged Status in networking-sfc package in Ubuntu: New Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in python-oslo.service package in Ubuntu: Triaged Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in magnum source package in Focal: Triaged Status in manila source package in Focal: Triaged Status in masakari source package in Focal: Triaged Status in mistral source package in Focal: Triaged Status in networking-bagpipe source package in Focal: Triaged Status in networking-hyperv source package in Focal: Triaged Status in networking-l2gw source package in Focal: Triaged Status in networking-mlnx source package in Focal: Triaged Status in networking-sfc source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in python-oslo.service source package in Focal: Triaged Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in ironic-inspector source package in Groovy: Triaged Status in magnum source package in Groovy: Triaged Status in manila source package in Groovy: Triaged Status in masakari source package in Groovy: Triaged Status in mistral source package in Groovy: Triaged Status in networking-bagpipe source package in Groovy: Triaged Status in networking-hyperv source package in Groovy: Triaged Status in networking-l2gw source package in Groovy: Triaged Status in networking-mlnx source package in Groovy: Triaged Status in networking-sfc source package in Groovy: New Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in python-oslo.service source package in Groovy: Triaged Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Changed in: manila (Ubuntu Groovy) Importance: Undecided => High ** Changed in: manila (Ubuntu Groovy) Status: New => Triaged ** Also affects: masakari Importance: Undecided Status: New ** Also affects: masakari (Ubuntu) Importance: Undecided Status: New ** Changed in: masakari (Ubuntu Focal) Importance: Undecided => High ** Changed in: masakari (Ubuntu Focal) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in OpenStack Shared File Systems Service (Manila): In Progress Status in masakari: In Progress Status in Mistral: New Status in BaGPipe: In Progress Status in networking-hyperv: In Progress Status in networking-l2gw: In Progress Status in Mellanox backend integration with Neutron (networking-mlnx): In Progress Status in networking-sfc: In Progress Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in oslo.service: In Progress Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in ironic-inspector package in Ubuntu: Triaged Status in magnum package in Ubuntu: Triaged Status in manila package in Ubuntu: Triaged Status in masakari package in Ubuntu: Triaged Status in mistral package in Ubuntu: Triaged Status in networking-bagpipe package in Ubuntu: Triaged Status in networking-hyperv package in Ubuntu: Triaged Status in networking-l2gw package in Ubuntu: Triaged Status in networking-mlnx package in Ubuntu: Triaged Status in networking-sfc package in Ubuntu: New Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in python-oslo.service package in Ubuntu: Triaged Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in magnum source package in Focal: Triaged Status in manila source package in Focal: Triaged Status in masakari source package in Focal: Triaged Status in mistral source package in Focal: Triaged Status in networking-bagpipe source package in Focal: Triaged Status in networking-hyperv source package in Focal: Triaged Status in networking-l2gw source package in Focal: Triaged Status in networking-mlnx source package in Focal: Triaged Status in networking-sfc source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in python-oslo.service source package in Focal: Triaged Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in ironic-inspector source package in Groovy: Triaged Status in magnum source package in Groovy: Triaged Status in manila source package in Groovy: Triaged Status in masakari source package in Groovy: Triaged Status in mistral source package in Groovy: Triaged Status in networking-bagpipe source package in Groovy: Triaged Status in networking-hyperv source package in Groovy: Triaged Status in networking-l2gw source package in Groovy: Triaged Status in networking-mlnx source package in Groovy: Triaged Status in networking-sfc source package in Groovy: New Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in python-oslo.service source package in Groovy: Triaged Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Also affects: manila Importance: Undecided Status: New ** Also affects: manila (Ubuntu) Importance: Undecided Status: New ** Changed in: manila (Ubuntu Focal) Importance: Undecided => High ** Changed in: manila (Ubuntu Focal) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in OpenStack Shared File Systems Service (Manila): In Progress Status in masakari: In Progress Status in Mistral: New Status in BaGPipe: In Progress Status in networking-hyperv: In Progress Status in networking-l2gw: In Progress Status in Mellanox backend integration with Neutron (networking-mlnx): In Progress Status in networking-sfc: In Progress Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in oslo.service: In Progress Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in ironic-inspector package in Ubuntu: Triaged Status in magnum package in Ubuntu: Triaged Status in manila package in Ubuntu: Triaged Status in masakari package in Ubuntu: Triaged Status in networking-bagpipe package in Ubuntu: Triaged Status in networking-hyperv package in Ubuntu: Triaged Status in networking-l2gw package in Ubuntu: Triaged Status in networking-mlnx package in Ubuntu: Triaged Status in networking-sfc package in Ubuntu: New Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in python-oslo.service package in Ubuntu: Triaged Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in magnum source package in Focal: Triaged Status in manila source package in Focal: Triaged Status in masakari source package in Focal: Triaged Status in networking-bagpipe source package in Focal: Triaged Status in networking-hyperv source package in Focal: Triaged Status in networking-l2gw source package in Focal: Triaged Status in networking-mlnx source package in Focal: Triaged Status in networking-sfc source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in python-oslo.service source package in Focal: Triaged Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in ironic-inspector source package in Groovy: Triaged Status in magnum source package in Groovy: Triaged Status in manila source package in Groovy: Triaged Status in masakari source package in Groovy: Triaged Status in networking-bagpipe source package in Groovy: Triaged Status in networking-hyperv source package in Groovy: Triaged Status in networking-l2gw source package in Groovy: Triaged Status in networking-mlnx source package in Groovy: Triaged Status in networking-sfc source package in Groovy: New Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in python-oslo.service source package in Groovy: Triaged Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Also affects: magnum (Ubuntu) Importance: Undecided Status: New ** Changed in: magnum (Ubuntu Focal) Importance: Undecided => High ** Changed in: magnum (Ubuntu Focal) Status: New => Triaged ** Changed in: magnum (Ubuntu Groovy) Importance: Undecided => High ** Changed in: magnum (Ubuntu Groovy) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in OpenStack Shared File Systems Service (Manila): New Status in BaGPipe: In Progress Status in networking-hyperv: In Progress Status in networking-l2gw: In Progress Status in Mellanox backend integration with Neutron (networking-mlnx): In Progress Status in networking-sfc: In Progress Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in oslo.service: In Progress Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in ironic-inspector package in Ubuntu: Triaged Status in magnum package in Ubuntu: Triaged Status in manila package in Ubuntu: New Status in networking-bagpipe package in Ubuntu: Triaged Status in networking-hyperv package in Ubuntu: Triaged Status in networking-l2gw package in Ubuntu: Triaged Status in networking-mlnx package in Ubuntu: Triaged Status in networking-sfc package in Ubuntu: New Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in python-oslo.service package in Ubuntu: Triaged Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in magnum source package in Focal: Triaged Status in manila source package in Focal: New Status in networking-bagpipe source package in Focal: Triaged Status in networking-hyperv source package in Focal: Triaged Status in networking-l2gw source package in Focal: Triaged Status in networking-mlnx source package in Focal: Triaged Status in networking-sfc source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in python-oslo.service source package in Focal: Triaged Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in ironic-inspector source package in Groovy: Triaged Status in magnum source package in Groovy: Triaged Status in manila source package in Groovy: New Status in networking-bagpipe source package in Groovy: Triaged Status in networking-hyperv source package in Groovy: Triaged Status in networking-l2gw source package in Groovy: Triaged Status in networking-mlnx source package in Groovy: Triaged Status in networking-sfc source package in Groovy: New Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in python-oslo.service source package in Groovy: Triaged Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the following appears to fix this: diff --git a/nova/monkey_patch.py b/nova/monkey_patch.py index a07ff91dac..bb7252c643 100644 --- a/nova/monkey_patch.py +++
[Yahoo-eng-team] [Bug 1876795] [NEW] `cloud-init analyze` should emit a useful error message if it finds no log lines to process
Public bug reported: When run on the log output from Amazon Linux 2 (see bug 1876323), `analyze` fails to find any log lines to process. Currently, it treats this as a normal situation and emits: $ cloud-init analyze show -i ~/Downloads/uhJNysgm.txt -- Boot Record 01 -- The total time elapsed since completing an event is printed after the "@" character. The time the event takes is printed after the "+" character. Total Time: 0.0 seconds 1 boot records analyzed It would be better if it emitted a clear error message that it didn't find any log lines. ** Affects: cloud-init Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1876795 Title: `cloud-init analyze` should emit a useful error message if it finds no log lines to process Status in cloud-init: New Bug description: When run on the log output from Amazon Linux 2 (see bug 1876323), `analyze` fails to find any log lines to process. Currently, it treats this as a normal situation and emits: $ cloud-init analyze show -i ~/Downloads/uhJNysgm.txt -- Boot Record 01 -- The total time elapsed since completing an event is printed after the "@" character. The time the event takes is printed after the "+" character. Total Time: 0.0 seconds 1 boot records analyzed It would be better if it emitted a clear error message that it didn't find any log lines. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1876795/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1876794] [NEW] `cloud-init analyze` depends on the upstream-shipped log format to function
Public bug reported: This is a more general version of bug 1876323: `cloud-init analyze` relies on the log format that upstream ships in order to detect log lines. This means that if logging configuration by users or downstream packagers diverges (as is the case on Amazon Linux 2, for example), then the `analyze` commands will not produce any useful output (or a useful error message). (Ubuntu uses this log format, so this bug does not present on Ubuntu.) ** Affects: cloud-init Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1876794 Title: `cloud-init analyze` depends on the upstream-shipped log format to function Status in cloud-init: New Bug description: This is a more general version of bug 1876323: `cloud-init analyze` relies on the log format that upstream ships in order to detect log lines. This means that if logging configuration by users or downstream packagers diverges (as is the case on Amazon Linux 2, for example), then the `analyze` commands will not produce any useful output (or a useful error message). (Ubuntu uses this log format, so this bug does not present on Ubuntu.) To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1876794/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Also affects: networking-hyperv (Ubuntu) Importance: Undecided Status: New ** Also affects: networking-l2gw Importance: Undecided Status: New ** Also affects: networking-l2gw (Ubuntu) Importance: Undecided Status: New ** Changed in: networking-l2gw (Ubuntu Focal) Importance: Undecided => High ** Changed in: networking-l2gw (Ubuntu Focal) Status: New => Triaged ** Changed in: networking-l2gw (Ubuntu Groovy) Importance: Undecided => High ** Changed in: networking-l2gw (Ubuntu Groovy) Status: New => Triaged ** Changed in: networking-hyperv (Ubuntu Focal) Importance: Undecided => High ** Changed in: networking-hyperv (Ubuntu Focal) Status: New => Triaged ** Changed in: networking-hyperv (Ubuntu Groovy) Importance: Undecided => High ** Changed in: networking-hyperv (Ubuntu Groovy) Status: New => Triaged ** Also affects: networking-mlnx Importance: Undecided Status: New ** Also affects: networking-mlnx (Ubuntu) Importance: Undecided Status: New ** Changed in: networking-mlnx (Ubuntu Focal) Importance: Undecided => High ** Changed in: networking-mlnx (Ubuntu Focal) Status: New => Triaged ** Changed in: networking-mlnx (Ubuntu Groovy) Importance: Undecided => High ** Changed in: networking-mlnx (Ubuntu Groovy) Status: New => Triaged ** Also affects: networking-sfc Importance: Undecided Status: New ** Also affects: networking-sfc (Ubuntu) Importance: Undecided Status: New ** Changed in: networking-sfc (Ubuntu Focal) Importance: Undecided => High ** Changed in: networking-sfc (Ubuntu Focal) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in BaGPipe: In Progress Status in networking-hyperv: In Progress Status in networking-l2gw: In Progress Status in Mellanox backend integration with Neutron (networking-mlnx): In Progress Status in networking-sfc: In Progress Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in oslo.service: In Progress Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in ironic-inspector package in Ubuntu: Triaged Status in networking-bagpipe package in Ubuntu: Triaged Status in networking-hyperv package in Ubuntu: Triaged Status in networking-l2gw package in Ubuntu: Triaged Status in networking-mlnx package in Ubuntu: Triaged Status in networking-sfc package in Ubuntu: New Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in python-oslo.service package in Ubuntu: Triaged Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in networking-bagpipe source package in Focal: Triaged Status in networking-hyperv source package in Focal: Triaged Status in networking-l2gw source package in Focal: Triaged Status in networking-mlnx source package in Focal: Triaged Status in networking-sfc source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in python-oslo.service source package in Focal: Triaged Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in ironic-inspector source package in Groovy: Triaged Status in networking-bagpipe source package in Groovy: Triaged Status in networking-hyperv source package in Groovy: Triaged Status in networking-l2gw source package in Groovy: Triaged Status in networking-mlnx source package in Groovy: Triaged Status in networking-sfc source package in Groovy: New Status in neutron source package in Groovy: Fix Released Status in nova source
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Also affects: networking-bagpipe Importance: Undecided Status: New ** Also affects: networking-bagpipe (Ubuntu) Importance: Undecided Status: New ** Changed in: networking-bagpipe (Ubuntu Focal) Importance: Undecided => High ** Changed in: networking-bagpipe (Ubuntu Focal) Status: New => Triaged ** Changed in: networking-bagpipe (Ubuntu Groovy) Importance: Undecided => High ** Changed in: networking-bagpipe (Ubuntu Groovy) Status: New => Triaged ** Also affects: networking-hyperv Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in BaGPipe: In Progress Status in networking-hyperv: In Progress Status in networking-l2gw: New Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in oslo.service: In Progress Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in ironic-inspector package in Ubuntu: Triaged Status in networking-bagpipe package in Ubuntu: Triaged Status in networking-hyperv package in Ubuntu: New Status in networking-l2gw package in Ubuntu: Triaged Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in python-oslo.service package in Ubuntu: Triaged Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in networking-bagpipe source package in Focal: Triaged Status in networking-hyperv source package in Focal: Triaged Status in networking-l2gw source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in python-oslo.service source package in Focal: Triaged Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in ironic-inspector source package in Groovy: Triaged Status in networking-bagpipe source package in Groovy: Triaged Status in networking-hyperv source package in Groovy: New Status in networking-l2gw source package in Groovy: Triaged Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in python-oslo.service source package in Groovy: Triaged Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the following appears to fix this: diff --git a/nova/monkey_patch.py b/nova/monkey_patch.py index a07ff91dac..bb7252c643 100644 --- a/nova/monkey_patch.py +++ b/nova/monkey_patch.py @@ -59,6 +59,9 @@ def _monkey_patch(): else: eventlet.monkey_patch() +import __original_module_threading +import threading +__original_module_threading.current_thread.__globals__['_active'] = threading._active # NOTE(rpodolyaka): import oslo_service first, so that it makes eventlet # hub use a monotonic clock to avoid issues with drifts of system time (see Similar patches to glance/cmd/api.py, glance/cmd/scrubber.py and glance/cmd/registry.py appears to fix it for glance. [Test Case] 'systemctl status ' should not show the
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Also affects: oslo.service Importance: Undecided Status: New ** Also affects: python-oslo.service (Ubuntu) Importance: Undecided Status: New ** Changed in: python-oslo.service (Ubuntu Focal) Importance: Undecided => High ** Changed in: python-oslo.service (Ubuntu Focal) Status: New => Triaged ** Changed in: python-oslo.service (Ubuntu Groovy) Importance: Undecided => High ** Changed in: python-oslo.service (Ubuntu Groovy) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in oslo.service: In Progress Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in ironic-inspector package in Ubuntu: Triaged Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in python-oslo.service package in Ubuntu: Triaged Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in python-oslo.service source package in Focal: Triaged Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in ironic-inspector source package in Groovy: Triaged Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in python-oslo.service source package in Groovy: Triaged Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the following appears to fix this: diff --git a/nova/monkey_patch.py b/nova/monkey_patch.py index a07ff91dac..bb7252c643 100644 --- a/nova/monkey_patch.py +++ b/nova/monkey_patch.py @@ -59,6 +59,9 @@ def _monkey_patch(): else: eventlet.monkey_patch() +import __original_module_threading +import threading +__original_module_threading.current_thread.__globals__['_active'] = threading._active # NOTE(rpodolyaka): import oslo_service first, so that it makes eventlet # hub use a monotonic clock to avoid issues with drifts of system time (see Similar patches to glance/cmd/api.py, glance/cmd/scrubber.py and glance/cmd/registry.py appears to fix it for glance. [Test Case] 'systemctl status ' should not show the failure reported in [Impact]. [Regression Potential] Should be low. The fix provided is a well-known solution to the problem. Openstack deployment + tempest regression testing will be performed to help reduce regression potential. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1863021/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Also affects: ironic-inspector (Ubuntu) Importance: Undecided Status: New ** Changed in: ironic-inspector (Ubuntu Focal) Importance: Undecided => High ** Changed in: ironic-inspector (Ubuntu Focal) Status: New => Triaged ** Changed in: ironic-inspector (Ubuntu Groovy) Importance: Undecided => High ** Changed in: ironic-inspector (Ubuntu Groovy) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in oslo.service: New Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in ironic-inspector package in Ubuntu: Triaged Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in python-oslo.service package in Ubuntu: Triaged Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in python-oslo.service source package in Focal: Triaged Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in ironic-inspector source package in Groovy: Triaged Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in python-oslo.service source package in Groovy: Triaged Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the following appears to fix this: diff --git a/nova/monkey_patch.py b/nova/monkey_patch.py index a07ff91dac..bb7252c643 100644 --- a/nova/monkey_patch.py +++ b/nova/monkey_patch.py @@ -59,6 +59,9 @@ def _monkey_patch(): else: eventlet.monkey_patch() +import __original_module_threading +import threading +__original_module_threading.current_thread.__globals__['_active'] = threading._active # NOTE(rpodolyaka): import oslo_service first, so that it makes eventlet # hub use a monotonic clock to avoid issues with drifts of system time (see Similar patches to glance/cmd/api.py, glance/cmd/scrubber.py and glance/cmd/registry.py appears to fix it for glance. [Test Case] 'systemctl status ' should not show the failure reported in [Impact]. [Regression Potential] Should be low. The fix provided is a well-known solution to the problem. Openstack deployment + tempest regression testing will be performed to help reduce regression potential. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1863021/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Also affects: ironic (Ubuntu) Importance: Undecided Status: New ** Changed in: ironic (Ubuntu Focal) Importance: Undecided => High ** Changed in: ironic (Ubuntu Focal) Status: New => Triaged ** Changed in: ironic (Ubuntu Groovy) Importance: Undecided => High ** Changed in: ironic (Ubuntu Groovy) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in ironic-inspector package in Ubuntu: Triaged Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in ironic-inspector source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in ironic-inspector source package in Groovy: Triaged Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the following appears to fix this: diff --git a/nova/monkey_patch.py b/nova/monkey_patch.py index a07ff91dac..bb7252c643 100644 --- a/nova/monkey_patch.py +++ b/nova/monkey_patch.py @@ -59,6 +59,9 @@ def _monkey_patch(): else: eventlet.monkey_patch() +import __original_module_threading +import threading +__original_module_threading.current_thread.__globals__['_active'] = threading._active # NOTE(rpodolyaka): import oslo_service first, so that it makes eventlet # hub use a monotonic clock to avoid issues with drifts of system time (see Similar patches to glance/cmd/api.py, glance/cmd/scrubber.py and glance/cmd/registry.py appears to fix it for glance. [Test Case] 'systemctl status ' should not show the failure reported in [Impact]. [Regression Potential] Should be low. The fix provided is a well-known solution to the problem. Openstack deployment + tempest regression testing will be performed to help reduce regression potential. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1863021/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Also affects: barbican (Ubuntu) Importance: Undecided Status: New ** Changed in: barbican (Ubuntu Focal) Importance: Undecided => High ** Changed in: barbican (Ubuntu Focal) Status: New => Triaged ** Changed in: barbican (Ubuntu Groovy) Importance: Undecided => High ** Changed in: barbican (Ubuntu Groovy) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in OpenStack Object Storage (swift): In Progress Status in barbican package in Ubuntu: Triaged Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in ironic package in Ubuntu: Triaged Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in swift package in Ubuntu: Triaged Status in barbican source package in Focal: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in ironic source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in swift source package in Focal: Triaged Status in barbican source package in Groovy: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in ironic source package in Groovy: Triaged Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the following appears to fix this: diff --git a/nova/monkey_patch.py b/nova/monkey_patch.py index a07ff91dac..bb7252c643 100644 --- a/nova/monkey_patch.py +++ b/nova/monkey_patch.py @@ -59,6 +59,9 @@ def _monkey_patch(): else: eventlet.monkey_patch() +import __original_module_threading +import threading +__original_module_threading.current_thread.__globals__['_active'] = threading._active # NOTE(rpodolyaka): import oslo_service first, so that it makes eventlet # hub use a monotonic clock to avoid issues with drifts of system time (see Similar patches to glance/cmd/api.py, glance/cmd/scrubber.py and glance/cmd/registry.py appears to fix it for glance. [Test Case] 'systemctl status ' should not show the failure reported in [Impact]. [Regression Potential] Should be low. The fix provided is a well-known solution to the problem. Openstack deployment + tempest regression testing will be performed to help reduce regression potential. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1863021/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
** Also affects: designate Importance: Undecided Status: New ** Also affects: designate (Ubuntu) Importance: Undecided Status: New ** Changed in: designate (Ubuntu Focal) Importance: Undecided => High ** Changed in: designate (Ubuntu Focal) Status: New => Triaged ** Changed in: designate (Ubuntu Groovy) Importance: Undecided => High ** Changed in: designate (Ubuntu Groovy) Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Designate: In Progress Status in Glance: Fix Released Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in OpenStack Object Storage (swift): In Progress Status in cinder package in Ubuntu: Fix Released Status in designate package in Ubuntu: Triaged Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in swift package in Ubuntu: Triaged Status in cinder source package in Focal: Triaged Status in designate source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in swift source package in Focal: Triaged Status in cinder source package in Groovy: Fix Released Status in designate source package in Groovy: Triaged Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the following appears to fix this: diff --git a/nova/monkey_patch.py b/nova/monkey_patch.py index a07ff91dac..bb7252c643 100644 --- a/nova/monkey_patch.py +++ b/nova/monkey_patch.py @@ -59,6 +59,9 @@ def _monkey_patch(): else: eventlet.monkey_patch() +import __original_module_threading +import threading +__original_module_threading.current_thread.__globals__['_active'] = threading._active # NOTE(rpodolyaka): import oslo_service first, so that it makes eventlet # hub use a monotonic clock to avoid issues with drifts of system time (see Similar patches to glance/cmd/api.py, glance/cmd/scrubber.py and glance/cmd/registry.py appears to fix it for glance. [Test Case] 'systemctl status ' should not show the failure reported in [Impact]. [Regression Potential] Should be low. The fix provided is a well-known solution to the problem. Openstack deployment + tempest regression testing will be performed to help reduce regression potential. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1863021/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1876772] [NEW] Failed to create resource provider
Public bug reported:
Hi,
We have an openstack ansible rocky 18.1.2 setup. It was working working fine.
As part of an upgrade
we have reinstalled it. Now we are unable to create any vms. We have four
compute nodes.
When we try to create a vm on a specific compute node, it is failing, showing
no host found in the nova-conductor and scheduler logs. When I observed the
nova-compute log on the compute, we have found the following errors:
-
2020-05-04 12:59:55.800 11245 ERROR nova.scheduler.client.report
[req-40cd6d9d-3fb5-4a85-b348-53705f22148e - - - - -]
[req-68c57117-a4b0-46b2-a75b-36e8d665da38] Failed to create resource provider
record in placement API for UUID 7e6fb27c-ed5b-4c2c-8373-c99e98da7bcc. Got 409:
{"errors": [{"status": 409, "request_id":
"req-68c57117-a4b0-46b2-a75b-36e8d665da38", "detail": "There was a conflict
when trying to complete your request.\n\n Conflicting resource provider name:
b2b.blr.example.cloud already exists. ", "title": "Conflict"}]}.
2020-05-04 12:59:55.801 11245 ERROR nova.compute.manager raise
exception.ResourceProviderCreationFailed(name=name)
2020-05-04 12:59:55.801 11245 ERROR nova.compute.manager
ResourceProviderCreationFailed: Failed to create resource provider
b2b.blr.example.cloud
2020-05-04 12:59:55.801 11245 ERROR nova.compute.manager
Can you help us solve this error ?
Thanks
Kumar
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1876772
Title:
Failed to create resource provider
Status in OpenStack Compute (nova):
New
Bug description:
Hi,
We have an openstack ansible rocky 18.1.2 setup. It was working working fine.
As part of an upgrade
we have reinstalled it. Now we are unable to create any vms. We have four
compute nodes.
When we try to create a vm on a specific compute node, it is failing, showing
no host found in the nova-conductor and scheduler logs. When I observed the
nova-compute log on the compute, we have found the following errors:
-
2020-05-04 12:59:55.800 11245 ERROR nova.scheduler.client.report
[req-40cd6d9d-3fb5-4a85-b348-53705f22148e - - - - -]
[req-68c57117-a4b0-46b2-a75b-36e8d665da38] Failed to create resource provider
record in placement API for UUID 7e6fb27c-ed5b-4c2c-8373-c99e98da7bcc. Got 409:
{"errors": [{"status": 409, "request_id":
"req-68c57117-a4b0-46b2-a75b-36e8d665da38", "detail": "There was a conflict
when trying to complete your request.\n\n Conflicting resource provider name:
b2b.blr.example.cloud already exists. ", "title": "Conflict"}]}.
2020-05-04 12:59:55.801 11245 ERROR nova.compute.manager raise
exception.ResourceProviderCreationFailed(name=name)
2020-05-04 12:59:55.801 11245 ERROR nova.compute.manager
ResourceProviderCreationFailed: Failed to create resource provider
b2b.blr.example.cloud
2020-05-04 12:59:55.801 11245 ERROR nova.compute.manager
Can you help us solve this error ?
Thanks
Kumar
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1876772/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1876419] Re: Failed to parse json file /etc/glance/metadefs/compute-vmware.json
Reviewed: https://review.opendev.org/724957 Committed: https://git.openstack.org/cgit/openstack/glance/commit/?id=ed3bb4e1fb07c2817b4b7ad2e3e069f78bd839ba Submitter: Zuul Branch:master commit ed3bb4e1fb07c2817b4b7ad2e3e069f78bd839ba Author: Takashi Natsume Date: Sat May 2 14:35:12 2020 +0900 Fix a failure to parse json file Change-Id: I53454c05710ebc45381315a20e10e627cbb65650 Signed-off-by: Takashi Natsume Closes-Bug: #1876419 ** Changed in: glance Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1876419 Title: Failed to parse json file /etc/glance/metadefs/compute-vmware.json Status in Glance: Fix Released Bug description: When starting devstack, the following error occurs. ERROR glance.db.sqlalchemy.metadata [-] Failed to parse json file /etc/glance/metadefs/compute-vmware.json while populating metadata due to: Expecting property name enclosed in double quotes: line 212 column 9 (char 8173): json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 212 column 9 (char 8173) Environment: Ubuntu 18.04.1 glance master commit 10f7130bcf9c66944b6deb0fcebf57bf8e3bf596 devstack master commit 67fd81a484b2e9153e39d583c03ffc929ef6703e To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1876419/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1876752] [NEW] neutron-ovn-db-sync-util fails with KeyError: 'ovn'
Public bug reported: As a result of https://review.opendev.org/#/c/708985/, the sync util can fail because it tries to look up the ovsdb connection from the mechanism_manager under 'ovn' but the sync util registers under 'ovn- sync'. ** Affects: neutron Importance: Undecided Assignee: Terry Wilson (otherwiseguy) Status: In Progress ** Tags: ovn -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1876752 Title: neutron-ovn-db-sync-util fails with KeyError: 'ovn' Status in neutron: In Progress Bug description: As a result of https://review.opendev.org/#/c/708985/, the sync util can fail because it tries to look up the ovsdb connection from the mechanism_manager under 'ovn' but the sync util registers under 'ovn- sync'. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1876752/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1863021] Re: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError
This bug was fixed in the package cinder - 2:16.0.0~b3~git2020041012 .eb915e2db-0ubuntu2 --- cinder (2:16.0.0~b3~git2020041012.eb915e2db-0ubuntu2) groovy; urgency=medium * d/p/monkey-patch-original-current-thread.patch: Cherry-picked from https://review.opendev.org/724754. This fixes neutron service failures with Python 3.8 (LP: #1863021). -- Corey Bryant Thu, 30 Apr 2020 16:44:03 -0400 ** Changed in: cinder (Ubuntu Groovy) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1863021 Title: [SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError Status in Cinder: In Progress Status in Glance: Fix Released Status in neutron: In Progress Status in OpenStack Compute (nova): Fix Released Status in OpenStack Object Storage (swift): In Progress Status in cinder package in Ubuntu: Fix Released Status in glance package in Ubuntu: Fix Released Status in heat package in Ubuntu: Triaged Status in neutron package in Ubuntu: Fix Released Status in nova package in Ubuntu: Fix Released Status in swift package in Ubuntu: Triaged Status in cinder source package in Focal: Triaged Status in glance source package in Focal: Fix Released Status in heat source package in Focal: Triaged Status in neutron source package in Focal: Triaged Status in nova source package in Focal: Fix Released Status in swift source package in Focal: Triaged Status in cinder source package in Groovy: Fix Released Status in glance source package in Groovy: Fix Released Status in heat source package in Groovy: Triaged Status in neutron source package in Groovy: Fix Released Status in nova source package in Groovy: Fix Released Status in swift source package in Groovy: Triaged Bug description: [Impact] This appears to be the same issue documented here: https://github.com/eventlet/eventlet/issues/592 However I seem to only hit this with python3.8. Basically nova, glance, neutron, and cinder services fail with: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Exception ignored in: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork assert len(_active) == 1 AssertionError: Patching nova/monkey_patch.py with the following appears to fix this: diff --git a/nova/monkey_patch.py b/nova/monkey_patch.py index a07ff91dac..bb7252c643 100644 --- a/nova/monkey_patch.py +++ b/nova/monkey_patch.py @@ -59,6 +59,9 @@ def _monkey_patch(): else: eventlet.monkey_patch() +import __original_module_threading +import threading +__original_module_threading.current_thread.__globals__['_active'] = threading._active # NOTE(rpodolyaka): import oslo_service first, so that it makes eventlet # hub use a monotonic clock to avoid issues with drifts of system time (see Similar patches to glance/cmd/api.py, glance/cmd/scrubber.py and glance/cmd/registry.py appears to fix it for glance. [Test Case] 'systemctl status ' should not show the failure reported in [Impact]. [Regression Potential] Should be low. The fix provided is a well-known solution to the problem. Openstack deployment + tempest regression testing will be performed to help reduce regression potential. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1863021/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1744670] Re: In pike ssl deployment horizon cnt retrieve volumes/snapshots and service data via cinderclient
** Changed in: python-cinderclient
Status: Confirmed => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1744670
Title:
In pike ssl deployment horizon cnt retrieve volumes/snapshots and
service data via cinderclient
Status in OpenStack Dashboard (Horizon):
Fix Released
Status in keystoneauth:
Incomplete
Status in python-cinderclient:
Fix Released
Bug description:
Ops packages versions:
openstack-cinder.noarch 11.0.1-1.el7
openstack-dashboard.noarch 12.0.1-1.el7
openstack-glance.noarch 15.0.0-2.el7
openstack-keystone.noarch 12.0.0-1.el7
openstack-neutron.noarch11.0.2-2.el7
openstack-neutron-common.noarch 11.0.2-2.el7
openstack-neutron-ml2.noarch11.0.2-2.el7
openstack-nova-api.noarch 16.0.3-2.el7
openstack-nova-common.noarch16.0.3-2.el7
openstack-nova-conductor.noarch 16.0.3-2.el7
openstack-nova-console.noarch 16.0.3-2.el7
openstack-nova-novncproxy.noarch16.0.3-2.el7
openstack-nova-placement-api.noarch 16.0.3-2.el7
openstack-nova-scheduler.noarch 16.0.3-2.el7
python2-cinderclient.noarch 3.1.0-1.el7
Only after applying hard-coded links to certificate in cinder-client
and keystone, the dashboard starts working:
/cinderclient/client.py
if self.timeout:
kwargs.setdefault('timeout', self.timeout)
self.http_log_req((url, method,), kwargs)
resp = requests.request(
method,
url,
+ cert = ("/etc/keystone/ssl/certs/signing_cert.pem",
"/etc/keystone/ssl/private/signing_key.pem"),
verify=self.verify_cert,
**kwargs)
self.http_log_resp(resp)
/keystoneauth1/session.py
def __init__(self, auth=None, session=None, original_ip=None, verify=True,
cert=None, timeout=None, user_agent=None,
redirect=_DEFAULT_REDIRECT_LIMIT, additional_headers=None,
app_name=None, app_version=None, additional_user_agent=None,
discovery_cache=None):
self.auth = auth
self.session = _construct_session(session)
self.original_ip = original_ip
self.verify = verify
- self.cert = cert
+ self.cert = ("/etc/keystone/ssl/certs/signing_cert.pem",
"/etc/keystone/ssl/private/signing_key.pem")
self.timeout = None
self.redirect = redirect
self.additional_headers = additional_headers or {}
self.app_name = app_name
self.app_version = app_version
self.additional_user_agent = additional_user_agent or []
self._determined_user_agent = None
if discovery_cache is None:
discovery_cache = {}
self._discovery_cache = discovery_cache
To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1744670/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1876198] Re: Setting unlimited qouta for zone
** Project changed: neutron => designate -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1876198 Title: Setting unlimited qouta for zone Status in Designate: New Bug description: in designate configuration Quota driver to use (string value) #quota_driver = storage # Number of zones allowed per tenant (integer value) #quota_zones = 10 # Number of recordsets allowed per zone (integer value) #quota_zone_recordsets = 500 # Number of records allowed per zone (integer value) #quota_zone_records = 500 # Number of records allowed per recordset (integer value) #quota_recordset_records = 20 # Number of recordsets allowed in a zone export (integer value) #quota_api_export_size = 1000 the user should be able to set any of the above configuration as -1 to denote that there is no limit on it but when that set now for quota_zone_recordsets = -1 the user will not be allowed to create recordsets at all To manage notifications about this bug go to: https://bugs.launchpad.net/designate/+bug/1876198/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
