Public bug reported: Running unit tests of Horizon in Debian Unstable leads to many unit test failures like this one below.
My instinct tells me that this is Python 3.12 related, but I'm not sure. Here's a pip freeze output with the installed build-dependency of the package: alabaster==0.7.12 appdirs==1.4.4 asgiref==3.7.2 async-timeout==4.0.3 attrs==23.1.0 autopage==0.4.0 Babel==2.10.3 blinker==1.6.2 calmjs==3.4.2 calmjs.parse==1.2.5 calmjs.types==1.0.1 certifi==2023.7.22 chardet==5.2.0 charset-normalizer==3.2.0 cliff==4.2.0 cmd2==2.4.3+ds coverage==7.2.7 cryptography==38.0.4 csscompressor==0.9.5 debtcollector==2.5.0 decorator==5.1.1 Deprecated==1.2.14 deprecation==2.0.7 Django==4.2.5 django-appconf==1.0.5 django-compressor==4.0 django-debreach==2.1.0 django-pyscss==2.0.2 dnspython==2.4.2 docutils==0.19 dogpile.cache==1.1.8 dulwich==0.21.6 enmerkar==0.7.1 eventlet==0.33.1 exceptiongroup==1.1.3 execnet==2.0.0 extras==1.0.0 fasteners==0.17.3 fixtures==4.0.1 flake8==5.0.4 freezegun==1.2.1 futurist==2.4.1 greenlet==2.0.2 h11==0.14.0 hacking==4.1.0 idna==3.3 imagesize==1.4.1 importlib-metadata==4.12.0 iniconfig==1.1.1 iso8601==1.0.2 jaraco.classes==3.2.1 jeepney==0.8.0 Jinja2==3.1.2 jmespath==1.0.1 jsonpatch==1.32 jsonpointer==2.3 jsonschema==4.10.3 keyring==24.2.0 keystoneauth1==5.3.0 lxml==4.9.3 Mako==1.2.4.dev0 MarkupSafe==2.1.3 mccabe==0.7.0 monotonic==1.6 more-itertools==10.1.0 msgpack==1.0.3 netaddr==0.8.0 netifaces==0.11.0 oauthlib==3.2.2 openstackdocstheme==1.20.0 openstacksdk==1.0.1 os-client-config==2.1.0 os-service-types==1.7.0 osc-lib==2.8.1 oslo.concurrency==5.1.1 oslo.config==9.1.1 oslo.context==5.1.1 oslo.i18n==6.0.0 oslo.log==5.2.0 oslo.policy==4.1.1 oslo.serialization==5.1.1 oslo.upgradecheck==2.1.1 oslo.utils==6.1.0 osprofiler==3.4.3 outcome==1.2.0 packaging==23.1 pbr==5.11.1 pep8==1.7.1 pluggy==1.3.0 ply==3.11 prettytable==3.6.0 pycodestyle==2.10.0 pyflakes==2.5.0 Pygments==2.15.1 pyinotify==0.9.6 PyJWT==2.7.0 pymongo==3.11.0 pyOpenSSL==23.0.0 pyparsing==3.1.1 pyperclip==1.8.2 pyrsistent==0.18.1 pyScss==1.4.0 pytest==7.4.2 pytest-django==4.5.2 pytest-xdist==3.3.1 python-cinderclient==9.3.0 python-dateutil==2.8.2 python-glanceclient==4.3.0 python-keystoneclient==5.1.0 python-memcached==1.58 python-neutronclient==9.0.0 python-novaclient==18.3.0 python-swiftclient==4.2.0 pytz==2023.3.post1 PyYAML==6.0.1 rcssmin==1.1.0 redis==4.3.4 requests==2.31.0 requestsexceptions==1.4.0 rfc3986==1.5.0 rjsmin==1.2.0 roman==3.3 SecretStorage==3.3.3 selenium==4.13.0 semantic-version==2.9.0 simplejson==3.19.1 six==1.16.0 sniffio==1.2.0 snowballstemmer==2.2.0 sortedcontainers==2.4.0 Sphinx==5.3.0 sqlparse==0.4.2 stevedore==5.1.0 testscenarios==0.5.0 testtools==2.5.0 trio==0.22.2 trio-websocket==0.10.3 urllib3==1.26.16 warlock==2.0.1 wcwidth==0.2.5 WebOb==1.8.6 wrapt==1.14.1 wsproto==1.2.0 XStatic==1.0.3 XStatic-Angular==1.8.2.2 XStatic-Angular-Bootstrap==2.5.0.0 XStatic-Angular-FileUpload==12.0.4.0 XStatic-Angular-Gettext==2.4.1.0 XStatic-Angular-lrdragndrop==1.0.2.2 XStatic-Angular-Schema-Form==0.8.13.0 XStatic-angular-ui-router==0.3.1.4 XStatic-Bootstrap-Datepicker==1.3.1.0 XStatic-Bootstrap-SCSS==3.4.1.0 XStatic-bootswatch==3.3.7.0 XStatic-D3==3.5.17.0 XStatic-Font-Awesome==4.7.0.0 XStatic-Hogan==2.0.0.2 XStatic-Jasmine==2.4.1.0 XStatic-jQuery==3.5.1.0 XStatic-JQuery-Migrate==3.3.2.1 XStatic-jquery-ui==1.12.0.1 XStatic-JQuery.quicksearch==2.0.4.1 XStatic-JQuery.TableSorter==2.14.5.1 XStatic-JSEncrypt==2.3.1.1 XStatic-Magic-Search==0.2.5.1 XStatic-mdi==1.6.50.2 XStatic-objectpath==1.2.1.0 XStatic-Rickshaw==1.5.0.2 XStatic-roboto-fontface==0.5.0.0 XStatic-smart-table==1.4.13.2 XStatic-Spin==1.2.8.2 XStatic-term.js==0.0.7.0 XStatic-tv4==1.2.7.0 xvfbwrapper==0.2.9 zipp==1.0.0 and here's a typical failure below. Note that there's maybe more than 3 dozen of issues like it: __________________ WorkflowsTests.test_workflow_registration ___________________ [gw3] linux -- Python 3.11.5 /usr/bin/python3.11 self = <horizon.test.unit.workflows.test_workflows.WorkflowsTests testMethod=test_workflow_registration> def test_workflow_registration(self): req = self.factory.get("/foo") flow = WorkflowForTesting(req) > self.assertQuerysetEqual(flow.steps, ['<StepOne: action_one>', '<StepTwo: action_two>']) horizon/test/unit/workflows/test_workflows.py:328: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3/dist-packages/django/test/testcases.py:1330: in assertQuerysetEqual return self.assertQuerySetEqual(*args, **kw) /usr/lib/python3/dist-packages/django/test/testcases.py:1346: in assertQuerySetEqual return self.assertEqual(list(items), values, msg=msg) E AssertionError: Lists differ: [<StepOne: action_one>, <StepTwo: action_two>] != ['<StepOne: action_one>', '<StepTwo: action_two>'] E E First differing element 0: E <StepOne: action_one> E '<StepOne: action_one>' E E - [<StepOne: action_one>, <StepTwo: action_two>] E + ['<StepOne: action_one>', '<StepTwo: action_two>'] E ? + + + + Probably, it used to be that Python returned strings, but now it's not? Anyways, calling repr() on each object fixes it, with something like this: self.assertQuerysetEqual(map(repr, flow.steps), ['<StepOne: action_one>', '<StepTwo: action_two>']) though, this breaks in earlier environment, as a map isn't a list, so this really fixes all cases: self.assertQuerysetEqual(list(map(repr, flow.steps)), ['<StepOne: action_one>', '<StepTwo: action_two>']) My patch here fixes them all: https://review.opendev.org/c/openstack/horizon/+/897310 Cheers, Thomas ** Affects: horizon Importance: Undecided Status: In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/2038474 Title: many unit tests issues with objects compared to strings Status in OpenStack Dashboard (Horizon): In Progress Bug description: Running unit tests of Horizon in Debian Unstable leads to many unit test failures like this one below. My instinct tells me that this is Python 3.12 related, but I'm not sure. Here's a pip freeze output with the installed build-dependency of the package: alabaster==0.7.12 appdirs==1.4.4 asgiref==3.7.2 async-timeout==4.0.3 attrs==23.1.0 autopage==0.4.0 Babel==2.10.3 blinker==1.6.2 calmjs==3.4.2 calmjs.parse==1.2.5 calmjs.types==1.0.1 certifi==2023.7.22 chardet==5.2.0 charset-normalizer==3.2.0 cliff==4.2.0 cmd2==2.4.3+ds coverage==7.2.7 cryptography==38.0.4 csscompressor==0.9.5 debtcollector==2.5.0 decorator==5.1.1 Deprecated==1.2.14 deprecation==2.0.7 Django==4.2.5 django-appconf==1.0.5 django-compressor==4.0 django-debreach==2.1.0 django-pyscss==2.0.2 dnspython==2.4.2 docutils==0.19 dogpile.cache==1.1.8 dulwich==0.21.6 enmerkar==0.7.1 eventlet==0.33.1 exceptiongroup==1.1.3 execnet==2.0.0 extras==1.0.0 fasteners==0.17.3 fixtures==4.0.1 flake8==5.0.4 freezegun==1.2.1 futurist==2.4.1 greenlet==2.0.2 h11==0.14.0 hacking==4.1.0 idna==3.3 imagesize==1.4.1 importlib-metadata==4.12.0 iniconfig==1.1.1 iso8601==1.0.2 jaraco.classes==3.2.1 jeepney==0.8.0 Jinja2==3.1.2 jmespath==1.0.1 jsonpatch==1.32 jsonpointer==2.3 jsonschema==4.10.3 keyring==24.2.0 keystoneauth1==5.3.0 lxml==4.9.3 Mako==1.2.4.dev0 MarkupSafe==2.1.3 mccabe==0.7.0 monotonic==1.6 more-itertools==10.1.0 msgpack==1.0.3 netaddr==0.8.0 netifaces==0.11.0 oauthlib==3.2.2 openstackdocstheme==1.20.0 openstacksdk==1.0.1 os-client-config==2.1.0 os-service-types==1.7.0 osc-lib==2.8.1 oslo.concurrency==5.1.1 oslo.config==9.1.1 oslo.context==5.1.1 oslo.i18n==6.0.0 oslo.log==5.2.0 oslo.policy==4.1.1 oslo.serialization==5.1.1 oslo.upgradecheck==2.1.1 oslo.utils==6.1.0 osprofiler==3.4.3 outcome==1.2.0 packaging==23.1 pbr==5.11.1 pep8==1.7.1 pluggy==1.3.0 ply==3.11 prettytable==3.6.0 pycodestyle==2.10.0 pyflakes==2.5.0 Pygments==2.15.1 pyinotify==0.9.6 PyJWT==2.7.0 pymongo==3.11.0 pyOpenSSL==23.0.0 pyparsing==3.1.1 pyperclip==1.8.2 pyrsistent==0.18.1 pyScss==1.4.0 pytest==7.4.2 pytest-django==4.5.2 pytest-xdist==3.3.1 python-cinderclient==9.3.0 python-dateutil==2.8.2 python-glanceclient==4.3.0 python-keystoneclient==5.1.0 python-memcached==1.58 python-neutronclient==9.0.0 python-novaclient==18.3.0 python-swiftclient==4.2.0 pytz==2023.3.post1 PyYAML==6.0.1 rcssmin==1.1.0 redis==4.3.4 requests==2.31.0 requestsexceptions==1.4.0 rfc3986==1.5.0 rjsmin==1.2.0 roman==3.3 SecretStorage==3.3.3 selenium==4.13.0 semantic-version==2.9.0 simplejson==3.19.1 six==1.16.0 sniffio==1.2.0 snowballstemmer==2.2.0 sortedcontainers==2.4.0 Sphinx==5.3.0 sqlparse==0.4.2 stevedore==5.1.0 testscenarios==0.5.0 testtools==2.5.0 trio==0.22.2 trio-websocket==0.10.3 urllib3==1.26.16 warlock==2.0.1 wcwidth==0.2.5 WebOb==1.8.6 wrapt==1.14.1 wsproto==1.2.0 XStatic==1.0.3 XStatic-Angular==1.8.2.2 XStatic-Angular-Bootstrap==2.5.0.0 XStatic-Angular-FileUpload==12.0.4.0 XStatic-Angular-Gettext==2.4.1.0 XStatic-Angular-lrdragndrop==1.0.2.2 XStatic-Angular-Schema-Form==0.8.13.0 XStatic-angular-ui-router==0.3.1.4 XStatic-Bootstrap-Datepicker==1.3.1.0 XStatic-Bootstrap-SCSS==3.4.1.0 XStatic-bootswatch==3.3.7.0 XStatic-D3==3.5.17.0 XStatic-Font-Awesome==4.7.0.0 XStatic-Hogan==2.0.0.2 XStatic-Jasmine==2.4.1.0 XStatic-jQuery==3.5.1.0 XStatic-JQuery-Migrate==3.3.2.1 XStatic-jquery-ui==1.12.0.1 XStatic-JQuery.quicksearch==2.0.4.1 XStatic-JQuery.TableSorter==2.14.5.1 XStatic-JSEncrypt==2.3.1.1 XStatic-Magic-Search==0.2.5.1 XStatic-mdi==1.6.50.2 XStatic-objectpath==1.2.1.0 XStatic-Rickshaw==1.5.0.2 XStatic-roboto-fontface==0.5.0.0 XStatic-smart-table==1.4.13.2 XStatic-Spin==1.2.8.2 XStatic-term.js==0.0.7.0 XStatic-tv4==1.2.7.0 xvfbwrapper==0.2.9 zipp==1.0.0 and here's a typical failure below. Note that there's maybe more than 3 dozen of issues like it: __________________ WorkflowsTests.test_workflow_registration ___________________ [gw3] linux -- Python 3.11.5 /usr/bin/python3.11 self = <horizon.test.unit.workflows.test_workflows.WorkflowsTests testMethod=test_workflow_registration> def test_workflow_registration(self): req = self.factory.get("/foo") flow = WorkflowForTesting(req) > self.assertQuerysetEqual(flow.steps, ['<StepOne: action_one>', '<StepTwo: action_two>']) horizon/test/unit/workflows/test_workflows.py:328: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3/dist-packages/django/test/testcases.py:1330: in assertQuerysetEqual return self.assertQuerySetEqual(*args, **kw) /usr/lib/python3/dist-packages/django/test/testcases.py:1346: in assertQuerySetEqual return self.assertEqual(list(items), values, msg=msg) E AssertionError: Lists differ: [<StepOne: action_one>, <StepTwo: action_two>] != ['<StepOne: action_one>', '<StepTwo: action_two>'] E E First differing element 0: E <StepOne: action_one> E '<StepOne: action_one>' E E - [<StepOne: action_one>, <StepTwo: action_two>] E + ['<StepOne: action_one>', '<StepTwo: action_two>'] E ? + + + + Probably, it used to be that Python returned strings, but now it's not? Anyways, calling repr() on each object fixes it, with something like this: self.assertQuerysetEqual(map(repr, flow.steps), ['<StepOne: action_one>', '<StepTwo: action_two>']) though, this breaks in earlier environment, as a map isn't a list, so this really fixes all cases: self.assertQuerysetEqual(list(map(repr, flow.steps)), ['<StepOne: action_one>', '<StepTwo: action_two>']) My patch here fixes them all: https://review.opendev.org/c/openstack/horizon/+/897310 Cheers, Thomas To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/2038474/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp