Re: [openstack-dev] [Ceilometer][WSME] Sphinx failing sporadically because of wsme autodoc extension
just an update, i had to re-add PYTHONHASHSEED = 0 to ceilometer. i didn't find the exact root cause of why WSME is affecting our doc gate but it appears WSME is also affected by new tox and random hashseed as it too suffers from random failures in UT. for now, i've added back HASHSEED so we should be unblocked cheers, gord ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer][WSME] Sphinx failing sporadically because of wsme autodoc extension
Gordon, exactly. Moreover, I think percentage of failings is something close to the 15-20% of all the runs - sometimes it passes for the change, and next run on this exactly the same commit will be the failed for some reason. Thanks Dina On Thu, Aug 21, 2014 at 10:46 PM, gordon chung g...@live.ca wrote: is it possible that it's not on all the nodes? seems like it passed here: https://review.openstack.org/#/c/109207/ but another patch at roughly the same time failed https://review.openstack.org/#/c/113549/ cheers, *gord* ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best regards, Dina Belova Software Engineer Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer][WSME] Sphinx failing sporadically because of wsme autodoc extension
Hi, I couldn’t reproduce this issue either. I’ve tried on precise and on a fresh trusty too, everything worked fine… Cheers, Ildikó From: Dina Belova [mailto:dbel...@mirantis.com] Sent: Friday, August 22, 2014 11:40 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Ceilometer][WSME] Sphinx failing sporadically because of wsme autodoc extension Gordon, exactly. Moreover, I think percentage of failings is something close to the 15-20% of all the runs - sometimes it passes for the change, and next run on this exactly the same commit will be the failed for some reason. Thanks Dina On Thu, Aug 21, 2014 at 10:46 PM, gordon chung g...@live.camailto:g...@live.ca wrote: is it possible that it's not on all the nodes? seems like it passed here: https://review.openstack.org/#/c/109207/ but another patch at roughly the same time failed https://review.openstack.org/#/c/113549/ cheers, gord ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best regards, Dina Belova Software Engineer Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer][WSME] Sphinx failing sporadically because of wsme autodoc extension
I couldn’t reproduce this issue either. I’ve tried on precise and on a fresh trusty too, everything worked fine… fun from the limited error message. it's because service path isn't found (https://github.com/stackforge/wsme/blob/master/wsmeext/sphinxext.py#L133-L140) and this code is returning None... so for some reason scan_services is not finding what it needs to find (https://github.com/stackforge/wsme/blob/master/wsmeext/sphinxext.py#L114-L130) in most cases but in the rare case, it actually does find a path looking at the build trends. there doesn't seem to be a noticeable trend and more a crapshoot whether a check passes or not:https://jenkins01.openstack.org/job/gate-ceilometer-docs/buildTimeTrendhttps://jenkins02.openstack.org/job/gate-ceilometer-docs/buildTimeTrendhttps://jenkins03.openstack.org/job/gate-ceilometer-docs/buildTimeTrendhttps://jenkins04.openstack.org/job/gate-ceilometer-docs/buildTimeTrendhttps://jenkins05.openstack.org/job/gate-ceilometer-docs/buildTimeTrendhttps://jenkins06.openstack.org/job/gate-ceilometer-docs/buildTimeTrendhttps://jenkins07.openstack.org/job/gate-ceilometer-docs/buildTimeTrend cheers,gord ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Ceilometer][WSME] Sphinx failing sporadically because of wsme autodoc extension
Yesterday, doc builds started failing sporadically in Ceilometer gate. http://logstash.openstack.org/#eyJzZWFyY2giOiJcIkVSUk9SOiBJbnZvY2F0aW9uRXJyb3I6ICcvaG9tZS9qZW5raW5zL3dvcmtzcGFjZS9nYXRlLWNlaWxvbWV0ZXItZG9jcy8udG94L3ZlbnYvYmluL3B5dGhvbiBzZXR1cC5weSBidWlsZF9zcGhpbngnXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjE3MjgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0MDg2MjQwMjcyMDF9 Can someone with more Sphinx-fu than me figure out why Sphinx is using a wsme extension where there is no wsme code? (The build fails when processing ceilometer.alarm module, example of a successful build: https://jenkins02.openstack.org/job/gate-ceilometer-docs/4412/consoleFull Cheers, Nejc ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer][WSME] Sphinx failing sporadically because of wsme autodoc extension
On Aug 21, 2014, at 8:30 AM, Nejc Saje ns...@redhat.com wrote: Yesterday, doc builds started failing sporadically in Ceilometer gate. http://logstash.openstack.org/#eyJzZWFyY2giOiJcIkVSUk9SOiBJbnZvY2F0aW9uRXJyb3I6ICcvaG9tZS9qZW5raW5zL3dvcmtzcGFjZS9nYXRlLWNlaWxvbWV0ZXItZG9jcy8udG94L3ZlbnYvYmluL3B5dGhvbiBzZXR1cC5weSBidWlsZF9zcGhpbngnXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjE3MjgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0MDg2MjQwMjcyMDF9 Can someone with more Sphinx-fu than me figure out why Sphinx is using a wsme extension where there is no wsme code? (The build fails when processing ceilometer.alarm module, example of a successful build: https://jenkins02.openstack.org/job/gate-ceilometer-docs/4412/consoleFull We recently updated the CI nodes to run a new image. Perhaps a dependency has changed in a way that is introducing this problem? Can you reproduce it locally? I’ve tried building the docs and was able to successfully build them on my server running Trusty. Doug Cheers, Nejc ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer][WSME] Sphinx failing sporadically because of wsme autodoc extension
On Thu, Aug 21, 2014, at 11:25 AM, Doug Hellmann wrote: On Aug 21, 2014, at 8:30 AM, Nejc Saje ns...@redhat.com wrote: Yesterday, doc builds started failing sporadically in Ceilometer gate. http://logstash.openstack.org/#eyJzZWFyY2giOiJcIkVSUk9SOiBJbnZvY2F0aW9uRXJyb3I6ICcvaG9tZS9qZW5raW5zL3dvcmtzcGFjZS9nYXRlLWNlaWxvbWV0ZXItZG9jcy8udG94L3ZlbnYvYmluL3B5dGhvbiBzZXR1cC5weSBidWlsZF9zcGhpbngnXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjE3MjgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0MDg2MjQwMjcyMDF9 Can someone with more Sphinx-fu than me figure out why Sphinx is using a wsme extension where there is no wsme code? (The build fails when processing ceilometer.alarm module, example of a successful build: https://jenkins02.openstack.org/job/gate-ceilometer-docs/4412/consoleFull We recently updated the CI nodes to run a new image. Perhaps a dependency has changed in a way that is introducing this problem? Can you reproduce it locally? I’ve tried building the docs and was able to successfully build them on my server running Trusty. Doug We made two changes. First we switched all projects to run tests like this on Trusty slave nodes. Ceilometer was switched long before the default switch so I don't think that change is related. Second we upgraded tox to version 1.7.2 on our test slaves. However, I didn't get the trusty images updated with tox 1.7.2 until late yesterday UTC time (close to midnight) and according to logstash the first occurrence of this error was earlier in the evening (around 1900UTC) so I am not convinced that is related either. Unfortunately I don't have any other good ideas for why this may be happening. Clark Cheers, Nejc ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer][WSME] Sphinx failing sporadically because of wsme autodoc extension
is it possible that it's not on all the nodes? seems like it passed here: https://review.openstack.org/#/c/109207/ but another patch at roughly the same time failed https://review.openstack.org/#/c/113549/ cheers, gord ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
