Re: [openstack-dev] [all] Reviewing coverage jobs set up
On 2016-08-04 09:36:18 -0400 (-0400), Jim Rollenhagen wrote: [...] > Nice! FWIW, it's also documented here: > http://docs.openstack.org/infra/manual/developers.html#automated-testing I just proposed a note clarifying the situation with URLs for merge commits too, since that seems to have been previously missing: https://review.openstack.org/351199 -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Reviewing coverage jobs set up
On Thu, Aug 04, 2016 at 09:24:15AM -0400, Doug Hellmann wrote: > Excerpts from Kiall Mac Innes's message of 2016-08-04 01:27:23 +0100: > > On 18/07/16 20:14, Jeremy Stanley wrote: > > > Note that this will only be true if the change's parent commit in > > > Gerrit was the branch tip at the time it landed. Otherwise (and > > > quite frequently in fact) you will need to identify the SHA of the > > > merge commit which was created at the time it merged and use that > > > instead to find the post job. > > Without wanting to diverge too much from the topic at hand, I believe this > > is why those of us who only occasionally want to look at post job output > > usually just give up! Keeping this in your head for the once every few > > months it's needed is hard ;) > > > > A change being merged is always (AFAIK) part and parcel with a review > > being closed, so - why not publish to /post/ (with some > > level of dir sharding)? > > > > Thanks, > > Kiall > > > > I could never remember the formula for constructing the URL either, > so I built this to help me: https://pypi.python.org/pypi/git-os-job Nice! FWIW, it's also documented here: http://docs.openstack.org/infra/manual/developers.html#automated-testing // jim __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Reviewing coverage jobs set up
Excerpts from Kiall Mac Innes's message of 2016-08-04 01:27:23 +0100: > On 18/07/16 20:14, Jeremy Stanley wrote: > > Note that this will only be true if the change's parent commit in > > Gerrit was the branch tip at the time it landed. Otherwise (and > > quite frequently in fact) you will need to identify the SHA of the > > merge commit which was created at the time it merged and use that > > instead to find the post job. > Without wanting to diverge too much from the topic at hand, I believe this > is why those of us who only occasionally want to look at post job output > usually just give up! Keeping this in your head for the once every few > months it's needed is hard ;) > > A change being merged is always (AFAIK) part and parcel with a review > being closed, so - why not publish to /post/ (with some > level of dir sharding)? > > Thanks, > Kiall > I could never remember the formula for constructing the URL either, so I built this to help me: https://pypi.python.org/pypi/git-os-job Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Reviewing coverage jobs set up
On 18/07/16 20:14, Jeremy Stanley wrote: Note that this will only be true if the change's parent commit in Gerrit was the branch tip at the time it landed. Otherwise (and quite frequently in fact) you will need to identify the SHA of the merge commit which was created at the time it merged and use that instead to find the post job. Without wanting to diverge too much from the topic at hand, I believe this is why those of us who only occasionally want to look at post job output usually just give up! Keeping this in your head for the once every few months it's needed is hard ;) A change being merged is always (AFAIK) part and parcel with a review being closed, so - why not publish to /post/ (with some level of dir sharding)? Thanks, Kiall __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Reviewing coverage jobs set up
The following coverage jobs are still wrongly setup, I've proposed a change now to remove them from our infra setup [1]: * cloudkitty-dashboard * murano-agent * nova-docker * os-net-config * poppy * sahara-dashboard * solum-dashboard * trove-dashboard * turbo-hipster https://review.openstack.org/349242 Andreas On 07/18/2016 08:09 PM, Andreas Jaeger wrote: > While looking at coverage jobs to enable them to allow use of > constraints in post jobs (something which has just been introduced and > needs some more testing before we take on the other jobs), I noticed > that we have quite a few coverage > jobs that are failing. > > In general, I think coverage jobs should be run as check job so > that you know how the coverage changes. Running them only in the post > job means that in practice nobody sees the output of the job. > > Out of 83 jobs having a post coverage job, 19 failed. See > below for list of failures and a list of all repos that have a coverage > post job setup. > > I suggest that projects review their coverage setup and decide whether > they want to run it as part of the check queue, fix it, or remove it. > > Btw. if you want to get the output of the post job, check the git SHA. > logs are found at ``http://logs.openstack.org/ commit SHA>/``. For example, if a change is committed with > the sha 'deadbeef123456', the logs will be found at > ``http://logs.openstack.org/de/deadbeef123456``. > > Here're the failures (including some that run successful but coverage > did not collect any data): > > cloudkitty-dashboard: > http://logs.openstack.org/11/113209883e3e9131602f35593bc0cb8880db19b9/post/cloudkitty-dashboard-coverage/c390a24/ > > designate-dashboard: > http://logs.openstack.org/56/5627ddb4a6eedb751fecced56d277961aac92436/post/designate-dashboard-coverage/73650dc/ > > horizon: > http://logs.openstack.org/8f/8f35c43bc5b4182d6e82a67cdc2beccd2364da8a/post/horizon-coverage/5535457/ > > monasca-ui: > http://logs.openstack.org/f0/f05e4992fb6ce722ac00c4aa6ad30ff89b453476/post/monasca-ui-coverage/af5105e/ > > murano-agent: > http://logs.openstack.org/14/1450eb39fb9f6dcbe1b70167d23d16e74f28dbd5/post/murano-agent-coverage/ee9c33a/ > > nodepool: > http://logs.openstack.org/e3/e345107476a82ddad82ea99e184398e0d0e7e85a/post/nodepool-coverage-db/8a0ee23/ > > nova: > http://logs.openstack.org/3f/3f700b5a5a498ba08e77378d34f059c3fa6845d8/post/nova-coverage-db/2bdabf8/ > > nova-docker: > http://logs.openstack.org/03/034a4842fc1ebba5912e02cff8cd197ae81eb0c3/post/nova-docker-coverage/879d790/ > > os-net-config: > http://logs.openstack.org/6b/6bb8412ef3d3f163d91f2884081b743f07a78f18/post/os-net-config-coverage/cba622a/ > > poppy: > http://logs.openstack.org/09/0948c854e4b8543f01d909437f09cfb23e71a5b0/post/poppy-coverage/ccbc8ad/ > > python-aodhclient: > http://logs.openstack.org/65/65d2e625ee3b359bae154a5da3931f48b48fe720/post/python-aodhclient-coverage/2d8a169/ > > python-gnocchiclient: > http://logs.openstack.org/81/81e1b91beb3d85760e574951e240a381d6a4d008/post/python-gnocchiclient-coverage/2d9a414/ > > python-monascaclient (this should be fixed with the enabling of > constraints): > http://logs.openstack.org/17/17b9eaa1ede715344ac9cb2049c63e25604476cb/post/python-monascaclient-coverage/2b44b1b/ > > sahara-dashboard: > http://logs.openstack.org/bc/bcf8e2938d7085799a3754a9ff6d245d73ec33ff/post/sahara-dashboard-coverage/7807a4e/console.html.gz > > sahara-tests: > http://logs.openstack.org/91/914bf0646f4de85107324f0aa05e856961bb0ee6/post/sahara-tests-coverage/b65352b/ > > solum-dashboard: > http://logs.openstack.org/82/8235df5a5ab0d48a6b407931c1d998102050b1b9/post/solum-dashboard-coverage/8d23679/ > > trove: > http://logs.openstack.org/4a/4ad0dfe88c6362356c8083d167f43ab495da661d/post/trove-coverage-db/990fd71/ > > trove-dashboard: > http://logs.openstack.org/01/01815715539fad40e06e974d6fdf840957051b69/post/trove-dashboard-coverage/908f8df/ > > turbo-hipster: > http://logs.openstack.org/7e/7ef12b758020176fb3e13e5bb0154084b1456797/post/turbo-hipster-coverage/ab2d17a/ > > > Full list of repos with coverage job defined: > > openstack-dev/hacking hacking-coverage > openstack-dev/pbr pbr-coverage > openstack-infra/bindep bindep-coverage > openstack-infra/grafyaml grafyaml-coverage > openstack-infra/jenkins-job-builder jenkins-job-builder-coverage > openstack-infra/nodepool nodepool-coverage-db > openstack-infra/python-storyboardclient python-storyboardclient-coverage > openstack-infra/shade shade-coverage > openstack-infra/storyboard storyboard-coverage-db > openstack-infra/zuul zuul-coverage-db > openstack/ara ara-coverage > openstack/ceilometermiddleware ceilometermiddleware-coverage > openstack/cloudbase-init cloudbase-init-coverage > openstack/cloudkitty cloudkitty-coverage > openstack/cloudkitty-dashboard cloudkitty-dashboard-coverage > openstack/designate designate-coverage-db > openstack/designate-dashboard designate-dashboard-coverage > openstack/heat heat-coverage-db > openstack/heat-t
Re: [openstack-dev] [all] Reviewing coverage jobs set up
On Mon, Jul 18 2016, Andreas Jaeger wrote: > While looking at coverage jobs to enable them to allow use of > constraints in post jobs (something which has just been introduced and > needs some more testing before we take on the other jobs), I noticed > that we have quite a few coverage > jobs that are failing. I had no idea these jobs existed and we never used them. So I think it's safe to disable them for telemetry projects and save some CPU time from the gate for other things. (Not saying the info is useless, but as far as we're concern, running it From time to time locally and trying to check what might not be covered is largely good enough.) -- Julien Danjou /* Free Software hacker https://julien.danjou.info */ signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Reviewing coverage jobs set up
On Mon, Jul 18, 2016 at 4:11 PM, Andreas Jaeger wrote: > You're welcome to send a patch for openstack-infra/project-config repo > and update zuul/layout.yaml to add the job to the check queue of nova. Done, thanks for pointing me in the right direction! https://review.openstack.org/#/c/343922/ I've added Matt Riedemann to the review since it's not really my call to make. Thanks again, --diana __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Reviewing coverage jobs set up
On 07/18/2016 10:03 PM, Diana Clarke wrote:
> On Mon, Jul 18, 2016 at 2:09 PM, Andreas Jaeger wrote:
>> In general, I think coverage jobs should be run as check job so
>> that you know how the coverage changes. Running them only in the post
>> job means that in practice nobody sees the output of the job.
>
> I look at the post queue coverage output for nova, and have often
> wished I could see the check queue coverage results. +1 for moving it
> to the check queue.
You're welcome to send a patch for openstack-infra/project-config repo
and update zuul/layout.yaml to add the job to the check queue of nova.
>> Here're the failures (including some that run successful but coverage
>>
>> nova:
>> http://logs.openstack.org/3f/3f700b5a5a498ba08e77378d34f059c3fa6845d8/post/nova-coverage-db/2bdabf8/
>
> I suspect you can cross nova off your list of failures soon. I believe
> those errors were addressed by the following:
>
> https://bugs.launchpad.net/nova/+bug/1603979
>
> Here's a recent passing coverage run for nova (from before those
> errors were introduced):
>
>
> http://logs.openstack.org/1a/1a074a72845d812666a904b9c17289a43bb32062/post/nova-coverage-db/2fe5506/cover/
Glad to see! I didn't dig deeper, just looked at today's status,
Andreas
--
Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Reviewing coverage jobs set up
On Mon, Jul 18, 2016 at 2:09 PM, Andreas Jaeger wrote: > In general, I think coverage jobs should be run as check job so > that you know how the coverage changes. Running them only in the post > job means that in practice nobody sees the output of the job. I look at the post queue coverage output for nova, and have often wished I could see the check queue coverage results. +1 for moving it to the check queue. > Here're the failures (including some that run successful but coverage > > nova: > http://logs.openstack.org/3f/3f700b5a5a498ba08e77378d34f059c3fa6845d8/post/nova-coverage-db/2bdabf8/ I suspect you can cross nova off your list of failures soon. I believe those errors were addressed by the following: https://bugs.launchpad.net/nova/+bug/1603979 Here's a recent passing coverage run for nova (from before those errors were introduced): http://logs.openstack.org/1a/1a074a72845d812666a904b9c17289a43bb32062/post/nova-coverage-db/2fe5506/cover/ Much appreciated, --diana __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Reviewing coverage jobs set up
see inline comment On Mon, Jul 18, 2016 at 2:09 PM, Andreas Jaeger wrote: > While looking at coverage jobs to enable them to allow use of > constraints in post jobs (something which has just been introduced and > needs some more testing before we take on the other jobs), I noticed > that we have quite a few coverage > jobs that are failing. > > In general, I think coverage jobs should be run as check job so > that you know how the coverage changes. Running them only in the post > job means that in practice nobody sees the output of the job. > Yes please! I never understood why they were run as post jobs. In keystone we've had it running as a check/gate job for a while without hiccups. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Reviewing coverage jobs set up
On 2016-07-18 20:09:54 +0200 (+0200), Andreas Jaeger wrote: [...] > Btw. if you want to get the output of the post job, check the git SHA. > logs are found at ``http://logs.openstack.org/ commit SHA>/``. For example, if a change is committed with > the sha 'deadbeef123456', the logs will be found at > ``http://logs.openstack.org/de/deadbeef123456``. [...] Note that this will only be true if the change's parent commit in Gerrit was the branch tip at the time it landed. Otherwise (and quite frequently in fact) you will need to identify the SHA of the merge commit which was created at the time it merged and use that instead to find the post job. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] Reviewing coverage jobs set up
While looking at coverage jobs to enable them to allow use of constraints in post jobs (something which has just been introduced and needs some more testing before we take on the other jobs), I noticed that we have quite a few coverage jobs that are failing. In general, I think coverage jobs should be run as check job so that you know how the coverage changes. Running them only in the post job means that in practice nobody sees the output of the job. Out of 83 jobs having a post coverage job, 19 failed. See below for list of failures and a list of all repos that have a coverage post job setup. I suggest that projects review their coverage setup and decide whether they want to run it as part of the check queue, fix it, or remove it. Btw. if you want to get the output of the post job, check the git SHA. logs are found at ``http://logs.openstack.org//``. For example, if a change is committed with the sha 'deadbeef123456', the logs will be found at ``http://logs.openstack.org/de/deadbeef123456``. Here're the failures (including some that run successful but coverage did not collect any data): cloudkitty-dashboard: http://logs.openstack.org/11/113209883e3e9131602f35593bc0cb8880db19b9/post/cloudkitty-dashboard-coverage/c390a24/ designate-dashboard: http://logs.openstack.org/56/5627ddb4a6eedb751fecced56d277961aac92436/post/designate-dashboard-coverage/73650dc/ horizon: http://logs.openstack.org/8f/8f35c43bc5b4182d6e82a67cdc2beccd2364da8a/post/horizon-coverage/5535457/ monasca-ui: http://logs.openstack.org/f0/f05e4992fb6ce722ac00c4aa6ad30ff89b453476/post/monasca-ui-coverage/af5105e/ murano-agent: http://logs.openstack.org/14/1450eb39fb9f6dcbe1b70167d23d16e74f28dbd5/post/murano-agent-coverage/ee9c33a/ nodepool: http://logs.openstack.org/e3/e345107476a82ddad82ea99e184398e0d0e7e85a/post/nodepool-coverage-db/8a0ee23/ nova: http://logs.openstack.org/3f/3f700b5a5a498ba08e77378d34f059c3fa6845d8/post/nova-coverage-db/2bdabf8/ nova-docker: http://logs.openstack.org/03/034a4842fc1ebba5912e02cff8cd197ae81eb0c3/post/nova-docker-coverage/879d790/ os-net-config: http://logs.openstack.org/6b/6bb8412ef3d3f163d91f2884081b743f07a78f18/post/os-net-config-coverage/cba622a/ poppy: http://logs.openstack.org/09/0948c854e4b8543f01d909437f09cfb23e71a5b0/post/poppy-coverage/ccbc8ad/ python-aodhclient: http://logs.openstack.org/65/65d2e625ee3b359bae154a5da3931f48b48fe720/post/python-aodhclient-coverage/2d8a169/ python-gnocchiclient: http://logs.openstack.org/81/81e1b91beb3d85760e574951e240a381d6a4d008/post/python-gnocchiclient-coverage/2d9a414/ python-monascaclient (this should be fixed with the enabling of constraints): http://logs.openstack.org/17/17b9eaa1ede715344ac9cb2049c63e25604476cb/post/python-monascaclient-coverage/2b44b1b/ sahara-dashboard: http://logs.openstack.org/bc/bcf8e2938d7085799a3754a9ff6d245d73ec33ff/post/sahara-dashboard-coverage/7807a4e/console.html.gz sahara-tests: http://logs.openstack.org/91/914bf0646f4de85107324f0aa05e856961bb0ee6/post/sahara-tests-coverage/b65352b/ solum-dashboard: http://logs.openstack.org/82/8235df5a5ab0d48a6b407931c1d998102050b1b9/post/solum-dashboard-coverage/8d23679/ trove: http://logs.openstack.org/4a/4ad0dfe88c6362356c8083d167f43ab495da661d/post/trove-coverage-db/990fd71/ trove-dashboard: http://logs.openstack.org/01/01815715539fad40e06e974d6fdf840957051b69/post/trove-dashboard-coverage/908f8df/ turbo-hipster: http://logs.openstack.org/7e/7ef12b758020176fb3e13e5bb0154084b1456797/post/turbo-hipster-coverage/ab2d17a/ Full list of repos with coverage job defined: openstack-dev/hacking hacking-coverage openstack-dev/pbr pbr-coverage openstack-infra/bindep bindep-coverage openstack-infra/grafyaml grafyaml-coverage openstack-infra/jenkins-job-builder jenkins-job-builder-coverage openstack-infra/nodepool nodepool-coverage-db openstack-infra/python-storyboardclient python-storyboardclient-coverage openstack-infra/shade shade-coverage openstack-infra/storyboard storyboard-coverage-db openstack-infra/zuul zuul-coverage-db openstack/ara ara-coverage openstack/ceilometermiddleware ceilometermiddleware-coverage openstack/cloudbase-init cloudbase-init-coverage openstack/cloudkitty cloudkitty-coverage openstack/cloudkitty-dashboard cloudkitty-dashboard-coverage openstack/designate designate-coverage-db openstack/designate-dashboard designate-dashboard-coverage openstack/heat heat-coverage-db openstack/heat-translator heat-translator-coverage openstack/horizon horizon-coverage openstack/ironic ironic-coverage-db openstack/ironic-lib ironic-lib-coverage openstack/keystonemiddleware keystonemiddleware-coverage openstack/kiloeyes kiloeyes-coverage openstack/manila manila-coverage-db openstack/monasca-ui monasca-ui-coverage openstack/murano murano-coverage-db openstack/murano-agent murano-agent-coverage openstack/neutron neutron-coverage openstack/neutron-dynamic-routing neutron-dynamic-routing-coverage openstack/neutron-fwaas neutron-fwaas-coverage openstack/neutron-vpnaas neutron-vpn
