Re: [openstack-dev] [qa] Tracking development of scenario tests
I've pushed https://review.openstack.org/#/c/56923/ trying to follow this protocol. Salvatore On 14 November 2013 16:38, Zhi Kun Liu liuzhi...@gmail.com wrote: +1, This is a great idea. We could consider it as a general process for all tests. 2013/11/14 Koderer, Marc m.kode...@telekom.de Hi all, I think we have quite the same issue with the neutron testing. I already put it on the agenda for the QA meeting for today. Let's make it to a general topic. Regards Marc From: Giulio Fidente [gfide...@redhat.com] Sent: Thursday, November 14, 2013 6:23 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [qa] Tracking development of scenario tests On 11/14/2013 12:24 AM, David Kranz wrote: 1. Developer checks in the zeroth version of a scenario test as work in progress. It contains a description of the test, and possibly work items. This will claim the area of the proposed scenario to avoid duplication and allow others to comment through gerrit. 2. The developer pushes new versions, removing work in progress if the code is in working state and a review is desired and/or others will be contributing to the scenario. 3. When finished, any process-oriented content such as progress tracking is removed and the test is ready for final review. +1 , the description will eventually contribute to documenting the scenarios yet the submitter (step 1) remains in charge of adding to the draft the reviewers how about we map at least one volunteer to each service (via the HACKING file) and ask submitters to add such a person as reviewer of its drafts when the tests touch the service? this should help avoid tests duplication. I very much like the idea of using gerrit for this -- Giulio Fidente GPG KEY: 08D733BA | IRC: giulivo ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tracking development of scenario tests
Hi, I read the qa-meeting log[1]. And I registered a blueprint[2] for tracking and avoiding duplication. I think if we put Partially Implements: blueprint add-scenario-tests-in-icehouse in the commit message, we can avoid the duplication and tracking the scenarios. Because the commit subject and the link will be wrote automatically in the whiteboard. However, I'm not sure whether we can associate with multiple blueprints such as BP:lbaas-scenario-tests and add-scenario-tests-in-icehouse though. Is this make sense? [1] http://eavesdrop.openstack.org/meetings/qa/2013/qa.2013-11-14-17.02.log.html [2] https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests-in-icehouse Any comments and suggestions are welcome. Best Regards, -- Masayuki Igawa On Mon, Nov 18, 2013 at 10:07 PM, Salvatore Orlando sorla...@nicira.com wrote: I've pushed https://review.openstack.org/#/c/56923/ trying to follow this protocol. Salvatore On 14 November 2013 16:38, Zhi Kun Liu liuzhi...@gmail.com wrote: +1, This is a great idea. We could consider it as a general process for all tests. 2013/11/14 Koderer, Marc m.kode...@telekom.de Hi all, I think we have quite the same issue with the neutron testing. I already put it on the agenda for the QA meeting for today. Let's make it to a general topic. Regards Marc From: Giulio Fidente [gfide...@redhat.com] Sent: Thursday, November 14, 2013 6:23 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [qa] Tracking development of scenario tests On 11/14/2013 12:24 AM, David Kranz wrote: 1. Developer checks in the zeroth version of a scenario test as work in progress. It contains a description of the test, and possibly work items. This will claim the area of the proposed scenario to avoid duplication and allow others to comment through gerrit. 2. The developer pushes new versions, removing work in progress if the code is in working state and a review is desired and/or others will be contributing to the scenario. 3. When finished, any process-oriented content such as progress tracking is removed and the test is ready for final review. +1 , the description will eventually contribute to documenting the scenarios yet the submitter (step 1) remains in charge of adding to the draft the reviewers how about we map at least one volunteer to each service (via the HACKING file) and ask submitters to add such a person as reviewer of its drafts when the tests touch the service? this should help avoid tests duplication. I very much like the idea of using gerrit for this -- Giulio Fidente GPG KEY: 08D733BA | IRC: giulivo ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tracking development of scenario tests
On 11/18/2013 09:34 AM, Masayuki Igawa wrote: Hi, I read the qa-meeting log[1]. And I registered a blueprint[2] for tracking and avoiding duplication. I think if we put Partially Implements: blueprint add-scenario-tests-in-icehouse in the commit message, we can avoid the duplication and tracking the scenarios. Because the commit subject and the link will be wrote automatically in the whiteboard. However, I'm not sure whether we can associate with multiple blueprints such as BP:lbaas-scenario-tests and add-scenario-tests-in-icehouse though. Is this make sense? [1] http://eavesdrop.openstack.org/meetings/qa/2013/qa.2013-11-14-17.02.log.html [2] https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests-in-icehouse Any comments and suggestions are welcome. Best Regards, -- Masayuki Igawa I think there could also be links to other blueprints either in the whiteboard or main section of the blueprint. At the meeting we just said there should be some way to get from the master blueprint to information about each new scenario being created. -David On Mon, Nov 18, 2013 at 10:07 PM, Salvatore Orlando sorla...@nicira.com wrote: I've pushed https://review.openstack.org/#/c/56923/ trying to follow this protocol. Salvatore On 14 November 2013 16:38, Zhi Kun Liu liuzhi...@gmail.com wrote: +1, This is a great idea. We could consider it as a general process for all tests. 2013/11/14 Koderer, Marc m.kode...@telekom.de Hi all, I think we have quite the same issue with the neutron testing. I already put it on the agenda for the QA meeting for today. Let's make it to a general topic. Regards Marc From: Giulio Fidente [gfide...@redhat.com] Sent: Thursday, November 14, 2013 6:23 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [qa] Tracking development of scenario tests On 11/14/2013 12:24 AM, David Kranz wrote: 1. Developer checks in the zeroth version of a scenario test as work in progress. It contains a description of the test, and possibly work items. This will claim the area of the proposed scenario to avoid duplication and allow others to comment through gerrit. 2. The developer pushes new versions, removing work in progress if the code is in working state and a review is desired and/or others will be contributing to the scenario. 3. When finished, any process-oriented content such as progress tracking is removed and the test is ready for final review. +1 , the description will eventually contribute to documenting the scenarios yet the submitter (step 1) remains in charge of adding to the draft the reviewers how about we map at least one volunteer to each service (via the HACKING file) and ask submitters to add such a person as reviewer of its drafts when the tests touch the service? this should help avoid tests duplication. I very much like the idea of using gerrit for this -- Giulio Fidente GPG KEY: 08D733BA | IRC: giulivo ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tracking development of scenario tests
Hi, David Thanks for your reply. On Tue, Nov 19, 2013 at 12:37 AM, David Kranz dkr...@redhat.com wrote: On 11/18/2013 09:34 AM, Masayuki Igawa wrote: Hi, I read the qa-meeting log[1]. And I registered a blueprint[2] for tracking and avoiding duplication. I think if we put Partially Implements: blueprint add-scenario-tests-in-icehouse in the commit message, we can avoid the duplication and tracking the scenarios. Because the commit subject and the link will be wrote automatically in the whiteboard. However, I'm not sure whether we can associate with multiple blueprints such as BP:lbaas-scenario-tests and add-scenario-tests-in-icehouse though. Is this make sense? [1] http://eavesdrop.openstack.org/meetings/qa/2013/qa.2013-11-14-17.02.log.html [2] https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests-in-icehouse Any comments and suggestions are welcome. Best Regards, -- Masayuki Igawa I think there could also be links to other blueprints either in the whiteboard or main section of the blueprint. At the meeting we just said there should be some way to get from the master blueprint to information about each new scenario being created. -David I've added three links on the whiteboard of the blueprint. https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests-in-icehouse --- * https://blueprints.launchpad.net/tempest/+spec/neutron-advanced-scenarios Advanced scenario tests for Neutron * https://blueprints.launchpad.net/tempest/+spec/lbaas-scenario-tests Add advanced scenario tests for Neutron LBaaS sevice * https://review.openstack.org/#/c/56923/ zeroth version of l3 topology scenario --- Every developers can modify the whiteboard. So developers can add their scenario to this white board by themselves or automatically. I hope this BP could be useful for tracking scenarios and avoiding duplicate development. Best Regards, -- Masayuki Igawa On Mon, Nov 18, 2013 at 10:07 PM, Salvatore Orlando sorla...@nicira.com wrote: I've pushed https://review.openstack.org/#/c/56923/ trying to follow this protocol. Salvatore On 14 November 2013 16:38, Zhi Kun Liu liuzhi...@gmail.com wrote: +1, This is a great idea. We could consider it as a general process for all tests. 2013/11/14 Koderer, Marc m.kode...@telekom.de Hi all, I think we have quite the same issue with the neutron testing. I already put it on the agenda for the QA meeting for today. Let's make it to a general topic. Regards Marc From: Giulio Fidente [gfide...@redhat.com] Sent: Thursday, November 14, 2013 6:23 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [qa] Tracking development of scenario tests On 11/14/2013 12:24 AM, David Kranz wrote: 1. Developer checks in the zeroth version of a scenario test as work in progress. It contains a description of the test, and possibly work items. This will claim the area of the proposed scenario to avoid duplication and allow others to comment through gerrit. 2. The developer pushes new versions, removing work in progress if the code is in working state and a review is desired and/or others will be contributing to the scenario. 3. When finished, any process-oriented content such as progress tracking is removed and the test is ready for final review. +1 , the description will eventually contribute to documenting the scenarios yet the submitter (step 1) remains in charge of adding to the draft the reviewers how about we map at least one volunteer to each service (via the HACKING file) and ask submitters to add such a person as reviewer of its drafts when the tests touch the service? this should help avoid tests duplication. I very much like the idea of using gerrit for this -- Giulio Fidente GPG KEY: 08D733BA | IRC: giulivo ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tracking development of scenario tests
+1, This is a great idea. We could consider it as a general process for all tests. 2013/11/14 Koderer, Marc m.kode...@telekom.de Hi all, I think we have quite the same issue with the neutron testing. I already put it on the agenda for the QA meeting for today. Let's make it to a general topic. Regards Marc From: Giulio Fidente [gfide...@redhat.com] Sent: Thursday, November 14, 2013 6:23 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [qa] Tracking development of scenario tests On 11/14/2013 12:24 AM, David Kranz wrote: 1. Developer checks in the zeroth version of a scenario test as work in progress. It contains a description of the test, and possibly work items. This will claim the area of the proposed scenario to avoid duplication and allow others to comment through gerrit. 2. The developer pushes new versions, removing work in progress if the code is in working state and a review is desired and/or others will be contributing to the scenario. 3. When finished, any process-oriented content such as progress tracking is removed and the test is ready for final review. +1 , the description will eventually contribute to documenting the scenarios yet the submitter (step 1) remains in charge of adding to the draft the reviewers how about we map at least one volunteer to each service (via the HACKING file) and ask submitters to add such a person as reviewer of its drafts when the tests touch the service? this should help avoid tests duplication. I very much like the idea of using gerrit for this -- Giulio Fidente GPG KEY: 08D733BA | IRC: giulivo ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [qa] Tracking development of scenario tests
It was clear at the summit that there is a pressing need for more scenario tests. A number of folks have volunteered to participate so we need a way to track progress and avoid duplication. We have not had great satisfaction using either bugs or blueprints, so Sean and I are proposing a more self-service approach and process: 1. Developer checks in the zeroth version of a scenario test as work in progress. It contains a description of the test, and possibly work items. This will claim the area of the proposed scenario to avoid duplication and allow others to comment through gerrit. 2. The developer pushes new versions, removing work in progress if the code is in working state and a review is desired and/or others will be contributing to the scenario. 3. When finished, any process-oriented content such as progress tracking is removed and the test is ready for final review. We can discuss this at the meeting tomorrow or you can reply with comments. -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
