Re: [openstack-dev] [all][infra] Proposed changes to unit-test setup
On 22 November 2016 18:56, John Dickinson wrote:
>
> On 22 Nov 2016, at 10:47, Andreas Jaeger wrote:
>
> > When we (infra) changed the unit test jobs to not set up databases by
> > default, we created special python-db and tox-db jobs that set up both
> > MySQL and PostgreSQL databases. And that complicated the setup of
> > those projects and lead to problems like setting projects up via
> > bindep for both databases even if one was used.
> >
> > We had last week an IRC discussion [1] and came up with the following
> > approach:
> >
> > Projects can use a tools/test-setup.sh script that is called from
> > our unit test (tox, python27, python34, python35) targets. The
> > script is executed as root and should set up the needed databases -
> > or whatever is needed. The script needs to reside in the repository -
> > and thus might need to get backported to older branches.
> >
> > This setup should be used for any kind of repo specific unit test
> > setup.
> >
> > Projects are suggested to add to their developer documents, e.g. the
> > README or CONTRIBUTING or TESTING file, the usage of
> > tools/testsetup.sh. Developers should be able to use the script to
> > set up prerequisites for unit tests locally.
> >
> > Long term goal is for projects to not use the -db jobs anymore, new
> > changes for them should not be accepted.
> >
> > This is implemented in project-config [2], an example usage in
> > nodepool [3,4], which leads to a cleanup [5].
> >
> > Further investigation shows that the special searchlight setup can be
> > solved with the same approach (searchlight change [6], project-config
> > [7]). Here it's interesting to note that moving the setup in the
> > repository, found a problem: The repo needs elasticsearch 1 for
> > liberty and 2 for newer branches, this can now be done inside the
> > repository.
> >
> > The xfs TMPDIR setup of swift [2] could been done in general this way
> > as well but that change needs to set TMPDIR for the unittests, passing
> > information from the set up builder to the tox builder. This is
> > currently not possible using only the proposed solution, and so would
> > still require a custom tox job. Alternative, this could be changed
> > with some other way of passing the value of TMPDIR between these
> > different invocations.
>
> (actually link [10])
>
> This sounds like a great idea that I would have loved to have in place a few
> weeks ago!
>
> One question: is there a limit to which tox environments will call this
> in-repo
> script? Above you list a few, but Swift and other projects have repo-specific
> environments that would need the setup as well. How will that work?
>
+1 this is a really useful feature.
Further to John's question, would it be possible for the test-setup.sh script
to be passed the job template name as an arg? That would allow customized setup
for specific test jobs. Right now I don’t have a specific use case in Swift but
I'm curious as to whether it is a possibility.
>
> >
> > Today, a change was proposed [8,9] that would setup docker for kolla
> > and kolla-ansible. I suggest to not merge it and instead use the same
> > approach here.
> >
> > Credits for the proposal go to Jeremy - and this got triggered by
> > comments by Jim. Thanks!
> >
> > Andreas
> >
> > [1]
> > http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack
> > -infra.2016-11-17.log.html#t2016-11-17T15:07:38
> >
> > [2] https://review.openstack.org/399105
> > [3] https://review.openstack.org/399079
> > [4] https://review.openstack.org/399177
> > [5] https://review.openstack.org/399180
> > [6] https://review.openstack.org/399159
> > [7] https://review.openstack.org/399169
> > [8] https://review.openstack.org/400128
> > [9] https://review.openstack.org/400474
> > [10] https://review.openstack.org/394600
> > --
> > Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
> > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> >GF: Felix Imendörffer, Jane Smithard, Graham Norton,
> >HRB 21284 (AG Nürnberg)
> > GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272
> > A126
> >
> >
>
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][infra] Proposed changes to unit-test setup
On 2016-11-22 19:47:32 +0100 (+0100), Andreas Jaeger wrote: [...] > Projects are suggested to add to their developer documents, e.g. the > README or CONTRIBUTING or TESTING file, the usage of > tools/testsetup.sh. Developers should be able to use the script to > set up prerequisites for unit tests locally. [...] Note that we've also got a fairly well-received addendum to the Consistent Testing Guidelines which aligns nicely with this option: https://review.openstack.org/397502 -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][infra] Proposed changes to unit-test setup
On 11/22/2016 07:56 PM, John Dickinson wrote:
>
>
> On 22 Nov 2016, at 10:47, Andreas Jaeger wrote:
>
>> When we (infra) changed the unit test jobs to not set up databases by
>> default, we created special python-db and tox-db jobs that set up both
>> MySQL and PostgreSQL databases. And that complicated the setup of those
>> projects and lead to problems like setting projects up via bindep for
>> both databases even if one was used.
>>
>> We had last week an IRC discussion [1] and came up with the following
>> approach:
>>
>> Projects can use a tools/test-setup.sh script that is called from
>> our unit test (tox, python27, python34, python35) targets. The
>> script is executed as root and should set up the needed databases -
>> or whatever is needed. The script needs to reside in the repository -
>> and thus might need to get backported to older branches.
>>
>> This setup should be used for any kind of repo specific unit test
>> setup.
>>
>> Projects are suggested to add to their developer documents, e.g. the
>> README or CONTRIBUTING or TESTING file, the usage of
>> tools/testsetup.sh. Developers should be able to use the script to
>> set up prerequisites for unit tests locally.
>>
>> Long term goal is for projects to not use the -db jobs anymore, new
>> changes for them should not be accepted.
>>
>> This is implemented in project-config [2], an example usage in
>> nodepool [3,4], which leads to a cleanup [5].
>>
>> Further investigation shows that the special searchlight setup can be
>> solved with the same approach (searchlight change [6], project-config
>> [7]). Here it's interesting to note that moving the setup in the
>> repository, found a problem: The repo needs elasticsearch 1 for
>> liberty and 2 for newer branches, this can now be done inside the
>> repository.
>>
>> The xfs TMPDIR setup of swift [2] could been done in general this way as
>> well but that change needs to set TMPDIR for the unittests, passing
>> information from the set up builder to the tox builder. This is
>> currently not possible using only the proposed solution, and so would
>> still require a custom tox job. Alternative, this could be changed with
>> some other way of passing the value of TMPDIR between these different
>> invocations.
>
> (actually link [10])
Indeed, thanks for correcting.
> This sounds like a great idea that I would have loved to have in place a few
> weeks ago!
>
> One question: is there a limit to which tox environments will call this
> in-repo script? Above you list a few, but Swift and other projects have
> repo-specific environments that would need the setup as well. How will that
> work?
So far it's the templates I mentioned. The custom swift jobs are not
setup this way and I'm fine with removing the custom jobs and use the
default jobs again.
In general, if additional jobs need the change from [2], we can add them
- let's discuss this one by one in a review,
Andreas
>
>>
>> Today, a change was proposed [8,9] that would setup docker for kolla
>> and kolla-ansible. I suggest to not merge it and instead use the same
>> approach here.
>>
>> Credits for the proposal go to Jeremy - and this got triggered by
>> comments by Jim. Thanks!
>>
>> Andreas
>>
>> [1]
>> http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-11-17.log.html#t2016-11-17T15:07:38
>>
>> [2] https://review.openstack.org/399105
>> [3] https://review.openstack.org/399079
>> [4] https://review.openstack.org/399177
>> [5] https://review.openstack.org/399180
>> [6] https://review.openstack.org/399159
>> [7] https://review.openstack.org/399169
>> [8] https://review.openstack.org/400128
>> [9] https://review.openstack.org/400474
>> [10] https://review.openstack.org/394600
>> --
>> Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>>HRB 21284 (AG Nürnberg)
>> GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
Re: [openstack-dev] [all][infra] Proposed changes to unit-test setup
On 22 Nov 2016, at 10:47, Andreas Jaeger wrote:
> When we (infra) changed the unit test jobs to not set up databases by
> default, we created special python-db and tox-db jobs that set up both
> MySQL and PostgreSQL databases. And that complicated the setup of those
> projects and lead to problems like setting projects up via bindep for
> both databases even if one was used.
>
> We had last week an IRC discussion [1] and came up with the following
> approach:
>
> Projects can use a tools/test-setup.sh script that is called from
> our unit test (tox, python27, python34, python35) targets. The
> script is executed as root and should set up the needed databases -
> or whatever is needed. The script needs to reside in the repository -
> and thus might need to get backported to older branches.
>
> This setup should be used for any kind of repo specific unit test
> setup.
>
> Projects are suggested to add to their developer documents, e.g. the
> README or CONTRIBUTING or TESTING file, the usage of
> tools/testsetup.sh. Developers should be able to use the script to
> set up prerequisites for unit tests locally.
>
> Long term goal is for projects to not use the -db jobs anymore, new
> changes for them should not be accepted.
>
> This is implemented in project-config [2], an example usage in
> nodepool [3,4], which leads to a cleanup [5].
>
> Further investigation shows that the special searchlight setup can be
> solved with the same approach (searchlight change [6], project-config
> [7]). Here it's interesting to note that moving the setup in the
> repository, found a problem: The repo needs elasticsearch 1 for
> liberty and 2 for newer branches, this can now be done inside the
> repository.
>
> The xfs TMPDIR setup of swift [2] could been done in general this way as
> well but that change needs to set TMPDIR for the unittests, passing
> information from the set up builder to the tox builder. This is
> currently not possible using only the proposed solution, and so would
> still require a custom tox job. Alternative, this could be changed with
> some other way of passing the value of TMPDIR between these different
> invocations.
(actually link [10])
This sounds like a great idea that I would have loved to have in place a few
weeks ago!
One question: is there a limit to which tox environments will call this in-repo
script? Above you list a few, but Swift and other projects have repo-specific
environments that would need the setup as well. How will that work?
>
> Today, a change was proposed [8,9] that would setup docker for kolla
> and kolla-ansible. I suggest to not merge it and instead use the same
> approach here.
>
> Credits for the proposal go to Jeremy - and this got triggered by
> comments by Jim. Thanks!
>
> Andreas
>
> [1]
> http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-11-17.log.html#t2016-11-17T15:07:38
>
> [2] https://review.openstack.org/399105
> [3] https://review.openstack.org/399079
> [4] https://review.openstack.org/399177
> [5] https://review.openstack.org/399180
> [6] https://review.openstack.org/399159
> [7] https://review.openstack.org/399169
> [8] https://review.openstack.org/400128
> [9] https://review.openstack.org/400474
> [10] https://review.openstack.org/394600
> --
> Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all][infra] Proposed changes to unit-test setup
When we (infra) changed the unit test jobs to not set up databases by
default, we created special python-db and tox-db jobs that set up both
MySQL and PostgreSQL databases. And that complicated the setup of those
projects and lead to problems like setting projects up via bindep for
both databases even if one was used.
We had last week an IRC discussion [1] and came up with the following
approach:
Projects can use a tools/test-setup.sh script that is called from
our unit test (tox, python27, python34, python35) targets. The
script is executed as root and should set up the needed databases -
or whatever is needed. The script needs to reside in the repository -
and thus might need to get backported to older branches.
This setup should be used for any kind of repo specific unit test
setup.
Projects are suggested to add to their developer documents, e.g. the
README or CONTRIBUTING or TESTING file, the usage of
tools/testsetup.sh. Developers should be able to use the script to
set up prerequisites for unit tests locally.
Long term goal is for projects to not use the -db jobs anymore, new
changes for them should not be accepted.
This is implemented in project-config [2], an example usage in
nodepool [3,4], which leads to a cleanup [5].
Further investigation shows that the special searchlight setup can be
solved with the same approach (searchlight change [6], project-config
[7]). Here it's interesting to note that moving the setup in the
repository, found a problem: The repo needs elasticsearch 1 for
liberty and 2 for newer branches, this can now be done inside the
repository.
The xfs TMPDIR setup of swift [2] could been done in general this way as
well but that change needs to set TMPDIR for the unittests, passing
information from the set up builder to the tox builder. This is
currently not possible using only the proposed solution, and so would
still require a custom tox job. Alternative, this could be changed with
some other way of passing the value of TMPDIR between these different
invocations.
Today, a change was proposed [8,9] that would setup docker for kolla
and kolla-ansible. I suggest to not merge it and instead use the same
approach here.
Credits for the proposal go to Jeremy - and this got triggered by
comments by Jim. Thanks!
Andreas
[1]
http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-11-17.log.html#t2016-11-17T15:07:38
[2] https://review.openstack.org/399105
[3] https://review.openstack.org/399079
[4] https://review.openstack.org/399177
[5] https://review.openstack.org/399180
[6] https://review.openstack.org/399159
[7] https://review.openstack.org/399169
[8] https://review.openstack.org/400128
[9] https://review.openstack.org/400474
[10] https://review.openstack.org/394600
--
Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
