Re: [openstack-dev] [Trove] Testing of new service types support
Hi Illia,
You're correct; until the work on establishing datastore types and versions as
a first class Trove concept is finished, which will hopefully be soon (see
Andrey Shestakov's pull request), testing non-MySQL datastore types will be
problematic.
A short term, fake-mode only solution could be accomplished fairly quickly as
follows: run the fake mode tests a third time in Tox with a new configuration
which allows for MongoDB.
If you look at tox.ini, you'll see that the integration tests run in fake mode
twice already:
{envpython} run_tests.py
{envpython} run_tests.py --test-config=etc/tests/xml.localhost.test.conf
The second invocation causes the trove-client to be used in XML mode,
effectively testing the XML client.
(Tangent: currently running the tests twice takes some time, even in fake mode-
however it will cost far less time once the following pull request is merged:
https://review.openstack.org/#/c/52490/)
If you look at run_tests.py, you'll see that on line 104 it accepts a trove
config file. If the run_tests.py script is updated to allow this value to be
specified optionally via the command line, you could create a variation on
etc/trove/trove.conf.test which specifies MongoDB. You'd then invoke
run_tests.py with a --group= argument to run some subset of the tests support
by the current Mongo DB code in fake mode.
Of course, this will do nothing to test the guest agent changes or confirm that
the end to end system actually works, but it could help test a lot of
incidental API and infrastructure database code.
As for real mode tests, I think we should wait until the datastore type /
version code is finished, at which point I know we'll all be eager to add
additional tests for these new datastores. Of course in the short term it
should be possible for you to change the code locally to build a Mongo DB image
as well as a Trove config file to support this and then just run some subset of
tests that works with Mongo.
Thanks,
Tim
From: Illia Khudoshyn [ikhudos...@mirantis.com]
Sent: Monday, October 21, 2013 9:42 AM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [Trove] Testing of new service types support
Hi all,
I've done with implementing the very first bits of MongoDB support in Trove
along with unit tests and faced an issue with proper testing of it.
It is well known that right now only one service type per installation is
supported by Trove (it is set in config). All testing infrastructure, including
Trove-integration codebase and jenkins jobs, seem to rely on that service type
as well. So it seems to be impossible to run all existing tests AND some
additional tests for MongoDB service type in one pass, at least until Trove
client will allow to pass service type (I know that there is ongoing work in
this area).
Please note, that all of the above is about functional and intergation testing
-- there is no issues with unit tests.
So the question is, should I first submit the code to Trove and then proceed
with updating Trove-integration or just put aside all that MongoDB stuff until
client (and -integration) will be ready?
PS AFAIK, there is some work on adding Cassandra and Riak (or Redis?) support
to Trove. These guys will likely face this issue as well.
--
Best regards,
Illia Khudoshyn,
Software Engineer, Mirantis, Inc.
38, Lenina ave. Kharkov, Ukraine
www.mirantis.comhttp://www.mirantis.ru/
www.mirantis.ruhttp://www.mirantis.ru/
Skype: gluke_work
ikhudos...@mirantis.commailto:ikhudos...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] Testing of new service types support
Hi Tim,
Thanks for a quick reply. I'll go with updating run_tests.py for now. Hope,
Andrey Shestakov's changes arrive soon.
Best wishes.
On Mon, Oct 21, 2013 at 7:01 PM, Tim Simpson tim.simp...@rackspace.comwrote:
Hi Illia,
You're correct; until the work on establishing datastore types and
versions as a first class Trove concept is finished, which will hopefully
be soon (see Andrey Shestakov's pull request), testing non-MySQL datastore
types will be problematic.
A short term, fake-mode only solution could be accomplished fairly
quickly as follows: run the fake mode tests a third time in Tox with a
new configuration which allows for MongoDB.
If you look at tox.ini, you'll see that the integration tests run in
fake mode twice already:
{envpython} run_tests.py
{envpython} run_tests.py --test-config=etc/tests/xml.localhost.test.conf
The second invocation causes the trove-client to be used in XML mode,
effectively testing the XML client.
(Tangent: currently running the tests twice takes some time, even in
fake mode- however it will cost far less time once the following pull
request is merged: https://review.openstack.org/#/c/52490/)
If you look at run_tests.py, you'll see that on line 104 it accepts a
trove config file. If the run_tests.py script is updated to allow this
value to be specified optionally via the command line, you could create a
variation on etc/trove/trove.conf.test which specifies MongoDB. You'd
then invoke run_tests.py with a --group= argument to run some subset of
the tests support by the current Mongo DB code in fake mode.
Of course, this will do nothing to test the guest agent changes or
confirm that the end to end system actually works, but it could help test a
lot of incidental API and infrastructure database code.
As for real mode tests, I think we should wait until the datastore type
/ version code is finished, at which point I know we'll all be eager to add
additional tests for these new datastores. Of course in the short term it
should be possible for you to change the code locally to build a Mongo DB
image as well as a Trove config file to support this and then just run some
subset of tests that works with Mongo.
Thanks,
Tim
--
*From:* Illia Khudoshyn [ikhudos...@mirantis.com]
*Sent:* Monday, October 21, 2013 9:42 AM
*To:* OpenStack Development Mailing List
*Subject:* [openstack-dev] [Trove] Testing of new service types support
Hi all,
I've done with implementing the very first bits of MongoDB support in
Trove along with unit tests and faced an issue with proper testing of it.
It is well known that right now only one service type per installation
is supported by Trove (it is set in config). All testing infrastructure,
including Trove-integration codebase and jenkins jobs, seem to rely on that
service type as well. So it seems to be impossible to run all existing
tests AND some additional tests for MongoDB service type in one pass, at
least until Trove client will allow to pass service type (I know that there
is ongoing work in this area).
Please note, that all of the above is about functional and intergation
testing -- there is no issues with unit tests.
So the question is, should I first submit the code to Trove and then
proceed with updating Trove-integration or just put aside all that MongoDB
stuff until client (and -integration) will be ready?
PS AFAIK, there is some work on adding Cassandra and Riak (or Redis?)
support to Trove. These guys will likely face this issue as well.
--
Best regards,
Illia Khudoshyn,
Software Engineer, Mirantis, Inc.
38, Lenina ave. Kharkov, Ukraine
www.mirantis.com http://www.mirantis.ru/
www.mirantis.ru
Skype: gluke_work
ikhudos...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Best regards,
Illia Khudoshyn,
Software Engineer, Mirantis, Inc.
38, Lenina ave. Kharkov, Ukraine
www.mirantis.com http://www.mirantis.ru/
www.mirantis.ru
Skype: gluke_work
ikhudos...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] Testing of new service types support
Top posting…
Id like to see these in the tempest tests. Im just getting started integrating
trove into tempest for testing, and there are some prerequisites that im
working thru with the infra team. Progress is being made though. Id rather not
see them go into 2 different test suites if we can just get them into the
tempest tests. Lets hope the stars line up so that you can start testing in
tempest. :)
On Oct 21, 2013, at 9:25 AM, Illia Khudoshyn wrote:
Hi Tim,
Thanks for a quick reply. I'll go with updating run_tests.py for now. Hope,
Andrey Shestakov's changes arrive soon.
Best wishes.
On Mon, Oct 21, 2013 at 7:01 PM, Tim Simpson tim.simp...@rackspace.com
wrote:
Hi Illia,
You're correct; until the work on establishing datastore types and versions
as a first class Trove concept is finished, which will hopefully be soon (see
Andrey Shestakov's pull request), testing non-MySQL datastore types will be
problematic.
A short term, fake-mode only solution could be accomplished fairly quickly as
follows: run the fake mode tests a third time in Tox with a new configuration
which allows for MongoDB.
If you look at tox.ini, you'll see that the integration tests run in fake
mode twice already:
{envpython} run_tests.py
{envpython} run_tests.py --test-config=etc/tests/xml.localhost.test.conf
The second invocation causes the trove-client to be used in XML mode,
effectively testing the XML client.
(Tangent: currently running the tests twice takes some time, even in fake
mode- however it will cost far less time once the following pull request is
merged: https://review.openstack.org/#/c/52490/)
If you look at run_tests.py, you'll see that on line 104 it accepts a trove
config file. If the run_tests.py script is updated to allow this value to be
specified optionally via the command line, you could create a variation on
etc/trove/trove.conf.test which specifies MongoDB. You'd then invoke
run_tests.py with a --group= argument to run some subset of the tests
support by the current Mongo DB code in fake mode.
Of course, this will do nothing to test the guest agent changes or confirm
that the end to end system actually works, but it could help test a lot of
incidental API and infrastructure database code.
As for real mode tests, I think we should wait until the datastore type /
version code is finished, at which point I know we'll all be eager to add
additional tests for these new datastores. Of course in the short term it
should be possible for you to change the code locally to build a Mongo DB
image as well as a Trove config file to support this and then just run some
subset of tests that works with Mongo.
Thanks,
Tim
From: Illia Khudoshyn [ikhudos...@mirantis.com]
Sent: Monday, October 21, 2013 9:42 AM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [Trove] Testing of new service types support
Hi all,
I've done with implementing the very first bits of MongoDB support in Trove
along with unit tests and faced an issue with proper testing of it.
It is well known that right now only one service type per installation is
supported by Trove (it is set in config). All testing infrastructure,
including Trove-integration codebase and jenkins jobs, seem to rely on that
service type as well. So it seems to be impossible to run all existing tests
AND some additional tests for MongoDB service type in one pass, at least
until Trove client will allow to pass service type (I know that there is
ongoing work in this area).
Please note, that all of the above is about functional and intergation
testing -- there is no issues with unit tests.
So the question is, should I first submit the code to Trove and then proceed
with updating Trove-integration or just put aside all that MongoDB stuff
until client (and -integration) will be ready?
PS AFAIK, there is some work on adding Cassandra and Riak (or Redis?) support
to Trove. These guys will likely face this issue as well.
--
Best regards,
Illia Khudoshyn,
Software Engineer, Mirantis, Inc.
38, Lenina ave. Kharkov, Ukraine
www.mirantis.com
www.mirantis.ru
Skype: gluke_work
ikhudos...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Best regards,
Illia Khudoshyn,
Software Engineer, Mirantis, Inc.
38, Lenina ave. Kharkov, Ukraine
www.mirantis.com
www.mirantis.ru
Skype: gluke_work
ikhudos...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
signature.asc
Description: Message signed with OpenPGP using GPGMail
___
OpenStack-dev mailing list
Re: [openstack-dev] [Trove] Testing of new service types support
Can't we say that about nearly any feature though? In theory we could put a
hold on any tests for feature work saying it
will need to be redone when Tempest integrated is finished.
Keep in mind what I'm suggesting here is a fairly trivial change to get some
validation via the existing fake mode / integration tests at a fairly small
cost.
From: Michael Basnight [mbasni...@gmail.com]
Sent: Monday, October 21, 2013 11:45 AM
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] [Trove] Testing of new service types support
Top posting…
Id like to see these in the tempest tests. Im just getting started integrating
trove into tempest for testing, and there are some prerequisites that im
working thru with the infra team. Progress is being made though. Id rather not
see them go into 2 different test suites if we can just get them into the
tempest tests. Lets hope the stars line up so that you can start testing in
tempest. :)
On Oct 21, 2013, at 9:25 AM, Illia Khudoshyn wrote:
Hi Tim,
Thanks for a quick reply. I'll go with updating run_tests.py for now. Hope,
Andrey Shestakov's changes arrive soon.
Best wishes.
On Mon, Oct 21, 2013 at 7:01 PM, Tim Simpson tim.simp...@rackspace.com
wrote:
Hi Illia,
You're correct; until the work on establishing datastore types and versions
as a first class Trove concept is finished, which will hopefully be soon (see
Andrey Shestakov's pull request), testing non-MySQL datastore types will be
problematic.
A short term, fake-mode only solution could be accomplished fairly quickly as
follows: run the fake mode tests a third time in Tox with a new configuration
which allows for MongoDB.
If you look at tox.ini, you'll see that the integration tests run in fake
mode twice already:
{envpython} run_tests.py
{envpython} run_tests.py --test-config=etc/tests/xml.localhost.test.conf
The second invocation causes the trove-client to be used in XML mode,
effectively testing the XML client.
(Tangent: currently running the tests twice takes some time, even in fake
mode- however it will cost far less time once the following pull request is
merged: https://review.openstack.org/#/c/52490/)
If you look at run_tests.py, you'll see that on line 104 it accepts a trove
config file. If the run_tests.py script is updated to allow this value to be
specified optionally via the command line, you could create a variation on
etc/trove/trove.conf.test which specifies MongoDB. You'd then invoke
run_tests.py with a --group= argument to run some subset of the tests
support by the current Mongo DB code in fake mode.
Of course, this will do nothing to test the guest agent changes or confirm
that the end to end system actually works, but it could help test a lot of
incidental API and infrastructure database code.
As for real mode tests, I think we should wait until the datastore type /
version code is finished, at which point I know we'll all be eager to add
additional tests for these new datastores. Of course in the short term it
should be possible for you to change the code locally to build a Mongo DB
image as well as a Trove config file to support this and then just run some
subset of tests that works with Mongo.
Thanks,
Tim
From: Illia Khudoshyn [ikhudos...@mirantis.com]
Sent: Monday, October 21, 2013 9:42 AM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [Trove] Testing of new service types support
Hi all,
I've done with implementing the very first bits of MongoDB support in Trove
along with unit tests and faced an issue with proper testing of it.
It is well known that right now only one service type per installation is
supported by Trove (it is set in config). All testing infrastructure,
including Trove-integration codebase and jenkins jobs, seem to rely on that
service type as well. So it seems to be impossible to run all existing tests
AND some additional tests for MongoDB service type in one pass, at least
until Trove client will allow to pass service type (I know that there is
ongoing work in this area).
Please note, that all of the above is about functional and intergation
testing -- there is no issues with unit tests.
So the question is, should I first submit the code to Trove and then proceed
with updating Trove-integration or just put aside all that MongoDB stuff
until client (and -integration) will be ready?
PS AFAIK, there is some work on adding Cassandra and Riak (or Redis?) support
to Trove. These guys will likely face this issue as well.
--
Best regards,
Illia Khudoshyn,
Software Engineer, Mirantis, Inc.
38, Lenina ave. Kharkov, Ukraine
www.mirantis.com
www.mirantis.ru
Skype: gluke_work
ikhudos...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] Testing of new service types support
On Oct 21, 2013, at 10:02 AM, Tim Simpson wrote: Can't we say that about nearly any feature though? In theory we could put a hold on any tests for feature work saying it will need to be redone when Tempest integrated is finished. Keep in mind what I'm suggesting here is a fairly trivial change to get some validation via the existing fake mode / integration tests at a fairly small cost. Of course we can do the old tests. And for this it might be the best thing. The problem i see is that we cant do real integration tests w/o this work, and i dont want to integrate a bunch of different service_types w/o tests that actually spin them up and run the guest, which is where 80% of the new code lives for a new service_type. Otherwise we are running fake-guest stuff that is not a good representation. For the api stuff, sure thats fine. i just think the overall coverage of the review will be quite low if we are only testing the API via fake code. signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] Testing of new service types support
For the api stuff, sure thats fine. i just think the overall coverage of the review will be quite low if we are only testing the API via fake code. We're in agreement here, I think. I will say though that if the people working on Mongo want to test it early, and go beyond simply using the client to manually confirm stuff, it should be possible to run the existing tests by building a different image and running a subset, such as --group=dbaas.guest.shutdown. IIRC those tests don't do much other than make an instance, see it turn to ACTIVE, and delete it. It would be a worthwhile spot test to see if it adheres to the bare-minimum Trove API. From: Michael Basnight [mbasni...@gmail.com] Sent: Monday, October 21, 2013 12:19 PM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Trove] Testing of new service types support On Oct 21, 2013, at 10:02 AM, Tim Simpson wrote: Can't we say that about nearly any feature though? In theory we could put a hold on any tests for feature work saying it will need to be redone when Tempest integrated is finished. Keep in mind what I'm suggesting here is a fairly trivial change to get some validation via the existing fake mode / integration tests at a fairly small cost. Of course we can do the old tests. And for this it might be the best thing. The problem i see is that we cant do real integration tests w/o this work, and i dont want to integrate a bunch of different service_types w/o tests that actually spin them up and run the guest, which is where 80% of the new code lives for a new service_type. Otherwise we are running fake-guest stuff that is not a good representation. For the api stuff, sure thats fine. i just think the overall coverage of the review will be quite low if we are only testing the API via fake code. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Trove] Testing of new service types support
Michael, Tim, Nice to see you, guys, agreed. But what should I do now? Dive into trove-integration? I guess there will be no use of mocked tests, coz I haven't actually written a single line of server side code. All the fun is in guest agent. Thanks For the api stuff, sure thats fine. i just think the overall coverage of the review will be quite low if we are only testing the API via fake code. We're in agreement here, I think. I will say though that if the people working on Mongo want to test it early, and go beyond simply using the client to manually confirm stuff, it should be possible to run the existing tests by building a different image and running a subset, such as --group=dbaas.guest.shutdown. IIRC those tests don't do much other than make an instance, see it turn to ACTIVE, and delete it. It would be a worthwhile spot test to see if it adheres to the bare-minimum Trove API. From: Michael Basnight [mbasni...@gmail.com javascript:;] Sent: Monday, October 21, 2013 12:19 PM To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [Trove] Testing of new service types support On Oct 21, 2013, at 10:02 AM, Tim Simpson wrote: Can't we say that about nearly any feature though? In theory we could put a hold on any tests for feature work saying it will need to be redone when Tempest integrated is finished. Keep in mind what I'm suggesting here is a fairly trivial change to get some validation via the existing fake mode / integration tests at a fairly small cost. Of course we can do the old tests. And for this it might be the best thing. The problem i see is that we cant do real integration tests w/o this work, and i dont want to integrate a bunch of different service_types w/o tests that actually spin them up and run the guest, which is where 80% of the new code lives for a new service_type. Otherwise we are running fake-guest stuff that is not a good representation. For the api stuff, sure thats fine. i just think the overall coverage of the review will be quite low if we are only testing the API via fake code. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org javascript:; http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best regards, Illia Khudoshyn, Software Engineer, Mirantis, Inc. 38, Lenina ave. Kharkov, Ukraine www.mirantis.com http://www.mirantis.ru/ www.mirantis.ru Skype: gluke_work ikhudos...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
