We should build a simple app using the Oslo libraries that we can place in the
oslo.messaging source tree to use for exercising the communication patterns of
the library with different drivers. Ideally that would be a single app (or set
of apps) that could be used to test all drivers, with tests written against the
app rather than the driver. Once we have the app and tests, we can define a new
gate job to run those tests just for oslo.messaging, with a different
configuration for each driver we support.
Doug
Provided a blueprint for it
https://blueprints.launchpad.net/oslo.messaging/+spec/oslo-functional-testing-apps
On 17.11.14 23:53, Doug Hellmann wrote:
On Nov 17, 2014, at 3:36 PM, Joshua Harlow <[email protected]> wrote:
Doug Hellmann wrote:
On Nov 17, 2014, at 10:01 AM, Denis Makogon<[email protected]> wrote:
On Mon, Nov 17, 2014 at 4:26 PM, James Page<[email protected]> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi Denis
On 17/11/14 07:43, Denis Makogon wrote:
During Paris Design summit oslo.messaging session was raised good
question about maintaining ZeroMQ driver in upstream (see section
“dropping ZeroMQ support in oslo.messaging” at [1]) . As we all
know, good thoughts are comming always after. I’d like to propose
several improvements in process of maintaining and developing of
ZeroMQ driver in upstream.
Contribution focus. As we all see, that there are enough patches
that are trying to address certain problems related to ZeroMQ
driver.
Few of them trying to add functional tests, which is definitely
good, but … there’s always ‘but’, they are not “gate”-able.
I'm not sure I understand you statement about them not being
"gate"-able - the functional/unit tests currently proposed for the zmq
driver run fine as part of the standard test suite execution - maybe
the confusion is over what 'functional' actually means, but in my
opinion until we have some level of testing of this driver, we can't
effectively make changes and fix bugs.
I do agree that there's a confusion what "functional testing" means.
Another thing, what the best solution is? Unit tests are welcome, but they
are still remain to be units (they are using mocks, etc.)
I'd try to define what 'fuctional testing' means for me. Functional testing
in oslo.messaging means that we've been using real service for messaging
(in this case - deployed 0mq). So, the simple definition, in term os
OpenStack integration, we should be able to run full Tempest test suit for
OpenStack services that are using oslo.messaging with enabled zmq driver.
Am i right or not?
That’s a good goal, but that’s not what I had in mind for in-tree functional
tests.
We should build a simple app using the Oslo libraries that we can place in the
oslo.messaging source tree to use for exercising the communication patterns of
the library with different drivers. Ideally that would be a single app (or set
of apps) that could be used to test all drivers, with tests written against the
app rather than the driver. Once we have the app and tests, we can define a new
gate job to run those tests just for oslo.messaging, with a different
configuration for each driver we support.
Doug
An interesting idea that might be useful that taskflow implemented/has done...
The examples @
https://github.com/openstack/taskflow/tree/master/taskflow/examples all get
tested during unit test runs to ensure they work as expected. This seems close
to your 'simple app' (where app is replaced with example), it might be nice to
have a similar approach for oslo.messaging that has 'examples' that are these
apps that get ran to probe the functionality of oslo.messaging (as well as
useful for documentation to show people how to use it, which is the other usage
taskflow has for these examples)
The hacky example tester could likely be shared (or refactored, since it
probably needs it),
https://github.com/openstack/taskflow/blob/master/taskflow/tests/test_examples.py#L91
Sure, that would be a good way to do it, too.
Doug
My proposal for this topic is to change contribution focus from
oslo.messaging by itself to OpenStack/Infra project and DevStack
(subsequently to devstack-gate too).
I guess there would be questions “why?”. I think the answer is
pretty obvious: we have driver that is not being tested at all
within DevStack and project integration.
This was discussed in the oslo.messaging summit session, and
re-enabling zeromq support in devstack is definately on my todo list,
but I don't think the should block landing of the currently proposed
unit tests on oslo.messaging.
For example https://review.openstack.org/#/c/128233/ says about adding
functional and units. I'm ok with units, but what about functional tests?
Which oslo.messaging gate job runs them?
Also i’d say that such focus re-orientation would be very useful
as source of use cases and bugs eventually. Here’s a list of what
we, as team, should do first:
1.
Ensure that DevStack can successfully:
1.
Install ZeroMQ.
2.
Configure each project to work with zmq driver from
oslo.messaging.
2.
Ensure that we can run successfully simple test plan for each
project (like boot VM, fill object store container, spin up volume,
etc.).
A better objective would be able to run a full tempest test as
conducted with the RabbitMQ driver IMHO.
I do agree with this too. But we should define step-by-step plan for this
type of testing. Since we want to see quick gate feedback adding full test
suit would be an overhead, at least for now.
ZeroMQ driver maintainers communityorganization. During design
session was raised question about who uses zmq driver in
production.
I’ve seen folks from Canonical and few other companies. So, here’s
my proposals around improving process of maintaining of given
driver:
1.
With respect to best practices of driver maintaining procedure, we
might need to set up community sub-group. What would it give to us
and to the project subsequently? It’s not pretty obvious, at least
for now, but i’d try to light out couple moments:
1.
continuous driver stability
2.
continuous community support (across all OpenStack Project that are
using same model: driver should have maintaining team, would it be
a company or community sub-group)
2.
As sub-group we would need to have our own weekly meeting. Separate
meeting would keep us, as sub-group, pretty focused on zmq driver
only (but it doesn’t mean that we should not participate in regular
meetings). Same question. What it would give us and to the project?
I’d say that the only one valid answer is: we’d not disturb other
folk that are not actually interested in given topic and in zqm
drive too.
I'd prefer that we continue to discuss ZMQ on the broader
oslo.messaging context; I'm keen that the OpenStack community
understands that we want ZMQ to be a first tier driver like qpid and
rmq, and I'm not convinced that pushing discussion out to a separate
sub-group enforces that message...
The only thing that i'm woried about is that we could eventually eat all
meeting time. That's why i try to build out drive maintaining/contribution
team for zmq drive since there were not so may hands when was raised
question about who uses zqm driver.
So, in the end, taking into account words above i’d like to get
feedback from all folks. I’m pretty open for discussion, and if
needed i can commit myself for driving such activities in
oslo.messaging.
- --
James Page
Ubuntu and Debian Developer
[email protected]
[email protected]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBCAAGBQJUagWDAAoJEL/srsug59jDWncP/2PVkA3tDHxLILjdyXKdLLy6
fsj5yovho45T9LtSrLXD1Y+CT3pQqGDnglB+J8kUBkX56zJLWzSH1szWfRo5Y4Ms
kI0c3K8LxJ6PT4+j/A5JzNt37IhAwBTJ25QcRdzAUgV06IZ3F9ctz9F9lW1GDx/q
u5XvctYacKWhXH/Z/5Y2g3VE2aJSZNlgLA3PxLZeUEQaREj7XeC5x77FZbBYHVI6
E8E8B2H5nf+wln80zIm5rax3vzGh0rZVT/fgUgVcQan33XaFl64zrimjhEUXHUVF
QHIVJ4PNVklqMAEliAq0JMe6ewo1rgbS6DOcB8yOD3RWNo+d/MwSbYiwM/iXI9ya
DpqXK0HVfSbXgoAAqNl2eP5TfvZCtlRk1h3hqhc+843c7i+i2psMZ2mVN6LeJKdt
7EvwY8xQErjKSbsmEjtV069ajjipP3hnmhyUwwJiFM2q8eKMIWRn+WDol88+f4Ke
NmguGjNzKkqqvWSS/IJVT+qHYEsm3GalLT1ZTDaagHpreIJ7vcXxSZTcoGLO6Nhs
445cPZcek6jS+lhf81S13+hmfA1ZgQW2f4Yz5hv15xn90K/OaWE2/Z9AFfsOGWOA
0FoyNY5FSGsNCG/km1BlfVSMWzB4wWpWzunMPFmwme/FoqAjvD4kt6kFKLu9DI1g
/L5WRZfi7Cu7eCC/X6c7
=NBH2
-----END PGP SIGNATURE-----
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Best regards,
Denis M.
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Sent with Postbox <http://www.getpostbox.com>
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
