We should build a simple app using the Oslo libraries that we can place in the 
oslo.messaging source tree to use for exercising the communication patterns of 
the library with different drivers. Ideally that would be a single app (or set 
of apps) that could be used to test all drivers, with tests written against the 
app rather than the driver. Once we have the app and tests, we can define a new 
gate job to run those tests just for oslo.messaging, with a different 
configuration for each driver we support.

Doug

Provided a blueprint for it https://blueprints.launchpad.net/oslo.messaging/+spec/oslo-functional-testing-apps

On 17.11.14 23:53, Doug Hellmann wrote:
On Nov 17, 2014, at 3:36 PM, Joshua Harlow <harlo...@outlook.com> wrote:

Doug Hellmann wrote:
On Nov 17, 2014, at 10:01 AM, Denis Makogon<dmako...@mirantis.com>  wrote:

On Mon, Nov 17, 2014 at 4:26 PM, James Page<james.p...@ubuntu.com>  wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Denis

On 17/11/14 07:43, Denis Makogon wrote:
During Paris Design summit oslo.messaging session was raised good
question about maintaining ZeroMQ driver in upstream (see section
“dropping ZeroMQ support in oslo.messaging” at [1]) . As we all
know, good thoughts are comming always after. I’d like to propose
several improvements in process of maintaining and developing of
ZeroMQ driver in upstream.

Contribution focus. As we all see, that there are enough patches
that are trying to address certain problems related to ZeroMQ
driver.

Few of them trying to add functional tests, which is definitely
good, but … there’s always ‘but’, they are not “gate”-able.
I'm not sure I understand you statement about them not being
"gate"-able - the functional/unit tests currently proposed for the zmq
driver run fine as part of the standard test suite execution - maybe
the confusion is over what 'functional' actually means, but in my
opinion until we have some level of testing of this driver, we can't
effectively make changes and fix bugs.

I do agree that there's a confusion what "functional testing" means.
Another thing, what the best solution is? Unit tests are welcome, but they
are still remain to be units (they are using mocks, etc.)
I'd try to define what 'fuctional testing' means for me. Functional testing
in oslo.messaging means that we've been using real service for messaging
(in this case - deployed 0mq). So, the simple definition, in term os
OpenStack integration, we should be able to run full Tempest test suit for
OpenStack services that are using oslo.messaging with enabled zmq driver.
Am i right or not?
That’s a good goal, but that’s not what I had in mind for in-tree functional 
tests.

We should build a simple app using the Oslo libraries that we can place in the 
oslo.messaging source tree to use for exercising the communication patterns of 
the library with different drivers. Ideally that would be a single app (or set 
of apps) that could be used to test all drivers, with tests written against the 
app rather than the driver. Once we have the app and tests, we can define a new 
gate job to run those tests just for oslo.messaging, with a different 
configuration for each driver we support.

Doug
An interesting idea that might be useful that taskflow implemented/has done...

The examples @ 
https://github.com/openstack/taskflow/tree/master/taskflow/examples all get 
tested during unit test runs to ensure they work as expected. This seems close 
to your 'simple app' (where app is replaced with example), it might be nice to 
have a similar approach for oslo.messaging that has 'examples' that are these 
apps that get ran to probe the functionality of oslo.messaging (as well as 
useful for documentation to show people how to use it, which is the other usage 
taskflow has for these examples)

The hacky example tester could likely be shared (or refactored, since it 
probably needs it), 
https://github.com/openstack/taskflow/blob/master/taskflow/tests/test_examples.py#L91
Sure, that would be a good way to do it, too.

Doug

My proposal for this topic is to change contribution focus from
oslo.messaging by itself to OpenStack/Infra project and DevStack
(subsequently to devstack-gate too).

I guess there would be questions “why?”.  I think the answer is
pretty obvious: we have driver that is not being tested at all
within DevStack and project integration.
This was discussed in the oslo.messaging summit session, and
re-enabling zeromq support in devstack is definately on my todo list,
but I don't think the should block landing of the currently proposed
unit tests on oslo.messaging.

For example https://review.openstack.org/#/c/128233/ says about adding
functional and units. I'm ok with units, but what about functional tests?
Which oslo.messaging gate job runs them?


Also i’d say that such focus re-orientation would be very useful
as source of use cases and bugs eventually. Here’s a list of what
we, as team, should do first:

1.

Ensure that DevStack can successfully:

1.

Install ZeroMQ.

2.

Configure  each project to work with zmq driver from
oslo.messaging.

2.

Ensure that we can run successfully simple test plan for each
project (like boot VM, fill object store container, spin up volume,
etc.).
A better objective would be able to run a full tempest test as
conducted with the RabbitMQ driver IMHO.


I do agree with this too. But we should define step-by-step plan for this
type of testing. Since we want to see quick gate feedback adding full test
suit would be an overhead, at least for now.


ZeroMQ driver maintainers communityorganization. During design
session was raised question about who uses zmq driver in
production.

I’ve seen folks from Canonical and few other companies. So, here’s
my proposals around improving process of maintaining of given
driver:

1.

With respect to best practices of driver maintaining procedure, we
might need to set up community sub-group. What would it give to us
and to the project subsequently? It’s not pretty obvious, at least
for now, but i’d try to light out couple moments:

1.

continuous driver stability

2.

continuous community support (across all OpenStack Project that are
using same model: driver should have maintaining team, would it be
a company or community sub-group)

2.

As sub-group we would need to have our own weekly meeting. Separate
meeting would keep us, as sub-group, pretty focused on zmq driver
only (but it doesn’t mean that we should not participate in regular
meetings). Same question. What it would give us and to the project?
I’d say that the only one valid answer is: we’d not disturb other
folk that are not actually interested in given topic and in zqm
drive too.
I'd prefer that we continue to discuss ZMQ on the broader
oslo.messaging context; I'm keen that the OpenStack community
understands that we want ZMQ to be a first tier driver like qpid and
rmq, and I'm not convinced that pushing discussion out to a separate
sub-group enforces that message...


The only thing that i'm woried about is that we could eventually eat all
meeting time. That's why i try to build out drive maintaining/contribution
team for zmq drive since there were not so may hands when was raised
question about who uses zqm driver.


So, in the end, taking into account words above i’d like to get
feedback from all folks. I’m pretty open for discussion, and if
needed i can commit myself for driving such activities in
oslo.messaging.
- --
James Page
Ubuntu and Debian Developer
james.p...@ubuntu.com
jamesp...@debian.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJUagWDAAoJEL/srsug59jDWncP/2PVkA3tDHxLILjdyXKdLLy6
fsj5yovho45T9LtSrLXD1Y+CT3pQqGDnglB+J8kUBkX56zJLWzSH1szWfRo5Y4Ms
kI0c3K8LxJ6PT4+j/A5JzNt37IhAwBTJ25QcRdzAUgV06IZ3F9ctz9F9lW1GDx/q
u5XvctYacKWhXH/Z/5Y2g3VE2aJSZNlgLA3PxLZeUEQaREj7XeC5x77FZbBYHVI6
E8E8B2H5nf+wln80zIm5rax3vzGh0rZVT/fgUgVcQan33XaFl64zrimjhEUXHUVF
QHIVJ4PNVklqMAEliAq0JMe6ewo1rgbS6DOcB8yOD3RWNo+d/MwSbYiwM/iXI9ya
DpqXK0HVfSbXgoAAqNl2eP5TfvZCtlRk1h3hqhc+843c7i+i2psMZ2mVN6LeJKdt
7EvwY8xQErjKSbsmEjtV069ajjipP3hnmhyUwwJiFM2q8eKMIWRn+WDol88+f4Ke
NmguGjNzKkqqvWSS/IJVT+qHYEsm3GalLT1ZTDaagHpreIJ7vcXxSZTcoGLO6Nhs
445cPZcek6jS+lhf81S13+hmfA1ZgQW2f4Yz5hv15xn90K/OaWE2/Z9AFfsOGWOA
0FoyNY5FSGsNCG/km1BlfVSMWzB4wWpWzunMPFmwme/FoqAjvD4kt6kFKLu9DI1g
/L5WRZfi7Cu7eCC/X6c7
=NBH2
-----END PGP SIGNATURE-----

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Best regards,
Denis M.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Sent with Postbox <http://www.getpostbox.com>

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to