There will definitely be tests that are specific to the client implementations of protocol version. I think client unit tests or systests are the correct place for these. If tests are pure JMS, then yes, moving them out to a non-client/protocol specific test pack makes sense.
This might also provide a good point at which to document the tests. So if a test is to be moved, it should be fully documented as is it moved. I would be happy to own the responsibility for cracking the Javadoc whip. The idea is that if we write a test against 0-8, against the surface of the product, that tests for some behaviour that we need to regression test as we move to 0-10, then that will help to compensate for the fact that the client software is being rewritten from scratch, and may therefore lose important bug fixes that are currently going into the 0-8 client on M2. In some cases writing a bug test against the product surface, versus writing a lower level unit test, might not be easy to do. For example, there is a test that checks that the store is free of rejected messages, or something like that. Not something you can do through the client API. Before launching in, and hand coding lots of tests, it is worth thinking about the test generation/fitness function idea that is described in my testing proposal. When I wrote the Immediate/Mandatory tests they were initially written because there was a bug with returned message processing. By varying the message size, whether it had no body, or a body, further bugs in this area were shown up. This led me to think about pulling out all of the different options and flags that can be used to describe a messaging scenario, and then thoroughly testing all reasonable combinations (or however many we can test without it taking forever); there may be bugs lurking in some particular combinations as has been previously shown to be the case. The Immediate/Mandatory tests hand code the specific set of combinations needed to test that scenario. However, I imagine writing a general purpose test, that is controlled by a set of Properties, and using the fitness function gets the set of assertions that apply to a given scenario. My starting point was all of the properties defined in PingPongProducer, which have been pulled out into a seperate class, MessagingTestConfigProperties. Once the general purpose test has been written, Immediate/Mandatory tests will be replaced by it. The general purpose test will also be able to run a lot of testing scenarios through the Coordinator as interop tests. These tests need to be run over a 'test circuit' which is a somewhat limited concept, but useful enough to test a lot of things. The analogy is an electrical test circuit, like you might have used at high school. There is a switch, a cell, a light bulb, a transistor etc. and you can wire it up to produce a limited range of circuits. The messaging equivalent has a sending side, a receiving side, and outward route, an inward route (not used by default), and is parametrized by message size, message rate, persistent/transient, transactional/non-tx and so on. The sending and receiving sides may consist of a single producer/consumer in-vm, or many producers/consumers across many test nodes. This circuit idea will not be suitable without extension, to cover more freestyle tests. The sort of tests that open a connection, create two sessions, send a message, close one session, send another and so on. Some of these can be pure JMS, which is good from the regression testing perspective. They may also benefit from using a testing base class to handle some configurable aspects of testing common to all such tests, such as in-vm vs remote, without using the 'test circuit' concept. Rupert On 27/09/2007, Arnaud Simon <[EMAIL PROTECTED]> wrote: > > As a starting point and for 0_10 testing purpose I was thinking about > changing the (pure JMS) tests from the client module so that to run them > either against an in_VM broker or against a remote broker (we can set > some maven parameters). The default behavior will obviously be to use an > in-VM broker. I am also adding some 0_10 specific test that will just > not run against an 8_0 (in-VM or remote) broker. > > Note that some of those tests (the one I am adding for example) are jms > specific. We may want to move them in /trunk/qpidtests later on. Or > maybe a better idea is to move them in /trunk/qpidtests right now. What > do you think? > > Arnaud > > > On Wed, 2007-09-26 at 19:36 +0100, Robert Greig wrote: > > On 26/09/2007, Rupert Smith <[EMAIL PROTECTED]> wrote: > > > Would anyone object if test framework under systests + integration > tests + > > > perftests + 'Immediate/MandatoryMessageTest', were merged with Arnauds > pure > > > JMS changes, and put under: > > > > > > /trunk/qpidtests ? > > > > > > This code will have no Qpid dependencies, except that there will need > to be > > > an interface created for an extension point for AMQP specific stuff > (like > > > immediate/mandatory flags). One nicety of this, is that the entire > test > > > framework will also be able to be run over other JMS products, for > > > comparative testing. > > > > From that description it sounds like a good idea. > > > > RG > >
