Jiri Danek commented on ARTEMIS-1330:

I guess you have a point, unchecked proliferation of duplicated and half-baked 
test suites would be a very bad thing. Still, while I was not able to 
investigate all failures I got in my modified test suite, what I did 
investigate and reported is I think real issues and some of them may be 
nontrivial (as opposed to e.g. QPIDJMS-312 which was certainly not serious.)

I am especially hoping that ARTEMIS-1337, ARTEMIS-1340, ARTEMIS-1339 and  
ARTEMIS-1329 will prove to be worth the effort required to investigate them.

So maybe trying to do this once gives the best results for minimum effort. I 
will continue to investigate these myself more deeply throughout the evenings 
and maybe i will find some specific bugs there.

> [DISCUSS] Run JMS tests over many protocols (through different JMS client 
> libraries)
> ------------------------------------------------------------------------------------
>                 Key: ARTEMIS-1330
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-1330
>             Project: ActiveMQ Artemis
>          Issue Type: Test
>          Components: AMQP, OpenWire
>    Affects Versions: 2.3.0
>            Reporter: Jiri Danek
>            Priority: Minor
> Currently, there are three sets of JMS tests in Apache ActiveMQ Artemis 
> codebase (that I know of)
> * 
> activemq-artemis/tests/jms-tests/src/test/java/org/apache/activemq/artemis/jms/tests/*
> * 
> activemq-artemis/tests/integration-tests/src/test/java/org/apache/activemq/artemis/tests/integration/jms/*
> * 
> activemq-artemis/tests/integration-tests/src/test/java/org/apache/activemq/artemis/tests/integration/amqp/*
> The first two are being run only through the Core JMS client, while the last 
> one is being run through Qpid JMS client (with few exceptions).
> It should be beneficial to be able to run all JMS tests through all three 
> protocols that Artemis supports that have a JMS client, these being
> * Core (artemis-jms-client),
> * OpenWire (activemq-client), and
> * AMQP (qpid-jms-client).
> I made an attempt to convert some of the tests to a parametrized test with 
> changeable connection factory. The way 
> {{org.apache.activemq.artemis.tests.integration.persistence.SyncSendTest}} 
> currently does it. See https://github.com/jdanekrh/activemq-artemis/pull/4
> There are the following obstackes
> * Some tests go beyond the JMS standardized APIs and cast the JMS object to 
> specific implementations in their chosen library.
> * Some tests require specific settings on the connection factory, these are 
> not portable across different JMS clients. There will have to be branching 
> depending on the specific factory currently in use, or something...
> * OpenWire client supports JMS 1.1, while Core and AMQP clients are JMS 2.0, 
> some tests will have to be skipped on OpenWire
> * OpenWire client has different semantics of {{receiver.receiveNoWait()}} 
> call; when there are no messages in prefetch cache, OpenWire returns null, 
> while other clients do a round trip to server to check for new messages 
> before giving up.
> There are some improvements which can be considered later, some of these are 
> partially implemented in various places
> * share broker between individual tests
> * use random name or random suffix for created addresses and queues use 
> suffix for queues/topics to ensure isolation
> * have broker uberconfig for all tests, or group tests by config they need, 
> so that reconfiguring broker is limited
> * that also allows tests to run in parallel
> * use random port for acceptor on broker, can run tests even more parallel, 
> and mitigates environmentally caused failures
> I intend to report the failures I found so far (see the code at the link 
> above) and I am reasonably certain they are actual failures. The rest I plan 
> to note in comments here, one test per comment.
> If you happen to have any comments and suggestions how to do this better, I'd 
> be glad to hear them.

This message was sent by Atlassian JIRA

Reply via email to