Re: PAX-EXAM - issue with Abstract classes in PerSuite reactor

2018-03-09 Thread Matej Ferencak 
Oh, OK,

we did know about the Wiki page, but didn't take it seriously, because 
otherwise it worked pretty well for us until we ran into this issue.

Thanks anyway, I guess we'll try to go on using a workaround and see how 
far it gets us.

Regards,
Matej.

On Friday, March 9, 2018 at 1:32:01 PM UTC+1, laeubi wrote:
>
> This strategy (PerSuite) is only supported in Java EE, CDI and Web modes. 
>
> Also please read the warning about "PerSuite" here: 
>
>
> https://ops4j1.jira.com/wiki/spaces/PAXEXAM4/pages/54263848/Reactor+Strategies
>  
>
> so it is very likely that it would produce inconsistent behaviour in 
> OSGi modes. 
>
> Am 09.03.2018 um 13:26 schrieb Matej Ferencak: 
> > Hi all, 
> > 
> > In our integration tests it’s common  to have an abstract class with 
> > @Test methods where common / boilerplate test code is executed, 
> > 
> > with subclasses which implement abstract methods for actual test 
> > specialization. 
> > 
> > We are also grouping tests in a suite, to minimize container startup 
> > overhead (we use Karaf). 
> > 
> > This means that practically one execution consists of a suite with one 
> > container running several probes, spread over several concrete class 
> > implementations of an abstract class with contains test methods. 
> > 
> > The problem we found, however, is that in such test suite, only one 
> > concrete subclass is ever executed (even though logs suggest otherwise), 
> > and all other subclasses are ignored. 
> > 
> > So the tests are executed N times (N = number of concrete classes), but 
> > each time only one (typically the last class in a suite) is actually 
> > probed (N times overall). 
> > 
> > The obvious workaround would be to use PerClass instead of PerSuite – 
> > but that would significantly increase our test execution time. 
> > 
> > Also, analysis of the bug showed that even this code path is not 100% 
> > „clean“. 
> > 
> > More detailed explanation including the test case can be found at: 
> > 
> > https://github.com/Traxpay/issues-paxexam-abstract-suite 
> > 
> > The project also includes a workaround which we are testing right now 
> > (see TraxpayProbeRunner.fillChildren() method). 
> > 
> > We can’t suggest an actual solution right now. 
> > 
> > Let me know if you need more info or if there is a better way to report 
> > this (Jira?). 
> > 
> > Thanks in advance, 
> > 
> > Matej. 
> > 
> > -- 
> > -- 
> > -- 
> > OPS4J - http://www.ops4j.org - op...@googlegroups.com  
> > 
> > --- 
> > You received this message because you are subscribed to the Google 
> > Groups "OPS4J" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> > an email to ops4j+un...@googlegroups.com  
> > . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
-- 
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

--- 
You received this message because you are subscribed to the Google Groups 
"OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ops4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: PAX-EXAM - issue with Abstract classes in PerSuite reactor

2018-03-09 Thread 'Christoph Läubrich' via OPS4J 

This strategy (PerSuite) is only supported in Java EE, CDI and Web modes.

Also please read the warning about "PerSuite" here:

https://ops4j1.jira.com/wiki/spaces/PAXEXAM4/pages/54263848/Reactor+Strategies

so it is very likely that it would produce inconsistent behaviour in 
OSGi modes.


Am 09.03.2018 um 13:26 schrieb Matej Ferencak:

Hi all,

In our integration tests it’s common  to have an abstract class with 
@Test methods where common / boilerplate test code is executed,


with subclasses which implement abstract methods for actual test 
specialization.


We are also grouping tests in a suite, to minimize container startup 
overhead (we use Karaf).


This means that practically one execution consists of a suite with one 
container running several probes, spread over several concrete class 
implementations of an abstract class with contains test methods.


The problem we found, however, is that in such test suite, only one 
concrete subclass is ever executed (even though logs suggest otherwise), 
and all other subclasses are ignored.


So the tests are executed N times (N = number of concrete classes), but 
each time only one (typically the last class in a suite) is actually 
probed (N times overall).


The obvious workaround would be to use PerClass instead of PerSuite – 
but that would significantly increase our test execution time.


Also, analysis of the bug showed that even this code path is not 100% 
„clean“.


More detailed explanation including the test case can be found at:

https://github.com/Traxpay/issues-paxexam-abstract-suite

The project also includes a workaround which we are testing right now 
(see TraxpayProbeRunner.fillChildren() method).


We can’t suggest an actual solution right now.

Let me know if you need more info or if there is a better way to report 
this (Jira?).


Thanks in advance,

Matej.

--
--
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

---
You received this message because you are subscribed to the Google 
Groups "OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to ops4j+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
--
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

--- 
You received this message because you are subscribed to the Google Groups "OPS4J" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to ops4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

PAX-EXAM - issue with Abstract classes in PerSuite reactor

2018-03-09 Thread Matej Ferencak 


Hi all,

 

In our integration tests it’s common  to have an abstract class with @Test 
methods where common / boilerplate test code is executed,

with subclasses which implement abstract methods for actual test 
specialization.

 

We are also grouping tests in a suite, to minimize container startup 
overhead (we use Karaf).

This means that practically one execution consists of a suite with one 
container running several probes, spread over several concrete class 
implementations of an abstract class with contains test methods.

 

The problem we found, however, is that in such test suite, only one 
concrete subclass is ever executed (even though logs suggest otherwise), 
and all other subclasses are ignored.

So the tests are executed N times (N = number of concrete classes), but 
each time only one (typically the last class in a suite) is actually probed 
(N times overall).

 

The obvious workaround would be to use PerClass instead of PerSuite – but 
that would significantly increase our test execution time. 

Also, analysis of the bug showed that even this code path is not 100% 
„clean“.

 

More detailed explanation including the test case can be found at:

https://github.com/Traxpay/issues-paxexam-abstract-suite

 

The project also includes a workaround which we are testing right now (see 
TraxpayProbeRunner.fillChildren() method).

We can’t suggest an actual solution right now.

 

Let me know if you need more info or if there is a better way to report 
this (Jira?).

 

Thanks in advance,

Matej.

 

-- 
-- 
--
OPS4J - http://www.ops4j.org - ops4j@googlegroups.com

--- 
You received this message because you are subscribed to the Google Groups 
"OPS4J" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ops4j+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.