Re: Enforcing minimal test coverage

2017-05-12 Thread Matt Ryan
On Thu, May 11, 2017 at 6:03 AM, Angela Schreiber wrote: > hi julian > > no i didn't, because i don't think it makes sense to enforce a running > mongoDB for being able to build oak-core. nor does it make sense to me to > require a coverage that can only be reached with the derby-rdb profile > en

Re: Enforcing minimal test coverage

2017-05-12 Thread Matt Ryan
Hi, Speaking generally I think this is a good idea for Oak, so I am appreciative of the proposal. I assume we are talking about enforcement via build automation, meaning there would be a step in the build that would compute the coverage number and fail the build if the coverage number is not met.

Re: Enforcing minimal test coverage

2017-05-11 Thread Julian Reschke
On 2017-05-11 14:03, Angela Schreiber wrote: hi julian no i didn't, because i don't think it makes sense to enforce a running mongoDB for being able to build oak-core. nor does it make sense to me to require a coverage that can only be reached with the derby-rdb profile enabled. Stepping back:

Re: Enforcing minimal test coverage

2017-05-11 Thread Bertrand Delacretaz
Hi Angela, On Thu, May 11, 2017 at 4:41 PM, Angela Schreiber wrote: > ...As far as I am concerned I only run the ITs if I know (or > suspect) a change to have a huge impact otherwise it just takes too > long... Ok! I just meant to expose our solution in case it's useful otherwise. Our setup

Re: Enforcing minimal test coverage

2017-05-11 Thread Angela Schreiber
Hi Bertrand Thanks for the pointer. As I explained in my reply to Michael it's not about the absolute numbers but rather about getting early feedback during development. As far as I am concerned I only run the ITs if I know (or suspect) a change to have a huge impact otherwise it just takes to

Re: Enforcing minimal test coverage

2017-05-11 Thread Bertrand Delacretaz
On Thu, May 11, 2017 at 2:46 PM, Michael Dürig wrote: > ...I guess this only includes the coverage from running unit tests and does > not > include integration tests... FWIW, In Sling we have a setup that computes the aggregate coverage of unit and integration tests that run in a single Maven mo

Re: Enforcing minimal test coverage

2017-05-11 Thread Angela Schreiber
hi michael exactly... it's just the default build without any extra profiles or requirements wrt running DBs or third party tools. as far as i am concerned this is neither about the absolute numbers nor about reaching some imaginary test coverage goal. it's about providing us with immediate feedb

Re: Enforcing minimal test coverage

2017-05-11 Thread Michael Dürig
On 11.05.17 12:05, Angela Schreiber wrote: - oak-segment-tar: 0.65 I guess this only includes the coverage from running unit tests and does not include integration tests. For segment tar the figures are actually much better with the latter, which is (another) indicator that segment tar nee

Re: Enforcing minimal test coverage

2017-05-11 Thread Angela Schreiber
hi julian no i didn't, because i don't think it makes sense to enforce a running mongoDB for being able to build oak-core. nor does it make sense to me to require a coverage that can only be reached with the derby-rdb profile enabled. but now that you mention: this is for me another hint that the

Re: Enforcing minimal test coverage

2017-05-11 Thread Julian Reschke
On 2017-05-11 12:05, Angela Schreiber wrote: hi oak-devs as a follow up to http://markmail.org/message/rzbqwwx3lilshct6 i would like to suggest that we enforce minimal test coverage not only with security modules but also for other oak code that is used in production. i gave it a try and identi

Enforcing minimal test coverage

2017-05-11 Thread Angela Schreiber
hi oak-devs as a follow up to http://markmail.org/message/rzbqwwx3lilshct6 i would like to suggest that we enforce minimal test coverage not only with security modules but also for other oak code that is used in production. i gave it a try and identified those modules that already have a somewhat