Thanks Andy. So the overall response times were only a bit faster as I mentioned (about 6%), but overall write activities were significantly faster as was to be expected given the changes made there, in some cases 40%
So yes, I definitely understand that there are aspects which cannot possibly ever be captured by junits. Many of the bugs we find come from system tests where servers gets killed, run into OOMEs, get beaten with a hammer and things like that. The other thing we do at some point during our verification testing is run the server for a longer time with lots of activity to monitor long-term stability. I have no results on that yet (not even for 2.7.1) as they require product stability itself (keep in mind that Jena is one small wheel here - even if works fine, lots of other things often don't) I definitely agree that the concept of RC cycles is good in principle, but I don't know if lightweight will do for us. As I explained before, we can only get comprehensive testing done with versions of Jena which I can safely deliver in our main source control stream. Because we publish milestone builds publically (on jazz.net if you're interested), each milestone needs legal approval for 3rd party libraries. I was told that for Apache libraries we can only do this for releases because of certain IBM assumptions about Apache releases (don't ask me what, IBM just has a certain relation with Apache which makes this the way it is). And this process fortunately does not take very long. In other words, if you introduce an RC, it would only help us if you perform the same tasks as you do with a regular release, make it available on the website like you do with a regular release, but just use a different name to indicate that it may not be mature. It would be a bit of a naming game, but it would take away the effect that you come out with a possibly buggy release, yet you give your consumers a chance to fully adopt and test the RC thoughts? Simon From: Andy Seaborne <[email protected]> To: [email protected] Date: 09/13/2012 07:27 AM Subject: Re: 2.7.4 release? On 12/09/12 20:49, Simon Helsen wrote: > On the testing front, we just got one of our clients to run a > scalability test (meaning they run with a large starting repository and > fire up multiple users to perform read/write activities) and they have not > observed any regressions compared to 2.7.1 and a slight overall speed > improvement (something like 6%, not stellar, but most importantly not > slower) Good to know. The crash-restart is the thing that's hardest to test for in the development test suite so feedback on that would be particularly useful. That's why a (lightweight) RC cycle struck me as important or else we'll end up in some kind of odd/event minor release cycle (odds are effectively RC's, evens are stable releases). Other people have been picking up the dev release as well. Andy
