Commented on the required QA time .
Regards,
Yumani
Tyrell Perera wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Yes we do. A postmortem follows FYI, team please feel free to add in;
Issues
======
1. The 2.2.1 versions of WSRequest and Dynamic-Codegen were accidentally
deleted from the maven repos by the WSAS team. The deletion of
Dynamic-Codegen 2.2.1 proved fatal.
Unfortunately we were under the impression that WSAS uses the latest
revisions of these from the trunk and versions them to match their
release version. This ended up in the 'reversion' of 2.2.1 versions from
our build to 2.2 (after discussions with Saminda). But WSAS apparently
uses their own branches of these projects, created from older svn revisions.
2. While integrating the security fixes done for wsas 2.2.1 at a
development level, we had to change a few of our own back end services
and the associated front end javascript. Although the javascript changes
were tested in firefox they were not verified in IE (NOT really a fault
of QA as explained below).
Explanation
===========
Due to the acute sense of urgency to release 1.0.1 (with fixes to
previous issues) before the webinar (10:30 PM LKT), and due to the time
we received WSAS 2.2.1 in the maven repos, QA never had a chance to do
the full length of smoke tests (which by the way requires at least 4
hours. We received WSAS 2.2.1 around 10:00 PM LKT at which time we
started our own final builds).
Had we given time for these smoke tests to be carried out in an orderly
fashion, both these issues would have been discovered and appropriate
measures could have been taken to fix them.
Preventative measures for future releases
=========================================
Give ample time for QA to carry out at least the smoke tests. I would
rather they are given the time to carry out the entire test suite
including the load tests. This usually adds around 5 hours to ETA
(Yumani please step in if this value is incorrect).
The smoke test suite I am having it a bit comprehensive (contains 40
test scenarios and 3 installation tests). It covers all happy paths
through admin console, annotations, host objects and installation tests.
It takes around 5-6 hours when smoke test cases for the admin console is
run on both IE and FF. For a release I am more confident if I can test
in both browsers separately. Because there are always browser specific
problems.
The load test cases I have for the current requirement can be completed
within 1 hour or less.
So I would say altogether around *1 day* would be great if you guys can
allow that time for me.
NOTE: As for 1.0.2 I can stick for the promised time lines. Tests for
this were already started yesterday.
This is why we have a code freeze in effect at the moment. Yumani will
run the required tests on the 1.0.2 artifacts till 12 noon tomorrow LKT.
Depending on the status at that time, we will either sign and release
the existing artifacts or rebuild.
Regards,
Tyrell
Sanjiva Weerawarana wrote:
Do we know what went wrong with the 1.0.1 release? We can't make the
same mistakes again.
Sanjiva.
Tyrell Perera wrote:
I believe we have everything in place for the 1.0.2 at the moment. The
issues produced by 1.0.1 are now fixed and I just committed the
necessary pom updates for 1.0.2
After a preliminary sanity check on the latest build, shall we please
enforce a code freeze on a named svn revision?
Tyrell
_______________________________________________
Mashup-dev mailing list
[email protected]
http://www.wso2.org/cgi-bin/mailman/listinfo/mashup-dev
- --
Tyrell Perera
Senior Software Engineer; WSO2, Inc.; http://www.wso2.com/
email: [EMAIL PROTECTED]; cell: +94 77 302 2505
"Oxygenating the Web Service Platform."
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHtG1XehFdPcgGx7oRAnWvAJ9/jcEs0bzBARaBQ6BpwwL/krMeaACgs29N
j6VmopnMzqOlF5r8HEaaclg=
=bQ1T
-----END PGP SIGNATURE-----
_______________________________________________
Mashup-dev mailing list
[email protected]
http://www.wso2.org/cgi-bin/mailman/listinfo/mashup-dev
_______________________________________________
Mashup-dev mailing list
[email protected]
http://www.wso2.org/cgi-bin/mailman/listinfo/mashup-dev
