Hi Sebastian, OK - I'll put it on the site tomorrow.
Best regards, Irina. On Tue, Mar 26, 2013 at 11:26 AM, [email protected] < [email protected]> wrote: > @Irina: I think I would put this document on the public community website. > The community website is bundled with each build in the directory > webapps/openmeetings/docs > That means that each version has its own dedivated version of the testing > plan. > Otherwise it is hard to tell which of those test methods belong to which > version of OpenMeetings. > > It is also possible to see different versions of the community website > through jenkins: > For example for the Version 2.0: > > https://builds.apache.org/view/M-R/view/OpenMeetings/job/OpenMeetings%202.0/ws/2.0/dist/red5/webapps/openmeetings/docs/index.html > For the version Version 2.1: > > https://builds.apache.org/view/M-R/view/OpenMeetings/job/Openmeetings%202.1/ws/2.1/dist/red5/webapps/openmeetings/docs/index.html > For trunk: > > https://builds.apache.org/job/openmeetings/ws/singlewebapp/dist/red5/webapps/openmeetings/docs/index.html > > I would suggest that we start with this document, and then try to automize > those tests step by step. > So hopefully one day we only run our test suite and no more manual testing > is needed. > > Sebastian > > > > > 2013/3/25 Irina Arkhipets <[email protected]> > > > Here is a link to the testplan for 2.1 release: > > > > > http://opensourcewebconferencing.blogspot.ru/2013/03/testplan-for-release-21-testing.html > > > > On Sun, Mar 24, 2013 at 2:05 PM, Irina Arkhipets > > <[email protected]>wrote: > > > > > Hi Sebastian, > > > > > > Some time ago I've created a test plan for Vasiliy where I tried to > cover > > > all the cases and which was revieved by Alexei and Maxim. > > > > > > It's currently on Russian, and probably is incomplete in some aspects. > > > I'll try to do translate it on English and share with others ASAP > (hope, > > > today later or tomorrow). Vasiliy will be responsible for the reports > > about > > > the tests execution. > > > . > > > Any yes, our fault is that we did not share it with the community from > > the > > > very beginning :(. > > > > > > You are right about the automated junit tests. I'll try to help Maxim > > with > > > this :) > > > > > > Best regards, > > > Irina. > > > > > > > > > > > > On Sun, Mar 24, 2013 at 11:15 AM, [email protected] < > > > [email protected]> wrote: > > > > > >> I would be already happy if we do the following: > > >> > > >> 1) Enable the Junit test to run automated (by using the Backup Import > > via > > >> JUnit as example) > > >> So that every committer can add new JUnit tests that run with every > > >> Nightly > > >> Build. > > >> > > >> 2) Start a list of test/use cases that should be performed with any > > >> release. > > >> Maybe there is already such a list ? What did Alexey, Artyom, Irina, > > >> Vasya, > > >> Yuliya test at all ? > > >> How did they manage the work of "testing", did they agree on any tests > > >> that > > >> need to be performed ? > > >> > > >> Sebastian > > >> > > >> > > >> > > >> 2013/3/24 Maxim Solodovnik <[email protected]> > > >> > > >> > I guess we need to improve+enlarge our automated tests and rely on > it > > in > > >> > the future. > > >> > Right now it is necessary to run manual tests :( > > >> > I'll try to write 1-2 tests per day/week (too much work right now > :(( > > ) > > >> > > > >> > > > >> > On Sun, Mar 24, 2013 at 11:52 AM, [email protected] < > > >> > [email protected]> wrote: > > >> > > > >> > > Wicket will help to do tests. However our client is 100% Flash > now. > > >> > > > > >> > > Do we want to run UI tests in the Flash UI or do we want to only > run > > >> > JUnit > > >> > > tests automated ? > > >> > > > > >> > > How can we define which JUnit tests run automated ? > > >> > > > > >> > > From my perspective the more we can test automated the less time > you > > >> > spend > > >> > > on even more painful tasks. > > >> > > Cause every test that is _not_ automated means that: > > >> > > - It is likely that nobody will do testing > > >> > > - A extremly painful process will start where we maintain a wiki > > >> > document > > >> > > that lists all tests (with all problems including, like nobody > takes > > >> care > > >> > > of those documents, nobody can really control if those tests have > > been > > >> > > performed at all or not, et cetera) > > >> > > > > >> > > So from my perspective putting some time into an automated test is > > >> still > > >> > > much less pain then trying now to re-run all those manual tests, > > mail > > >> > ping > > >> > > pong and discussion with every release that we do. > > >> > > > > >> > > Sebastian > > >> > > > > >> > > > > >> > > > > >> > > 2013/3/24 Maxim Solodovnik <[email protected]> > > >> > > > > >> > > > It is hard for me to answer such long letters :))) > > >> > > > > > >> > > > >> Yeah, well how should any user do a test if there is no > public > > >> demo > > >> > > > I'm not sure what is the status of demo.dataved.ru, it allows > > "self > > >> > > > registration" and it is up 24/7, but you are right, there were > no > > >> "Call > > >> > > to > > >> > > > test". But I was sure My emails like "people I'm going to > release, > > >> > please > > >> > > > stop me if it is too early" is sort of call to test it and let > me > > >> > > know.... > > >> > > > > > >> > > > I agree on "automated testing", I promise I'll add out tests to > > the > > >> > build > > >> > > > (I forgot about it, will create JIRA issue). > > >> > > > > > >> > > > since we will be on Wicket we can finally start writing tests on > > >> our UI > > >> > > > similar to their tests (never tried that) > > >> > > > I do like automated tests, it is just not my favorite task :) > > >> > > > > > >> > > > according to our (and Apache guide) > > >> > > > http://openmeetings.apache.org/ReleaseGuide.html > > >> > > > " > > >> > > > > > >> > > > *Before voting +1 PMC members are required to download the > signed > > >> > source > > >> > > > code package, compile it as provided, and test the resulting > > >> executable > > >> > > on > > >> > > > their own platform, along with also verifying that the package > > meets > > >> > the > > >> > > > requirements of the ASF policy on releases.* > > >> > > > > > >> > > > " > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > On Sun, Mar 24, 2013 at 11:17 AM, [email protected] < > > >> > > > [email protected]> wrote: > > >> > > > > > >> > > > > We did extensive testing of 2.1 (Alexey, Artyom, Irina, Vasya, > > >> > Yuliya) > > >> > > > > => where did they perform the tests? I thought we would invite > > the > > >> > > > > community to help us testing. > > >> > > > > > > >> > > > > *1) there were no issues reported by users* > > >> > > > > Yeah, well how should any user do a test if there is no public > > >> demo? > > >> > I > > >> > > > also > > >> > > > > did not hear any call on the user mailing list that users are > > >> invited > > >> > > to > > >> > > > > test. > > >> > > > > *2) We better release 2.1.1 or 2.2 in a month than wait > another > > 6 > > >> > > months* > > >> > > > > I agree on that. But our past agreement was more like "dev > > >> complete > > >> > => > > >> > > > > release". That model will not work for our future. > > >> > > > > And I want to make sure that everybody involved understands > > that. > > >> > > > > > > >> > > > > IMHO our lack of automated testing and the need for a manual > > test > > >> / > > >> > > click > > >> > > > > through of all the features is one of the biggest issues in > our > > >> > current > > >> > > > > project. > > >> > > > > For example I do not understand why the JUnit test for the > > backup > > >> > > import > > >> > > > > was never integrated into the Nightly builds? I mean all that > > work > > >> > that > > >> > > > > you've put into that. Simply nobody uses it now. > > >> > > > > It would be such a nice thing to wake up every morning and see > > >> what > > >> > > test > > >> > > > > fails and what to look at? I guess there are only a couple of > > bits > > >> > > > missing > > >> > > > > to get the backup import running automated but I don't > > understand > > >> > what > > >> > > > > keeps us away from doing that? > > >> > > > > > > >> > > > > Similar for the rest of the Junit tests. Of couse a good > amount > > of > > >> > the > > >> > > > > tests are just outdated. > > >> > > > > But if there would be at least a minimal subset of tests that > > run > > >> > > > > automated, that would be an improment by 100%, cause at the > > >> moment, > > >> > > just > > >> > > > > zero tests run automated. > > >> > > > > This will become even more interesting with Wicket, where you > > can > > >> > test > > >> > > a > > >> > > > > lot of the UI stuff with simple JUnit tests. > > >> > > > > The manual work that Alexey, Artyom, Irina, Vasya, Yuliya and > > >> anybody > > >> > > > else > > >> > > > > involved has done for 2.1 > > >> > > > > => Will need to happen with every release. 2.1.1, 2.2, ... > > >> > > > > An approach like "A feature that has been tested in the > release > > >> 2.1 > > >> > > needs > > >> > > > > no more testing in a release 2.1.1 (or 2.2)". I will not agree > > on > > >> > that > > >> > > in > > >> > > > > any sense. Every release does need a full test. > > >> > > > > And IMHO this approach will not scale at all with the growing > > >> number > > >> > of > > >> > > > > committers. > > >> > > > > > > >> > > > > It would be great if we start thinking about what we will do > to > > >> > improve > > >> > > > > that in the future? > > >> > > > > > > >> > > > > The tools are basically there but it seems like nobody > involved > > in > > >> > the > > >> > > > > project believes that automated tests make sense (except me) ? > > >> > > > > > > >> > > > > From @Alexey I know that he believes only additions to the > > feature > > >> > add > > >> > > > > value to the end product. And it seems like "testing" is not a > > >> > > "feature" > > >> > > > > that adds any value to the end user from that perspective. > > >> > > > > So my questions would be: Do we really want to do the same > > amount > > >> of > > >> > > > manual > > >> > > > > click-through tests that we do now with every release ?! > > >> > > > > I mean: Am I the only person sick of downloading every release > > and > > >> > > > clicking > > >> > > > > through every feature 30 minutes to give a "+1" ?! > > >> > > > > > > >> > > > > Sebastian > > >> > > > > > > >> > > > > > > >> > > > > 2013/3/24 Maxim Solodovnik <[email protected]> > > >> > > > > > > >> > > > > > We did extensive testing of 2.1 (Alexey, Artyom, Irina, > Vasya, > > >> > > Yuliya) > > >> > > > > > additional causes are: > > >> > > > > > 1) there were no issues reported by users > > >> > > > > > 2) We better release 2.1.1 or 2.2 in a month than wait > > another 6 > > >> > > months > > >> > > > > > > > >> > > > > > ps Apach Wicket has 1 month release cycle .... I believe we > > >> should > > >> > > have > > >> > > > > 2-3 > > >> > > > > > month > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > On Sun, Mar 24, 2013 at 10:20 AM, [email protected] < > > >> > > > > > [email protected]> wrote: > > >> > > > > > > > >> > > > > > > Hi Maxim, > > >> > > > > > > > > >> > > > > > > I was wondering if the testing phase that I thought we > have > > >> > agreed > > >> > > on > > >> > > > > > > already happen? > > >> > > > > > > Or is there another reason why you initiated this RC? > > >> > > > > > > > > >> > > > > > > Sebastian > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > 2013/3/23 Maxim Solodovnik <[email protected]> > > >> > > > > > > > > >> > > > > > > > Dear OpenMeetings Community, > > >> > > > > > > > > > >> > > > > > > > I would like to start a vote about releasing Apache > > >> > OpenMeetings > > >> > > > > 2.1.0 > > >> > > > > > > RC3 > > >> > > > > > > > > > >> > > > > > > > RC2 was rejected due to broken audio/video setup panel > > >> > > > > > > > > > >> > > > > > > > Main changes are covered in the Readme: > > >> > > > > > > > > > >> > > > > > > > > > >> > http://svn.apache.org/repos/asf/openmeetings/tags/2.1RC3/README > > >> > > > > > > > > > >> > > > > > > > Full Changelog: > > >> > > > > > > > > > >> > > > http://svn.apache.org/repos/asf/openmeetings/tags/2.1RC3/CHANGELOG > > >> > > > > > > > > > >> > > > > > > > Release artefacts: > > >> > > > > > > > > > >> https://dist.apache.org/repos/dist/dev/openmeetings/2.1/rc3/ > > >> > > > > > > > > > >> > > > > > > > Tag: > > >> http://svn.apache.org/repos/asf/openmeetings/tags/2.1RC3/ > > >> > > > > > > > > > >> > > > > > > > PGP release keys (signed using C467526E): > > >> > > > > > > > > > >> > https://dist.apache.org/repos/dist/dev/openmeetings/2.1/rc3/KEYS > > >> > > > > > > > > > >> > > > > > > > Vote will be open for 72 hours. > > >> > > > > > > > > > >> > > > > > > > [ ] +1 approve > > >> > > > > > > > [ ] +0 no opinion > > >> > > > > > > > [ ] -1 disapprove (and reason why) > > >> > > > > > > > > > >> > > > > > > > My vote is +1. > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > -- > > >> > > > > > > > WBR > > >> > > > > > > > Maxim aka solomax > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > -- > > >> > > > > > > Sebastian Wagner > > >> > > > > > > https://twitter.com/#!/dead_lock > > >> > > > > > > http://www.webbase-design.de > > >> > > > > > > http://www.wagner-sebastian.com > > >> > > > > > > [email protected] > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > -- > > >> > > > > > WBR > > >> > > > > > Maxim aka solomax > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > -- > > >> > > > > Sebastian Wagner > > >> > > > > https://twitter.com/#!/dead_lock > > >> > > > > http://www.webbase-design.de > > >> > > > > http://www.wagner-sebastian.com > > >> > > > > [email protected] > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > -- > > >> > > > WBR > > >> > > > Maxim aka solomax > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > -- > > >> > > Sebastian Wagner > > >> > > https://twitter.com/#!/dead_lock > > >> > > http://www.webbase-design.de > > >> > > http://www.wagner-sebastian.com > > >> > > [email protected] > > >> > > > > >> > > > >> > > > >> > > > >> > -- > > >> > WBR > > >> > Maxim aka solomax > > >> > > > >> > > >> > > >> > > >> -- > > >> Sebastian Wagner > > >> https://twitter.com/#!/dead_lock > > >> http://www.webbase-design.de > > >> http://www.wagner-sebastian.com > > >> [email protected] > > >> > > > > > > > > > > > > -- > Sebastian Wagner > https://twitter.com/#!/dead_lock > http://www.webbase-design.de > http://www.wagner-sebastian.com > [email protected] >
