Hi Sebastian, Done for 2.1 branch and trunk:
http://openmeetings.apache.org/ManualTesting.html Best regards, Irina. On Tue, Mar 26, 2013 at 11:09 PM, Irina Arkhipets <[email protected] > wrote: > Hi Sebastian, > > OK - I'll put it on the site tomorrow. > > Best regards, > Irina. > > > On Tue, Mar 26, 2013 at 11:26 AM, [email protected] < > [email protected]> wrote: > >> @Irina: I think I would put this document on the public community website. >> The community website is bundled with each build in the directory >> webapps/openmeetings/docs >> That means that each version has its own dedivated version of the testing >> plan. >> Otherwise it is hard to tell which of those test methods belong to which >> version of OpenMeetings. >> >> It is also possible to see different versions of the community website >> through jenkins: >> For example for the Version 2.0: >> >> https://builds.apache.org/view/M-R/view/OpenMeetings/job/OpenMeetings%202.0/ws/2.0/dist/red5/webapps/openmeetings/docs/index.html >> For the version Version 2.1: >> >> https://builds.apache.org/view/M-R/view/OpenMeetings/job/Openmeetings%202.1/ws/2.1/dist/red5/webapps/openmeetings/docs/index.html >> For trunk: >> >> https://builds.apache.org/job/openmeetings/ws/singlewebapp/dist/red5/webapps/openmeetings/docs/index.html >> >> I would suggest that we start with this document, and then try to automize >> those tests step by step. >> So hopefully one day we only run our test suite and no more manual testing >> is needed. >> >> Sebastian >> >> >> >> >> 2013/3/25 Irina Arkhipets <[email protected]> >> >> > Here is a link to the testplan for 2.1 release: >> > >> > >> http://opensourcewebconferencing.blogspot.ru/2013/03/testplan-for-release-21-testing.html >> > >> > On Sun, Mar 24, 2013 at 2:05 PM, Irina Arkhipets >> > <[email protected]>wrote: >> > >> > > Hi Sebastian, >> > > >> > > Some time ago I've created a test plan for Vasiliy where I tried to >> cover >> > > all the cases and which was revieved by Alexei and Maxim. >> > > >> > > It's currently on Russian, and probably is incomplete in some aspects. >> > > I'll try to do translate it on English and share with others ASAP >> (hope, >> > > today later or tomorrow). Vasiliy will be responsible for the reports >> > about >> > > the tests execution. >> > > . >> > > Any yes, our fault is that we did not share it with the community from >> > the >> > > very beginning :(. >> > > >> > > You are right about the automated junit tests. I'll try to help Maxim >> > with >> > > this :) >> > > >> > > Best regards, >> > > Irina. >> > > >> > > >> > > >> > > On Sun, Mar 24, 2013 at 11:15 AM, [email protected] < >> > > [email protected]> wrote: >> > > >> > >> I would be already happy if we do the following: >> > >> >> > >> 1) Enable the Junit test to run automated (by using the Backup Import >> > via >> > >> JUnit as example) >> > >> So that every committer can add new JUnit tests that run with every >> > >> Nightly >> > >> Build. >> > >> >> > >> 2) Start a list of test/use cases that should be performed with any >> > >> release. >> > >> Maybe there is already such a list ? What did Alexey, Artyom, Irina, >> > >> Vasya, >> > >> Yuliya test at all ? >> > >> How did they manage the work of "testing", did they agree on any >> tests >> > >> that >> > >> need to be performed ? >> > >> >> > >> Sebastian >> > >> >> > >> >> > >> >> > >> 2013/3/24 Maxim Solodovnik <[email protected]> >> > >> >> > >> > I guess we need to improve+enlarge our automated tests and rely on >> it >> > in >> > >> > the future. >> > >> > Right now it is necessary to run manual tests :( >> > >> > I'll try to write 1-2 tests per day/week (too much work right now >> :(( >> > ) >> > >> > >> > >> > >> > >> > On Sun, Mar 24, 2013 at 11:52 AM, [email protected] < >> > >> > [email protected]> wrote: >> > >> > >> > >> > > Wicket will help to do tests. However our client is 100% Flash >> now. >> > >> > > >> > >> > > Do we want to run UI tests in the Flash UI or do we want to only >> run >> > >> > JUnit >> > >> > > tests automated ? >> > >> > > >> > >> > > How can we define which JUnit tests run automated ? >> > >> > > >> > >> > > From my perspective the more we can test automated the less time >> you >> > >> > spend >> > >> > > on even more painful tasks. >> > >> > > Cause every test that is _not_ automated means that: >> > >> > > - It is likely that nobody will do testing >> > >> > > - A extremly painful process will start where we maintain a wiki >> > >> > document >> > >> > > that lists all tests (with all problems including, like nobody >> takes >> > >> care >> > >> > > of those documents, nobody can really control if those tests have >> > been >> > >> > > performed at all or not, et cetera) >> > >> > > >> > >> > > So from my perspective putting some time into an automated test >> is >> > >> still >> > >> > > much less pain then trying now to re-run all those manual tests, >> > mail >> > >> > ping >> > >> > > pong and discussion with every release that we do. >> > >> > > >> > >> > > Sebastian >> > >> > > >> > >> > > >> > >> > > >> > >> > > 2013/3/24 Maxim Solodovnik <[email protected]> >> > >> > > >> > >> > > > It is hard for me to answer such long letters :))) >> > >> > > > >> > >> > > > >> Yeah, well how should any user do a test if there is no >> public >> > >> demo >> > >> > > > I'm not sure what is the status of demo.dataved.ru, it allows >> > "self >> > >> > > > registration" and it is up 24/7, but you are right, there were >> no >> > >> "Call >> > >> > > to >> > >> > > > test". But I was sure My emails like "people I'm going to >> release, >> > >> > please >> > >> > > > stop me if it is too early" is sort of call to test it and let >> me >> > >> > > know.... >> > >> > > > >> > >> > > > I agree on "automated testing", I promise I'll add out tests to >> > the >> > >> > build >> > >> > > > (I forgot about it, will create JIRA issue). >> > >> > > > >> > >> > > > since we will be on Wicket we can finally start writing tests >> on >> > >> our UI >> > >> > > > similar to their tests (never tried that) >> > >> > > > I do like automated tests, it is just not my favorite task :) >> > >> > > > >> > >> > > > according to our (and Apache guide) >> > >> > > > http://openmeetings.apache.org/ReleaseGuide.html >> > >> > > > " >> > >> > > > >> > >> > > > *Before voting +1 PMC members are required to download the >> signed >> > >> > source >> > >> > > > code package, compile it as provided, and test the resulting >> > >> executable >> > >> > > on >> > >> > > > their own platform, along with also verifying that the package >> > meets >> > >> > the >> > >> > > > requirements of the ASF policy on releases.* >> > >> > > > >> > >> > > > " >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > On Sun, Mar 24, 2013 at 11:17 AM, [email protected] < >> > >> > > > [email protected]> wrote: >> > >> > > > >> > >> > > > > We did extensive testing of 2.1 (Alexey, Artyom, Irina, >> Vasya, >> > >> > Yuliya) >> > >> > > > > => where did they perform the tests? I thought we would >> invite >> > the >> > >> > > > > community to help us testing. >> > >> > > > > >> > >> > > > > *1) there were no issues reported by users* >> > >> > > > > Yeah, well how should any user do a test if there is no >> public >> > >> demo? >> > >> > I >> > >> > > > also >> > >> > > > > did not hear any call on the user mailing list that users are >> > >> invited >> > >> > > to >> > >> > > > > test. >> > >> > > > > *2) We better release 2.1.1 or 2.2 in a month than wait >> another >> > 6 >> > >> > > months* >> > >> > > > > I agree on that. But our past agreement was more like "dev >> > >> complete >> > >> > => >> > >> > > > > release". That model will not work for our future. >> > >> > > > > And I want to make sure that everybody involved understands >> > that. >> > >> > > > > >> > >> > > > > IMHO our lack of automated testing and the need for a manual >> > test >> > >> / >> > >> > > click >> > >> > > > > through of all the features is one of the biggest issues in >> our >> > >> > current >> > >> > > > > project. >> > >> > > > > For example I do not understand why the JUnit test for the >> > backup >> > >> > > import >> > >> > > > > was never integrated into the Nightly builds? I mean all that >> > work >> > >> > that >> > >> > > > > you've put into that. Simply nobody uses it now. >> > >> > > > > It would be such a nice thing to wake up every morning and >> see >> > >> what >> > >> > > test >> > >> > > > > fails and what to look at? I guess there are only a couple of >> > bits >> > >> > > > missing >> > >> > > > > to get the backup import running automated but I don't >> > understand >> > >> > what >> > >> > > > > keeps us away from doing that? >> > >> > > > > >> > >> > > > > Similar for the rest of the Junit tests. Of couse a good >> amount >> > of >> > >> > the >> > >> > > > > tests are just outdated. >> > >> > > > > But if there would be at least a minimal subset of tests that >> > run >> > >> > > > > automated, that would be an improment by 100%, cause at the >> > >> moment, >> > >> > > just >> > >> > > > > zero tests run automated. >> > >> > > > > This will become even more interesting with Wicket, where you >> > can >> > >> > test >> > >> > > a >> > >> > > > > lot of the UI stuff with simple JUnit tests. >> > >> > > > > The manual work that Alexey, Artyom, Irina, Vasya, Yuliya and >> > >> anybody >> > >> > > > else >> > >> > > > > involved has done for 2.1 >> > >> > > > > => Will need to happen with every release. 2.1.1, 2.2, ... >> > >> > > > > An approach like "A feature that has been tested in the >> release >> > >> 2.1 >> > >> > > needs >> > >> > > > > no more testing in a release 2.1.1 (or 2.2)". I will not >> agree >> > on >> > >> > that >> > >> > > in >> > >> > > > > any sense. Every release does need a full test. >> > >> > > > > And IMHO this approach will not scale at all with the growing >> > >> number >> > >> > of >> > >> > > > > committers. >> > >> > > > > >> > >> > > > > It would be great if we start thinking about what we will do >> to >> > >> > improve >> > >> > > > > that in the future? >> > >> > > > > >> > >> > > > > The tools are basically there but it seems like nobody >> involved >> > in >> > >> > the >> > >> > > > > project believes that automated tests make sense (except me) >> ? >> > >> > > > > >> > >> > > > > From @Alexey I know that he believes only additions to the >> > feature >> > >> > add >> > >> > > > > value to the end product. And it seems like "testing" is not >> a >> > >> > > "feature" >> > >> > > > > that adds any value to the end user from that perspective. >> > >> > > > > So my questions would be: Do we really want to do the same >> > amount >> > >> of >> > >> > > > manual >> > >> > > > > click-through tests that we do now with every release ?! >> > >> > > > > I mean: Am I the only person sick of downloading every >> release >> > and >> > >> > > > clicking >> > >> > > > > through every feature 30 minutes to give a "+1" ?! >> > >> > > > > >> > >> > > > > Sebastian >> > >> > > > > >> > >> > > > > >> > >> > > > > 2013/3/24 Maxim Solodovnik <[email protected]> >> > >> > > > > >> > >> > > > > > We did extensive testing of 2.1 (Alexey, Artyom, Irina, >> Vasya, >> > >> > > Yuliya) >> > >> > > > > > additional causes are: >> > >> > > > > > 1) there were no issues reported by users >> > >> > > > > > 2) We better release 2.1.1 or 2.2 in a month than wait >> > another 6 >> > >> > > months >> > >> > > > > > >> > >> > > > > > ps Apach Wicket has 1 month release cycle .... I believe we >> > >> should >> > >> > > have >> > >> > > > > 2-3 >> > >> > > > > > month >> > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > > > > On Sun, Mar 24, 2013 at 10:20 AM, [email protected] < >> > >> > > > > > [email protected]> wrote: >> > >> > > > > > >> > >> > > > > > > Hi Maxim, >> > >> > > > > > > >> > >> > > > > > > I was wondering if the testing phase that I thought we >> have >> > >> > agreed >> > >> > > on >> > >> > > > > > > already happen? >> > >> > > > > > > Or is there another reason why you initiated this RC? >> > >> > > > > > > >> > >> > > > > > > Sebastian >> > >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > 2013/3/23 Maxim Solodovnik <[email protected]> >> > >> > > > > > > >> > >> > > > > > > > Dear OpenMeetings Community, >> > >> > > > > > > > >> > >> > > > > > > > I would like to start a vote about releasing Apache >> > >> > OpenMeetings >> > >> > > > > 2.1.0 >> > >> > > > > > > RC3 >> > >> > > > > > > > >> > >> > > > > > > > RC2 was rejected due to broken audio/video setup panel >> > >> > > > > > > > >> > >> > > > > > > > Main changes are covered in the Readme: >> > >> > > > > > > > >> > >> > > > > > > > >> > >> > http://svn.apache.org/repos/asf/openmeetings/tags/2.1RC3/README >> > >> > > > > > > > >> > >> > > > > > > > Full Changelog: >> > >> > > > > > > > >> > >> > > >> http://svn.apache.org/repos/asf/openmeetings/tags/2.1RC3/CHANGELOG >> > >> > > > > > > > >> > >> > > > > > > > Release artefacts: >> > >> > > > > > > > >> > >> https://dist.apache.org/repos/dist/dev/openmeetings/2.1/rc3/ >> > >> > > > > > > > >> > >> > > > > > > > Tag: >> > >> http://svn.apache.org/repos/asf/openmeetings/tags/2.1RC3/ >> > >> > > > > > > > >> > >> > > > > > > > PGP release keys (signed using C467526E): >> > >> > > > > > > > >> > >> > https://dist.apache.org/repos/dist/dev/openmeetings/2.1/rc3/KEYS >> > >> > > > > > > > >> > >> > > > > > > > Vote will be open for 72 hours. >> > >> > > > > > > > >> > >> > > > > > > > [ ] +1 approve >> > >> > > > > > > > [ ] +0 no opinion >> > >> > > > > > > > [ ] -1 disapprove (and reason why) >> > >> > > > > > > > >> > >> > > > > > > > My vote is +1. >> > >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > -- >> > >> > > > > > > > WBR >> > >> > > > > > > > Maxim aka solomax >> > >> > > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > -- >> > >> > > > > > > Sebastian Wagner >> > >> > > > > > > https://twitter.com/#!/dead_lock >> > >> > > > > > > http://www.webbase-design.de >> > >> > > > > > > http://www.wagner-sebastian.com >> > >> > > > > > > [email protected] >> > >> > > > > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > > > > -- >> > >> > > > > > WBR >> > >> > > > > > Maxim aka solomax >> > >> > > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > -- >> > >> > > > > Sebastian Wagner >> > >> > > > > https://twitter.com/#!/dead_lock >> > >> > > > > http://www.webbase-design.de >> > >> > > > > http://www.wagner-sebastian.com >> > >> > > > > [email protected] >> > >> > > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > -- >> > >> > > > WBR >> > >> > > > Maxim aka solomax >> > >> > > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > -- >> > >> > > Sebastian Wagner >> > >> > > https://twitter.com/#!/dead_lock >> > >> > > http://www.webbase-design.de >> > >> > > http://www.wagner-sebastian.com >> > >> > > [email protected] >> > >> > > >> > >> > >> > >> > >> > >> > >> > >> > -- >> > >> > WBR >> > >> > Maxim aka solomax >> > >> > >> > >> >> > >> >> > >> >> > >> -- >> > >> Sebastian Wagner >> > >> https://twitter.com/#!/dead_lock >> > >> http://www.webbase-design.de >> > >> http://www.wagner-sebastian.com >> > >> [email protected] >> > >> >> > > >> > > >> > >> >> >> >> -- >> Sebastian Wagner >> https://twitter.com/#!/dead_lock >> http://www.webbase-design.de >> http://www.wagner-sebastian.com >> [email protected] >> > >
