On 08/16/2013 07:41 AM, Justin Rohrman wrote:
This sounds like a great idea to me. The timing would certainly be tight
but I think with the VMs and Vagrant usage we should be good. I'll ping
Michael and see what he thinks.
It is a great idea and it is technically ambitious. Participants must
understand that they need to do some homework before the weekend. It is
all well documented (and I will go through the process & docs again
before the end of this month) but it takes time and good bandwidth to
get your environment downloaded and installed.
Date: Thu, 15 Aug 2013 14:12:08 -0700
From: Chris McMahon <[email protected]
<mailto:[email protected]>>
To: "QA (software quality assurance) for Wikimedia projects."
<[email protected] <mailto:[email protected]>>
Subject: Re: [QA] Weekend testing americas on September 7th
Message-ID:
<cajohbhturqx7g5ehk1q2ant+gtfzp4gpvqj3xjzh2_x2fsk...@mail.gmail.com
<mailto:cajohbhturqx7g5ehk1q2ant%[email protected]>>
Content-Type: text/plain; charset="iso-8859-1"
On Thu, Aug 15, 2013 at 6:43 AM, Justin Rohrman <[email protected]
<mailto:[email protected]>> wrote:
> Let's hear it :)
>
Here goes...
There is a certain tension in the testing community between
advocates for
UI/browser test automation and advocates for human "sapient"
testing. I'd
like to go there.
Specifically, I'd like to do this with Weekend Testing:
* Pick a complex feature for which browser test automation exists.
Right
now I'm thinking CirrusSearch or VisualEditor. Because we're using
Cucumber, the test Scenarios should be understandable by people who
are not
programmers. Because we're using the page_object Ruby gem, the guts
of the
tests should be readable by people with programming experience in just
about any language. Regardless of level of expertise, we should be
able
to provide everyone a local test environment from which to run the
automated tests against WMF hosts either natively or with a
VirtualBox VM
configured via vagrant (http://www.mediawiki.org/wiki/Vagrant). We have
done this before:
online: https://www.mediawiki.org/wiki/Meetings/2013-07-18
live: https://www.mediawiki.org/wiki/Meetings/2013-06-27
Given a robust set of automated browser tests and the ability to run
them,
let us then:
* Identify deficiencies in the automated test coverage for the feature.
This may or may not include analysis of the test code itself, but
would
certainly at least include analysis of the ATDD-style stated feature
coverage in the Cucumber Scenarios. Do we have any technical debt
in our
test code?
* Identify test charters for which automated testing is not possible and
which are only testable by actual human beings. Are there tests that
cannot be automated, and is such testing worthwhile?
Outcomes:
* Participants will be able to analyze ATDD-style automated browser
tests
* Participants will be able to run Cucumber + page_object browser
tests and
analyze the results of those tests
* Participants will be able to demonstrate automated browser test
practices
with examples from the open WMF browser test code
* Participants will be able to begin to contribute to WMF testing
efforts
if they wish, whether automated or not
This might be too ambitious. I'm pretty sure this would be the most
technically challenging session in the history of WTA. OTOH, we've
already
built the infrastructure to do this kind of thing, let's spread the word
about what is possible.
-Chris
>
> Message: 2
>> Date: Wed, 14 Aug 2013 08:56:54 -0700
>> From: Chris McMahon <[email protected]
<mailto:[email protected]>>
>> To: "QA (software quality assurance) for Wikimedia projects."
>>
>> <[email protected] <mailto:[email protected]>>
>> Subject: Re: [QA] Weekend testing americas on September 7th
>> Message-ID:
>> <
>>
cajohbhqzwk5h9wegs-serhe0fazrss08wjigw-unb7abe-x...@mail.gmail.com
<mailto:cajohbhqzwk5h9wegs-serhe0fazrss08wjigw-unb7abe-x...@mail.gmail.com>>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>>
>> I actually have an idea that I think would be of interest to WTA
and of
>> benefit to WMF, but I'd like to encourage others to reply first..
>>
>> -C
>>
>>
>>
>>
> _______________________________________________
> QA mailing list
> [email protected] <mailto:[email protected]>
> https://lists.wikimedia.org/mailman/listinfo/qa
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.wikimedia.org/pipermail/qa/attachments/20130815/e6688311/attachment-0001.html>
------------------------------
_______________________________________________
QA mailing list
[email protected] <mailto:[email protected]>
https://lists.wikimedia.org/mailman/listinfo/qa
End of QA Digest, Vol 4, Issue 13
*********************************
_______________________________________________
QA mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/qa
--
Quim Gil
Technical Contributor Coordinator @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil
_______________________________________________
QA mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/qa
