Agreed, but it would be good for this mechanism to be as decoupled from
the tests themselves as possible. This way it can be reused and improved
by other people (outside of WMF).
On 04/22/2014 08:58 AM, Chris McMahon wrote:
Yes, but I think it might be possible to share the underlying
mechanism of taking screen shots among all the projects that want to
do that, for whatever purposes they have.
-Chris
On Tue, Apr 22, 2014 at 8:53 AM, Jon Robson <[email protected]
<mailto:[email protected]>> wrote:
This wouldn't just be when a test fails. It would be used to detect
regressions in appearance at given steps....
e.g. Given I go to the login page When I trigger an error Then I see
an error message And the login page looks exactly like it was a week
ago.
On Tue, Apr 22, 2014 at 7:20 AM, Chris McMahon
<[email protected] <mailto:[email protected]>> wrote:
>
>
> Language people also working on screen shots:
> https://gerrit.wikimedia.org/r/#/c/127899/
>
>
> On Mon, Apr 21, 2014 at 2:25 PM, Juliusz Gonera
<[email protected] <mailto:[email protected]>>
> wrote:
>>
>> Nope, it's an early prototype. It basically only makes an
updated Cucumber
>> test fail if new screenshot differs from reference screenshot
by more than
>> X%. Adding visual comparison of two screenshots would be the
next step and
>> would not be too hard. I might work on it one evening this week.
>>
>>
>>
>> On 04/21/2014 11:42 AM, Tomasz Finc wrote:
>>>
>>> Great. Eager to see us test with this.
>>>
>>> Do we have any of its early screenshot comparisons up?
>>>
>>> --tomasz
>>>
>>> On Fri, Apr 18, 2014 at 4:33 PM, Juliusz Gonera
<[email protected] <mailto:[email protected]>>
>>> wrote:
>>>>
>>>> I conducted a little bit of research on visual regression
testing,
>>>> mainly
>>>> for the mobile team for now, but it could be easily reused
for other
>>>> teams.
>>>> I had a look at three existing solutions that seem somewhat
popular and
>>>> are
>>>> actively developed:
>>>>
>>>> * Wraith (https://github.com/BBC-News/wraith)
>>>> * PhantomCSS (https://github.com/Huddle/PhantomCSS)
>>>> * Huxley (https://github.com/facebook/huxley)
>>>>
>>>> They all have their own pros and cons, but in my opinion they
share one
>>>> important disadvantage: they can't be easily integrated with
our current
>>>> browser testing setup. For all the aforementioned tools we
would have to
>>>> create from scratch a completely separate set of tests just
for visual
>>>> regression testing instead of extending our existing browser
tests.
>>>>
>>>> I spent a few hours in my spare time tinkering with an
alternative idea
>>>> which would enable us to add visual regression testing to our
>>>> Cucumber/Watir
>>>> tests. The result is a small prototype available at
>>>> https://github.com/jgonera/photographer. There is no docs or
anything
>>>> yet,
>>>> but I prepared a simple demo patch for MobileFrontend:
>>>> https://gerrit.wikimedia.org/r/#/c/126878/.
>>>>
>>>> The idea is to add a new method for Cucumber steps (snap)
that takes a
>>>> screenshot of the current browser state and compares it with a
>>>> screenshot
>>>> taken in one of the previous test runs. To update screenshots
that are
>>>> used
>>>> as a reference you run tests with env var
PHOTOGRAPHER=update. If newly
>>>> taken screenshot differs by too many pixels from an old one,
the test
>>>> will
>>>> fail.
>>>>
>>>> It's still only an early prototype, but I'd appreciate any
comments
>>>> about
>>>> this idea.
>>>>
>>>> --
>>>> Juliusz
>>>>
>>>> _______________________________________________
>>>> Mobile-l mailing list
>>>> [email protected]
<mailto:[email protected]>
>>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>
>>
>>
>> _______________________________________________
>> QA mailing list
>> [email protected] <mailto:[email protected]>
>> https://lists.wikimedia.org/mailman/listinfo/qa
>
>
>
> _______________________________________________
> QA mailing list
> [email protected] <mailto:[email protected]>
> https://lists.wikimedia.org/mailman/listinfo/qa
>
_______________________________________________
QA mailing list
[email protected] <mailto:[email protected]>
https://lists.wikimedia.org/mailman/listinfo/qa
_______________________________________________
QA mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/qa
_______________________________________________
QA mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/qa
