This wouldn't just be when a test fails. It would be used to detect regressions in appearance at given steps.... e.g. Given I go to the login page When I trigger an error Then I see an error message And the login page looks exactly like it was a week ago.
On Tue, Apr 22, 2014 at 7:20 AM, Chris McMahon <[email protected]> wrote: > > > Language people also working on screen shots: > https://gerrit.wikimedia.org/r/#/c/127899/ > > > On Mon, Apr 21, 2014 at 2:25 PM, Juliusz Gonera <[email protected]> > wrote: >> >> Nope, it's an early prototype. It basically only makes an updated Cucumber >> test fail if new screenshot differs from reference screenshot by more than >> X%. Adding visual comparison of two screenshots would be the next step and >> would not be too hard. I might work on it one evening this week. >> >> >> >> On 04/21/2014 11:42 AM, Tomasz Finc wrote: >>> >>> Great. Eager to see us test with this. >>> >>> Do we have any of its early screenshot comparisons up? >>> >>> --tomasz >>> >>> On Fri, Apr 18, 2014 at 4:33 PM, Juliusz Gonera <[email protected]> >>> wrote: >>>> >>>> I conducted a little bit of research on visual regression testing, >>>> mainly >>>> for the mobile team for now, but it could be easily reused for other >>>> teams. >>>> I had a look at three existing solutions that seem somewhat popular and >>>> are >>>> actively developed: >>>> >>>> * Wraith (https://github.com/BBC-News/wraith) >>>> * PhantomCSS (https://github.com/Huddle/PhantomCSS) >>>> * Huxley (https://github.com/facebook/huxley) >>>> >>>> They all have their own pros and cons, but in my opinion they share one >>>> important disadvantage: they can't be easily integrated with our current >>>> browser testing setup. For all the aforementioned tools we would have to >>>> create from scratch a completely separate set of tests just for visual >>>> regression testing instead of extending our existing browser tests. >>>> >>>> I spent a few hours in my spare time tinkering with an alternative idea >>>> which would enable us to add visual regression testing to our >>>> Cucumber/Watir >>>> tests. The result is a small prototype available at >>>> https://github.com/jgonera/photographer. There is no docs or anything >>>> yet, >>>> but I prepared a simple demo patch for MobileFrontend: >>>> https://gerrit.wikimedia.org/r/#/c/126878/. >>>> >>>> The idea is to add a new method for Cucumber steps (snap) that takes a >>>> screenshot of the current browser state and compares it with a >>>> screenshot >>>> taken in one of the previous test runs. To update screenshots that are >>>> used >>>> as a reference you run tests with env var PHOTOGRAPHER=update. If newly >>>> taken screenshot differs by too many pixels from an old one, the test >>>> will >>>> fail. >>>> >>>> It's still only an early prototype, but I'd appreciate any comments >>>> about >>>> this idea. >>>> >>>> -- >>>> Juliusz >>>> >>>> _______________________________________________ >>>> Mobile-l mailing list >>>> [email protected] >>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l >> >> >> >> _______________________________________________ >> QA mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/qa > > > > _______________________________________________ > QA mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/qa > _______________________________________________ QA mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/qa
