[chromium-dev] Layout Tests Task Force syncup meeting notes 11/16/2009
Layout Tests Task Force syncup 11/16/2009 - Number of failing tests: 635 - dashboard: http://chromiumlttf.appspot.com/ - bunch of rebaseline fixes coming up - last week was not as productive because mstone 4.0 bugs were being worked on - we can see that new failures accumulated last week because of this - this is a sign of sustainability issues we should think about - overarching goal of LTTF is maintaining compatibility - compatibility with Safari Mac - compatibility w/ the web - have we been spending too much time on test harness, rebaselines, etc? - sustainability - how do we make this a non-infinite-length task force? - we should start brainstorming about sustainability - some ideas: - assign different priorities to different tests - we will start a Wave to discuss... - https://wave.google.com/a/google.com/?AuthEventSource=SSO#minimized:nav,minimized:contact,minimized:search,restored:wave:google.com!w%252B-7O-i99aC OKR status: - Fix all Windows layout tests: make test_expectations.txt only contain items that we will never fix, features we have not yet implemented, or bugs less than one week old that are a result of a recent WebKit merge. - see sustainability issues discussed above - All LTTF bugs that we fix are fixed across all three platforms (Win, Mac, Linux), unless they depend on features not yet implemented. - seem to be doing well on this - Fix any crashing layout tests within a week of occurrence. - doing okay, people tend to jump in fairly quickly - Work with the green tree task force to eliminate flakiness of layout tests. - could we run tests twice to hide/avoid the flakiness? - people often wait for it to run again anyway - this could be very useful - number of flaky tests is probably going to go up before they go down - Set up a public dashboard which tracks the number of failing layout tests over time on the Chromium site. - done -- Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Chrome Layout Tests Task Force meeting notes 10/26
*Layout Tests Task Force meeting - 10/26/2009* Action Items - *everyone on LTTF*: look at dpranke's dashboard ( http://chromiumlttf.appspot.com/), send feedback - *dglazkov*: send hayato and rolandsteiner the Wave that the Rock Lobster Brigade is using to communicate Notes - how involved is everybody? - dglazkov: 100% (not doing any mstone4 bugs) - victorw: 100% (not doing any mstone4 bugs) - japhet: 1 m4 bug, 50% this week (build sheriff), 80-90% after that - dpranke (wfh sick): been working on dashboard (see below), will add something to capture history, will be categorizing bugs in bugs db, then fixing bugs fulltime starting tmrw - dimich: worked on stuff related to workers (but not layout tests) - levin: 50% this week (working on m4 crasher) - jianli: normally 20-40% each week (has been working on m4 bug) - senorblanco: 30% (rest is skia bugs) - rolandsteiner: 70% (taking care of old bugs, and machine translation) - yutak: 100% (last week was webkit gardener) - hayato: new to the team, 80% - hamaji: 70% - other Tokyo folks are indeed working on LTTF and fixing tests - had syncup with task force leads last week - net number of fixed layout tests has been ~180 in one month :) - look at dpranke's dashboard (http://chromiumlttf.appspot.com/) - please take a look, send feedback - if anyone is working on a specific issue, make sure to assign the bug to yourself first to avoid overlap - Rock lobster brigade - victorw + japhet + dglazkov: work on the *bottom* of test_expectations (things that came through recently) - lots of them are fresh, easy fixes, rebaselines, etc. (low hanging fruit) - idea: have new folks work on the easy fixes - good for learning the codebase! - AI for dglazkov: send hayato and rolandsteiner the Wave that the Rock Lobster Brigade is using to communicate - is three people enough? do we want more people? - it would be useful if canary could tell us which tests are new, which tests are regressions - in the last 1-2 weeks, rock lobster brigade *has *been able to keep up with the # of tests coming into the bottom (but it does fluctuate a lot) - questions, concerns, complaints, suggestions - there are unassigned LTTF bugs - people should grab them - feedback on meeting format? how often to have this? - every 2 weeks? - let's do another one two weeks from now (4:30pm Pacific), and then re-evaluate --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Chrome Layout Tests Task Force progress update
*Failing Test Count* * * The number of failing WebKit layout tests (WinXP) has been reduced to *622*. (We had ~800 when we started keeping track.) *LTTF Dashboard* dpranke@ has put together a nice dashboardhttp://chromiumlttf.appspot.com/ showing some important metrics, along with corresponding issue tracker links. These include: - flaky/fail/skip test counts - tests with invalid bug ids - non LayoutTests-labeled bugs - other good info Check it out at: http://chromiumlttf.appspot.com/. (Click on each line to expand.) * * *Recent Work* http://tinyurl.com/lttfstatus has the details, but some of the things being worked on include: - getClientRects - accessibilityController - CSS2.1 test suites - drag-related tests - EventSendingController - setTimeout - PlainTextController - canvas - Skia - paste-xml - type=search input -- Jeff, on behalf of the LTTF --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Chrome Layout Tests Task Force status updates 10/6
Hello, Attached are the meeting notes from the most recent non-existent Layout Tests Task Force meeting. (Yes, that was a joke.) In lieu of meeting notes, here, once again, are the status updates from the intrepid members of the LTTF. tinyurl.com/LTTFstatus At last count, we are down to 627 failing tests (on Windows). Woo! cheers, Jeff --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~--- attachment: meeting-notes.png
[chromium-dev] Chrome layout tests task force status updates 9/30
Hi all, The Layout Tests Task Force has been busy! If you're interested in what folks have been working on, take a look at this spreadsheet: http://tinyurl.com/LTTFstatus best, Jeff --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] [LTTF] Layout Tests Task Force
Hello, The Chromium team will be putting a lot of focus over the next few months on WebKit layout tests. In fact, we've created a Layout Tests Task Force, and anyone is welcome to contribute. As you know, layout tests are used to check the correctness of the renderer. Existing documentation on how to set up and run the tests is here: http://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tests . We've also set up a page which describes how the Task Force operates. In a nutshell, there are two exciting roles that anyone is free to choose from: Finders, and Fixers. http://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tests/task-force There's no shortage of layout tests to tackle, so go for it! We'll try to set up something to help us visualize our progress over time better. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] [LTTF] Goals for the Layout Tests Task Force
Hello, As mentioned previously, a Layout Tests Task Force has been created to tackle the noble problem of fixing all of the WebKit layout tests that Chromium is currently failing. (Documentation on the task force is here: http://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tests/task-force ) We have also assembled a set of goals for the next three months: http://sites.google.com/a/chromium.org/dev/developers/testing/webkit-layout-tests/goals They are pasted here for convenience: - Fix all Windows layout tests: make test_expectations.txt only contain items that we will never fix, features we have not yet implemented, or bugs less than one week old that are a result of a recent WebKit merge. - All LTTF bugs that we fix are fixed across all three platforms (Win, Mac, Linux), unless they depend on features not yet implemented. - Fix any crashing layout tests within a week of occurrence. - Work with the green tree task force to eliminate flakiness of layout tests. - Set up a public dashboard which tracks the number of failing layout tests over time on the Chromium site. Feedback is more than welcome! --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] [LTTF] Status updates
Hello, Are you on the Layout Tests Task Force and wondering what your fellow taskforcers have been up to? Are you interested in joining the task force and want to see what everyone else has been working on? In the spirit of transparency, we're going to try collecting and sending out weekly status updates on what each member of the LTTF is doing. http://tinyurl.com/LTTFstatus I will email at the beginning of each week to remind people, and again later in the week to encourage people to take a look. Feel free to add yourself to the list if you join the task force! As always, I'd love to hear feedback, so if you think this is useless, or you think this is great, or you prefer email instead of spreadsheet, biweekly instead of weekly, etc., feel free to reply to me individually or publicly. Jeff --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
