On Thu, Apr 9, 2009 at 6:49 PM, Ojan Vafai <[email protected]> wrote: > At a quick glance, this looks great. I didn't look over every bug, but the > ones I did look at look good.
Yep. You'll want some sort of default description for the ones that have none. 200+ bugs is certainly too many, but that's no reason not to file them. (Sorry, couldn't resist. Seriously, yes, definitely file them. Better in the issue tracker than getting lost.) > It would be great to check in a version of this script that we could run > when a number of tests fail (e.g. when doing a bad webkit merge). That way, > we can add them all to the local test_expectations.txt file and have it spit > out the new results. > Fixing the file we have and moving forward are slightly different use cases, but yes. In the long run, we shouldn't need any script to fix an existing bug-less expectations file, only the part that adds bugs for newly added tests. I'm not sure whether that part should be controlled by adding the tests to the file and having the script file bugs, or making it a fully interactive "app": give it a list of files and a description, and it both changes the file and creates a bug. > Really, it would be great if the script filed bugs and then just modified > test_expectations.txt directly (without committing it). Also, the script > should remove any comments it moves into bug descriptions. We should get to > a point where all the comments about layout tests are in the bug > descriptions themselves instead of in this file. > I disagree with this last part. I'd prefer a brief description to remain in the file, with any details in the bug. Certainly that's needed for WONTFIX bugs, where we may not have a bug since there's no work to be tracked, but it's helpful for others too. I've found it frustrating and time-consuming to track down when I see big blocks of failures with no explanations at all. Think of it like the svn checkin comment: enough to have an idea what's going on right there where you need it, with more detail in the bug for when you're really digging. - Pam > I think it would be good to get the script checked in first and then run it > on the existing test_expectations.txt file. > > Ojan > > > On Thu, Apr 9, 2009 at 6:42 PM, Glenn Wilson <[email protected]> wrote: > >> Hi Pam & Ojan, >> I wrote a script that would extract all of the layout tests from >> test_expectations.txt that we haven't marked as WONTFIX and don't have a bug >> number. I also tried a simple heuristic to get the context of the layout >> test via nearby comments....it's not perfect, and I'll have to change some >> of them by hand, but many of the merge comments are getting picked up. >> >> I've also hooked up our library for creating demetrius bugs, so getting >> bugs made for these should be a matter of running the script with different >> arguments (I hope). >> >> What are your thoughts? Is these as descriptive/accurate as we need? Is >> 200+ bugs too many? >> >> Thanks! >> Glenn >> >> >> > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
