[chromium-dev] Re: 2-day merges and the cleanup schedule
If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson bre...@chromium.org wrote: I'm currently doing a 2-day merge rotation.As part of this, various layout tests are regressing or getting added that I'm inserting into the tests_fixable list. But, like every other layout test fixer, after my merges are done, I'll finally go back to my old work and not think about it any more. This is how we get a monotonically increasing list of broken tests at the end of the tests fixable. I'm pretty certain this happens faster than the tests are getting fixed because nobody wants to fix them. I'm sort of tempted to just fix the ones my merge broke now, but that will put me behind for my next merge, so I don't do that. I propose a different rotation schedule for WebKit merges. You would do one day of merges, and the next day you would have to clean up all the regressed and new layout tests your merge introduced. The layout tests usually aren't that hard, so it normally wouldn't take the whole day. This way we can be in a good steady state of WebKit merges. It should also have a big advantage for fixing the broken tests since the things that changed, the build numbers involved, etc. are still fresh in the merger's head, and it won't be like approaching a random layout test from the list with no context. The disadvantage of doing this is that we have to find people to do the merges faster (we should probably formalize the schedule), and you'll lose some advantage the second day of having all the instructions and gotchas of the merge fresh in your mind. I was thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but that seems too heavyweight for the people volunteering to do this. Anybody have any comments? Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
I think the merger should be responsible for ensuring that any layout test fallout from the merge gets resolved. This doesn't mean necessary fixing everything, but rather it can be mean reaching out to others for help. I think the merger has to have incentive not to create a big mess with the merge and to also make sure the job gets done completely :-) -Darin On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet s...@chromium.org wrote: If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson bre...@chromium.org wrote: I'm currently doing a 2-day merge rotation.As part of this, various layout tests are regressing or getting added that I'm inserting into the tests_fixable list. But, like every other layout test fixer, after my merges are done, I'll finally go back to my old work and not think about it any more. This is how we get a monotonically increasing list of broken tests at the end of the tests fixable. I'm pretty certain this happens faster than the tests are getting fixed because nobody wants to fix them. I'm sort of tempted to just fix the ones my merge broke now, but that will put me behind for my next merge, so I don't do that. I propose a different rotation schedule for WebKit merges. You would do one day of merges, and the next day you would have to clean up all the regressed and new layout tests your merge introduced. The layout tests usually aren't that hard, so it normally wouldn't take the whole day. This way we can be in a good steady state of WebKit merges. It should also have a big advantage for fixing the broken tests since the things that changed, the build numbers involved, etc. are still fresh in the merger's head, and it won't be like approaching a random layout test from the list with no context. The disadvantage of doing this is that we have to find people to do the merges faster (we should probably formalize the schedule), and you'll lose some advantage the second day of having all the instructions and gotchas of the merge fresh in your mind. I was thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but that seems too heavyweight for the people volunteering to do this. Anybody have any comments? Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
Right. Ultimately, we need to have a buildbot visible on build.webkit.orgthat reports layout test failures to others in the webkit community. Then, we can pester the webkit contributor to help fix the problems they caused. At the very least, we'll have the big advantage of knowing precisely which change caused a problem. This applies not only to layout tests, but also to performance regressions and stability (reliability test runs will be triggered by WebKit commits!). -Darin On Wed, Mar 4, 2009 at 5:05 PM, Ojan Vafai o...@chromium.org wrote: The process I was picturing is that we'd have a WebKit sheriff. That person would be in charge of keeping the build with latest webkit green and updating the webkit revision we pull once a day. If someone else needs a newer webkit revision (e.g. something they just committed), they can work with the WebKit sherriff to pull it in. But basically, it would be that person's job to either rebaseline failing tests or add them to the fixable list and assign them to someone to fix. Ojan On Wed, Mar 4, 2009 at 5:01 PM, David Levin le...@google.com wrote: Yes, so what would be the process then? (Will we still have mergers whose job is mostly to handle new and modified layout tests?) On Wed, Mar 4, 2009 at 4:58 PM, Ojan Vafai o...@chromium.org wrote: Even once pulling in a new webkit is as simple as changing a revision number in DEPS, we'll still have new and modified layout tests that we'll need to deal with. On Wed, Mar 4, 2009 at 4:44 PM, David Levin le...@google.com wrote: Aren't we trying to get rid of the merge process (soon-ish)? If so, how would layout tests get fixed at that point? Dave On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher da...@chromium.orgwrote: I think the merger should be responsible for ensuring that any layout test fallout from the merge gets resolved. This doesn't mean necessary fixing everything, but rather it can be mean reaching out to others for help. I think the merger has to have incentive not to create a big mess with the merge and to also make sure the job gets done completely :-) -Darin On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet s...@chromium.orgwrote: If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson bre...@chromium.org wrote: I'm currently doing a 2-day merge rotation.As part of this, various layout tests are regressing or getting added that I'm inserting into the tests_fixable list. But, like every other layout test fixer, after my merges are done, I'll finally go back to my old work and not think about it any more. This is how we get a monotonically increasing list of broken tests at the end of the tests fixable. I'm pretty certain this happens faster than the tests are getting fixed because nobody wants to fix them. I'm sort of tempted to just fix the ones my merge broke now, but that will put me behind for my next merge, so I don't do that. I propose a different rotation schedule for WebKit merges. You would do one day of merges, and the next day you would have to clean up all the regressed and new layout tests your merge introduced. The layout tests usually aren't that hard, so it normally wouldn't take the whole day. This way we can be in a good steady state of WebKit merges. It should also have a big advantage for fixing the broken tests since the things that changed, the build numbers involved, etc. are still fresh in the merger's head, and it won't be like approaching a random layout test from the list with no context. The disadvantage of doing this is that we have to find people to do the merges faster (we should probably formalize the schedule), and you'll lose some advantage the second day of having all the instructions and gotchas of the merge fresh in your mind. I was thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but that seems too heavyweight for the people volunteering to do this. Anybody have any comments? Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev
[chromium-dev] Re: 2-day merges and the cleanup schedule
Right. I just mean that the merger should take care to ensure that all of the easy fixes are done and that the rest are accounted for somehow, probably via regression bugs. If that's the current process, then great. It just sounded like it wasn't. -Darin On Wed, Mar 4, 2009 at 4:51 PM, Pam Greene p...@chromium.org wrote: Define resolved. Is filing and assigning a bug sufficient? It's not efficient to have whoever volunteers to do a merge personally pester people to fix the resulting test errors forever after. Whatever we do, we're going to have broken layout tests after a merge. Easy fixes -- re-baselining the ones that are actually all right, and filing specific bugs for the rest -- are definitely part of the merge task. But we don't want to put too much blame on the messenger: if a bunch of tests really break, or new tests don't pass, it's hardly the fault of the person who happened to bring the changes in. - Pam On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher da...@chromium.org wrote: I think the merger should be responsible for ensuring that any layout test fallout from the merge gets resolved. This doesn't mean necessary fixing everything, but rather it can be mean reaching out to others for help. I think the merger has to have incentive not to create a big mess with the merge and to also make sure the job gets done completely :-) -Darin On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet s...@chromium.org wrote: If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson bre...@chromium.org wrote: I'm currently doing a 2-day merge rotation.As part of this, various layout tests are regressing or getting added that I'm inserting into the tests_fixable list. But, like every other layout test fixer, after my merges are done, I'll finally go back to my old work and not think about it any more. This is how we get a monotonically increasing list of broken tests at the end of the tests fixable. I'm pretty certain this happens faster than the tests are getting fixed because nobody wants to fix them. I'm sort of tempted to just fix the ones my merge broke now, but that will put me behind for my next merge, so I don't do that. I propose a different rotation schedule for WebKit merges. You would do one day of merges, and the next day you would have to clean up all the regressed and new layout tests your merge introduced. The layout tests usually aren't that hard, so it normally wouldn't take the whole day. This way we can be in a good steady state of WebKit merges. It should also have a big advantage for fixing the broken tests since the things that changed, the build numbers involved, etc. are still fresh in the merger's head, and it won't be like approaching a random layout test from the list with no context. The disadvantage of doing this is that we have to find people to do the merges faster (we should probably formalize the schedule), and you'll lose some advantage the second day of having all the instructions and gotchas of the merge fresh in your mind. I was thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but that seems too heavyweight for the people volunteering to do this. Anybody have any comments? Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
I'm ok with saying that you do the merge one day and have another day to cleanup layout test issues. Once we're not merging anymore though, I think we should just have 1-2 people oncall for 1 week at a time to keep the webkit(webkit.org) build green and to pull a new revision at least once a day (this latter part we can automate pretty easily as well). Ojan On Wed, Mar 4, 2009 at 5:10 PM, Darin Fisher da...@chromium.org wrote: Right. I just mean that the merger should take care to ensure that all of the easy fixes are done and that the rest are accounted for somehow, probably via regression bugs. If that's the current process, then great. It just sounded like it wasn't. -Darin On Wed, Mar 4, 2009 at 4:51 PM, Pam Greene p...@chromium.org wrote: Define resolved. Is filing and assigning a bug sufficient? It's not efficient to have whoever volunteers to do a merge personally pester people to fix the resulting test errors forever after. Whatever we do, we're going to have broken layout tests after a merge. Easy fixes -- re-baselining the ones that are actually all right, and filing specific bugs for the rest -- are definitely part of the merge task. But we don't want to put too much blame on the messenger: if a bunch of tests really break, or new tests don't pass, it's hardly the fault of the person who happened to bring the changes in. - Pam On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher da...@chromium.org wrote: I think the merger should be responsible for ensuring that any layout test fallout from the merge gets resolved. This doesn't mean necessary fixing everything, but rather it can be mean reaching out to others for help. I think the merger has to have incentive not to create a big mess with the merge and to also make sure the job gets done completely :-) -Darin On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet s...@chromium.org wrote: If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson bre...@chromium.org wrote: I'm currently doing a 2-day merge rotation.As part of this, various layout tests are regressing or getting added that I'm inserting into the tests_fixable list. But, like every other layout test fixer, after my merges are done, I'll finally go back to my old work and not think about it any more. This is how we get a monotonically increasing list of broken tests at the end of the tests fixable. I'm pretty certain this happens faster than the tests are getting fixed because nobody wants to fix them. I'm sort of tempted to just fix the ones my merge broke now, but that will put me behind for my next merge, so I don't do that. I propose a different rotation schedule for WebKit merges. You would do one day of merges, and the next day you would have to clean up all the regressed and new layout tests your merge introduced. The layout tests usually aren't that hard, so it normally wouldn't take the whole day. This way we can be in a good steady state of WebKit merges. It should also have a big advantage for fixing the broken tests since the things that changed, the build numbers involved, etc. are still fresh in the merger's head, and it won't be like approaching a random layout test from the list with no context. The disadvantage of doing this is that we have to find people to do the merges faster (we should probably formalize the schedule), and you'll lose some advantage the second day of having all the instructions and gotchas of the merge fresh in your mind. I was thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but that seems too heavyweight for the people volunteering to do this. Anybody have any comments? Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
I'm not sure how closely that's followed. Me, I did merges Monday and Tuesday, and layout-test cleanup (baselines and bugs) today. - Pam On Wed, Mar 4, 2009 at 5:10 PM, Darin Fisher da...@chromium.org wrote: Right. I just mean that the merger should take care to ensure that all of the easy fixes are done and that the rest are accounted for somehow, probably via regression bugs. If that's the current process, then great. It just sounded like it wasn't. -Darin On Wed, Mar 4, 2009 at 4:51 PM, Pam Greene p...@chromium.org wrote: Define resolved. Is filing and assigning a bug sufficient? It's not efficient to have whoever volunteers to do a merge personally pester people to fix the resulting test errors forever after. Whatever we do, we're going to have broken layout tests after a merge. Easy fixes -- re-baselining the ones that are actually all right, and filing specific bugs for the rest -- are definitely part of the merge task. But we don't want to put too much blame on the messenger: if a bunch of tests really break, or new tests don't pass, it's hardly the fault of the person who happened to bring the changes in. - Pam On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher da...@chromium.org wrote: I think the merger should be responsible for ensuring that any layout test fallout from the merge gets resolved. This doesn't mean necessary fixing everything, but rather it can be mean reaching out to others for help. I think the merger has to have incentive not to create a big mess with the merge and to also make sure the job gets done completely :-) -Darin On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet s...@chromium.org wrote: If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson bre...@chromium.org wrote: I'm currently doing a 2-day merge rotation.As part of this, various layout tests are regressing or getting added that I'm inserting into the tests_fixable list. But, like every other layout test fixer, after my merges are done, I'll finally go back to my old work and not think about it any more. This is how we get a monotonically increasing list of broken tests at the end of the tests fixable. I'm pretty certain this happens faster than the tests are getting fixed because nobody wants to fix them. I'm sort of tempted to just fix the ones my merge broke now, but that will put me behind for my next merge, so I don't do that. I propose a different rotation schedule for WebKit merges. You would do one day of merges, and the next day you would have to clean up all the regressed and new layout tests your merge introduced. The layout tests usually aren't that hard, so it normally wouldn't take the whole day. This way we can be in a good steady state of WebKit merges. It should also have a big advantage for fixing the broken tests since the things that changed, the build numbers involved, etc. are still fresh in the merger's head, and it won't be like approaching a random layout test from the list with no context. The disadvantage of doing this is that we have to find people to do the merges faster (we should probably formalize the schedule), and you'll lose some advantage the second day of having all the instructions and gotchas of the merge fresh in your mind. I was thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but that seems too heavyweight for the people volunteering to do this. Anybody have any comments? Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
Added layout-test cleanup steps to the instructions. - Pam On Wed, Mar 4, 2009 at 6:08 PM, David Levin le...@chromium.org wrote: I know that I haven't filed chromium bugs for layout test failures in the past because it isn't in the instructions (I thought adding the the tests_fixable was sufficient.) http://dev.chromium.org/developers/how-tos/webkit-merge-1 Though I do usually have at least a partial 3rd day of items for clean up: upstreaming, some times filing bugs for new reliability crashes, etc. Dave On Wed, Mar 4, 2009 at 5:30 PM, Pam Greene p...@chromium.org wrote: I'm not sure how closely that's followed. Me, I did merges Monday and Tuesday, and layout-test cleanup (baselines and bugs) today. - Pam On Wed, Mar 4, 2009 at 5:10 PM, Darin Fisher da...@chromium.org wrote: Right. I just mean that the merger should take care to ensure that all of the easy fixes are done and that the rest are accounted for somehow, probably via regression bugs. If that's the current process, then great. It just sounded like it wasn't. -Darin On Wed, Mar 4, 2009 at 4:51 PM, Pam Greene p...@chromium.org wrote: Define resolved. Is filing and assigning a bug sufficient? It's not efficient to have whoever volunteers to do a merge personally pester people to fix the resulting test errors forever after. Whatever we do, we're going to have broken layout tests after a merge. Easy fixes -- re-baselining the ones that are actually all right, and filing specific bugs for the rest -- are definitely part of the merge task. But we don't want to put too much blame on the messenger: if a bunch of tests really break, or new tests don't pass, it's hardly the fault of the person who happened to bring the changes in. - Pam On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher da...@chromium.orgwrote: I think the merger should be responsible for ensuring that any layout test fallout from the merge gets resolved. This doesn't mean necessary fixing everything, but rather it can be mean reaching out to others for help. I think the merger has to have incentive not to create a big mess with the merge and to also make sure the job gets done completely :-) -Darin On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet s...@chromium.orgwrote: If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson bre...@chromium.org wrote: I'm currently doing a 2-day merge rotation.As part of this, various layout tests are regressing or getting added that I'm inserting into the tests_fixable list. But, like every other layout test fixer, after my merges are done, I'll finally go back to my old work and not think about it any more. This is how we get a monotonically increasing list of broken tests at the end of the tests fixable. I'm pretty certain this happens faster than the tests are getting fixed because nobody wants to fix them. I'm sort of tempted to just fix the ones my merge broke now, but that will put me behind for my next merge, so I don't do that. I propose a different rotation schedule for WebKit merges. You would do one day of merges, and the next day you would have to clean up all the regressed and new layout tests your merge introduced. The layout tests usually aren't that hard, so it normally wouldn't take the whole day. This way we can be in a good steady state of WebKit merges. It should also have a big advantage for fixing the broken tests since the things that changed, the build numbers involved, etc. are still fresh in the merger's head, and it won't be like approaching a random layout test from the list with no context. The disadvantage of doing this is that we have to find people to do the merges faster (we should probably formalize the schedule), and you'll lose some advantage the second day of having all the instructions and gotchas of the merge fresh in your mind. I was thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but that seems too heavyweight for the people volunteering to do this. Anybody have any comments? Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or
[chromium-dev] Re: 2-day merges and the cleanup schedule
When fixing layout tests only means re-baselining, that's easy. But sometimes they break (or new ones fail) for deeper reasons, and the person doing the merge may not be the right one to make the fix (or may not be able to fix them in one day). So perhaps clean up in this context means re-baseline if that's all it needs, and file individual bugs on specific people for bigger brokenness. Also, to clarify, are you proposing that we only merge every other day, or that we have two people assigned each day: one to merge and one to clean up the previous day's layout-test breakage? If the latter, we could also split the job in the other direction, and have one person merging two days in a row and one fixing up the test list both days. I could imagine people's tastes running more to one job or the other, and we don't really care who does what as long as it gets done. - Pam On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson bre...@chromium.org wrote: I'm currently doing a 2-day merge rotation.As part of this, various layout tests are regressing or getting added that I'm inserting into the tests_fixable list. But, like every other layout test fixer, after my merges are done, I'll finally go back to my old work and not think about it any more. This is how we get a monotonically increasing list of broken tests at the end of the tests fixable. I'm pretty certain this happens faster than the tests are getting fixed because nobody wants to fix them. I'm sort of tempted to just fix the ones my merge broke now, but that will put me behind for my next merge, so I don't do that. I propose a different rotation schedule for WebKit merges. You would do one day of merges, and the next day you would have to clean up all the regressed and new layout tests your merge introduced. The layout tests usually aren't that hard, so it normally wouldn't take the whole day. This way we can be in a good steady state of WebKit merges. It should also have a big advantage for fixing the broken tests since the things that changed, the build numbers involved, etc. are still fresh in the merger's head, and it won't be like approaching a random layout test from the list with no context. The disadvantage of doing this is that we have to find people to do the merges faster (we should probably formalize the schedule), and you'll lose some advantage the second day of having all the instructions and gotchas of the merge fresh in your mind. I was thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but that seems too heavyweight for the people volunteering to do this. Anybody have any comments? Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
As someone sitting on the other side only fixing layout tests this sounds great. The current mode of operation where a merge is landed and others are left to fix the new failures is very disheartening in that we'll make progress only to have a new merge land breaking the progress we made. I now know how a mailman feels! -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson bre...@chromium.org wrote: I'm currently doing a 2-day merge rotation.As part of this, various layout tests are regressing or getting added that I'm inserting into the tests_fixable list. But, like every other layout test fixer, after my merges are done, I'll finally go back to my old work and not think about it any more. This is how we get a monotonically increasing list of broken tests at the end of the tests fixable. I'm pretty certain this happens faster than the tests are getting fixed because nobody wants to fix them. I'm sort of tempted to just fix the ones my merge broke now, but that will put me behind for my next merge, so I don't do that. I propose a different rotation schedule for WebKit merges. You would do one day of merges, and the next day you would have to clean up all the regressed and new layout tests your merge introduced. The layout tests usually aren't that hard, so it normally wouldn't take the whole day. This way we can be in a good steady state of WebKit merges. It should also have a big advantage for fixing the broken tests since the things that changed, the build numbers involved, etc. are still fresh in the merger's head, and it won't be like approaching a random layout test from the list with no context. The disadvantage of doing this is that we have to find people to do the merges faster (we should probably formalize the schedule), and you'll lose some advantage the second day of having all the instructions and gotchas of the merge fresh in your mind. I was thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but that seems too heavyweight for the people volunteering to do this. Anybody have any comments? Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene p...@chromium.org wrote: When fixing layout tests only means re-baselining, that's easy. But sometimes they break (or new ones fail) for deeper reasons, and the person doing the merge may not be the right one to make the fix (or may not be able to fix them in one day). So perhaps clean up in this context means re-baseline if that's all it needs, and file individual bugs on specific people for bigger brokenness. I think our tendency to file bugs and forget about it is part of the problem. I am at least as guilty as anybody else. I think the merger should have the responsibility to get their regressions fixed. They will have to talk with some other people to get input. If they aren't the right person to fix a problem, it should be their responsibility as part of the cleanup to make sure that the right person has been assigned to it and is actually working on it. When people are assigned merge bugs, they should be treated as important regressions and prioritized over other work. We currently had a whole lot of layout tests bugs filed that are getting no love. The only way to not keep getting behind is to be much more proactive. Also, to clarify, are you proposing that we only merge every other day, or that we have two people assigned each day: one to merge and one to clean up the previous day's layout-test breakage? If the latter, we could also split the job in the other direction, and have one person merging two days in a row and one fixing up the test list both days. I could imagine people's tastes running more to one job or the other, and we don't really care who does what as long as it gets done. I'm proposing overlapping so we merge every day. I think there is an advantage in having the same person who did the merge do the fixing. This hopefully also makes the merge less tedious since you have different tasks your two days. Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
Generally +1, except I just imagined the situation where the merger collides with the fixer. So maybe no overlapping, just do the merge every other day? :DG On Thu, Jan 15, 2009 at 1:31 PM, Brett Wilson bre...@chromium.org wrote: On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene p...@chromium.org wrote: When fixing layout tests only means re-baselining, that's easy. But sometimes they break (or new ones fail) for deeper reasons, and the person doing the merge may not be the right one to make the fix (or may not be able to fix them in one day). So perhaps clean up in this context means re-baseline if that's all it needs, and file individual bugs on specific people for bigger brokenness. I think our tendency to file bugs and forget about it is part of the problem. I am at least as guilty as anybody else. I think the merger should have the responsibility to get their regressions fixed. They will have to talk with some other people to get input. If they aren't the right person to fix a problem, it should be their responsibility as part of the cleanup to make sure that the right person has been assigned to it and is actually working on it. When people are assigned merge bugs, they should be treated as important regressions and prioritized over other work. We currently had a whole lot of layout tests bugs filed that are getting no love. The only way to not keep getting behind is to be much more proactive. Also, to clarify, are you proposing that we only merge every other day, or that we have two people assigned each day: one to merge and one to clean up the previous day's layout-test breakage? If the latter, we could also split the job in the other direction, and have one person merging two days in a row and one fixing up the test list both days. I could imagine people's tastes running more to one job or the other, and we don't really care who does what as long as it gets done. I'm proposing overlapping so we merge every day. I think there is an advantage in having the same person who did the merge do the fixing. This hopefully also makes the merge less tedious since you have different tasks your two days. Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
On Thu, Jan 15, 2009 at 1:31 PM, Brett Wilson bre...@chromium.org wrote: On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene p...@chromium.org wrote: When fixing layout tests only means re-baselining, that's easy. But sometimes they break (or new ones fail) for deeper reasons, and the person doing the merge may not be the right one to make the fix (or may not be able to fix them in one day). So perhaps clean up in this context means re-baseline if that's all it needs, and file individual bugs on specific people for bigger brokenness. I think our tendency to file bugs and forget about it is part of the problem. I am at least as guilty as anybody else. I think the merger should have the responsibility to get their regressions fixed. They will have to talk with some other people to get input. If they aren't the right person to fix a problem, it should be their responsibility as part of the cleanup to make sure that the right person has been assigned to it and is actually working on it. Taking responsibility, check. Making sure someone is assigned, check. Fixing tricky regressions yourself may not be the most efficient use of time, and would be a strong deterrent to volunteer for a merge. Merging isn't much fun; fixing layout tests isn't much fun. Let's spread the pain where suitable, rather than piling it all on the person who volunteers for one part. In fact, I'd ask the merger to fix the easy problems (tests changed, new tests need baselines) before committing the merge. When people are assigned merge bugs, they should be treated as important regressions and prioritized over other work. We currently had a whole lot of layout tests bugs filed that are getting no love. The only way to not keep getting behind is to be much more proactive. Absolutely. Also, to clarify, are you proposing that we only merge every other day, or that we have two people assigned each day: one to merge and one to clean up the previous day's layout-test breakage? If the latter, we could also split the job in the other direction, and have one person merging two days in a row and one fixing up the test list both days. I could imagine people's tastes running more to one job or the other, and we don't really care who does what as long as it gets done. I'm proposing overlapping so we merge every day. I think there is an advantage in having the same person who did the merge do the fixing. This hopefully also makes the merge less tedious since you have different tasks your two days. Sounds fine. People can always trade if they want. - Pam Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
I definitely get that a merge is no trivial amount of work, and my expectations are probably unrealistic without more than one person working on a merge at a time. I was hoping for a magic bullet, but those are only in movies:( -Scott PS If anyone not doing merges feels like the people doing merges should do more, please volunteer to do a merge so at least you'll know what it is like -- Sorry Scott but I'm looking at you :) On Thu, Jan 15, 2009 at 2:26 PM, Pam Greene p...@chromium.org wrote: On Thu, Jan 15, 2009 at 1:31 PM, Brett Wilson bre...@chromium.org wrote: On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene p...@chromium.org wrote: When fixing layout tests only means re-baselining, that's easy. But sometimes they break (or new ones fail) for deeper reasons, and the person doing the merge may not be the right one to make the fix (or may not be able to fix them in one day). So perhaps clean up in this context means re-baseline if that's all it needs, and file individual bugs on specific people for bigger brokenness. I think our tendency to file bugs and forget about it is part of the problem. I am at least as guilty as anybody else. I think the merger should have the responsibility to get their regressions fixed. They will have to talk with some other people to get input. If they aren't the right person to fix a problem, it should be their responsibility as part of the cleanup to make sure that the right person has been assigned to it and is actually working on it. Taking responsibility, check. Making sure someone is assigned, check. Fixing tricky regressions yourself may not be the most efficient use of time, and would be a strong deterrent to volunteer for a merge. Merging isn't much fun; fixing layout tests isn't much fun. Let's spread the pain where suitable, rather than piling it all on the person who volunteers for one part. In fact, I'd ask the merger to fix the easy problems (tests changed, new tests need baselines) before committing the merge. When people are assigned merge bugs, they should be treated as important regressions and prioritized over other work. We currently had a whole lot of layout tests bugs filed that are getting no love. The only way to not keep getting behind is to be much more proactive. Absolutely. Also, to clarify, are you proposing that we only merge every other day, or that we have two people assigned each day: one to merge and one to clean up the previous day's layout-test breakage? If the latter, we could also split the job in the other direction, and have one person merging two days in a row and one fixing up the test list both days. I could imagine people's tastes running more to one job or the other, and we don't really care who does what as long as it gets done. I'm proposing overlapping so we merge every day. I think there is an advantage in having the same person who did the merge do the fixing. This hopefully also makes the merge less tedious since you have different tasks your two days. Sounds fine. People can always trade if they want. - Pam Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
This is veering wildly off-topic, but I think the key to solving merge regressions is in moving to an integration model. With the integration model, we can integrate one WebKit changeset at a time, and clearly identify the regressions. This would go a long way in identifying the cause and addressing the issue. Unfortunately, currently we have engineers fixing layout tests and making changes to our WebKit fork, which makes integration model ... complicated. What if we go with an integration-merge model, which works as follows: 1) we have a separate WebKit git repository (hosted on someone's machine even), which has a branch with all of our changes applied as one (or more) patch. 2) you pull WebKit changes, rebase (in other words, re-apply our change patches), see if the changes broke anything. 3) if they did, identify the changeset that breaks stuff by using git bisect (nifty stuff, I might say), then mark it in text_fixable and file a bug 4) merge integrated changes into main svn repository. ... or something like that? Somebody take this idea and flesh it out to make it excellent! :) :DG On Thu, Jan 15, 2009 at 2:51 PM, Scott Violet s...@chromium.org wrote: I definitely get that a merge is no trivial amount of work, and my expectations are probably unrealistic without more than one person working on a merge at a time. I was hoping for a magic bullet, but those are only in movies:( -Scott PS If anyone not doing merges feels like the people doing merges should do more, please volunteer to do a merge so at least you'll know what it is like -- Sorry Scott but I'm looking at you :) On Thu, Jan 15, 2009 at 2:26 PM, Pam Greene p...@chromium.org wrote: On Thu, Jan 15, 2009 at 1:31 PM, Brett Wilson bre...@chromium.org wrote: On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene p...@chromium.org wrote: When fixing layout tests only means re-baselining, that's easy. But sometimes they break (or new ones fail) for deeper reasons, and the person doing the merge may not be the right one to make the fix (or may not be able to fix them in one day). So perhaps clean up in this context means re-baseline if that's all it needs, and file individual bugs on specific people for bigger brokenness. I think our tendency to file bugs and forget about it is part of the problem. I am at least as guilty as anybody else. I think the merger should have the responsibility to get their regressions fixed. They will have to talk with some other people to get input. If they aren't the right person to fix a problem, it should be their responsibility as part of the cleanup to make sure that the right person has been assigned to it and is actually working on it. Taking responsibility, check. Making sure someone is assigned, check. Fixing tricky regressions yourself may not be the most efficient use of time, and would be a strong deterrent to volunteer for a merge. Merging isn't much fun; fixing layout tests isn't much fun. Let's spread the pain where suitable, rather than piling it all on the person who volunteers for one part. In fact, I'd ask the merger to fix the easy problems (tests changed, new tests need baselines) before committing the merge. When people are assigned merge bugs, they should be treated as important regressions and prioritized over other work. We currently had a whole lot of layout tests bugs filed that are getting no love. The only way to not keep getting behind is to be much more proactive. Absolutely. Also, to clarify, are you proposing that we only merge every other day, or that we have two people assigned each day: one to merge and one to clean up the previous day's layout-test breakage? If the latter, we could also split the job in the other direction, and have one person merging two days in a row and one fixing up the test list both days. I could imagine people's tastes running more to one job or the other, and we don't really care who does what as long as it gets done. I'm proposing overlapping so we merge every day. I think there is an advantage in having the same person who did the merge do the fixing. This hopefully also makes the merge less tedious since you have different tasks your two days. Sounds fine. People can always trade if they want. - Pam Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
