[chromium-dev] Re: 2-day merges and the cleanup schedule
Added layout-test cleanup steps to the instructions. - Pam On Wed, Mar 4, 2009 at 6:08 PM, David Levin wrote: > I know that I haven't filed chromium bugs for layout test failures in the > past because it isn't in the instructions (I thought adding the the > tests_fixable was sufficient.) > > http://dev.chromium.org/developers/how-tos/webkit-merge-1 > > > Though I do usually have at least a partial 3rd day of items for clean up: > upstreaming, some times filing bugs for new reliability crashes, etc. > Dave > > > On Wed, Mar 4, 2009 at 5:30 PM, Pam Greene wrote: > >> I'm not sure how closely that's followed. Me, I did merges Monday and >> Tuesday, and layout-test cleanup (baselines and bugs) today. >> - Pam >> >> >> On Wed, Mar 4, 2009 at 5:10 PM, Darin Fisher wrote: >> >>> Right. I just mean that the merger should take care to ensure that all >>> of the easy "fixes" are done and that the rest are accounted for somehow, >>> probably via regression bugs. If that's the current process, then great. >>> It just sounded like it wasn't. >>> -Darin >>> >>> >>> >>> On Wed, Mar 4, 2009 at 4:51 PM, Pam Greene wrote: >>> Define "resolved". Is filing and assigning a bug sufficient? It's not efficient to have whoever volunteers to do a merge personally pester people to fix the resulting test errors forever after. Whatever we do, we're going to have broken layout tests after a merge. Easy "fixes" -- re-baselining the ones that are actually all right, and filing specific bugs for the rest -- are definitely part of the merge task. But we don't want to put too much blame on the messenger: if a bunch of tests really break, or new tests don't pass, it's hardly the fault of the person who happened to bring the changes in. - Pam On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: > I think the merger should be responsible for ensuring that any layout > test fallout from the merge gets resolved. This doesn't mean necessary > fixing everything, but rather it can be mean reaching out to others for > help. > I think the merger has to have incentive not to create a big mess with > the merge and to also make sure the job gets done completely :-) > > -Darin > > > On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: > >> >> If we are ever to keep up to date with layout tests we need to come to >> a consensus on this. Here's the current set of proposals: >> >> 1. The merge becomes a two day activity. First day is merge, second >> day is fixing any failing layout tests. >> 2. We tag team people: first person does merge, next day another >> person fixes any failing layout tests. >> 3. One person for merge (like we do now), and failures are handled by >> owners of the layout tests. This assumes we can identify owners for >> buckets of layout tests so that folks know they are on the spot for a >> failing test. >> >> At this point we only care about regressions since 1.0, but that'll >> change soon. >> >> Can we make a decision on this at next weeks WebKit meeting? >> >> -Scott >> >> On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson >> wrote: >> > >> > I'm currently doing a 2-day merge rotation.As part of this, various >> > layout tests are regressing or getting added that I'm inserting into >> > the tests_fixable list. >> > >> > But, like every other layout test fixer, after my merges are done, >> > I'll finally go back to my old work and not think about it any more. >> > This is how we get a monotonically increasing list of broken tests >> at >> > the end of the tests fixable. I'm pretty certain this happens faster >> > than the tests are getting fixed because nobody wants to fix them. >> I'm >> > sort of tempted to just fix the ones my merge broke now, but that >> will >> > put me behind for my next merge, so I don't do that. >> > >> > I propose a different rotation schedule for WebKit merges. You would >> > do one day of merges, and the next day you would have to clean up >> all >> > the regressed and new layout tests your merge introduced. The layout >> > tests usually aren't that hard, so it normally wouldn't take the >> whole >> > day. This way we can be in a good steady state of WebKit merges. It >> > should also have a big advantage for fixing the broken tests since >> the >> > things that changed, the build numbers involved, etc. are still >> fresh >> > in the merger's head, and it won't be like approaching a random >> layout >> > test from the list with no context. >> > >> > The disadvantage of doing this is that we have to find people to do >> > the merges faster (we should probably formalize the schedule), and >> > you'll lose some advantage the second day of having all the >> > i
[chromium-dev] Re: 2-day merges and the cleanup schedule
I know that I haven't filed chromium bugs for layout test failures in the past because it isn't in the instructions (I thought adding the the tests_fixable was sufficient.) http://dev.chromium.org/developers/how-tos/webkit-merge-1 Though I do usually have at least a partial 3rd day of items for clean up: upstreaming, some times filing bugs for new reliability crashes, etc. Dave On Wed, Mar 4, 2009 at 5:30 PM, Pam Greene wrote: > I'm not sure how closely that's followed. Me, I did merges Monday and > Tuesday, and layout-test cleanup (baselines and bugs) today. > - Pam > > > On Wed, Mar 4, 2009 at 5:10 PM, Darin Fisher wrote: > >> Right. I just mean that the merger should take care to ensure that all of >> the easy "fixes" are done and that the rest are accounted for somehow, >> probably via regression bugs. If that's the current process, then great. >> It just sounded like it wasn't. >> -Darin >> >> >> >> On Wed, Mar 4, 2009 at 4:51 PM, Pam Greene wrote: >> >>> Define "resolved". Is filing and assigning a bug sufficient? It's not >>> efficient to have whoever volunteers to do a merge personally pester people >>> to fix the resulting test errors forever after. >>> Whatever we do, we're going to have broken layout tests after a merge. >>> Easy "fixes" -- re-baselining the ones that are actually all right, and >>> filing specific bugs for the rest -- are definitely part of the merge task. >>> But we don't want to put too much blame on the messenger: if a bunch of >>> tests really break, or new tests don't pass, it's hardly the fault of the >>> person who happened to bring the changes in. >>> >>> - Pam >>> >>> On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: >>> I think the merger should be responsible for ensuring that any layout test fallout from the merge gets resolved. This doesn't mean necessary fixing everything, but rather it can be mean reaching out to others for help. I think the merger has to have incentive not to create a big mess with the merge and to also make sure the job gets done completely :-) -Darin On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: > > If we are ever to keep up to date with layout tests we need to come to > a consensus on this. Here's the current set of proposals: > > 1. The merge becomes a two day activity. First day is merge, second > day is fixing any failing layout tests. > 2. We tag team people: first person does merge, next day another > person fixes any failing layout tests. > 3. One person for merge (like we do now), and failures are handled by > owners of the layout tests. This assumes we can identify owners for > buckets of layout tests so that folks know they are on the spot for a > failing test. > > At this point we only care about regressions since 1.0, but that'll > change soon. > > Can we make a decision on this at next weeks WebKit meeting? > > -Scott > > On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson > wrote: > > > > I'm currently doing a 2-day merge rotation.As part of this, various > > layout tests are regressing or getting added that I'm inserting into > > the tests_fixable list. > > > > But, like every other layout test fixer, after my merges are done, > > I'll finally go back to my old work and not think about it any more. > > This is how we get a monotonically increasing list of broken tests at > > the end of the tests fixable. I'm pretty certain this happens faster > > than the tests are getting fixed because nobody wants to fix them. > I'm > > sort of tempted to just fix the ones my merge broke now, but that > will > > put me behind for my next merge, so I don't do that. > > > > I propose a different rotation schedule for WebKit merges. You would > > do one day of merges, and the next day you would have to clean up all > > the regressed and new layout tests your merge introduced. The layout > > tests usually aren't that hard, so it normally wouldn't take the > whole > > day. This way we can be in a good steady state of WebKit merges. It > > should also have a big advantage for fixing the broken tests since > the > > things that changed, the build numbers involved, etc. are still fresh > > in the merger's head, and it won't be like approaching a random > layout > > test from the list with no context. > > > > The disadvantage of doing this is that we have to find people to do > > the merges faster (we should probably formalize the schedule), and > > you'll lose some advantage the second day of having all the > > instructions and gotchas of the merge fresh in your mind. I was > > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but > > that seems too heavyweight for the people volunteering to do this. > > > > Anybody have any com
[chromium-dev] Re: 2-day merges and the cleanup schedule
I'm not sure how closely that's followed. Me, I did merges Monday and Tuesday, and layout-test cleanup (baselines and bugs) today. - Pam On Wed, Mar 4, 2009 at 5:10 PM, Darin Fisher wrote: > Right. I just mean that the merger should take care to ensure that all of > the easy "fixes" are done and that the rest are accounted for somehow, > probably via regression bugs. If that's the current process, then great. > It just sounded like it wasn't. > -Darin > > > > On Wed, Mar 4, 2009 at 4:51 PM, Pam Greene wrote: > >> Define "resolved". Is filing and assigning a bug sufficient? It's not >> efficient to have whoever volunteers to do a merge personally pester people >> to fix the resulting test errors forever after. >> Whatever we do, we're going to have broken layout tests after a merge. >> Easy "fixes" -- re-baselining the ones that are actually all right, and >> filing specific bugs for the rest -- are definitely part of the merge task. >> But we don't want to put too much blame on the messenger: if a bunch of >> tests really break, or new tests don't pass, it's hardly the fault of the >> person who happened to bring the changes in. >> >> - Pam >> >> On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: >> >>> I think the merger should be responsible for ensuring that any layout >>> test fallout from the merge gets resolved. This doesn't mean necessary >>> fixing everything, but rather it can be mean reaching out to others for >>> help. >>> I think the merger has to have incentive not to create a big mess with >>> the merge and to also make sure the job gets done completely :-) >>> >>> -Darin >>> >>> >>> On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: >>> If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson wrote: > > I'm currently doing a 2-day merge rotation.As part of this, various > layout tests are regressing or getting added that I'm inserting into > the tests_fixable list. > > But, like every other layout test fixer, after my merges are done, > I'll finally go back to my old work and not think about it any more. > This is how we get a monotonically increasing list of broken tests at > the end of the tests fixable. I'm pretty certain this happens faster > than the tests are getting fixed because nobody wants to fix them. I'm > sort of tempted to just fix the ones my merge broke now, but that will > put me behind for my next merge, so I don't do that. > > I propose a different rotation schedule for WebKit merges. You would > do one day of merges, and the next day you would have to clean up all > the regressed and new layout tests your merge introduced. The layout > tests usually aren't that hard, so it normally wouldn't take the whole > day. This way we can be in a good steady state of WebKit merges. It > should also have a big advantage for fixing the broken tests since the > things that changed, the build numbers involved, etc. are still fresh > in the merger's head, and it won't be like approaching a random layout > test from the list with no context. > > The disadvantage of doing this is that we have to find people to do > the merges faster (we should probably formalize the schedule), and > you'll lose some advantage the second day of having all the > instructions and gotchas of the merge fresh in your mind. I was > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but > that seems too heavyweight for the people volunteering to do this. > > Anybody have any comments? > Brett > > > > >>> >>> >>> >>> >> > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
I'm ok with saying that you do the merge one day and have another day to cleanup layout test issues. Once we're not merging anymore though, I think we should just have 1-2 people oncall for 1 week at a time to keep the webkit(webkit.org) build green and to pull a new revision at least once a day (this latter part we can automate pretty easily as well). Ojan On Wed, Mar 4, 2009 at 5:10 PM, Darin Fisher wrote: > Right. I just mean that the merger should take care to ensure that all of > the easy "fixes" are done and that the rest are accounted for somehow, > probably via regression bugs. If that's the current process, then great. > It just sounded like it wasn't. > -Darin > > > > On Wed, Mar 4, 2009 at 4:51 PM, Pam Greene wrote: > >> Define "resolved". Is filing and assigning a bug sufficient? It's not >> efficient to have whoever volunteers to do a merge personally pester people >> to fix the resulting test errors forever after. >> Whatever we do, we're going to have broken layout tests after a merge. >> Easy "fixes" -- re-baselining the ones that are actually all right, and >> filing specific bugs for the rest -- are definitely part of the merge task. >> But we don't want to put too much blame on the messenger: if a bunch of >> tests really break, or new tests don't pass, it's hardly the fault of the >> person who happened to bring the changes in. >> >> - Pam >> >> On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: >> >>> I think the merger should be responsible for ensuring that any layout >>> test fallout from the merge gets resolved. This doesn't mean necessary >>> fixing everything, but rather it can be mean reaching out to others for >>> help. >>> I think the merger has to have incentive not to create a big mess with >>> the merge and to also make sure the job gets done completely :-) >>> >>> -Darin >>> >>> >>> On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: >>> If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson wrote: > > I'm currently doing a 2-day merge rotation.As part of this, various > layout tests are regressing or getting added that I'm inserting into > the tests_fixable list. > > But, like every other layout test fixer, after my merges are done, > I'll finally go back to my old work and not think about it any more. > This is how we get a monotonically increasing list of broken tests at > the end of the tests fixable. I'm pretty certain this happens faster > than the tests are getting fixed because nobody wants to fix them. I'm > sort of tempted to just fix the ones my merge broke now, but that will > put me behind for my next merge, so I don't do that. > > I propose a different rotation schedule for WebKit merges. You would > do one day of merges, and the next day you would have to clean up all > the regressed and new layout tests your merge introduced. The layout > tests usually aren't that hard, so it normally wouldn't take the whole > day. This way we can be in a good steady state of WebKit merges. It > should also have a big advantage for fixing the broken tests since the > things that changed, the build numbers involved, etc. are still fresh > in the merger's head, and it won't be like approaching a random layout > test from the list with no context. > > The disadvantage of doing this is that we have to find people to do > the merges faster (we should probably formalize the schedule), and > you'll lose some advantage the second day of having all the > instructions and gotchas of the merge fresh in your mind. I was > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but > that seems too heavyweight for the people volunteering to do this. > > Anybody have any comments? > Brett > > > > >>> >>> >>> >> > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~
[chromium-dev] Re: 2-day merges and the cleanup schedule
Right. I just mean that the merger should take care to ensure that all of the easy "fixes" are done and that the rest are accounted for somehow, probably via regression bugs. If that's the current process, then great. It just sounded like it wasn't. -Darin On Wed, Mar 4, 2009 at 4:51 PM, Pam Greene wrote: > Define "resolved". Is filing and assigning a bug sufficient? It's not > efficient to have whoever volunteers to do a merge personally pester people > to fix the resulting test errors forever after. > Whatever we do, we're going to have broken layout tests after a merge. Easy > "fixes" -- re-baselining the ones that are actually all right, and filing > specific bugs for the rest -- are definitely part of the merge task. But we > don't want to put too much blame on the messenger: if a bunch of tests > really break, or new tests don't pass, it's hardly the fault of the person > who happened to bring the changes in. > > - Pam > > On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: > >> I think the merger should be responsible for ensuring that any layout test >> fallout from the merge gets resolved. This doesn't mean necessary fixing >> everything, but rather it can be mean reaching out to others for help. >> I think the merger has to have incentive not to create a big mess with the >> merge and to also make sure the job gets done completely :-) >> >> -Darin >> >> >> On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: >> >>> >>> If we are ever to keep up to date with layout tests we need to come to >>> a consensus on this. Here's the current set of proposals: >>> >>> 1. The merge becomes a two day activity. First day is merge, second >>> day is fixing any failing layout tests. >>> 2. We tag team people: first person does merge, next day another >>> person fixes any failing layout tests. >>> 3. One person for merge (like we do now), and failures are handled by >>> owners of the layout tests. This assumes we can identify owners for >>> buckets of layout tests so that folks know they are on the spot for a >>> failing test. >>> >>> At this point we only care about regressions since 1.0, but that'll >>> change soon. >>> >>> Can we make a decision on this at next weeks WebKit meeting? >>> >>> -Scott >>> >>> On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson >>> wrote: >>> > >>> > I'm currently doing a 2-day merge rotation.As part of this, various >>> > layout tests are regressing or getting added that I'm inserting into >>> > the tests_fixable list. >>> > >>> > But, like every other layout test fixer, after my merges are done, >>> > I'll finally go back to my old work and not think about it any more. >>> > This is how we get a monotonically increasing list of broken tests at >>> > the end of the tests fixable. I'm pretty certain this happens faster >>> > than the tests are getting fixed because nobody wants to fix them. I'm >>> > sort of tempted to just fix the ones my merge broke now, but that will >>> > put me behind for my next merge, so I don't do that. >>> > >>> > I propose a different rotation schedule for WebKit merges. You would >>> > do one day of merges, and the next day you would have to clean up all >>> > the regressed and new layout tests your merge introduced. The layout >>> > tests usually aren't that hard, so it normally wouldn't take the whole >>> > day. This way we can be in a good steady state of WebKit merges. It >>> > should also have a big advantage for fixing the broken tests since the >>> > things that changed, the build numbers involved, etc. are still fresh >>> > in the merger's head, and it won't be like approaching a random layout >>> > test from the list with no context. >>> > >>> > The disadvantage of doing this is that we have to find people to do >>> > the merges faster (we should probably formalize the schedule), and >>> > you'll lose some advantage the second day of having all the >>> > instructions and gotchas of the merge fresh in your mind. I was >>> > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but >>> > that seems too heavyweight for the people volunteering to do this. >>> > >>> > Anybody have any comments? >>> > Brett >>> > >>> > > >>> > >>> >>> >>> >> >> >> >> > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
Right. Ultimately, we need to have a buildbot visible on build.webkit.orgthat reports layout test failures to others in the webkit community. Then, we can pester the webkit contributor to help fix the problems they caused. At the very least, we'll have the big advantage of knowing precisely which change caused a problem. This applies not only to layout tests, but also to performance regressions and stability (reliability test runs will be triggered by WebKit commits!). -Darin On Wed, Mar 4, 2009 at 5:05 PM, Ojan Vafai wrote: > The process I was picturing is that we'd have a WebKit sheriff. That person > would be in charge of keeping the build with latest webkit green and > updating the webkit revision we pull once a day. If someone else needs a > newer webkit revision (e.g. something they just committed), they can work > with the WebKit sherriff to pull it in. But basically, it would be that > person's job to either rebaseline failing tests or add them to the fixable > list and assign them to someone to fix. > Ojan > > > On Wed, Mar 4, 2009 at 5:01 PM, David Levin wrote: > >> Yes, so what would be the process then? (Will we still have "mergers" >> whose job is mostly to handle new and modified layout tests?) >> >> On Wed, Mar 4, 2009 at 4:58 PM, Ojan Vafai wrote: >> >>> Even once pulling in a new webkit is as simple as changing a revision >>> number in DEPS, we'll still have new and modified layout tests that we'll >>> need to deal with. >>> >>> >>> On Wed, Mar 4, 2009 at 4:44 PM, David Levin wrote: >>> Aren't we trying to get rid of the merge process (soon-ish)? If so, how would layout tests get fixed at that point? Dave On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: > I think the merger should be responsible for ensuring that any layout > test fallout from the merge gets resolved. This doesn't mean necessary > fixing everything, but rather it can be mean reaching out to others for > help. > I think the merger has to have incentive not to create a big mess with > the merge and to also make sure the job gets done completely :-) > > -Darin > > > On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: > >> >> If we are ever to keep up to date with layout tests we need to come to >> a consensus on this. Here's the current set of proposals: >> >> 1. The merge becomes a two day activity. First day is merge, second >> day is fixing any failing layout tests. >> 2. We tag team people: first person does merge, next day another >> person fixes any failing layout tests. >> 3. One person for merge (like we do now), and failures are handled by >> owners of the layout tests. This assumes we can identify owners for >> buckets of layout tests so that folks know they are on the spot for a >> failing test. >> >> At this point we only care about regressions since 1.0, but that'll >> change soon. >> >> Can we make a decision on this at next weeks WebKit meeting? >> >> -Scott >> >> On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson >> wrote: >> > >> > I'm currently doing a 2-day merge rotation.As part of this, various >> > layout tests are regressing or getting added that I'm inserting into >> > the tests_fixable list. >> > >> > But, like every other layout test fixer, after my merges are done, >> > I'll finally go back to my old work and not think about it any more. >> > This is how we get a monotonically increasing list of broken tests >> at >> > the end of the tests fixable. I'm pretty certain this happens faster >> > than the tests are getting fixed because nobody wants to fix them. >> I'm >> > sort of tempted to just fix the ones my merge broke now, but that >> will >> > put me behind for my next merge, so I don't do that. >> > >> > I propose a different rotation schedule for WebKit merges. You would >> > do one day of merges, and the next day you would have to clean up >> all >> > the regressed and new layout tests your merge introduced. The layout >> > tests usually aren't that hard, so it normally wouldn't take the >> whole >> > day. This way we can be in a good steady state of WebKit merges. It >> > should also have a big advantage for fixing the broken tests since >> the >> > things that changed, the build numbers involved, etc. are still >> fresh >> > in the merger's head, and it won't be like approaching a random >> layout >> > test from the list with no context. >> > >> > The disadvantage of doing this is that we have to find people to do >> > the merges faster (we should probably formalize the schedule), and >> > you'll lose some advantage the second day of having all the >> > instructions and gotchas of the merge fresh in your mind. I was >> > thinking of a 3 day schedule (2 days of merging
[chromium-dev] Re: 2-day merges and the cleanup schedule
The process I was picturing is that we'd have a WebKit sheriff. That person would be in charge of keeping the build with latest webkit green and updating the webkit revision we pull once a day. If someone else needs a newer webkit revision (e.g. something they just committed), they can work with the WebKit sherriff to pull it in. But basically, it would be that person's job to either rebaseline failing tests or add them to the fixable list and assign them to someone to fix. Ojan On Wed, Mar 4, 2009 at 5:01 PM, David Levin wrote: > Yes, so what would be the process then? (Will we still have "mergers" > whose job is mostly to handle new and modified layout tests?) > > On Wed, Mar 4, 2009 at 4:58 PM, Ojan Vafai wrote: > >> Even once pulling in a new webkit is as simple as changing a revision >> number in DEPS, we'll still have new and modified layout tests that we'll >> need to deal with. >> >> >> On Wed, Mar 4, 2009 at 4:44 PM, David Levin wrote: >> >>> Aren't we trying to get rid of the merge process (soon-ish)? >>> If so, how would layout tests get fixed at that point? >>> >>> Dave >>> >>> On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: >>> I think the merger should be responsible for ensuring that any layout test fallout from the merge gets resolved. This doesn't mean necessary fixing everything, but rather it can be mean reaching out to others for help. I think the merger has to have incentive not to create a big mess with the merge and to also make sure the job gets done completely :-) -Darin On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: > > If we are ever to keep up to date with layout tests we need to come to > a consensus on this. Here's the current set of proposals: > > 1. The merge becomes a two day activity. First day is merge, second > day is fixing any failing layout tests. > 2. We tag team people: first person does merge, next day another > person fixes any failing layout tests. > 3. One person for merge (like we do now), and failures are handled by > owners of the layout tests. This assumes we can identify owners for > buckets of layout tests so that folks know they are on the spot for a > failing test. > > At this point we only care about regressions since 1.0, but that'll > change soon. > > Can we make a decision on this at next weeks WebKit meeting? > > -Scott > > On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson > wrote: > > > > I'm currently doing a 2-day merge rotation.As part of this, various > > layout tests are regressing or getting added that I'm inserting into > > the tests_fixable list. > > > > But, like every other layout test fixer, after my merges are done, > > I'll finally go back to my old work and not think about it any more. > > This is how we get a monotonically increasing list of broken tests at > > the end of the tests fixable. I'm pretty certain this happens faster > > than the tests are getting fixed because nobody wants to fix them. > I'm > > sort of tempted to just fix the ones my merge broke now, but that > will > > put me behind for my next merge, so I don't do that. > > > > I propose a different rotation schedule for WebKit merges. You would > > do one day of merges, and the next day you would have to clean up all > > the regressed and new layout tests your merge introduced. The layout > > tests usually aren't that hard, so it normally wouldn't take the > whole > > day. This way we can be in a good steady state of WebKit merges. It > > should also have a big advantage for fixing the broken tests since > the > > things that changed, the build numbers involved, etc. are still fresh > > in the merger's head, and it won't be like approaching a random > layout > > test from the list with no context. > > > > The disadvantage of doing this is that we have to find people to do > > the merges faster (we should probably formalize the schedule), and > > you'll lose some advantage the second day of having all the > > instructions and gotchas of the merge fresh in your mind. I was > > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but > > that seems too heavyweight for the people volunteering to do this. > > > > Anybody have any comments? > > Brett > > > > > > > > > > >>> >>> >>> >>> >> > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
Yes, so what would be the process then? (Will we still have "mergers" whose job is mostly to handle new and modified layout tests?) On Wed, Mar 4, 2009 at 4:58 PM, Ojan Vafai wrote: > Even once pulling in a new webkit is as simple as changing a revision > number in DEPS, we'll still have new and modified layout tests that we'll > need to deal with. > > > On Wed, Mar 4, 2009 at 4:44 PM, David Levin wrote: > >> Aren't we trying to get rid of the merge process (soon-ish)? >> If so, how would layout tests get fixed at that point? >> >> Dave >> >> On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: >> >>> I think the merger should be responsible for ensuring that any layout >>> test fallout from the merge gets resolved. This doesn't mean necessary >>> fixing everything, but rather it can be mean reaching out to others for >>> help. >>> I think the merger has to have incentive not to create a big mess with >>> the merge and to also make sure the job gets done completely :-) >>> >>> -Darin >>> >>> >>> On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: >>> If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson wrote: > > I'm currently doing a 2-day merge rotation.As part of this, various > layout tests are regressing or getting added that I'm inserting into > the tests_fixable list. > > But, like every other layout test fixer, after my merges are done, > I'll finally go back to my old work and not think about it any more. > This is how we get a monotonically increasing list of broken tests at > the end of the tests fixable. I'm pretty certain this happens faster > than the tests are getting fixed because nobody wants to fix them. I'm > sort of tempted to just fix the ones my merge broke now, but that will > put me behind for my next merge, so I don't do that. > > I propose a different rotation schedule for WebKit merges. You would > do one day of merges, and the next day you would have to clean up all > the regressed and new layout tests your merge introduced. The layout > tests usually aren't that hard, so it normally wouldn't take the whole > day. This way we can be in a good steady state of WebKit merges. It > should also have a big advantage for fixing the broken tests since the > things that changed, the build numbers involved, etc. are still fresh > in the merger's head, and it won't be like approaching a random layout > test from the list with no context. > > The disadvantage of doing this is that we have to find people to do > the merges faster (we should probably formalize the schedule), and > you'll lose some advantage the second day of having all the > instructions and gotchas of the merge fresh in your mind. I was > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but > that seems too heavyweight for the people volunteering to do this. > > Anybody have any comments? > Brett > > > > >>> >>> >>> >> >> >> >> > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
Even once pulling in a new webkit is as simple as changing a revision number in DEPS, we'll still have new and modified layout tests that we'll need to deal with. On Wed, Mar 4, 2009 at 4:44 PM, David Levin wrote: > Aren't we trying to get rid of the merge process (soon-ish)? > If so, how would layout tests get fixed at that point? > > Dave > > On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: > >> I think the merger should be responsible for ensuring that any layout test >> fallout from the merge gets resolved. This doesn't mean necessary fixing >> everything, but rather it can be mean reaching out to others for help. >> I think the merger has to have incentive not to create a big mess with the >> merge and to also make sure the job gets done completely :-) >> >> -Darin >> >> >> On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: >> >>> >>> If we are ever to keep up to date with layout tests we need to come to >>> a consensus on this. Here's the current set of proposals: >>> >>> 1. The merge becomes a two day activity. First day is merge, second >>> day is fixing any failing layout tests. >>> 2. We tag team people: first person does merge, next day another >>> person fixes any failing layout tests. >>> 3. One person for merge (like we do now), and failures are handled by >>> owners of the layout tests. This assumes we can identify owners for >>> buckets of layout tests so that folks know they are on the spot for a >>> failing test. >>> >>> At this point we only care about regressions since 1.0, but that'll >>> change soon. >>> >>> Can we make a decision on this at next weeks WebKit meeting? >>> >>> -Scott >>> >>> On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson >>> wrote: >>> > >>> > I'm currently doing a 2-day merge rotation.As part of this, various >>> > layout tests are regressing or getting added that I'm inserting into >>> > the tests_fixable list. >>> > >>> > But, like every other layout test fixer, after my merges are done, >>> > I'll finally go back to my old work and not think about it any more. >>> > This is how we get a monotonically increasing list of broken tests at >>> > the end of the tests fixable. I'm pretty certain this happens faster >>> > than the tests are getting fixed because nobody wants to fix them. I'm >>> > sort of tempted to just fix the ones my merge broke now, but that will >>> > put me behind for my next merge, so I don't do that. >>> > >>> > I propose a different rotation schedule for WebKit merges. You would >>> > do one day of merges, and the next day you would have to clean up all >>> > the regressed and new layout tests your merge introduced. The layout >>> > tests usually aren't that hard, so it normally wouldn't take the whole >>> > day. This way we can be in a good steady state of WebKit merges. It >>> > should also have a big advantage for fixing the broken tests since the >>> > things that changed, the build numbers involved, etc. are still fresh >>> > in the merger's head, and it won't be like approaching a random layout >>> > test from the list with no context. >>> > >>> > The disadvantage of doing this is that we have to find people to do >>> > the merges faster (we should probably formalize the schedule), and >>> > you'll lose some advantage the second day of having all the >>> > instructions and gotchas of the merge fresh in your mind. I was >>> > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but >>> > that seems too heavyweight for the people volunteering to do this. >>> > >>> > Anybody have any comments? >>> > Brett >>> > >>> > > >>> > >>> >>> >>> >> >> >> > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
Define "resolved". Is filing and assigning a bug sufficient? It's not efficient to have whoever volunteers to do a merge personally pester people to fix the resulting test errors forever after. Whatever we do, we're going to have broken layout tests after a merge. Easy "fixes" -- re-baselining the ones that are actually all right, and filing specific bugs for the rest -- are definitely part of the merge task. But we don't want to put too much blame on the messenger: if a bunch of tests really break, or new tests don't pass, it's hardly the fault of the person who happened to bring the changes in. - Pam On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: > I think the merger should be responsible for ensuring that any layout test > fallout from the merge gets resolved. This doesn't mean necessary fixing > everything, but rather it can be mean reaching out to others for help. > I think the merger has to have incentive not to create a big mess with the > merge and to also make sure the job gets done completely :-) > > -Darin > > > On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: > >> >> If we are ever to keep up to date with layout tests we need to come to >> a consensus on this. Here's the current set of proposals: >> >> 1. The merge becomes a two day activity. First day is merge, second >> day is fixing any failing layout tests. >> 2. We tag team people: first person does merge, next day another >> person fixes any failing layout tests. >> 3. One person for merge (like we do now), and failures are handled by >> owners of the layout tests. This assumes we can identify owners for >> buckets of layout tests so that folks know they are on the spot for a >> failing test. >> >> At this point we only care about regressions since 1.0, but that'll change >> soon. >> >> Can we make a decision on this at next weeks WebKit meeting? >> >> -Scott >> >> On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson >> wrote: >> > >> > I'm currently doing a 2-day merge rotation.As part of this, various >> > layout tests are regressing or getting added that I'm inserting into >> > the tests_fixable list. >> > >> > But, like every other layout test fixer, after my merges are done, >> > I'll finally go back to my old work and not think about it any more. >> > This is how we get a monotonically increasing list of broken tests at >> > the end of the tests fixable. I'm pretty certain this happens faster >> > than the tests are getting fixed because nobody wants to fix them. I'm >> > sort of tempted to just fix the ones my merge broke now, but that will >> > put me behind for my next merge, so I don't do that. >> > >> > I propose a different rotation schedule for WebKit merges. You would >> > do one day of merges, and the next day you would have to clean up all >> > the regressed and new layout tests your merge introduced. The layout >> > tests usually aren't that hard, so it normally wouldn't take the whole >> > day. This way we can be in a good steady state of WebKit merges. It >> > should also have a big advantage for fixing the broken tests since the >> > things that changed, the build numbers involved, etc. are still fresh >> > in the merger's head, and it won't be like approaching a random layout >> > test from the list with no context. >> > >> > The disadvantage of doing this is that we have to find people to do >> > the merges faster (we should probably formalize the schedule), and >> > you'll lose some advantage the second day of having all the >> > instructions and gotchas of the merge fresh in your mind. I was >> > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but >> > that seems too heavyweight for the people volunteering to do this. >> > >> > Anybody have any comments? >> > Brett >> > >> > > >> > >> >> >> > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
Aren't we trying to get rid of the merge process (soon-ish)? If so, how would layout tests get fixed at that point? Dave On Wed, Mar 4, 2009 at 3:18 PM, Darin Fisher wrote: > I think the merger should be responsible for ensuring that any layout test > fallout from the merge gets resolved. This doesn't mean necessary fixing > everything, but rather it can be mean reaching out to others for help. > I think the merger has to have incentive not to create a big mess with the > merge and to also make sure the job gets done completely :-) > > -Darin > > > On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: > >> >> If we are ever to keep up to date with layout tests we need to come to >> a consensus on this. Here's the current set of proposals: >> >> 1. The merge becomes a two day activity. First day is merge, second >> day is fixing any failing layout tests. >> 2. We tag team people: first person does merge, next day another >> person fixes any failing layout tests. >> 3. One person for merge (like we do now), and failures are handled by >> owners of the layout tests. This assumes we can identify owners for >> buckets of layout tests so that folks know they are on the spot for a >> failing test. >> >> At this point we only care about regressions since 1.0, but that'll change >> soon. >> >> Can we make a decision on this at next weeks WebKit meeting? >> >> -Scott >> >> On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson >> wrote: >> > >> > I'm currently doing a 2-day merge rotation.As part of this, various >> > layout tests are regressing or getting added that I'm inserting into >> > the tests_fixable list. >> > >> > But, like every other layout test fixer, after my merges are done, >> > I'll finally go back to my old work and not think about it any more. >> > This is how we get a monotonically increasing list of broken tests at >> > the end of the tests fixable. I'm pretty certain this happens faster >> > than the tests are getting fixed because nobody wants to fix them. I'm >> > sort of tempted to just fix the ones my merge broke now, but that will >> > put me behind for my next merge, so I don't do that. >> > >> > I propose a different rotation schedule for WebKit merges. You would >> > do one day of merges, and the next day you would have to clean up all >> > the regressed and new layout tests your merge introduced. The layout >> > tests usually aren't that hard, so it normally wouldn't take the whole >> > day. This way we can be in a good steady state of WebKit merges. It >> > should also have a big advantage for fixing the broken tests since the >> > things that changed, the build numbers involved, etc. are still fresh >> > in the merger's head, and it won't be like approaching a random layout >> > test from the list with no context. >> > >> > The disadvantage of doing this is that we have to find people to do >> > the merges faster (we should probably formalize the schedule), and >> > you'll lose some advantage the second day of having all the >> > instructions and gotchas of the merge fresh in your mind. I was >> > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but >> > that seems too heavyweight for the people volunteering to do this. >> > >> > Anybody have any comments? >> > Brett >> > >> > > >> > >> >> >> > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
I think the merger should be responsible for ensuring that any layout test fallout from the merge gets resolved. This doesn't mean necessary fixing everything, but rather it can be mean reaching out to others for help. I think the merger has to have incentive not to create a big mess with the merge and to also make sure the job gets done completely :-) -Darin On Wed, Mar 4, 2009 at 10:06 AM, Scott Violet wrote: > > If we are ever to keep up to date with layout tests we need to come to > a consensus on this. Here's the current set of proposals: > > 1. The merge becomes a two day activity. First day is merge, second > day is fixing any failing layout tests. > 2. We tag team people: first person does merge, next day another > person fixes any failing layout tests. > 3. One person for merge (like we do now), and failures are handled by > owners of the layout tests. This assumes we can identify owners for > buckets of layout tests so that folks know they are on the spot for a > failing test. > > At this point we only care about regressions since 1.0, but that'll change > soon. > > Can we make a decision on this at next weeks WebKit meeting? > > -Scott > > On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson wrote: > > > > I'm currently doing a 2-day merge rotation.As part of this, various > > layout tests are regressing or getting added that I'm inserting into > > the tests_fixable list. > > > > But, like every other layout test fixer, after my merges are done, > > I'll finally go back to my old work and not think about it any more. > > This is how we get a monotonically increasing list of broken tests at > > the end of the tests fixable. I'm pretty certain this happens faster > > than the tests are getting fixed because nobody wants to fix them. I'm > > sort of tempted to just fix the ones my merge broke now, but that will > > put me behind for my next merge, so I don't do that. > > > > I propose a different rotation schedule for WebKit merges. You would > > do one day of merges, and the next day you would have to clean up all > > the regressed and new layout tests your merge introduced. The layout > > tests usually aren't that hard, so it normally wouldn't take the whole > > day. This way we can be in a good steady state of WebKit merges. It > > should also have a big advantage for fixing the broken tests since the > > things that changed, the build numbers involved, etc. are still fresh > > in the merger's head, and it won't be like approaching a random layout > > test from the list with no context. > > > > The disadvantage of doing this is that we have to find people to do > > the merges faster (we should probably formalize the schedule), and > > you'll lose some advantage the second day of having all the > > instructions and gotchas of the merge fresh in your mind. I was > > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but > > that seems too heavyweight for the people volunteering to do this. > > > > Anybody have any comments? > > Brett > > > > > > > > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
If we are ever to keep up to date with layout tests we need to come to a consensus on this. Here's the current set of proposals: 1. The merge becomes a two day activity. First day is merge, second day is fixing any failing layout tests. 2. We tag team people: first person does merge, next day another person fixes any failing layout tests. 3. One person for merge (like we do now), and failures are handled by owners of the layout tests. This assumes we can identify owners for buckets of layout tests so that folks know they are on the spot for a failing test. At this point we only care about regressions since 1.0, but that'll change soon. Can we make a decision on this at next weeks WebKit meeting? -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson wrote: > > I'm currently doing a 2-day merge rotation.As part of this, various > layout tests are regressing or getting added that I'm inserting into > the tests_fixable list. > > But, like every other layout test fixer, after my merges are done, > I'll finally go back to my old work and not think about it any more. > This is how we get a monotonically increasing list of broken tests at > the end of the tests fixable. I'm pretty certain this happens faster > than the tests are getting fixed because nobody wants to fix them. I'm > sort of tempted to just fix the ones my merge broke now, but that will > put me behind for my next merge, so I don't do that. > > I propose a different rotation schedule for WebKit merges. You would > do one day of merges, and the next day you would have to clean up all > the regressed and new layout tests your merge introduced. The layout > tests usually aren't that hard, so it normally wouldn't take the whole > day. This way we can be in a good steady state of WebKit merges. It > should also have a big advantage for fixing the broken tests since the > things that changed, the build numbers involved, etc. are still fresh > in the merger's head, and it won't be like approaching a random layout > test from the list with no context. > > The disadvantage of doing this is that we have to find people to do > the merges faster (we should probably formalize the schedule), and > you'll lose some advantage the second day of having all the > instructions and gotchas of the merge fresh in your mind. I was > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but > that seems too heavyweight for the people volunteering to do this. > > Anybody have any comments? > Brett > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
This is veering wildly off-topic, but I think the key to solving merge regressions is in moving to an integration model. With the integration model, we can integrate one WebKit changeset at a time, and clearly identify the regressions. This would go a long way in identifying the cause and addressing the issue. Unfortunately, currently we have engineers fixing layout tests and making changes to our WebKit fork, which makes integration model ... complicated. What if we go with an integration->merge model, which works as follows: 1) we have a separate WebKit git repository (hosted on someone's machine even), which has a branch with all of our changes applied as one (or more) patch. 2) you pull WebKit changes, rebase (in other words, re-apply our change patches), see if the changes broke anything. 3) if they did, identify the changeset that breaks stuff by using git bisect (nifty stuff, I might say), then mark it in text_fixable and file a bug 4) merge integrated changes into main svn repository. ... or something like that? Somebody take this idea and flesh it out to make it excellent! :) :DG< On Thu, Jan 15, 2009 at 2:51 PM, Scott Violet wrote: > > I definitely get that a merge is no trivial amount of work, and my > expectations are probably unrealistic without more than one person > working on a merge at a time. I was hoping for a magic bullet, but > those are only in movies:( > > -Scott > >> PS If anyone not doing merges feels like the people doing merges should do >> more, please volunteer to do a merge so at least you'll know what it is like >> -- Sorry Scott but I'm looking at you :) >> >> On Thu, Jan 15, 2009 at 2:26 PM, Pam Greene wrote: >>> >>> On Thu, Jan 15, 2009 at 1:31 PM, Brett Wilson wrote: On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene wrote: > When fixing layout tests only means re-baselining, that's easy. But > sometimes they break (or new ones fail) for deeper reasons, and the > person > doing the merge may not be the right one to make the fix (or may not be > able > to fix them in one day). So perhaps "clean up" in this context means > "re-baseline if that's all it needs, and file individual bugs on > specific > people for bigger brokenness". I think our tendency to file bugs and forget about it is part of the problem. I am at least as guilty as anybody else. I think the merger should have the responsibility to get their regressions fixed. They will have to talk with some other people to get input. If they aren't the right person to fix a problem, it should be their responsibility as part of the cleanup to make sure that the right person has been assigned to it and is actually working on it. >>> >>> Taking responsibility, check. Making sure someone is assigned, check. >>> Fixing tricky regressions yourself may not be the most efficient use of >>> time, and would be a strong deterrent to volunteer for a merge. Merging >>> isn't much fun; fixing layout tests isn't much fun. Let's spread the pain >>> where suitable, rather than piling it all on the person who volunteers for >>> one part. >>> In fact, I'd ask the merger to fix the "easy" problems (tests changed, new >>> tests need baselines) before committing the merge. >>> When people are assigned merge bugs, they should be treated as important regressions and prioritized over other work. We currently had a whole lot of layout tests bugs filed that are getting no love. The only way to not keep getting behind is to be much more proactive. >>> >>> Absolutely. >>> > Also, to clarify, are you proposing that we only merge every other day, > or > that we have two people assigned each day: one to merge and one to > clean up > the previous day's layout-test breakage? If the latter, we could also > split > the job in the other direction, and have one person merging two days in > a > row and one fixing up the test list both days. I could imagine > people's > tastes running more to one job or the other, and we don't really care > who > does what as long as it gets done. I'm proposing overlapping so we merge every day. I think there is an advantage in having the same person who did the merge do the fixing. This hopefully also makes the merge less tedious since you have different tasks your two days. >>> >>> Sounds fine. People can always trade if they want. >>> - Pam >>> Brett >>> >>> >>> >> >> >> > >> > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
I definitely get that a merge is no trivial amount of work, and my expectations are probably unrealistic without more than one person working on a merge at a time. I was hoping for a magic bullet, but those are only in movies:( -Scott > PS If anyone not doing merges feels like the people doing merges should do > more, please volunteer to do a merge so at least you'll know what it is like > -- Sorry Scott but I'm looking at you :) > > On Thu, Jan 15, 2009 at 2:26 PM, Pam Greene wrote: >> >> On Thu, Jan 15, 2009 at 1:31 PM, Brett Wilson wrote: >>> >>> On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene wrote: >>> > When fixing layout tests only means re-baselining, that's easy. But >>> > sometimes they break (or new ones fail) for deeper reasons, and the >>> > person >>> > doing the merge may not be the right one to make the fix (or may not be >>> > able >>> > to fix them in one day). So perhaps "clean up" in this context means >>> > "re-baseline if that's all it needs, and file individual bugs on >>> > specific >>> > people for bigger brokenness". >>> >>> I think our tendency to file bugs and forget about it is part of the >>> problem. I am at least as guilty as anybody else. I think the merger >>> should have the responsibility to get their regressions fixed. They >>> will have to talk with some other people to get input. If they aren't >>> the right person to fix a problem, it should be their responsibility >>> as part of the cleanup to make sure that the right person has been >>> assigned to it and is actually working on it. >> >> Taking responsibility, check. Making sure someone is assigned, check. >> Fixing tricky regressions yourself may not be the most efficient use of >> time, and would be a strong deterrent to volunteer for a merge. Merging >> isn't much fun; fixing layout tests isn't much fun. Let's spread the pain >> where suitable, rather than piling it all on the person who volunteers for >> one part. >> In fact, I'd ask the merger to fix the "easy" problems (tests changed, new >> tests need baselines) before committing the merge. >> >>> >>> When people are assigned merge bugs, they should be treated as >>> important regressions and prioritized over other work. We currently >>> had a whole lot of layout tests bugs filed that are getting no love. >>> The only way to not keep getting behind is to be much more proactive. >> >> Absolutely. >> >>> >>> > Also, to clarify, are you proposing that we only merge every other day, >>> > or >>> > that we have two people assigned each day: one to merge and one to >>> > clean up >>> > the previous day's layout-test breakage? If the latter, we could also >>> > split >>> > the job in the other direction, and have one person merging two days in >>> > a >>> > row and one fixing up the test list both days. I could imagine >>> > people's >>> > tastes running more to one job or the other, and we don't really care >>> > who >>> > does what as long as it gets done. >>> >>> I'm proposing overlapping so we merge every day. I think there is an >>> advantage in having the same person who did the merge do the fixing. >>> This hopefully also makes the merge less tedious since you have >>> different tasks your two days. >> >> Sounds fine. People can always trade if they want. >> - Pam >> >>> >>> Brett >>> >>> >> >> >> > > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
I don't think this is a good idea. With respect to two days of merging, it was a good thing (for me). For one thing, the first day was an absolute pain and it took me well into the second day to get it checked in. On the other hand, the second merge was a breeze by comparison (no added/remove files, no method name changes needed in own code, etc.) It took me a least three days. After merging various unit test failures got blaimed on the merge so I tracked down those failures and submitted fixes (and they weren't the fault of the merge but instead due to a unit test re-sorting that happened to occur at the same time.) In addition, after the merge, you need to upstream the changes that you had to do back to webkit. Not too hard but it adds time. With respect to layout test, we could do two things: 1. Have a layout test fixers that rotate like merges. That seems like what Brett is proposing anyway but tacking it on to the people doing the merges. 2. Have folks who are familiar with the areas look at the breaks. If the fix is easy, then it should be fast to rebaseline, etc. (If it isn't easy, then the people doing merges, aren't going to do it any faster.) If you had stuck me on fixing layout tests after the merge, I would have spent a day on it and likely accomplished not much (nothing) due to my lack of familiarity with Chromium code (sorry, I mostly work on WebKit). Dave PS If anyone not doing merges feels like the people doing merges should do more, please volunteer to do a merge so at least you'll know what it is like -- Sorry Scott but I'm looking at you :) On Thu, Jan 15, 2009 at 2:26 PM, Pam Greene wrote: > On Thu, Jan 15, 2009 at 1:31 PM, Brett Wilson wrote: > >> >> On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene wrote: >> > When fixing layout tests only means re-baselining, that's easy. But >> > sometimes they break (or new ones fail) for deeper reasons, and the >> person >> > doing the merge may not be the right one to make the fix (or may not be >> able >> > to fix them in one day). So perhaps "clean up" in this context means >> > "re-baseline if that's all it needs, and file individual bugs on >> specific >> > people for bigger brokenness". >> >> I think our tendency to file bugs and forget about it is part of the >> problem. I am at least as guilty as anybody else. I think the merger >> should have the responsibility to get their regressions fixed. They >> will have to talk with some other people to get input. If they aren't >> the right person to fix a problem, it should be their responsibility >> as part of the cleanup to make sure that the right person has been >> assigned to it and is actually working on it. > > > Taking responsibility, check. Making sure someone is assigned, check. > Fixing tricky regressions yourself may not be the most efficient use of > time, and would be a strong deterrent to volunteer for a merge. Merging > isn't much fun; fixing layout tests isn't much fun. Let's spread the pain > where suitable, rather than piling it all on the person who volunteers for > one part. > > In fact, I'd ask the merger to fix the "easy" problems (tests changed, new > tests need baselines) before committing the merge. > > >> When people are assigned merge bugs, they should be treated as >> important regressions and prioritized over other work. We currently >> had a whole lot of layout tests bugs filed that are getting no love. >> The only way to not keep getting behind is to be much more proactive. > > > Absolutely. > > >> > Also, to clarify, are you proposing that we only merge every other day, >> or >> > that we have two people assigned each day: one to merge and one to clean >> up >> > the previous day's layout-test breakage? If the latter, we could also >> split >> > the job in the other direction, and have one person merging two days in >> a >> > row and one fixing up the test list both days. I could imagine people's >> > tastes running more to one job or the other, and we don't really care >> who >> > does what as long as it gets done. >> >> I'm proposing overlapping so we merge every day. I think there is an >> advantage in having the same person who did the merge do the fixing. >> This hopefully also makes the merge less tedious since you have >> different tasks your two days. > > > Sounds fine. People can always trade if they want. > > - Pam > > >> Brett >> >> >> > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
On Thu, Jan 15, 2009 at 1:31 PM, Brett Wilson wrote: > > On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene wrote: > > When fixing layout tests only means re-baselining, that's easy. But > > sometimes they break (or new ones fail) for deeper reasons, and the > person > > doing the merge may not be the right one to make the fix (or may not be > able > > to fix them in one day). So perhaps "clean up" in this context means > > "re-baseline if that's all it needs, and file individual bugs on specific > > people for bigger brokenness". > > I think our tendency to file bugs and forget about it is part of the > problem. I am at least as guilty as anybody else. I think the merger > should have the responsibility to get their regressions fixed. They > will have to talk with some other people to get input. If they aren't > the right person to fix a problem, it should be their responsibility > as part of the cleanup to make sure that the right person has been > assigned to it and is actually working on it. Taking responsibility, check. Making sure someone is assigned, check. Fixing tricky regressions yourself may not be the most efficient use of time, and would be a strong deterrent to volunteer for a merge. Merging isn't much fun; fixing layout tests isn't much fun. Let's spread the pain where suitable, rather than piling it all on the person who volunteers for one part. In fact, I'd ask the merger to fix the "easy" problems (tests changed, new tests need baselines) before committing the merge. > When people are assigned merge bugs, they should be treated as > important regressions and prioritized over other work. We currently > had a whole lot of layout tests bugs filed that are getting no love. > The only way to not keep getting behind is to be much more proactive. Absolutely. > > Also, to clarify, are you proposing that we only merge every other day, > or > > that we have two people assigned each day: one to merge and one to clean > up > > the previous day's layout-test breakage? If the latter, we could also > split > > the job in the other direction, and have one person merging two days in a > > row and one fixing up the test list both days. I could imagine people's > > tastes running more to one job or the other, and we don't really care who > > does what as long as it gets done. > > I'm proposing overlapping so we merge every day. I think there is an > advantage in having the same person who did the merge do the fixing. > This hopefully also makes the merge less tedious since you have > different tasks your two days. Sounds fine. People can always trade if they want. - Pam > Brett > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
Generally +1, except I just imagined the situation where the merger collides with the fixer. So maybe no overlapping, just do the merge every other day? :DG< On Thu, Jan 15, 2009 at 1:31 PM, Brett Wilson wrote: > > On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene wrote: >> When fixing layout tests only means re-baselining, that's easy. But >> sometimes they break (or new ones fail) for deeper reasons, and the person >> doing the merge may not be the right one to make the fix (or may not be able >> to fix them in one day). So perhaps "clean up" in this context means >> "re-baseline if that's all it needs, and file individual bugs on specific >> people for bigger brokenness". > > I think our tendency to file bugs and forget about it is part of the > problem. I am at least as guilty as anybody else. I think the merger > should have the responsibility to get their regressions fixed. They > will have to talk with some other people to get input. If they aren't > the right person to fix a problem, it should be their responsibility > as part of the cleanup to make sure that the right person has been > assigned to it and is actually working on it. > > When people are assigned merge bugs, they should be treated as > important regressions and prioritized over other work. We currently > had a whole lot of layout tests bugs filed that are getting no love. > The only way to not keep getting behind is to be much more proactive. > > >> Also, to clarify, are you proposing that we only merge every other day, or >> that we have two people assigned each day: one to merge and one to clean up >> the previous day's layout-test breakage? If the latter, we could also split >> the job in the other direction, and have one person merging two days in a >> row and one fixing up the test list both days. I could imagine people's >> tastes running more to one job or the other, and we don't really care who >> does what as long as it gets done. > > I'm proposing overlapping so we merge every day. I think there is an > advantage in having the same person who did the merge do the fixing. > This hopefully also makes the merge less tedious since you have > different tasks your two days. > > Brett > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
On Thu, Jan 15, 2009 at 1:20 PM, Pam Greene wrote: > When fixing layout tests only means re-baselining, that's easy. But > sometimes they break (or new ones fail) for deeper reasons, and the person > doing the merge may not be the right one to make the fix (or may not be able > to fix them in one day). So perhaps "clean up" in this context means > "re-baseline if that's all it needs, and file individual bugs on specific > people for bigger brokenness". I think our tendency to file bugs and forget about it is part of the problem. I am at least as guilty as anybody else. I think the merger should have the responsibility to get their regressions fixed. They will have to talk with some other people to get input. If they aren't the right person to fix a problem, it should be their responsibility as part of the cleanup to make sure that the right person has been assigned to it and is actually working on it. When people are assigned merge bugs, they should be treated as important regressions and prioritized over other work. We currently had a whole lot of layout tests bugs filed that are getting no love. The only way to not keep getting behind is to be much more proactive. > Also, to clarify, are you proposing that we only merge every other day, or > that we have two people assigned each day: one to merge and one to clean up > the previous day's layout-test breakage? If the latter, we could also split > the job in the other direction, and have one person merging two days in a > row and one fixing up the test list both days. I could imagine people's > tastes running more to one job or the other, and we don't really care who > does what as long as it gets done. I'm proposing overlapping so we merge every day. I think there is an advantage in having the same person who did the merge do the fixing. This hopefully also makes the merge less tedious since you have different tasks your two days. Brett --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
As someone sitting on the other side only fixing layout tests this sounds great. The current mode of operation where a merge is landed and others are left to fix the new failures is very disheartening in that we'll make progress only to have a new merge land breaking the progress we made. I now know how a mailman feels! -Scott On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson wrote: > > I'm currently doing a 2-day merge rotation.As part of this, various > layout tests are regressing or getting added that I'm inserting into > the tests_fixable list. > > But, like every other layout test fixer, after my merges are done, > I'll finally go back to my old work and not think about it any more. > This is how we get a monotonically increasing list of broken tests at > the end of the tests fixable. I'm pretty certain this happens faster > than the tests are getting fixed because nobody wants to fix them. I'm > sort of tempted to just fix the ones my merge broke now, but that will > put me behind for my next merge, so I don't do that. > > I propose a different rotation schedule for WebKit merges. You would > do one day of merges, and the next day you would have to clean up all > the regressed and new layout tests your merge introduced. The layout > tests usually aren't that hard, so it normally wouldn't take the whole > day. This way we can be in a good steady state of WebKit merges. It > should also have a big advantage for fixing the broken tests since the > things that changed, the build numbers involved, etc. are still fresh > in the merger's head, and it won't be like approaching a random layout > test from the list with no context. > > The disadvantage of doing this is that we have to find people to do > the merges faster (we should probably formalize the schedule), and > you'll lose some advantage the second day of having all the > instructions and gotchas of the merge fresh in your mind. I was > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but > that seems too heavyweight for the people volunteering to do this. > > Anybody have any comments? > Brett > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: 2-day merges and the cleanup schedule
When fixing layout tests only means re-baselining, that's easy. But sometimes they break (or new ones fail) for deeper reasons, and the person doing the merge may not be the right one to make the fix (or may not be able to fix them in one day). So perhaps "clean up" in this context means "re-baseline if that's all it needs, and file individual bugs on specific people for bigger brokenness". Also, to clarify, are you proposing that we only merge every other day, or that we have two people assigned each day: one to merge and one to clean up the previous day's layout-test breakage? If the latter, we could also split the job in the other direction, and have one person merging two days in a row and one fixing up the test list both days. I could imagine people's tastes running more to one job or the other, and we don't really care who does what as long as it gets done. - Pam On Thu, Jan 15, 2009 at 1:03 PM, Brett Wilson wrote: > > I'm currently doing a 2-day merge rotation.As part of this, various > layout tests are regressing or getting added that I'm inserting into > the tests_fixable list. > > But, like every other layout test fixer, after my merges are done, > I'll finally go back to my old work and not think about it any more. > This is how we get a monotonically increasing list of broken tests at > the end of the tests fixable. I'm pretty certain this happens faster > than the tests are getting fixed because nobody wants to fix them. I'm > sort of tempted to just fix the ones my merge broke now, but that will > put me behind for my next merge, so I don't do that. > > I propose a different rotation schedule for WebKit merges. You would > do one day of merges, and the next day you would have to clean up all > the regressed and new layout tests your merge introduced. The layout > tests usually aren't that hard, so it normally wouldn't take the whole > day. This way we can be in a good steady state of WebKit merges. It > should also have a big advantage for fixing the broken tests since the > things that changed, the build numbers involved, etc. are still fresh > in the merger's head, and it won't be like approaching a random layout > test from the list with no context. > > The disadvantage of doing this is that we have to find people to do > the merges faster (we should probably formalize the schedule), and > you'll lose some advantage the second day of having all the > instructions and gotchas of the merge fresh in your mind. I was > thinking of a 3 day schedule (2 days of merging, 1 day of fixing) but > that seems too heavyweight for the people volunteering to do this. > > Anybody have any comments? > Brett > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
