Re: [openstack-dev] revert hacking to 0.8 series
Another idea I had: It appears like a lot of the current hacking issues can be automatically fixed. So it would be really nice if the hacking tool included a fix for most of the problems it finds. I started https://review.openstack.org/#/c/68988/ but don't have the time to do a full tool, but it would be nice if the hacking tool/proposal bot/other would propose fixes by itself for most of the 'autofixable' issues (like imports being in the wrong order or not in the right grouping). That'd help in avoiding a ton of less useful commits to fix those same issues manually. -Josh -Original Message- From: Sean Dague Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Monday, June 16, 2014 at 5:15 AM To: "OpenStack Development Mailing List (not for usage questions)" Subject: [openstack-dev] revert hacking to 0.8 series >Hacking 0.9 series was released pretty late for Juno. The entire check >queue was flooded this morning with requirements proposals failing pep8 >because of it (so at 6am EST we were waiting 1.5 hrs for a check node). > >The previous soft policy with pep8 updates was that we set a pep8 >version basically release week, and changes stopped being done for style >after first milestone. > >I think in the spirit of that we should revert the hacking requirements >update back to the 0.8 series for Juno. We're past milestone 1, so >shouldn't be working on style only fixes at this point. > >Proposed review here - https://review.openstack.org/#/c/100231/ > >I also think in future hacking major releases need to happen within one >week of release, or not at all for that series. > > -Sean > >-- >Sean Dague >http://dague.net > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] revert hacking to 0.8 series
On Mon, Jun 16, 2014 at 2:54 PM, Angus Salkeld wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 17/06/14 02:46, Ben Nemec wrote: > > On 06/16/2014 08:37 AM, Thierry Carrez wrote: > >> Sean Dague wrote: > >>> Hacking 0.9 series was released pretty late for Juno. The entire > >>> check queue was flooded this morning with requirements proposals > >>> failing pep8 because of it (so at 6am EST we were waiting 1.5 hrs > >>> for a check node). > >>> > >>> The previous soft policy with pep8 updates was that we set a > >>> pep8 version basically release week, and changes stopped being > >>> done for style after first milestone. > >>> > >>> I think in the spirit of that we should revert the hacking > >>> requirements update back to the 0.8 series for Juno. We're past > >>> milestone 1, so shouldn't be working on style only fixes at this > >>> point. > >>> > >>> Proposed review here - https://review.openstack.org/#/c/100231/ > >>> > >>> I also think in future hacking major releases need to happen > >>> within one week of release, or not at all for that series. > > > >> We may also have reached a size where changing style rules is just > >> too costly, whatever the moment in the cycle. I think it's good > >> that we have rules to enforce a minimum of common style, but the > >> added value of those extra rules is limited, while their impact on > >> the common gate grows as we add more projects. > > > > A few thoughts: > > > > 1) I disagree with the proposition that hacking updates can only > > happen in the first week after release. I get that there needs to be > > a cutoff, but I don't think one week is reasonable. Even if we > > release in the first week, you're still going to be dealing with > > hacking updates for the rest of the cycle as projects adopt the new > > rules at their leisure. I don't like retroactively applying milestone > > 1 as a cutoff either, although I could see making that the policy > > going forward. > > Can't we move to a mode of enabling rules instead of ignoring them? > If we did this in tox.ini then it wouldn't matter when you release > hacking. > > [hacking] > errors = H306,... > ignore = H101 > > So if you upgraded hacking you would not get the new checks generating > errors, but only warnings. > This is out side the scope of hacking in and in the domain of the flake8 project, although we can easily contribute upstream to it. As for warnings, I assume you mean we log the issue but don't gate on it, I don't think anyone would pay attention to them, so not sure what the value is. > > I guess the list of rules to error on would get big, but maybe we could > have > some short cuts (H3*,H2*)? > > At least the projects are a bit more in control of what rule they add. > - -Angus > > > > > 2) Given that most of the changes involved in fixing the new failures > > are trivial, I think we should encourage combining the fixes into one > > commit. We _really_ don't need separate commits to fix H305 and H307. > > This doesn't help much with the reviewer load, but it should reduce > > the gate load somewhat. It violates the one change-one commit rule, > > but "A foolish consistency..." > > > > 3) We should start requiring specs for all new hacking rules to make > > sure we have consensus (I think oslo-specs is the place for this). 2 > > +2's doesn't really accomplish that. We also may need to raise the > > bar for inclusion of new rules - while I agree with all of the new > > ones added in hacking .9, I wonder if some of them are really necessary. > > > > 4) I don't think we're at a point where we should freeze hacking > > completely, however. The import grouping and long line wrapping > > checks in particular are things that reviewers have to enforce today, > > and that has a significant, if less well-defined, cost too. If we're > > really going to say those rules can't be enforced by hacking then we > > need to remove them from our hacking guidelines and start the long > > process of educating reviewers to stop requiring them. I'd rather > > just deal with the pain of adding them to hacking one time and never > > have to think about them again. I'm less convinced the other two that > > were added in .9 are necessary, but in any case these are discussions > > that should happen in spec reviews going forward. > > > > 5) We may want to come up with some way to globally disable pep8 > > checks we don't want to enforce, since we don't have any control over > > that but probably don't want to just stop updating pep8. That could > > make the pain of these updates much less. > > > > I could probably come up with a few more, but this is already too > > wall-of-texty for my tastes. :-) > > > > -Ben > > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > -BEGIN PGP SIGNATURE- > Version: GnuPG v1 > Comment: Using GnuPG with T
Re: [openstack-dev] revert hacking to 0.8 series
On Mon, Jun 16, 2014 at 2:56 PM, Doug Hellmann wrote: > On Mon, Jun 16, 2014 at 2:23 PM, Joe Gordon wrote: > > > > On Jun 16, 2014 9:44 AM, "Ben Nemec" wrote: > >> > >> -BEGIN PGP SIGNED MESSAGE- > >> Hash: SHA1 > >> > >> On 06/16/2014 08:37 AM, Thierry Carrez wrote: > >> > Sean Dague wrote: > >> >> Hacking 0.9 series was released pretty late for Juno. The entire > >> >> check queue was flooded this morning with requirements proposals > >> >> failing pep8 because of it (so at 6am EST we were waiting 1.5 hrs > >> >> for a check node). > > > > This is a general issue with global requirements, the number of jobs we > run > > and the number of available nodes. Let's solve the general case. > > > >> >> > >> >> The previous soft policy with pep8 updates was that we set a > >> >> pep8 version basically release week, and changes stopped being > >> >> done for style after first milestone. > >> >> > >> >> I think in the spirit of that we should revert the hacking > >> >> requirements update back to the 0.8 series for Juno. We're past > >> >> milestone 1, so shouldn't be working on style only fixes at this > >> >> point. > >> >> > >> >> Proposed review here - https://review.openstack.org/#/c/100231/ > >> >> > >> >> I also think in future hacking major releases need to happen > >> >> within one week of release, or not at all for that series. > >> > > >> > We may also have reached a size where changing style rules is just > >> > too costly, whatever the moment in the cycle. I think it's good > >> > that we have rules to enforce a minimum of common style, but the > >> > added value of those extra rules is limited, while their impact on > >> > the common gate grows as we add more projects. > >> > >> A few thoughts: > >> > >> 1) I disagree with the proposition that hacking updates can only > >> happen in the first week after release. I get that there needs to be > >> a cutoff, but I don't think one week is reasonable. Even if we > >> release in the first week, you're still going to be dealing with > >> hacking updates for the rest of the cycle as projects adopt the new > >> rules at their leisure. I don't like retroactively applying milestone > >> 1 as a cutoff either, although I could see making that the policy > >> going forward. > >> > > > > ++ > > > >> 2) Given that most of the changes involved in fixing the new failures > >> are trivial, I think we should encourage combining the fixes into one > >> commit. We _really_ don't need separate commits to fix H305 and H307. > >> This doesn't help much with the reviewer load, but it should reduce > >> the gate load somewhat. It violates the one change-one commit rule, > >> but "A foolish consistency..." > >> > > > > ++ > > > >> 3) We should start requiring specs for all new hacking rules to make > >> sure we have consensus (I think oslo-specs is the place for this). 2 > >> +2's doesn't really accomplish that. We also may need to raise the > >> bar for inclusion of new rules - while I agree with all of the new > >> ones added in hacking .9, I wonder if some of them are really necessary. > >> > > > > I would rather just have more folks review hacking patches then add a > specs > > repo. A specs repo is overkill, IMHO, hacking doesn't have that many > patches > > per cycle. In general when adding a rule to hacking it has to already be > in > > HACKING.rst and/or needs a ML post. > > You wouldn't need a new repo. Hacking is part of the Oslo program, so > you would use oslo-specs just like the other Oslo projects. > > The process you describe is what we've done historically, but as we > standardize on specs repos we will want to eventually reconsider it. > If the point of posting to the mailing list is to encourage > discussion, then it seems like that step could be replaced with a > spec. > Perhaps, I still am not sure how that really helps. > > Doug > > > > >> 4) I don't think we're at a point where we should freeze hacking > >> completely, however. The import grouping and long line wrapping > >> checks in particular are things that reviewers have to enforce today, > >> and that has a significant, if less well-defined, cost too. If we're > >> really going to say those rules can't be enforced by hacking then we > >> need to remove them from our hacking guidelines and start the long > >> process of educating reviewers to stop requiring them. I'd rather > >> just deal with the pain of adding them to hacking one time and never > >> have to think about them again. I'm less convinced the other two that > >> were added in .9 are necessary, but in any case these are discussions > >> that should happen in spec reviews going forward. > >> > >> 5) We may want to come up with some way to globally disable pep8 > >> checks we don't want to enforce, since we don't have any control over > >> that but probably don't want to just stop updating pep8. That could > >> make the pain of these updates much less. > >> > >> I could probably come up with a few more, but this i
Re: [openstack-dev] revert hacking to 0.8 series
On 06/16/2014 05:46 PM, Ben Nemec wrote: > On 06/16/2014 04:46 PM, Joe Gordon wrote: >> On Mon, Jun 16, 2014 at 2:33 PM, Sean Dague wrote: >> >>> On 06/16/2014 05:06 PM, Ben Nemec wrote: On 06/16/2014 12:01 PM, Sean Dague wrote: > On 06/16/2014 12:44 PM, Ben Nemec wrote: >> On 06/16/2014 08:37 AM, Thierry Carrez wrote: >>> Sean Dague wrote: Hacking 0.9 series was released pretty late for Juno. The entire check queue was flooded this morning with requirements proposals failing pep8 because of it (so at 6am EST we were waiting 1.5 hrs for a check node). The previous soft policy with pep8 updates was that we set a pep8 version basically release week, and changes stopped being done for style after first milestone. I think in the spirit of that we should revert the hacking requirements update back to the 0.8 series for Juno. We're past milestone 1, so shouldn't be working on style only fixes at this point. Proposed review here - https://review.openstack.org/#/c/100231/ I also think in future hacking major releases need to happen within one week of release, or not at all for that series. >> >>> We may also have reached a size where changing style rules is >>> just too costly, whatever the moment in the cycle. I think it's >>> good that we have rules to enforce a minimum of common style, >>> but the added value of those extra rules is limited, while >>> their impact on the common gate grows as we add more projects. >> >> A few thoughts: >> >> 1) I disagree with the proposition that hacking updates can only >> happen in the first week after release. I get that there needs >> to be a cutoff, but I don't think one week is reasonable. Even >> if we release in the first week, you're still going to be dealing >> with hacking updates for the rest of the cycle as projects adopt >> the new rules at their leisure. I don't like retroactively >> applying milestone 1 as a cutoff either, although I could see >> making that the policy going forward. >> >> 2) Given that most of the changes involved in fixing the new >> failures are trivial, I think we should encourage combining the >> fixes into one commit. We _really_ don't need separate commits >> to fix H305 and H307. This doesn't help much with the reviewer >> load, but it should reduce the gate load somewhat. It violates >> the one change-one commit rule, but "A foolish consistency..." > The challenge is that hacking updates are basically giant merge > conflict engines. If there is any significant amount of code > outstanding in a project, landing hacking only changes basically > means requiring much of the outstanding code to rebase. > So it's actually expensive in a way that doesn't jump out > immediately. The cost of landing hacking isn't just the code of > reviewing the hacking patches, it's also the cost of the extra > roundtrips on outstanding patches. If a project has that many violations of a new hacking rule then I'd say they should just leave it disabled. It already happens and I'd rather have that than throw out a rule just because some projects haven't been following it. >>> >>> That would be fine if the propose bot *also* proposed the tox.ini >>> ignores, which it does not. >>> >>> So it pushes a largely by definition broken patch that consumes >>> resources, and will require a human to take it over. If a person did >>> this on a regular basis, we'd have a stern talking to them. Or at least >>> a light making fun of them. >>> >>> It feels like robo auto propose should be limited to the class of things >>> where the patch is 99% sure to pass. >>> >> >> I feel the same way on this one, for some reason I was originally under the >> impression that hacking had to manually be rolled out because of the >> reasons above. > > I wonder if we should just make future hacking checks opt-in. Normally > I'd be opposed to that because in general people ignore opt-ins, but we > seem to have enough people using style changes as low-hanging-fruit to > get started that it might work in this case. Whoops, forgot to credit Angus for the suggestion: http://lists.openstack.org/pipermail/openstack-dev/2014-June/037877.html > >> >> >>> >>> Which I think is really the crux of my distaste for hacking breaking >>> updates auto proposing to projects. >>> >>> -Sean >>> >>> -- >>> Sean Dague >>> http://dague.net >>> >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.o
Re: [openstack-dev] revert hacking to 0.8 series
On 06/16/2014 04:46 PM, Joe Gordon wrote: > On Mon, Jun 16, 2014 at 2:33 PM, Sean Dague wrote: > >> On 06/16/2014 05:06 PM, Ben Nemec wrote: >>> On 06/16/2014 12:01 PM, Sean Dague wrote: On 06/16/2014 12:44 PM, Ben Nemec wrote: > On 06/16/2014 08:37 AM, Thierry Carrez wrote: >> Sean Dague wrote: >>> Hacking 0.9 series was released pretty late for Juno. The >>> entire check queue was flooded this morning with requirements >>> proposals failing pep8 because of it (so at 6am EST we were >>> waiting 1.5 hrs for a check node). >>> >>> The previous soft policy with pep8 updates was that we set a >>> pep8 version basically release week, and changes stopped >>> being done for style after first milestone. >>> >>> I think in the spirit of that we should revert the hacking >>> requirements update back to the 0.8 series for Juno. We're >>> past milestone 1, so shouldn't be working on style only fixes >>> at this point. >>> >>> Proposed review here - >>> https://review.openstack.org/#/c/100231/ >>> >>> I also think in future hacking major releases need to happen >>> within one week of release, or not at all for that series. > >> We may also have reached a size where changing style rules is >> just too costly, whatever the moment in the cycle. I think it's >> good that we have rules to enforce a minimum of common style, >> but the added value of those extra rules is limited, while >> their impact on the common gate grows as we add more projects. > > A few thoughts: > > 1) I disagree with the proposition that hacking updates can only > happen in the first week after release. I get that there needs > to be a cutoff, but I don't think one week is reasonable. Even > if we release in the first week, you're still going to be dealing > with hacking updates for the rest of the cycle as projects adopt > the new rules at their leisure. I don't like retroactively > applying milestone 1 as a cutoff either, although I could see > making that the policy going forward. > > 2) Given that most of the changes involved in fixing the new > failures are trivial, I think we should encourage combining the > fixes into one commit. We _really_ don't need separate commits > to fix H305 and H307. This doesn't help much with the reviewer > load, but it should reduce the gate load somewhat. It violates > the one change-one commit rule, but "A foolish consistency..." >>> The challenge is that hacking updates are basically giant merge conflict engines. If there is any significant amount of code outstanding in a project, landing hacking only changes basically means requiring much of the outstanding code to rebase. >>> So it's actually expensive in a way that doesn't jump out immediately. The cost of landing hacking isn't just the code of reviewing the hacking patches, it's also the cost of the extra roundtrips on outstanding patches. >>> >>> If a project has that many violations of a new hacking rule then I'd >>> say they should just leave it disabled. It already happens and I'd >>> rather have that than throw out a rule just because some projects >>> haven't been following it. >> >> That would be fine if the propose bot *also* proposed the tox.ini >> ignores, which it does not. >> >> So it pushes a largely by definition broken patch that consumes >> resources, and will require a human to take it over. If a person did >> this on a regular basis, we'd have a stern talking to them. Or at least >> a light making fun of them. >> >> It feels like robo auto propose should be limited to the class of things >> where the patch is 99% sure to pass. >> > > I feel the same way on this one, for some reason I was originally under the > impression that hacking had to manually be rolled out because of the > reasons above. I wonder if we should just make future hacking checks opt-in. Normally I'd be opposed to that because in general people ignore opt-ins, but we seem to have enough people using style changes as low-hanging-fruit to get started that it might work in this case. > > >> >> Which I think is really the crux of my distaste for hacking breaking >> updates auto proposing to projects. >> >> -Sean >> >> -- >> Sean Dague >> http://dague.net >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] revert hacking to 0.8 series
On Mon, Jun 16, 2014 at 2:23 PM, Joe Gordon wrote: > > On Jun 16, 2014 9:44 AM, "Ben Nemec" wrote: >> >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> On 06/16/2014 08:37 AM, Thierry Carrez wrote: >> > Sean Dague wrote: >> >> Hacking 0.9 series was released pretty late for Juno. The entire >> >> check queue was flooded this morning with requirements proposals >> >> failing pep8 because of it (so at 6am EST we were waiting 1.5 hrs >> >> for a check node). > > This is a general issue with global requirements, the number of jobs we run > and the number of available nodes. Let's solve the general case. > >> >> >> >> The previous soft policy with pep8 updates was that we set a >> >> pep8 version basically release week, and changes stopped being >> >> done for style after first milestone. >> >> >> >> I think in the spirit of that we should revert the hacking >> >> requirements update back to the 0.8 series for Juno. We're past >> >> milestone 1, so shouldn't be working on style only fixes at this >> >> point. >> >> >> >> Proposed review here - https://review.openstack.org/#/c/100231/ >> >> >> >> I also think in future hacking major releases need to happen >> >> within one week of release, or not at all for that series. >> > >> > We may also have reached a size where changing style rules is just >> > too costly, whatever the moment in the cycle. I think it's good >> > that we have rules to enforce a minimum of common style, but the >> > added value of those extra rules is limited, while their impact on >> > the common gate grows as we add more projects. >> >> A few thoughts: >> >> 1) I disagree with the proposition that hacking updates can only >> happen in the first week after release. I get that there needs to be >> a cutoff, but I don't think one week is reasonable. Even if we >> release in the first week, you're still going to be dealing with >> hacking updates for the rest of the cycle as projects adopt the new >> rules at their leisure. I don't like retroactively applying milestone >> 1 as a cutoff either, although I could see making that the policy >> going forward. >> > > ++ > >> 2) Given that most of the changes involved in fixing the new failures >> are trivial, I think we should encourage combining the fixes into one >> commit. We _really_ don't need separate commits to fix H305 and H307. >> This doesn't help much with the reviewer load, but it should reduce >> the gate load somewhat. It violates the one change-one commit rule, >> but "A foolish consistency..." >> > > ++ > >> 3) We should start requiring specs for all new hacking rules to make >> sure we have consensus (I think oslo-specs is the place for this). 2 >> +2's doesn't really accomplish that. We also may need to raise the >> bar for inclusion of new rules - while I agree with all of the new >> ones added in hacking .9, I wonder if some of them are really necessary. >> > > I would rather just have more folks review hacking patches then add a specs > repo. A specs repo is overkill, IMHO, hacking doesn't have that many patches > per cycle. In general when adding a rule to hacking it has to already be in > HACKING.rst and/or needs a ML post. You wouldn't need a new repo. Hacking is part of the Oslo program, so you would use oslo-specs just like the other Oslo projects. The process you describe is what we've done historically, but as we standardize on specs repos we will want to eventually reconsider it. If the point of posting to the mailing list is to encourage discussion, then it seems like that step could be replaced with a spec. Doug > >> 4) I don't think we're at a point where we should freeze hacking >> completely, however. The import grouping and long line wrapping >> checks in particular are things that reviewers have to enforce today, >> and that has a significant, if less well-defined, cost too. If we're >> really going to say those rules can't be enforced by hacking then we >> need to remove them from our hacking guidelines and start the long >> process of educating reviewers to stop requiring them. I'd rather >> just deal with the pain of adding them to hacking one time and never >> have to think about them again. I'm less convinced the other two that >> were added in .9 are necessary, but in any case these are discussions >> that should happen in spec reviews going forward. >> >> 5) We may want to come up with some way to globally disable pep8 >> checks we don't want to enforce, since we don't have any control over >> that but probably don't want to just stop updating pep8. That could >> make the pain of these updates much less. >> >> I could probably come up with a few more, but this is already too >> wall-of-texty for my tastes. :-) >> >> - -Ben >> -BEGIN PGP SIGNATURE- >> Version: GnuPG v1 >> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ >> >> iQEcBAEBAgAGBQJTnx7wAAoJEDehGd0Fy7uqoYAH/0KmxmR873Qn2Kti7LIEUNp4 >> 1FJaBOX09ItxkkvyNRpcsIQu4fWycm60CckSOLfB7rgxgIjgsVkiZ9puE6oCmj2o >
Re: [openstack-dev] revert hacking to 0.8 series
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/06/14 02:46, Ben Nemec wrote: > On 06/16/2014 08:37 AM, Thierry Carrez wrote: >> Sean Dague wrote: >>> Hacking 0.9 series was released pretty late for Juno. The entire >>> check queue was flooded this morning with requirements proposals >>> failing pep8 because of it (so at 6am EST we were waiting 1.5 hrs >>> for a check node). >>> >>> The previous soft policy with pep8 updates was that we set a >>> pep8 version basically release week, and changes stopped being >>> done for style after first milestone. >>> >>> I think in the spirit of that we should revert the hacking >>> requirements update back to the 0.8 series for Juno. We're past >>> milestone 1, so shouldn't be working on style only fixes at this >>> point. >>> >>> Proposed review here - https://review.openstack.org/#/c/100231/ >>> >>> I also think in future hacking major releases need to happen >>> within one week of release, or not at all for that series. > >> We may also have reached a size where changing style rules is just >> too costly, whatever the moment in the cycle. I think it's good >> that we have rules to enforce a minimum of common style, but the >> added value of those extra rules is limited, while their impact on >> the common gate grows as we add more projects. > > A few thoughts: > > 1) I disagree with the proposition that hacking updates can only > happen in the first week after release. I get that there needs to be > a cutoff, but I don't think one week is reasonable. Even if we > release in the first week, you're still going to be dealing with > hacking updates for the rest of the cycle as projects adopt the new > rules at their leisure. I don't like retroactively applying milestone > 1 as a cutoff either, although I could see making that the policy > going forward. Can't we move to a mode of enabling rules instead of ignoring them? If we did this in tox.ini then it wouldn't matter when you release hacking. [hacking] errors = H306,... ignore = H101 So if you upgraded hacking you would not get the new checks generating errors, but only warnings. I guess the list of rules to error on would get big, but maybe we could have some short cuts (H3*,H2*)? At least the projects are a bit more in control of what rule they add. - -Angus > > 2) Given that most of the changes involved in fixing the new failures > are trivial, I think we should encourage combining the fixes into one > commit. We _really_ don't need separate commits to fix H305 and H307. > This doesn't help much with the reviewer load, but it should reduce > the gate load somewhat. It violates the one change-one commit rule, > but "A foolish consistency..." > > 3) We should start requiring specs for all new hacking rules to make > sure we have consensus (I think oslo-specs is the place for this). 2 > +2's doesn't really accomplish that. We also may need to raise the > bar for inclusion of new rules - while I agree with all of the new > ones added in hacking .9, I wonder if some of them are really necessary. > > 4) I don't think we're at a point where we should freeze hacking > completely, however. The import grouping and long line wrapping > checks in particular are things that reviewers have to enforce today, > and that has a significant, if less well-defined, cost too. If we're > really going to say those rules can't be enforced by hacking then we > need to remove them from our hacking guidelines and start the long > process of educating reviewers to stop requiring them. I'd rather > just deal with the pain of adding them to hacking one time and never > have to think about them again. I'm less convinced the other two that > were added in .9 are necessary, but in any case these are discussions > that should happen in spec reviews going forward. > > 5) We may want to come up with some way to globally disable pep8 > checks we don't want to enforce, since we don't have any control over > that but probably don't want to just stop updating pep8. That could > make the pain of these updates much less. > > I could probably come up with a few more, but this is already too > wall-of-texty for my tastes. :-) > > -Ben > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTn2eqAAoJEFrDYBLxZjWo9dkIAJ55WTdVZgIHEFJGp+7Px8jC FxsBzRvDKeDTN6ONXUtE82G10ru6UR0HNndfhgbdEVQSazdcavbd/Q0AG+tmDyaE 7PBUpJ3bVIQVpJQ9tz/Xo4dqvsZhsOZBo28iLJyShU+VYy05I16WCGpsS0NUlD95 ND78vjwUCNnjbzkOgBjt6V0QsuWpEZynIR6PfRkUJaaT+gFtrhAG7n4aQmgYnJri 9huTnEjyyg9KldlinxLxVP9nk2uVoKD7sfDAvREAjstFeRK4tVcdspB6xxPkfTKA RDAG1tGT0yvD3VtgqajFlJvImUyV7YN3/zXyXxKeb0301ouWpFeyiSZ1jqsK6Nc= =/Tmf -END PGP SIGNATURE- ___ OpenStack-dev mailing list Op
Re: [openstack-dev] revert hacking to 0.8 series
On Mon, Jun 16, 2014 at 2:33 PM, Sean Dague wrote: > On 06/16/2014 05:06 PM, Ben Nemec wrote: > > On 06/16/2014 12:01 PM, Sean Dague wrote: > >> On 06/16/2014 12:44 PM, Ben Nemec wrote: > >>> On 06/16/2014 08:37 AM, Thierry Carrez wrote: > Sean Dague wrote: > > Hacking 0.9 series was released pretty late for Juno. The > > entire check queue was flooded this morning with requirements > > proposals failing pep8 because of it (so at 6am EST we were > > waiting 1.5 hrs for a check node). > > > > The previous soft policy with pep8 updates was that we set a > > pep8 version basically release week, and changes stopped > > being done for style after first milestone. > > > > I think in the spirit of that we should revert the hacking > > requirements update back to the 0.8 series for Juno. We're > > past milestone 1, so shouldn't be working on style only fixes > > at this point. > > > > Proposed review here - > > https://review.openstack.org/#/c/100231/ > > > > I also think in future hacking major releases need to happen > > within one week of release, or not at all for that series. > >>> > We may also have reached a size where changing style rules is > just too costly, whatever the moment in the cycle. I think it's > good that we have rules to enforce a minimum of common style, > but the added value of those extra rules is limited, while > their impact on the common gate grows as we add more projects. > >>> > >>> A few thoughts: > >>> > >>> 1) I disagree with the proposition that hacking updates can only > >>> happen in the first week after release. I get that there needs > >>> to be a cutoff, but I don't think one week is reasonable. Even > >>> if we release in the first week, you're still going to be dealing > >>> with hacking updates for the rest of the cycle as projects adopt > >>> the new rules at their leisure. I don't like retroactively > >>> applying milestone 1 as a cutoff either, although I could see > >>> making that the policy going forward. > >>> > >>> 2) Given that most of the changes involved in fixing the new > >>> failures are trivial, I think we should encourage combining the > >>> fixes into one commit. We _really_ don't need separate commits > >>> to fix H305 and H307. This doesn't help much with the reviewer > >>> load, but it should reduce the gate load somewhat. It violates > >>> the one change-one commit rule, but "A foolish consistency..." > > > >> The challenge is that hacking updates are basically giant merge > >> conflict engines. If there is any significant amount of code > >> outstanding in a project, landing hacking only changes basically > >> means requiring much of the outstanding code to rebase. > > > >> So it's actually expensive in a way that doesn't jump out > >> immediately. The cost of landing hacking isn't just the code of > >> reviewing the hacking patches, it's also the cost of the extra > >> roundtrips on outstanding patches. > > > > If a project has that many violations of a new hacking rule then I'd > > say they should just leave it disabled. It already happens and I'd > > rather have that than throw out a rule just because some projects > > haven't been following it. > > That would be fine if the propose bot *also* proposed the tox.ini > ignores, which it does not. > > So it pushes a largely by definition broken patch that consumes > resources, and will require a human to take it over. If a person did > this on a regular basis, we'd have a stern talking to them. Or at least > a light making fun of them. > > It feels like robo auto propose should be limited to the class of things > where the patch is 99% sure to pass. > I feel the same way on this one, for some reason I was originally under the impression that hacking had to manually be rolled out because of the reasons above. > > Which I think is really the crux of my distaste for hacking breaking > updates auto proposing to projects. > > -Sean > > -- > Sean Dague > http://dague.net > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] revert hacking to 0.8 series
On 06/16/2014 05:06 PM, Ben Nemec wrote: > On 06/16/2014 12:01 PM, Sean Dague wrote: >> On 06/16/2014 12:44 PM, Ben Nemec wrote: >>> On 06/16/2014 08:37 AM, Thierry Carrez wrote: Sean Dague wrote: > Hacking 0.9 series was released pretty late for Juno. The > entire check queue was flooded this morning with requirements > proposals failing pep8 because of it (so at 6am EST we were > waiting 1.5 hrs for a check node). > > The previous soft policy with pep8 updates was that we set a > pep8 version basically release week, and changes stopped > being done for style after first milestone. > > I think in the spirit of that we should revert the hacking > requirements update back to the 0.8 series for Juno. We're > past milestone 1, so shouldn't be working on style only fixes > at this point. > > Proposed review here - > https://review.openstack.org/#/c/100231/ > > I also think in future hacking major releases need to happen > within one week of release, or not at all for that series. >>> We may also have reached a size where changing style rules is just too costly, whatever the moment in the cycle. I think it's good that we have rules to enforce a minimum of common style, but the added value of those extra rules is limited, while their impact on the common gate grows as we add more projects. >>> >>> A few thoughts: >>> >>> 1) I disagree with the proposition that hacking updates can only >>> happen in the first week after release. I get that there needs >>> to be a cutoff, but I don't think one week is reasonable. Even >>> if we release in the first week, you're still going to be dealing >>> with hacking updates for the rest of the cycle as projects adopt >>> the new rules at their leisure. I don't like retroactively >>> applying milestone 1 as a cutoff either, although I could see >>> making that the policy going forward. >>> >>> 2) Given that most of the changes involved in fixing the new >>> failures are trivial, I think we should encourage combining the >>> fixes into one commit. We _really_ don't need separate commits >>> to fix H305 and H307. This doesn't help much with the reviewer >>> load, but it should reduce the gate load somewhat. It violates >>> the one change-one commit rule, but "A foolish consistency..." > >> The challenge is that hacking updates are basically giant merge >> conflict engines. If there is any significant amount of code >> outstanding in a project, landing hacking only changes basically >> means requiring much of the outstanding code to rebase. > >> So it's actually expensive in a way that doesn't jump out >> immediately. The cost of landing hacking isn't just the code of >> reviewing the hacking patches, it's also the cost of the extra >> roundtrips on outstanding patches. > > If a project has that many violations of a new hacking rule then I'd > say they should just leave it disabled. It already happens and I'd > rather have that than throw out a rule just because some projects > haven't been following it. That would be fine if the propose bot *also* proposed the tox.ini ignores, which it does not. So it pushes a largely by definition broken patch that consumes resources, and will require a human to take it over. If a person did this on a regular basis, we'd have a stern talking to them. Or at least a light making fun of them. It feels like robo auto propose should be limited to the class of things where the patch is 99% sure to pass. Which I think is really the crux of my distaste for hacking breaking updates auto proposing to projects. -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] revert hacking to 0.8 series
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/16/2014 12:01 PM, Sean Dague wrote: > On 06/16/2014 12:44 PM, Ben Nemec wrote: >> On 06/16/2014 08:37 AM, Thierry Carrez wrote: >>> Sean Dague wrote: Hacking 0.9 series was released pretty late for Juno. The entire check queue was flooded this morning with requirements proposals failing pep8 because of it (so at 6am EST we were waiting 1.5 hrs for a check node). The previous soft policy with pep8 updates was that we set a pep8 version basically release week, and changes stopped being done for style after first milestone. I think in the spirit of that we should revert the hacking requirements update back to the 0.8 series for Juno. We're past milestone 1, so shouldn't be working on style only fixes at this point. Proposed review here - https://review.openstack.org/#/c/100231/ I also think in future hacking major releases need to happen within one week of release, or not at all for that series. >> >>> We may also have reached a size where changing style rules is >>> just too costly, whatever the moment in the cycle. I think it's >>> good that we have rules to enforce a minimum of common style, >>> but the added value of those extra rules is limited, while >>> their impact on the common gate grows as we add more projects. >> >> A few thoughts: >> >> 1) I disagree with the proposition that hacking updates can only >> happen in the first week after release. I get that there needs >> to be a cutoff, but I don't think one week is reasonable. Even >> if we release in the first week, you're still going to be dealing >> with hacking updates for the rest of the cycle as projects adopt >> the new rules at their leisure. I don't like retroactively >> applying milestone 1 as a cutoff either, although I could see >> making that the policy going forward. >> >> 2) Given that most of the changes involved in fixing the new >> failures are trivial, I think we should encourage combining the >> fixes into one commit. We _really_ don't need separate commits >> to fix H305 and H307. This doesn't help much with the reviewer >> load, but it should reduce the gate load somewhat. It violates >> the one change-one commit rule, but "A foolish consistency..." > > The challenge is that hacking updates are basically giant merge > conflict engines. If there is any significant amount of code > outstanding in a project, landing hacking only changes basically > means requiring much of the outstanding code to rebase. > > So it's actually expensive in a way that doesn't jump out > immediately. The cost of landing hacking isn't just the code of > reviewing the hacking patches, it's also the cost of the extra > roundtrips on outstanding patches. If a project has that many violations of a new hacking rule then I'd say they should just leave it disabled. It already happens and I'd rather have that than throw out a rule just because some projects haven't been following it. > >> 3) We should start requiring specs for all new hacking rules to >> make sure we have consensus (I think oslo-specs is the place for >> this). 2 +2's doesn't really accomplish that. We also may need >> to raise the bar for inclusion of new rules - while I agree with >> all of the new ones added in hacking .9, I wonder if some of them >> are really necessary. > >> 4) I don't think we're at a point where we should freeze hacking >> completely, however. The import grouping and long line wrapping >> checks in particular are things that reviewers have to enforce >> today, and that has a significant, if less well-defined, cost >> too. If we're really going to say those rules can't be enforced >> by hacking then we need to remove them from our hacking >> guidelines and start the long process of educating reviewers to >> stop requiring them. I'd rather just deal with the pain of >> adding them to hacking one time and never have to think about >> them again. I'm less convinced the other two that were added in >> .9 are necessary, but in any case these are discussions that >> should happen in spec reviews going forward. > > I think both of those cases are really nits to the point that they > aren't worth enforcing. They won't change the correctness of the > code. And barely change the readability. Yeah, I suppose that's my personal dislike of \ line continuations talking. I don't like the idea of dropping our import guidelines though - I've seen code that doesn't follow them and it's not nice to read. Of course, I'm horribly biased on that too since I wrote the check. Then again, it's something we've been enforcing manually all along, so in theory it shouldn't be that difficult to fix either. > > There are differences with things like the is None checks, or > python 3 checks, which change correctness, or prevent subtle bugs. > But I think we're now getting to a level of cleanliness enforcement > that t
Re: [openstack-dev] revert hacking to 0.8 series
On Jun 16, 2014 10:02 AM, "Sean Dague" wrote: > > On 06/16/2014 12:44 PM, Ben Nemec wrote: > > On 06/16/2014 08:37 AM, Thierry Carrez wrote: > >> Sean Dague wrote: > >>> Hacking 0.9 series was released pretty late for Juno. The entire > >>> check queue was flooded this morning with requirements proposals > >>> failing pep8 because of it (so at 6am EST we were waiting 1.5 hrs > >>> for a check node). > >>> > >>> The previous soft policy with pep8 updates was that we set a > >>> pep8 version basically release week, and changes stopped being > >>> done for style after first milestone. > >>> > >>> I think in the spirit of that we should revert the hacking > >>> requirements update back to the 0.8 series for Juno. We're past > >>> milestone 1, so shouldn't be working on style only fixes at this > >>> point. > >>> > >>> Proposed review here - https://review.openstack.org/#/c/100231/ > >>> > >>> I also think in future hacking major releases need to happen > >>> within one week of release, or not at all for that series. > > > >> We may also have reached a size where changing style rules is just > >> too costly, whatever the moment in the cycle. I think it's good > >> that we have rules to enforce a minimum of common style, but the > >> added value of those extra rules is limited, while their impact on > >> the common gate grows as we add more projects. > > > > A few thoughts: > > > > 1) I disagree with the proposition that hacking updates can only > > happen in the first week after release. I get that there needs to be > > a cutoff, but I don't think one week is reasonable. Even if we > > release in the first week, you're still going to be dealing with > > hacking updates for the rest of the cycle as projects adopt the new > > rules at their leisure. I don't like retroactively applying milestone > > 1 as a cutoff either, although I could see making that the policy > > going forward. > > > > 2) Given that most of the changes involved in fixing the new failures > > are trivial, I think we should encourage combining the fixes into one > > commit. We _really_ don't need separate commits to fix H305 and H307. > > This doesn't help much with the reviewer load, but it should reduce > > the gate load somewhat. It violates the one change-one commit rule, > > but "A foolish consistency..." > > The challenge is that hacking updates are basically giant merge conflict > engines. If there is any significant amount of code outstanding in a > project, landing hacking only changes basically means requiring much of > the outstanding code to rebase. > > So it's actually expensive in a way that doesn't jump out immediately. > The cost of landing hacking isn't just the code of reviewing the hacking > patches, it's also the cost of the extra roundtrips on outstanding patches. When a project chooses to enforce rules us decoupled with when hacking if released. So this sounds like a related yet different issue. > > > 3) We should start requiring specs for all new hacking rules to make > > sure we have consensus (I think oslo-specs is the place for this). 2 > > +2's doesn't really accomplish that. We also may need to raise the > > bar for inclusion of new rules - while I agree with all of the new > > ones added in hacking .9, I wonder if some of them are really necessary. > > > 4) I don't think we're at a point where we should freeze hacking > > completely, however. The import grouping and long line wrapping > > checks in particular are things that reviewers have to enforce today, > > and that has a significant, if less well-defined, cost too. If we're > > really going to say those rules can't be enforced by hacking then we > > need to remove them from our hacking guidelines and start the long > > process of educating reviewers to stop requiring them. I'd rather > > just deal with the pain of adding them to hacking one time and never > > have to think about them again. I'm less convinced the other two that > > were added in .9 are necessary, but in any case these are discussions > > that should happen in spec reviews going forward. > > I think both of those cases are really nits to the point that they > aren't worth enforcing. They won't change the correctness of the code. > And barely change the readability. > > There are differences with things like the is None checks, or python 3 > checks, which change correctness, or prevent subtle bugs. But I think > we're now getting to a level of cleanliness enforcement that trumps > functionally working. > > > 5) We may want to come up with some way to globally disable pep8 > > checks we don't want to enforce, since we don't have any control over > > that but probably don't want to just stop updating pep8. That could > > make the pain of these updates much less. > > Actually, if you look at python novaclient, the doc string checks, which > are really niggly, and part of hacking are the biggest fails. It's much > less upstream that's doing this to us. > > > I could probably come up w
Re: [openstack-dev] revert hacking to 0.8 series
On Jun 16, 2014 9:44 AM, "Ben Nemec" wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 06/16/2014 08:37 AM, Thierry Carrez wrote: > > Sean Dague wrote: > >> Hacking 0.9 series was released pretty late for Juno. The entire > >> check queue was flooded this morning with requirements proposals > >> failing pep8 because of it (so at 6am EST we were waiting 1.5 hrs > >> for a check node). This is a general issue with global requirements, the number of jobs we run and the number of available nodes. Let's solve the general case. > >> > >> The previous soft policy with pep8 updates was that we set a > >> pep8 version basically release week, and changes stopped being > >> done for style after first milestone. > >> > >> I think in the spirit of that we should revert the hacking > >> requirements update back to the 0.8 series for Juno. We're past > >> milestone 1, so shouldn't be working on style only fixes at this > >> point. > >> > >> Proposed review here - https://review.openstack.org/#/c/100231/ > >> > >> I also think in future hacking major releases need to happen > >> within one week of release, or not at all for that series. > > > > We may also have reached a size where changing style rules is just > > too costly, whatever the moment in the cycle. I think it's good > > that we have rules to enforce a minimum of common style, but the > > added value of those extra rules is limited, while their impact on > > the common gate grows as we add more projects. > > A few thoughts: > > 1) I disagree with the proposition that hacking updates can only > happen in the first week after release. I get that there needs to be > a cutoff, but I don't think one week is reasonable. Even if we > release in the first week, you're still going to be dealing with > hacking updates for the rest of the cycle as projects adopt the new > rules at their leisure. I don't like retroactively applying milestone > 1 as a cutoff either, although I could see making that the policy > going forward. > ++ > 2) Given that most of the changes involved in fixing the new failures > are trivial, I think we should encourage combining the fixes into one > commit. We _really_ don't need separate commits to fix H305 and H307. > This doesn't help much with the reviewer load, but it should reduce > the gate load somewhat. It violates the one change-one commit rule, > but "A foolish consistency..." > ++ > 3) We should start requiring specs for all new hacking rules to make > sure we have consensus (I think oslo-specs is the place for this). 2 > +2's doesn't really accomplish that. We also may need to raise the > bar for inclusion of new rules - while I agree with all of the new > ones added in hacking .9, I wonder if some of them are really necessary. > I would rather just have more folks review hacking patches then add a specs repo. A specs repo is overkill, IMHO, hacking doesn't have that many patches per cycle. In general when adding a rule to hacking it has to already be in HACKING.rst and/or needs a ML post. > 4) I don't think we're at a point where we should freeze hacking > completely, however. The import grouping and long line wrapping > checks in particular are things that reviewers have to enforce today, > and that has a significant, if less well-defined, cost too. If we're > really going to say those rules can't be enforced by hacking then we > need to remove them from our hacking guidelines and start the long > process of educating reviewers to stop requiring them. I'd rather > just deal with the pain of adding them to hacking one time and never > have to think about them again. I'm less convinced the other two that > were added in .9 are necessary, but in any case these are discussions > that should happen in spec reviews going forward. > > 5) We may want to come up with some way to globally disable pep8 > checks we don't want to enforce, since we don't have any control over > that but probably don't want to just stop updating pep8. That could > make the pain of these updates much less. > > I could probably come up with a few more, but this is already too > wall-of-texty for my tastes. :-) > > - -Ben > -BEGIN PGP SIGNATURE- > Version: GnuPG v1 > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQEcBAEBAgAGBQJTnx7wAAoJEDehGd0Fy7uqoYAH/0KmxmR873Qn2Kti7LIEUNp4 > 1FJaBOX09ItxkkvyNRpcsIQu4fWycm60CckSOLfB7rgxgIjgsVkiZ9puE6oCmj2o > Lhe5DhjYA2ROu9h8i0vmzYDnAKeu/WuRGtgyLSElUXeuiLpSrBcEA/03GpkCGiAP > 1muAkVgv2oxDDwsaLwL7MmFrlZ1MPTP97lAfsfHbwbsOM5YMuPrRz9PirgHPBtTV > 59UyofCGEBTtJKmJRLzRDZyDwTux5xrrc/cefer5GFLQH0ZbxOU1HHFESyc5wFVJ > tI/3nPlbFpqCUtgmnQc8k3lX3d2H1Qr9UfCvYlJFTN1TmPmHmK378ioi81HoAVo= > =tqtf > -END PGP SIGNATURE- > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenS
Re: [openstack-dev] revert hacking to 0.8 series
This sounds totally reasonable. +1 to keeping style-specific changes consistent across a release. — Morgan Fainberg From: Clint Byrum cl...@fewbar.com Reply: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: June 16, 2014 at 10:51:31 To: openstack-dev openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] revert hacking to 0.8 series Excerpts from Sean Dague's message of 2014-06-16 05:15:54 -0700: > Hacking 0.9 series was released pretty late for Juno. The entire check > queue was flooded this morning with requirements proposals failing pep8 > because of it (so at 6am EST we were waiting 1.5 hrs for a check node). > > The previous soft policy with pep8 updates was that we set a pep8 > version basically release week, and changes stopped being done for style > after first milestone. > > I think in the spirit of that we should revert the hacking requirements > update back to the 0.8 series for Juno. We're past milestone 1, so > shouldn't be working on style only fixes at this point. > > Proposed review here - https://review.openstack.org/#/c/100231/ > > I also think in future hacking major releases need to happen within one > week of release, or not at all for that series. > +1. Hacking is supposed to help us avoid redundant nit-picking in reviews. If it places any large burden on developers, whether by merge conflicting or backing up CI, it is a failure IMO. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] revert hacking to 0.8 series
Excerpts from Sean Dague's message of 2014-06-16 05:15:54 -0700: > Hacking 0.9 series was released pretty late for Juno. The entire check > queue was flooded this morning with requirements proposals failing pep8 > because of it (so at 6am EST we were waiting 1.5 hrs for a check node). > > The previous soft policy with pep8 updates was that we set a pep8 > version basically release week, and changes stopped being done for style > after first milestone. > > I think in the spirit of that we should revert the hacking requirements > update back to the 0.8 series for Juno. We're past milestone 1, so > shouldn't be working on style only fixes at this point. > > Proposed review here - https://review.openstack.org/#/c/100231/ > > I also think in future hacking major releases need to happen within one > week of release, or not at all for that series. > +1. Hacking is supposed to help us avoid redundant nit-picking in reviews. If it places any large burden on developers, whether by merge conflicting or backing up CI, it is a failure IMO. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] revert hacking to 0.8 series
On 06/16/2014 12:44 PM, Ben Nemec wrote: > On 06/16/2014 08:37 AM, Thierry Carrez wrote: >> Sean Dague wrote: >>> Hacking 0.9 series was released pretty late for Juno. The entire >>> check queue was flooded this morning with requirements proposals >>> failing pep8 because of it (so at 6am EST we were waiting 1.5 hrs >>> for a check node). >>> >>> The previous soft policy with pep8 updates was that we set a >>> pep8 version basically release week, and changes stopped being >>> done for style after first milestone. >>> >>> I think in the spirit of that we should revert the hacking >>> requirements update back to the 0.8 series for Juno. We're past >>> milestone 1, so shouldn't be working on style only fixes at this >>> point. >>> >>> Proposed review here - https://review.openstack.org/#/c/100231/ >>> >>> I also think in future hacking major releases need to happen >>> within one week of release, or not at all for that series. > >> We may also have reached a size where changing style rules is just >> too costly, whatever the moment in the cycle. I think it's good >> that we have rules to enforce a minimum of common style, but the >> added value of those extra rules is limited, while their impact on >> the common gate grows as we add more projects. > > A few thoughts: > > 1) I disagree with the proposition that hacking updates can only > happen in the first week after release. I get that there needs to be > a cutoff, but I don't think one week is reasonable. Even if we > release in the first week, you're still going to be dealing with > hacking updates for the rest of the cycle as projects adopt the new > rules at their leisure. I don't like retroactively applying milestone > 1 as a cutoff either, although I could see making that the policy > going forward. > > 2) Given that most of the changes involved in fixing the new failures > are trivial, I think we should encourage combining the fixes into one > commit. We _really_ don't need separate commits to fix H305 and H307. > This doesn't help much with the reviewer load, but it should reduce > the gate load somewhat. It violates the one change-one commit rule, > but "A foolish consistency..." The challenge is that hacking updates are basically giant merge conflict engines. If there is any significant amount of code outstanding in a project, landing hacking only changes basically means requiring much of the outstanding code to rebase. So it's actually expensive in a way that doesn't jump out immediately. The cost of landing hacking isn't just the code of reviewing the hacking patches, it's also the cost of the extra roundtrips on outstanding patches. > 3) We should start requiring specs for all new hacking rules to make > sure we have consensus (I think oslo-specs is the place for this). 2 > +2's doesn't really accomplish that. We also may need to raise the > bar for inclusion of new rules - while I agree with all of the new > ones added in hacking .9, I wonder if some of them are really necessary. > 4) I don't think we're at a point where we should freeze hacking > completely, however. The import grouping and long line wrapping > checks in particular are things that reviewers have to enforce today, > and that has a significant, if less well-defined, cost too. If we're > really going to say those rules can't be enforced by hacking then we > need to remove them from our hacking guidelines and start the long > process of educating reviewers to stop requiring them. I'd rather > just deal with the pain of adding them to hacking one time and never > have to think about them again. I'm less convinced the other two that > were added in .9 are necessary, but in any case these are discussions > that should happen in spec reviews going forward. I think both of those cases are really nits to the point that they aren't worth enforcing. They won't change the correctness of the code. And barely change the readability. There are differences with things like the is None checks, or python 3 checks, which change correctness, or prevent subtle bugs. But I think we're now getting to a level of cleanliness enforcement that trumps functionally working. > 5) We may want to come up with some way to globally disable pep8 > checks we don't want to enforce, since we don't have any control over > that but probably don't want to just stop updating pep8. That could > make the pain of these updates much less. Actually, if you look at python novaclient, the doc string checks, which are really niggly, and part of hacking are the biggest fails. It's much less upstream that's doing this to us. > I could probably come up with a few more, but this is already too > wall-of-texty for my tastes. :-) > > -Ben > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital sign
Re: [openstack-dev] revert hacking to 0.8 series
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/16/2014 08:37 AM, Thierry Carrez wrote: > Sean Dague wrote: >> Hacking 0.9 series was released pretty late for Juno. The entire >> check queue was flooded this morning with requirements proposals >> failing pep8 because of it (so at 6am EST we were waiting 1.5 hrs >> for a check node). >> >> The previous soft policy with pep8 updates was that we set a >> pep8 version basically release week, and changes stopped being >> done for style after first milestone. >> >> I think in the spirit of that we should revert the hacking >> requirements update back to the 0.8 series for Juno. We're past >> milestone 1, so shouldn't be working on style only fixes at this >> point. >> >> Proposed review here - https://review.openstack.org/#/c/100231/ >> >> I also think in future hacking major releases need to happen >> within one week of release, or not at all for that series. > > We may also have reached a size where changing style rules is just > too costly, whatever the moment in the cycle. I think it's good > that we have rules to enforce a minimum of common style, but the > added value of those extra rules is limited, while their impact on > the common gate grows as we add more projects. A few thoughts: 1) I disagree with the proposition that hacking updates can only happen in the first week after release. I get that there needs to be a cutoff, but I don't think one week is reasonable. Even if we release in the first week, you're still going to be dealing with hacking updates for the rest of the cycle as projects adopt the new rules at their leisure. I don't like retroactively applying milestone 1 as a cutoff either, although I could see making that the policy going forward. 2) Given that most of the changes involved in fixing the new failures are trivial, I think we should encourage combining the fixes into one commit. We _really_ don't need separate commits to fix H305 and H307. This doesn't help much with the reviewer load, but it should reduce the gate load somewhat. It violates the one change-one commit rule, but "A foolish consistency..." 3) We should start requiring specs for all new hacking rules to make sure we have consensus (I think oslo-specs is the place for this). 2 +2's doesn't really accomplish that. We also may need to raise the bar for inclusion of new rules - while I agree with all of the new ones added in hacking .9, I wonder if some of them are really necessary. 4) I don't think we're at a point where we should freeze hacking completely, however. The import grouping and long line wrapping checks in particular are things that reviewers have to enforce today, and that has a significant, if less well-defined, cost too. If we're really going to say those rules can't be enforced by hacking then we need to remove them from our hacking guidelines and start the long process of educating reviewers to stop requiring them. I'd rather just deal with the pain of adding them to hacking one time and never have to think about them again. I'm less convinced the other two that were added in .9 are necessary, but in any case these are discussions that should happen in spec reviews going forward. 5) We may want to come up with some way to globally disable pep8 checks we don't want to enforce, since we don't have any control over that but probably don't want to just stop updating pep8. That could make the pain of these updates much less. I could probably come up with a few more, but this is already too wall-of-texty for my tastes. :-) - -Ben -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTnx7wAAoJEDehGd0Fy7uqoYAH/0KmxmR873Qn2Kti7LIEUNp4 1FJaBOX09ItxkkvyNRpcsIQu4fWycm60CckSOLfB7rgxgIjgsVkiZ9puE6oCmj2o Lhe5DhjYA2ROu9h8i0vmzYDnAKeu/WuRGtgyLSElUXeuiLpSrBcEA/03GpkCGiAP 1muAkVgv2oxDDwsaLwL7MmFrlZ1MPTP97lAfsfHbwbsOM5YMuPrRz9PirgHPBtTV 59UyofCGEBTtJKmJRLzRDZyDwTux5xrrc/cefer5GFLQH0ZbxOU1HHFESyc5wFVJ tI/3nPlbFpqCUtgmnQc8k3lX3d2H1Qr9UfCvYlJFTN1TmPmHmK378ioi81HoAVo= =tqtf -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] revert hacking to 0.8 series
Sean Dague wrote: > Hacking 0.9 series was released pretty late for Juno. The entire check > queue was flooded this morning with requirements proposals failing pep8 > because of it (so at 6am EST we were waiting 1.5 hrs for a check node). > > The previous soft policy with pep8 updates was that we set a pep8 > version basically release week, and changes stopped being done for style > after first milestone. > > I think in the spirit of that we should revert the hacking requirements > update back to the 0.8 series for Juno. We're past milestone 1, so > shouldn't be working on style only fixes at this point. > > Proposed review here - https://review.openstack.org/#/c/100231/ > > I also think in future hacking major releases need to happen within one > week of release, or not at all for that series. We may also have reached a size where changing style rules is just too costly, whatever the moment in the cycle. I think it's good that we have rules to enforce a minimum of common style, but the added value of those extra rules is limited, while their impact on the common gate grows as we add more projects. -- Thierry Carrez (ttx) signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
