[gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread hasufell
I'd like to discuss whether we should allow/encourage stabilization
commits to be less atomic. They often bloat the history, make it hard to
skim through the summaries list and people who are looking for
stabilization probably do 'git log -- ' anyway, no? In
addition, I'm not sure the bug information where people post "stable"
comments is very useful.

At least, the previous commits on
  app-leechcraft/

could have been category-commits, since they all refer to the same thing
and the same bug.

I'd go so far to say allow people to do commits like:
"""
amd64 stabilizations


"""
possibly pre-pending the rough domain like "kde", if any. I think kde
herd already does that, no?

Unless someone thinks the mass-100+ commits are really useful in the
visible history.



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread Rich Freeman
On Mon, Oct 19, 2015 at 7:55 AM, Dirkjan Ochtman  wrote:
> On Mon, Oct 19, 2015 at 1:21 PM, hasufell  wrote:
>> I'd go so far to say allow people to do commits like:
>> """
>> amd64 stabilizations
>>
>> 
>> """
>> possibly pre-pending the rough domain like "kde", if any. I think kde
>> herd already does that, no?
>
> Sounds sane to me.

I think that standardizing how we comment on bulk-stabilization
commits makes more sense than making them less atomic.  Not getting
half a KDE update is actually one of the nice selling features of git.
Plus, in the event of a disaster it also makes rollback easier.

But, by all means we should update the wiki to recommend the standard
way to document these changes.

-- 
Rich



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread Dirkjan Ochtman
On Mon, Oct 19, 2015 at 1:21 PM, hasufell  wrote:
> I'd go so far to say allow people to do commits like:
> """
> amd64 stabilizations
>
> 
> """
> possibly pre-pending the rough domain like "kde", if any. I think kde
> herd already does that, no?

Sounds sane to me.

Cheers,

Dirkjan



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread Matthew Thode
On 10/19/2015 10:04 AM, hasufell wrote:
> On 10/19/2015 04:37 PM, Ian Stakenvicius wrote:
>>
>>
>>
>> It may be my lack of coffee this morning, but I think you and
>> hasufell are saying the same thing but using "making commits less
>> atomic" conversely.
>>
>> Just so i make sure i'm understanding this right, hasufell's
>> suggestion is to, instead of rolling a single "atomic" commit for
>> every package being stabilized under a tracker bug, that the whole
>> set of packages gets stabilized via one commit.  Thus ensuring one
>> doesn't get half a kde update, and rollbacks can be done at a single
>> commit level, etc.
>>
>> Do I have this right?
>>
>> (note, since all of these package changes are for a particular
>> single purpose imo it would still be an atomic commit)
>>
>>
> 
> Well yes. But you could go one step further and argue that we can allow
> the same thing when ago's scripts make 300 commits for 300 stabilization
> bugs at once (same category or not).
> 
> The question is if stabilization needs to be atomic history-wise. It is
> nothing you revert or cherry-pick anyway and you could consider it a
> global commit too with the subsystem "stable arch".
> 
while I think that one commit per bug is preferred, having multiple bugs
in one commit is ok as well.  Some of us already do this sometimes when
updating packages (multiple birds with one stone and all that).

-- 
-- Matthew Thode (prometheanfire)



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread hasufell
On 10/19/2015 04:37 PM, Ian Stakenvicius wrote:
> 
> 
> 
> It may be my lack of coffee this morning, but I think you and
> hasufell are saying the same thing but using "making commits less
> atomic" conversely.
> 
> Just so i make sure i'm understanding this right, hasufell's
> suggestion is to, instead of rolling a single "atomic" commit for
> every package being stabilized under a tracker bug, that the whole
> set of packages gets stabilized via one commit.  Thus ensuring one
> doesn't get half a kde update, and rollbacks can be done at a single
> commit level, etc.
> 
> Do I have this right?
> 
> (note, since all of these package changes are for a particular
> single purpose imo it would still be an atomic commit)
> 
> 

Well yes. But you could go one step further and argue that we can allow
the same thing when ago's scripts make 300 commits for 300 stabilization
bugs at once (same category or not).

The question is if stabilization needs to be atomic history-wise. It is
nothing you revert or cherry-pick anyway and you could consider it a
global commit too with the subsystem "stable arch".



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread Ian Stakenvicius
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 19/10/15 11:04 AM, hasufell wrote:
> On 10/19/2015 04:37 PM, Ian Stakenvicius wrote:
>> 
>> 
>> 
>> It may be my lack of coffee this morning, but I think you and 
>> hasufell are saying the same thing but using "making commits 
>> less atomic" conversely.
>> 
>> Just so i make sure i'm understanding this right, hasufell's 
>> suggestion is to, instead of rolling a single "atomic" commit 
>> for every package being stabilized under a tracker bug, that 
>> the whole set of packages gets stabilized via one commit.
>> Thus ensuring one doesn't get half a kde update, and rollbacks
>> can be done at a single commit level, etc.
>> 
>> Do I have this right?
>> 
>> (note, since all of these package changes are for a particular
>>  single purpose imo it would still be an atomic commit)
>> 
>> 
> 
> Well yes. But you could go one step further and argue that we
> can allow the same thing when ago's scripts make 300 commits for
> 300 stabilization bugs at once (same category or not).
> 
> The question is if stabilization needs to be atomic
> history-wise. It is nothing you revert or cherry-pick anyway and
> you could consider it a global commit too with the subsystem
> "stable arch".
> 


Ahh, so what you're referring to here is stabilization of multiple
unrelated packages in a single commit..  ok..  i'm not so
comfortable with that idea.. BUT, nothing stopped us from doing this
with CVS (although the mapping of commit between CVS and GIT aren't
exactly 1:1), so i guess it's fine..?

What about simply keeping things as they are but not strictly
enforcing them when they are used in a manner like this for special
cases, such as ago's stabilizations or other security@ or arch team
keyword-related commits?

-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iF4EAREIAAYFAlYlC/wACgkQAJxUfCtlWe1XSgD/fa4M2E7k4asOeUGgLEt2um6m
9NovN22eVUeLbSvtnLoBALT4+vhXqYhi3K3ytFv6dcfcKFpiYMbuWuMNu2YrVRj9
=Ef9v
-END PGP SIGNATURE-



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread Ian Stakenvicius
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 19/10/15 08:21 AM, Rich Freeman wrote:
> On Mon, Oct 19, 2015 at 7:55 AM, Dirkjan Ochtman 
> wrote:
>> On Mon, Oct 19, 2015 at 1:21 PM, hasufell 
>> wrote:
>>> I'd go so far to say allow people to do commits like: """ 
>>> amd64 stabilizations
>>> 
>>>  """ possibly pre-pending the rough
>>> domain like "kde", if any. I think kde herd already does
>>> that, no?
>> 
>> Sounds sane to me.
> 
> I think that standardizing how we comment on bulk-stabilization 
> commits makes more sense than making them less atomic.  Not
> getting half a KDE update is actually one of the nice selling
> features of git. Plus, in the event of a disaster it also makes
> rollback easier.
> 
> But, by all means we should update the wiki to recommend the
> standard way to document these changes.
> 


It may be my lack of coffee this morning, but I think you and
hasufell are saying the same thing but using "making commits less
atomic" conversely.

Just so i make sure i'm understanding this right, hasufell's
suggestion is to, instead of rolling a single "atomic" commit for
every package being stabilized under a tracker bug, that the whole
set of packages gets stabilized via one commit.  Thus ensuring one
doesn't get half a kde update, and rollbacks can be done at a single
commit level, etc.

Do I have this right?

(note, since all of these package changes are for a particular
single purpose imo it would still be an atomic commit)

-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iF4EAREIAAYFAlYlACsACgkQAJxUfCtlWe1yuQD+KaeYsBnQdxL/jCA7AywJwRW4
Iv6LSjNSgMAgYJRCtU8BANz5MrAh8uzqdA03oWetvISXz50nSDa0LuS3XebBZCfi
=UBQF
-END PGP SIGNATURE-



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread Rich Freeman
On Mon, Oct 19, 2015 at 1:13 PM, hasufell  wrote:
>
> We already know that. But if e.g. ago runs his scripts at 00:00 with
> ~300 packages stabilized, the history (without git command line) on
> github/gitweb will be fun to read (and people DO that).
>

It doesn't seem like it would have been any better in the cvs days,
but I guess that isn't a reason to reject this on its own.

If this was about changing the copyright headers in all the ebuilds in
the tree I'd say that this is a million related trivial changes that
can be merged.  Nobody needs to see that in the history broken out.

However, stabilizing a single package really is an impactful change.
The fact that you're doing 100 of them at one time doesn't really
diminish the impact of each one.  Any of them could break a system or
need to be reverted.

If they're being done at once because they're all part of some library
stabilization then I'd combine them into a commit, because they are
actually related.

Maybe what is needed is better tools for tagging/filtering history?

-- 
Rich



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread hasufell
On 10/19/2015 07:37 PM, Rich Freeman wrote:
> 
> However, stabilizing a single package really is an impactful change.
> The fact that you're doing 100 of them at one time doesn't really
> diminish the impact of each one.  Any of them could break a system or
> need to be reverted.
> 

Since when do we allow reverting stabilization? The package needs to be
fixed and possibly revbumped instead.



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread Rich Freeman
On Mon, Oct 19, 2015 at 1:40 PM, hasufell  wrote:
> On 10/19/2015 07:37 PM, Rich Freeman wrote:
>>
>> However, stabilizing a single package really is an impactful change.
>> The fact that you're doing 100 of them at one time doesn't really
>> diminish the impact of each one.  Any of them could break a system or
>> need to be reverted.
>>
>
> Since when do we allow reverting stabilization? The package needs to be
> fixed and possibly revbumped instead.
>

It would really depend on the nature of the break.  If it is a serious
upstream problem and no fix is available, then reverting might be the
only practical solution.  It is of course not a preferred solution.

-- 
Rich



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread hasufell
On 10/19/2015 07:52 PM, Rich Freeman wrote:
> On Mon, Oct 19, 2015 at 1:40 PM, hasufell  wrote:
>> On 10/19/2015 07:37 PM, Rich Freeman wrote:
>>>
>>> However, stabilizing a single package really is an impactful change.
>>> The fact that you're doing 100 of them at one time doesn't really
>>> diminish the impact of each one.  Any of them could break a system or
>>> need to be reverted.
>>>
>>
>> Since when do we allow reverting stabilization? The package needs to be
>> fixed and possibly revbumped instead.
>>
> 
> It would really depend on the nature of the break.  If it is a serious
> upstream problem and no fix is available, then reverting might be the
> only practical solution.  It is of course not a preferred solution.
> 

I don't think we depend on 'git revert' in that case. KEYWORDS are
trivial changes (in terms of file diffs).



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread Rich Freeman
On Mon, Oct 19, 2015 at 1:55 PM, hasufell  wrote:
> On 10/19/2015 07:52 PM, Rich Freeman wrote:
>> On Mon, Oct 19, 2015 at 1:40 PM, hasufell  wrote:
>>> On 10/19/2015 07:37 PM, Rich Freeman wrote:

 However, stabilizing a single package really is an impactful change.
 The fact that you're doing 100 of them at one time doesn't really
 diminish the impact of each one.  Any of them could break a system or
 need to be reverted.

>>>
>>> Since when do we allow reverting stabilization? The package needs to be
>>> fixed and possibly revbumped instead.
>>>
>>
>> It would really depend on the nature of the break.  If it is a serious
>> upstream problem and no fix is available, then reverting might be the
>> only practical solution.  It is of course not a preferred solution.
>>
>
> I don't think we depend on 'git revert' in that case. KEYWORDS are
> trivial changes (in terms of file diffs).
>

True, as long as they're truly standalone.

Maybe the compromise is:
1.  Groups of related stabilizations should be grouped.
2.  Groups of only unrelated stabilizations can also be grouped.
3.  You must not try to mix #1 and #2 in the same commit.

As you say individual packages are easy to revert anyway.  However, we
wouldn't want 100 of those to be mixed in with 50 packages that all
needed to be coordinated, because those 50 aren't easy to revert.

-- 
Rich



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread Rich Freeman
On Mon, Oct 19, 2015 at 11:27 AM, Ian Stakenvicius  wrote:
>
> Ahh, so what you're referring to here is stabilization of multiple
> unrelated packages in a single commit..  ok..  i'm not so
> comfortable with that idea..

Nor am I.  A commit should be a set of related changes.  Stabilizing
all of KDE-n in one commit makes a lot of sense.  Stabilizing 5 random
packages in one commit doesn't make sense.  By all means push them all
at once, but don't commit them all at once.  It isn't like we have to
pay for each commit.

I also don't have a problem with fixing multiple bugs in one commit.
In an ideal world you'd do that on a branch and then do a merge, and I
don't have a problem with that either, but I get that we tend to not
work that way.  If you want every commit to stand on its own and
you're porting to a new EAPI and fixing 3 bugs at the same time, I
don't expect maintainers to rewrite their bugs into the new code to
port it to the new EAPI, then fix each bug in turn.

> BUT, nothing stopped us from doing this
> with CVS (although the mapping of commit between CVS and GIT aren't
> exactly 1:1), so i guess it's fine..?

In cvs all commits were at the file level, even if you happened to
create more than one as a single operation.  If you did one commit
that touched 100 ebuilds, you were actually doing 100 commits, and
there is nothing that really ties those 100 commits together and by
the time it gets to rsync you might only get 50 of them if the timing
is right.

So, this actually is a new problem, or rather benefit.

>
> What about simply keeping things as they are but not strictly
> enforcing them when they are used in a manner like this for special
> cases, such as ago's stabilizations or other security@ or arch team
> keyword-related commits?
>

I don't think we're strictly enforcing anything now but I could be
wrong.  I think we should have guidelines that recommend best
practices and try to stick to them.  If there is a really good reason
to do things differently, that is why we call them "guidelines."

-- 
Rich



Re: [gentoo-dev] stabilization commits and atomicity

2015-10-19 Thread hasufell
On 10/19/2015 07:08 PM, Rich Freeman wrote:
> On Mon, Oct 19, 2015 at 11:27 AM, Ian Stakenvicius  wrote:
>>
>> Ahh, so what you're referring to here is stabilization of multiple
>> unrelated packages in a single commit..  ok..  i'm not so
>> comfortable with that idea..
> 
> Nor am I.  A commit should be a set of related changes.  Stabilizing
> all of KDE-n in one commit makes a lot of sense.  Stabilizing 5 random
> packages in one commit doesn't make sense.  By all means push them all
> at once, but don't commit them all at once.  It isn't like we have to
> pay for each commit.
> 

We already know that. But if e.g. ago runs his scripts at 00:00 with
~300 packages stabilized, the history (without git command line) on
github/gitweb will be fun to read (and people DO that).

The argument is that those are related changes to the subsystem "stable
arch" (and affect not random ebuild details, but stable arch only, as in
KEYWORDS). Ofc, people can still create atomic commits if the
stabilization is security related.