[gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Duncan]

Rich Freeman posted on Mon, 31 Jul 2017 20:55:05 -0400 as excerpted:

> On Mon, Jul 31, 2017 at 8:24 PM, Duncan <1i5t5.dun...@cox.net> wrote:
>> Rich Freeman posted on Mon, 31 Jul 2017 11:11:24 -0400 as excerpted:
>>
>>> On Mon, Jul 31, 2017 at 10:52 AM, Alec Warner 
>>> wrote:

 [T]he conclusion I was hoping to draw is that one
 has 2 repos instead of 1.

 1) Rolling.
 2) Stable.

>>> This seems like it would be fairly painful to maintain.
>>
>> FWIW, the gentoo/kde team effectively do this right now
>>
> The difficulty isn't in moving the ebuilds around.
> 
> The difficulty is in knowing WHICH ebuilds to move around.
> 
> In the case of KDE it is the maintainers doing the maintaining,

Very good point.  Thanks.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Rich Freeman]

On Mon, Jul 31, 2017 at 8:24 PM, Duncan <1i5t5.dun...@cox.net> wrote:
> Rich Freeman posted on Mon, 31 Jul 2017 11:11:24 -0400 as excerpted:
>
>> On Mon, Jul 31, 2017 at 10:52 AM, Alec Warner 
>> wrote:
>>>
>>>
>>> Sorry, to be clear the conclusion I was hoping to draw is that one has
>>> 2 repos instead of 1.
>>>
>>> 1) Rolling.
>>> 2) Stable.
>>>
>>> Rolling is typical ~arch Gentoo. People in rolling can do whatever they
>>> want; they can't affect stable at all.
>>>
>>> Stable is an entirely separate repo, a fork, where CPVs are pulled from
>>> Rolling into Stable. If Stable wants to keep a gnarly old version of
>>> some package around; great! But the rolling people don't have to care.
>>>
>>>
>> This seems like it would be fairly painful to maintain.  You'd need to
>> constantly pull in new packages, and prune out old ones.  It would
>> duplicate many of the functions maintainers already do.  I doubt anybody
>> would go to the trouble to make this happen.
>
> FWIW, the gentoo/kde team effectively do this right now, tho only with kde
> packages and some of their deps, and it's live/prerelease/release-staging
> vs ~arch/stable, not ~arch vs stable.  But the amount of work is surely
> similar, and they've been doing it now for a number of years and over a
> major kde version bump, an upstream svn/git upgrade and general upstream
> remodularization.
>

The difficulty isn't in moving the ebuilds around.

The difficulty is in knowing WHICH ebuilds to move around.

In the case of KDE it is the maintainers doing the maintaining, so
they already understand which versions should move.  They've all been
tested, so I suspect it is likely a lift and place of the entire
thing.

In the proposed multi-repository approach the maintainers would not be
the ones doing the moving.

Now, I guess you could have a snapshot/release-based approach.  Take a
snapshot of the ENTIRE ~arch tree.  Then do whatever level of QA, and
after a delay move the ENTIRE ~arch tree to stable.  The problem with
this is that you'll probably pick that oddball version of some package
that is about to be replaced and isn't a good stable candidate, and so
on.  It also would be difficult to actually test it all.

And if you're going to get the maintainers to move all their own
stuff, then you're just giving them extra work compared to just using
the KEYWORDS variable.

In the current state the maintainer is at the heart of the
stabilization process, so the person who already needs to understand
the individual package is the one deciding which versions go stable.
Duplicating this level of knowledge would not be straightforward.

-- 
Rich

[gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Duncan]

Rich Freeman posted on Mon, 31 Jul 2017 11:11:24 -0400 as excerpted:

> On Mon, Jul 31, 2017 at 10:52 AM, Alec Warner 
> wrote:
>>
>>
>> Sorry, to be clear the conclusion I was hoping to draw is that one has
>> 2 repos instead of 1.
>>
>> 1) Rolling.
>> 2) Stable.
>>
>> Rolling is typical ~arch Gentoo. People in rolling can do whatever they
>> want; they can't affect stable at all.
>>
>> Stable is an entirely separate repo, a fork, where CPVs are pulled from
>> Rolling into Stable. If Stable wants to keep a gnarly old version of
>> some package around; great! But the rolling people don't have to care.
>>
>>
> This seems like it would be fairly painful to maintain.  You'd need to
> constantly pull in new packages, and prune out old ones.  It would
> duplicate many of the functions maintainers already do.  I doubt anybody
> would go to the trouble to make this happen.

FWIW, the gentoo/kde team effectively do this right now, tho only with kde 
packages and some of their deps, and it's live/prerelease/release-staging 
vs ~arch/stable, not ~arch vs stable.  But the amount of work is surely 
similar, and they've been doing it now for a number of years and over a 
major kde version bump, an upstream svn/git upgrade and general upstream 
remodularization.

They seem to have the method and routine /down/, and I'm sure many of the 
lessons they've learned could be used were such a main repo split to be 
undertaken, but I honestly have no idea whether they'd consider the 
effort huge or "painful to maintain", only that they do it -- pretty  
effectively if I might add from my own consumption of both the main tree 
and kde overlay.

And to address the concern over users with mixed ~arch/stable usage, as a 
user effectively doing it but with mixed ~arch-main/live-kde usage, the 
trouble of having to pull and update from both trees, managing masks, 
etc, isn't actually that bad at all, particularly given the fact that the 
main mask/unmask sets are maintained (automatically via project script) 
in the kde repo so all I have to do is symlink appropriately and add an 
occasionally temporarily overlooked one to my local exception file.


For gentoo/kde it would seem to have been worth it, but you'd have to ask 
them if it's "painful" for them.

So it's certainly doable, maintainable over years and major changes, and 
consumable, as gentoo/kde devs and their users have been and continues to 
demonstrate. =:^)  The /big/ question then is only whether that model's 
actually a good fit for the wider gentoo culture, and I still have my 
doubts on that one.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Walter Dnes]

On Fri, Jul 28, 2017 at 05:56:25PM -0400, William L. Thomson Jr. wrote

> If upstream does a new release, fixes bugs. Gentoo marks a previous
> release stable. It is stabilizing a package with issues fixed upstream.
> That does not make sense. Gentoo issues maybe good, but not upstreams.
> 
> I maintained packages like iText which used to have a 30 day release
> cycle. Up till recently Jetty was about the same. As a end user, I
> needed the bug fixes. Not the delay for it be marked stable.
> 
> I stopped running Redhat long ago due to time to vet updates. I run
> Gentoo for the speed of being able to package and test out new code.

  What I get out of this discussion is that some people prefer to run
~arch versus stable arch.  I have no problem with that.  But I do object
to dropping "stable" altogether.  I personally run stable with the rare
occasional unstable package, where it's either not available as stable,
or the unstable version fixes a bug in the stable version.  And just for
kicks I'm running gcc 6.3.0.

  It's one thing to rush-stabilize a new package that fixes a security
hole.  But I don't see the point of rush-stabilizing everything "just
because".  I recommend mostly keeping our current setup, with one
change, i.e. allowing security-fix ebuilds to go "stable" immediately.

-- 
Walter Dnes 
I don't run "desktop environments"; I run useful applications

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[William L. Thomson Jr.]

On Fri, 28 Jul 2017 21:45:57 + (UTC)
Duncan <1i5t5.dun...@cox.net> wrote:

> William L. Thomson Jr. posted on Fri, 28 Jul 2017 16:10:42 -0400 as
> excerpted:
> 
> > It seems odd that upstream will release a package. Just for
> > downstream to consider it not stable. Did it get messed up during
> > packaging? Did it get messed up by the distro? The whole lag thing
> > does not make sense for Gentoo. Sooner released and tested on
> > Gentoo. Sooner bugs can be founded, reported back to upstream, etc.
> > Speeds up development. That is Gentoo's role in FOSS IMHO.  
> 
> Not so odd.  Gentoo's arch-stable has a different meaning than
> upstream's stable.  As a long time gentooer I'm surprised you weren't
> aware of this already.

If upstream does a new release, fixes bugs. Gentoo marks a previous
release stable. It is stabilizing a package with issues fixed upstream.
That does not make sense. Gentoo issues maybe good, but not upstreams.

I maintained packages like iText which used to have a 30 day release
cycle. Up till recently Jetty was about the same. As a end user, I
needed the bug fixes. Not the delay for it be marked stable.

I stopped running Redhat long ago due to time to vet updates. I run
Gentoo for the speed of being able to package and test out new code.

-- 
William L. Thomson Jr.


pgpjA1gvTVSJQ.pgp
Description: OpenPGP digital signature

[gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Duncan]

William L. Thomson Jr. posted on Fri, 28 Jul 2017 16:10:42 -0400 as
excerpted:

> It seems odd that upstream will release a package. Just for downstream
> to consider it not stable. Did it get messed up during packaging? Did it
> get messed up by the distro? The whole lag thing does not make sense for
> Gentoo. Sooner released and tested on Gentoo. Sooner bugs can be
> founded, reported back to upstream, etc. Speeds up development. That is
> Gentoo's role in FOSS IMHO.

Not so odd.  Gentoo's arch-stable has a different meaning than upstream's 
stable.  As a long time gentooer I'm surprised you weren't aware of this 
already.

In general (individual packages may differ somewhat on a case-by-case 
basis, one variant being packages where gentoo /is/ upstream and ~arch is 
thus used for upstream unstable testing to some degree)...

Gentoo's stable doesn't really relate to upstream stable except that 
upstream betas and the like (well, short-term ones, not the ones that 
last years without a non-beta release...) aren't normally considered 
stable candidates because they're not released by upstream as such.  
Instead, gentoo's stable means that the packaging -- the ebuild script, 
the eclasses it calls, and the eapi as encoded in pms and deployed by the 
pm -- is considered tested and stable on a particular arch.  Gentoo-
stable also includes proper integration, making sure the header files, 
libs, configuration, documentation, etc, all end up in the place gentooers 
expect them to be, that they build with our particular toolset, etc.

Similarly, upstream-unstable isn't supposed to be a candidate for ~arch 
either, because ~arch is supposed to be upstream stable that simply 
hasn't yet had enough testing of its gentoo packaging and integration in 
ordered for that particular package to be declared stable.

That's one reason why ~arch normally works so well for those prepared to 
manually deal with the occasional packaging or integration hiccup, missed 
or incorrect dependency, failed merge due to conflict with existing files 
because upstream moved something and the package hasn't yet adapted to 
it, etc -- it's releases that upstream has already declared reasonably 
stable, that simply aren't yet sufficiently tested in terms of the gentoo 
packaging and integration, and if a user's willing to deal with the 
occasional hiccup there it should otherwise be as stable as the upstream 
declaration.

Which is why people like me find ~arch quite stable -- it /should/ be for 
us, as it's upstream stable, and we're prepared to deal with the minor 
dependency and integration issues, etc, that the process of gentoo 
stabilization is intended to fix.

The problem this thread is seeking to at least incrementally tweak toward 
the better is that the above only works for people on stable, when people 
actually declare a package stable when there's no serious bugs left after 
the testing period.  Sometimes they forget, packages drop thru the 
cracks, etc, and stable users get further behind than they would be if 
policies were actually being followed.

And perhaps more to the point, on the minor archs which tend to be the 
bottleneck, sometimes there's simply not the resources, either 
sufficiently interested people with the time (who are volunteers after 
all, so interest and time can't be commanded) or arch hardware or both, 
available to do those stabilizations.  The proposal here hopes to 
automate much of the process as well as standardize it, hopefully 
alleviating that bottleneck.

Tho as I've said before, as one who /is/ prepared to deal with the 
occasional packaging or integration issue and thus finds ~arch perfectly 
usable, I'd personally have no problem with dropping stable, it's just 
that I know it to be a practical impossibility, because were we to do so, 
instead of freeing that time to work on what's now ~arch, we'd simply 
lose most of the volunteers who have a major interest in stable.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Hans de Graaff]

On Tue, 2017-07-25 at 11:03 +0200, Agostino Sarubbo wrote:
> 
> 1) Don't file keywordreq, since noone work on them. File directly
> stablereq.

This does not make sense to me.

If we want to go this route we should probably state a policy instead
that new dependencies for already keyworded packages automatically get
those keywords as well, even if untested. For packages with stable
keywords this would provide a chance to find issues before the package
is marked stable.

For KEYWORDREQ bugs we could also enlist our users. As a maintainer of
dev-ruby packages I'd be happy to add any keywords based on a "emerge
--info" and "build.log with FEATURES=test" combo added to a KEYWORDREQ
bug. Putting out a call for action and an easy way for users to scan
open KEYWORDREQ bugs for their arch might put a good dent in these.

Hans

signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Daniel Campbell]

On 07/25/2017 06:22 AM, Rich Freeman wrote:
> On Tue, Jul 25, 2017 at 9:10 AM, Michael Palimaka  
> wrote:
>>
>> The 30 day waiting period is useful for smoking out major upstream bugs,
>> but can't replace stabilisation integration testing. For example,
>> package foobar may build fine in ~arch but fails in stable because it
>> needs a newer libbaz.
>>
> 
> I think this is a good reason why everything should be at least
> build-tested on a stable tree before getting stabilized.  Now, that
> might not be on each arch if it is truly arch-independent (and if
> arches keep the dependencies reasonably in sync).
> 
> This might be a situation where a compromise could exist.  Have some
> kind of flag (in metadata, or maybe the ebuild) that indicates that
> the maintainer believes the package is low-risk to stabilize purely on
> a build test.  Then after 30 days in testing a tinderbox could
> build-test the package and stabilize it automatically.
> 
> If the package is considered at-risk then it goes through manual
> testing, either by the maintainer or an arch team.
> 
> This will also encourage the teams doing testing to actually do more
> testing on the packages that would benefit from it.
> 
> We wouldn't set hard criteria but leave it up to maintainer
> discretion, with a guideline being that past performance is probably a
> good predictor of future results.
> 
This reads like a practical use of both developer time and machine time.
Build testing at a minimum, imo, is necessary if the word "stable" is
going to mean anything for us. So +1.

Since there are bound to be fewer manually tested packages than
automatic "it builds, ship it" packages, I think it would make a bit
more sense to add a "manually test this please" tag to metadata.xml,
rather than expect auto-stabilizers to have additional tags, which will
outnumber the manual packages and inflate the size of the tree (albeit
slightly).

Since maintainers also manage their packages in various ways, could we
extend this to a general  element? Maintainers can specify how
they'd prefer bugs or commits to be done, and an additional element to
indicate hand-testing. This would solve two problems instead of just
one: indicate a package is ready for auto-stabilization, and give a
single, canonical location for a maintainer to put maintenance policy.

Just my 2¢,

~zlg
-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Rich Freeman]

On Tue, Jul 25, 2017 at 3:45 PM, Markus Meier  wrote:
> On Tue, 25 Jul 2017 11:03:30 +0200
> Agostino Sarubbo  wrote:
>
>> On Monday 24 July 2017 22:22:23 Sergei Trofimovich wrote:
>> > 1. lack of automation
>> I'd summarize the techical steps into:
>> 1) get the list of packages
>> 2) test
>> 3) commit to git
>> 4) write on bugzilla
>>
>> 1 is done by getatoms https://github.com/kensington/bugbot
>> 2 is done by the tester in the manner he prefer
>> 3 no official tool available, I used a modified version of
>> https://gitweb.gentoo.org/proj/arch-tools.git/tree/batch-stabilize.py
>> which is still based on CVS
>> 4 no official tool available, I used my own bash script which calls
>> pybugz
>>
>> So, points 3 and 4 needs to be improved, I have the idea on how the
>> script should look, but I have no time to do it and no python
>> knowledge. I can assist everyone that candidate itself to make the
>> tool/script like I did with kensington when he made getatoms.
>
> for 3 and 4 there's the keyword.sh script in my overlay (under scripts)
> that has been working for ages (at least for me)...
>

It is a slightly different process, but there is also the situation
where an arch is slow to respond to a stablereq and the maintainer
wishes to drop keywords.

In that case a script is needed which will remove stable keywords on
all reverse deps of the package.

Back when the council approved dropping keywords in these situations
it seemed to be that the effort required to do so was the main barrier
to maintainers taking advantage of the policy.  Awareness might be
another issue, as I don't think it really got documented outside of
the summaries.

-- 
Rich

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Markus Meier]

On Tue, 25 Jul 2017 11:03:30 +0200
Agostino Sarubbo  wrote:

> On Monday 24 July 2017 22:22:23 Sergei Trofimovich wrote:
> > 1. lack of automation  
> I'd summarize the techical steps into:
> 1) get the list of packages
> 2) test
> 3) commit to git
> 4) write on bugzilla
> 
> 1 is done by getatoms https://github.com/kensington/bugbot
> 2 is done by the tester in the manner he prefer
> 3 no official tool available, I used a modified version of 
> https://gitweb.gentoo.org/proj/arch-tools.git/tree/batch-stabilize.py
> which is still based on CVS
> 4 no official tool available, I used my own bash script which calls
> pybugz
>
> So, points 3 and 4 needs to be improved, I have the idea on how the
> script should look, but I have no time to do it and no python
> knowledge. I can assist everyone that candidate itself to make the
> tool/script like I did with kensington when he made getatoms.

for 3 and 4 there's the keyword.sh script in my overlay (under scripts)
that has been working for ages (at least for me)...


Regards,
Markus

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Peter Stuge]

Michael Palimaka wrote:
> The 30 day waiting period is useful for smoking out major upstream bugs,
> but can't replace stabilisation integration testing. For example,
> package foobar may build fine in ~arch but fails in stable because it
> needs a newer libbaz.

So that's either because of an ebuild bug (incorrect DEPEND) or an
upstream bug (e.g. incorrect PKG_CONFIG_CHECK in configure.ac), right?

It seems to me that waiting (random()=30) days and then testing
against (random()=current-version) libbaz in stable isn't amazing. :)

The ebuild bug could be detected automatically for upstreams using
configure.ac and maybe also cmake.

The upstream bug, well, that's a bit more tricky.


I'll try to write a thoughtful response in the other part of this
thread later. Gotta run now. Thanks everyone for your insights!


//Peter

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Michał Górny]

On wto, 2017-07-25 at 22:59 +1000, Michael Palimaka wrote:
> On 07/25/2017 07:22 AM, Sergei Trofimovich wrote:
> > 2. Q: How to make arch testing faster and easier?
> > 
> >A: - KEYWORDREQ/STABLEREQ bugs not marked as "runtime testing
> > required" will be automatically tested and keyworded.
> > 
> > [handwave] automated tinderbox setup would help a lot
> > to now upfront what fails to built, fails tests.
> 
> I've had similar thoughts about this and have already been working on
> some tooling for this.
> 
> We would need to establish exactly what criteria must be met for an
> automated test to be considered as successful.
> 
> Here's a sample report that my tool produces:
> https://dev.gentoo.org/~kensington/tinderbox/sys-apps/dbus/dbus-1.6.18-r1/df017e14-bd68-47e2-9738-554e7ba1cf10.html
> 
> In this case, would it be enough that it builds and tests pass? What
> about the QA issues? Do we need someone to review them to determine if
> they should block stabilisation, or if they're even a regression or not?
> 

There are different kinds of QA issues and they have different
significance. You can ignore some of them, some others are more
important.

GLEP 65 defines a 'eqatag' function that the checks can use to provide
machine-readable results for the checks. You should work with that
instead of parsing the verbose messages.

-- 
Best regards,
Michał Górny


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Pacho Ramos]

El mar, 25-07-2017 a las 23:10 +1000, Michael Palimaka escribió:
> On 07/25/2017 05:22 PM, Dirkjan Ochtman wrote:
> > First, the assumption in our processes seems to be that many or
> > important bugs will be due to architecture-specific differences, and I
> > wonder if that assumption really holds up. Do arch testers for a smaller
> > arch often find problems that were not noticed on one of the larger
> > arches? With the languages and tools that we have today, it seems like
> > for many of our packages, bugs due to architectural differences
> > represent a minority of the problems we found. In this case, the whole
> > idea of per-arch stabilization does not really make sense, and doing
> > away with that idea could drastically shortcut our process.
> 
> This would be really interesting to know.

Anyway, I think it depends on the arch you are running. I remember to have seen
specific issues for ia64, hppa, ppc64 or arm. But, for example, I agree that,
*at present time*, I don't remember to have seen a package failing on x86 and
not on amd64 for example (well, I now remember a past systemd upstream runtime
bug that was catched in testing period ;)). 

Then, I guess it depends on each arch. For example, for x86 it could be probably
done if things work on amd64 :/. Between ppc and ppc64 I don't know. For the
others, I don't think that we can extrapolate between amd64 and ia64 for example
(I remember important runtime issues to be catched only affecting ia64 for
example).

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Pacho Ramos]

El mar, 25-07-2017 a las 22:59 +1000, Michael Palimaka escribió:
> On 07/25/2017 07:22 AM, Sergei Trofimovich wrote:
> > 2. Q: How to make arch testing faster and easier?
> > 
> >    A: - KEYWORDREQ/STABLEREQ bugs not marked as "runtime testing
> > required" will be automatically tested and keyworded.
> > 
> > [handwave] automated tinderbox setup would help a lot
> > to now upfront what fails to built, fails tests.
> 
> I've had similar thoughts about this and have already been working on
> some tooling for this.
> 
> We would need to establish exactly what criteria must be met for an
> automated test to be considered as successful.
> 
> Here's a sample report that my tool produces:
> https://dev.gentoo.org/~kensington/tinderbox/sys-apps/dbus/dbus-1.6.18-r1/df01
> 7e14-bd68-47e2-9738-554e7ba1cf10.html
> 
> In this case, would it be enough that it builds and tests pass? What
> about the QA issues?

Personally, I don't feel QA issues as major enough to block a stabilization,
usually they won't cause major issues for stable users and, if they do, for sure
they shouldn't be only QA issues :/

>  Do we need someone to review them to determine if
> they should block stabilisation, or if they're even a regression or not?
> 

Regarding general regressions, that is probably the harder point to handle
automatically. In the past, the scripts in arch-tools.git were avoiding to open
automated stable bug reports for packages having opened bugs (excluding
enhancement and QA bug reports), but that approach is not too good as, for
example, having a bug report asking for a version bump but tagged as "normal"
and not "enhancement" will lead to the bug not being opened :S

Then, I am unsure about if that part can be done automatically :/

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Rich Freeman]

On Tue, Jul 25, 2017 at 9:10 AM, Michael Palimaka  wrote:
>
> The 30 day waiting period is useful for smoking out major upstream bugs,
> but can't replace stabilisation integration testing. For example,
> package foobar may build fine in ~arch but fails in stable because it
> needs a newer libbaz.
>

I think this is a good reason why everything should be at least
build-tested on a stable tree before getting stabilized.  Now, that
might not be on each arch if it is truly arch-independent (and if
arches keep the dependencies reasonably in sync).

This might be a situation where a compromise could exist.  Have some
kind of flag (in metadata, or maybe the ebuild) that indicates that
the maintainer believes the package is low-risk to stabilize purely on
a build test.  Then after 30 days in testing a tinderbox could
build-test the package and stabilize it automatically.

If the package is considered at-risk then it goes through manual
testing, either by the maintainer or an arch team.

This will also encourage the teams doing testing to actually do more
testing on the packages that would benefit from it.

We wouldn't set hard criteria but leave it up to maintainer
discretion, with a guideline being that past performance is probably a
good predictor of future results.

-- 
Rich

[gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Michael Palimaka]

On 07/25/2017 05:22 PM, Dirkjan Ochtman wrote:
> First, the assumption in our processes seems to be that many or
> important bugs will be due to architecture-specific differences, and I
> wonder if that assumption really holds up. Do arch testers for a smaller
> arch often find problems that were not noticed on one of the larger
> arches? With the languages and tools that we have today, it seems like
> for many of our packages, bugs due to architectural differences
> represent a minority of the problems we found. In this case, the whole
> idea of per-arch stabilization does not really make sense, and doing
> away with that idea could drastically shortcut our process.

This would be really interesting to know.

> Second, I believe a lot of the value in our stable tree comes *just*
> from the requirement that stabilization is only requested after 30 days
> without major bugs/changes in the unstable tree. Assuming there are
> enough users of a package on unstable, that means important bugs can be
> shaken out before a version hits stable. This could mean that
> potentially, even if we inverted our entire model to say we
> "automatically" stabilize after a 30-day period without major bugs, we
> hit most of the value of the stable tree with again drastically reduced
> pain/work.

The 30 day waiting period is useful for smoking out major upstream bugs,
but can't replace stabilisation integration testing. For example,
package foobar may build fine in ~arch but fails in stable because it
needs a newer libbaz.

[gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Michael Palimaka]

On 07/25/2017 07:22 AM, Sergei Trofimovich wrote:
> 2. Q: How to make arch testing faster and easier?
> 
>A: - KEYWORDREQ/STABLEREQ bugs not marked as "runtime testing
> required" will be automatically tested and keyworded.
> 
> [handwave] automated tinderbox setup would help a lot
> to now upfront what fails to built, fails tests.

I've had similar thoughts about this and have already been working on
some tooling for this.

We would need to establish exactly what criteria must be met for an
automated test to be considered as successful.

Here's a sample report that my tool produces:
https://dev.gentoo.org/~kensington/tinderbox/sys-apps/dbus/dbus-1.6.18-r1/df017e14-bd68-47e2-9738-554e7ba1cf10.html

In this case, would it be enough that it builds and tests pass? What
about the QA issues? Do we need someone to review them to determine if
they should block stabilisation, or if they're even a regression or not?

[gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Agostino Sarubbo]

Hello Sergei,

thanks to bring into the topic which nowadays is a common point of discussion 
:)


On Monday 24 July 2017 22:22:23 Sergei Trofimovich wrote:
> 1. lack of automation
I'd summarize the techical steps into:
1) get the list of packages
2) test
3) commit to git
4) write on bugzilla

1 is done by getatoms https://github.com/kensington/bugbot
2 is done by the tester in the manner he prefer
3 no official tool available, I used a modified version of 
https://gitweb.gentoo.org/proj/arch-tools.git/tree/batch-stabilize.py which is 
still based on CVS
4 no official tool available, I used my own bash script which calls pybugz

So, points 3 and 4 needs to be improved, I have the idea on how the script 
should look, but I have no time to do it and no python knowledge. I can assist 
everyone that candidate itself to make the tool/script like I did with 
kensington when he made getatoms.

> 2. lack of manpower

lack of manpower, so in my opinion reduce a bit the workload. I proposed 
something in one of my last mail to -dev, the following refers to the arches 
with very less manpower:
1) Don't file keywordreq, since noone work on them. File directly stablereq.
2) Reduce the number of the stable packages on those arches
3) Make a more visible list (like this list in term of 
visibility:https://qa-reports.gentoo.org/output/gentoo-ci/output.html) of the 
arches-dependent bugs so that everyone can contribute to maintain alive the 
exotic arches.

-- 
Agostino Sarubbo
Gentoo Linux Developer

Re: [gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Hans de Graaff]

On Tue, 2017-07-25 at 04:34 +, Duncan wrote:
> 
> Automating stabilization and automated keyword dropping on timeouts
> seems 
> the only other practical choice, as unfortunately, "stale" is what
> we 
> have today in practice, if not in name.

Looking at https://repology.org/statistics stable isn't quite that
stale compared to other distributions. We're not doing great either, so
we should certainly try to improve.

Hans

signature.asc
Description: This is a digitally signed message part

[gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Duncan]

Rich Freeman posted on Mon, 24 Jul 2017 19:52:40 -0400 as excerpted:

> On Mon, Jul 24, 2017 at 7:22 PM, Peter Stuge  wrote:
>>
>> I hold a perhaps radical view: I would like to simply remove stable.
>>
>> I continue to feel that maintaining two worlds (stable+unstable)
>> carries with it an unneccessary cost.
>>
>>
> The question is whether devs would start being more conservative with
> ~arch if it essentially turned into the new stable?
> 
> If ~arch doesn't break then we're probably delaying updates too much.
> If it does start breaking and we don't have any alternative, we'll
> probably start losing users who just can't deal with their systems
> breaking.
> 
> Personally I'd rather see stable stick around.  If it isn't updated
> often that isn't a big deal (to me at least).

Indeed, while along with Peter I have little personal use for stable 
(~arch is my stable, live-git my unstable, and stale arch, well, stale), 
I've come to realize over the years that there's enough gentooers, both 
users and devs, that do stable, that killing it isn't going to be the 
boon people only looking at all that "wasted" effort might believe it to 
be.

Instead, were gentoo to lose stable, it'd ultimately shrink as both users 
and devs that previously found gentoo stable the most effective 'scratch' 
to their 'configurable stability itch', were forced to look elsewhere to 
scratch that itch.  While there's a small chance it'd be an incremental 
gain for gentoo ~arch, there's a far larger chance it'd be the beginning 
of the end as without stable, the gentoo community could easily shrink 
into unsustainability -- too few people ever considering being users to 
produce enough incoming developers to maintain gentoo ~arch at anything 
close to the level we have now.


OTOH, there may be a case to be made for the implications of Rich's 
suggestion... and mine above.  Arguably just lose the pretense and simply 
rename stable -> stale, and let people that want/need it continue to deal 
with it on those terms.  At least that way gentoo security advisories, 
etc could then be for "gentoo stale", and as such wouldn't look so dated 
when they come out half a year after the upstream public vulnerability 
and patch and/or unaffected release announcements, because that's what it 
took to stabilize the patched version on some platform or other that was 
holding up the glsa.

Automating stabilization and automated keyword dropping on timeouts seems 
the only other practical choice, as unfortunately, "stale" is what we 
have today in practice, if not in name.

So yes, I support the initiative. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: [RFC] Future of gentoo's stable and unstable trees: what are your thoughts?

[Pacho Ramos]

El lun, 24-07-2017 a las 22:22 +0100, Sergei Trofimovich escribió:
> 4. Q: How to push more packages into STABLE?
> 
>    A: File automatic STABLEREQ bugs more aggressively if no known bugs
>   exist for a package version. The rough workflow is the following:
> 
>   - Grab a list of candidates for stabilization (will need
>     additional tooling)
>   - File STABLEREQ against maintainer only first.
>   - Maintainer will have a 2-week timeout to either proceed with
>     request:
>     - add "runtime testing required fields", CC relevant arches to
>   start stabilization
>     - or set blocking bugs against STABLEREQ to stop stabilization
>   - After 2-week timeout tooling automatically CCes arches and
>     stabilization happens (ideally with minimal manual work)
>   - Profit!

I think the tools for this were already developed... but they were relying on
some of us remembering to run them from time to time :/
https://gitweb.gentoo.org/proj/arch-tools.git/tree/file-stabilization-bugs.py
https://gitweb.gentoo.org/proj/arch-tools.git/tree/maintainer-timeout.py
https://gitweb.gentoo.org/proj/arch-tools.git/tree/stabilization-candidates.py

Thanks specially for sharing https://wiki.gentoo.org/wiki/Package_testing#Tools
link, it summarizes all the process really well :O

Also, it seems that tatt- is able to do most of the work... then only thing
that maybe has no official script to get it, for example, what would be the best
way of getting the list of bug numbers that are pending to handle?

For example, if I go to the list manually and pick 10 random bugs, it's ok to
run tatt and copy every bug number but, if the arch queue has 300 pending
bugs. Do we have a way to simply get a plain text with all the bug numbers from
bugzilla interface? Having links per arch pointing to that list of bugs with
arches CCed and sanity-check=+ would be probably helpful ;)

Thanks