Re: [gentoo-dev] avoiding urgent stabilizations

[Ed W]

On 26/02/2011 15:57, Enrico Weigelt wrote:

* Ed Wli...@wildgooses.com  schrieb:


I'm just building some embedded devices on the side using
gentoo and my minimal builds are only a few MB?

How to do you get out all the buildtime stuff (portage, toolchain, etc) ?


Seems like your complaint is that you have gentoo installs which are
full featured with a toolchain and portage, which you are comparing to
an installation you built with a different tool that doesn't have a
toolchain installed?

Well, with Briegel, I'm strictly differenciating between the build
host and the target image. Everything's crosscompiled by design.
So I can explicitly define which packages need to be in the final
image (and only those are built). The build host can by any fairly
recent GNU/Linux system (happens to be Gentoo at my site)


I think you are missing the point that the gentoo build system is quite 
happily building you a destination build with as little as one package 
in it? Whether that destination boots is a function of careful package 
choice, but the point is you are definitely not constrained from 
building a highly minimal target if you wish?


Just from the top of my head I should imagine this would likely boot:

mkdir -p /newroot/{proc,dev,sys}
cp -a /dev/{null,zero} /newroot/dev/
emerge —root /newroot glibc shadow baselayout openrc util-linux udev


If you want to go more minimal then I have a uclibc chroot that I use as 
a build environment, the first line of my build script for my minimal 
embedded device is simply:


emerge --root /embedded baselayout uclibc busybox

I haven't checked, but give or take creating /proc, /sys and some 
minimal devices in /dev, the above should boot fine.




If you were booting into a container based virtualisation then you 
don't need to manage /dev and mounting devices and so you can get even 
more minimal than the top example I posted (ie I don't think you need 
udev, etc)


Obviously such minimal builds will have minimal package management, but 
this may not be a problem if you have a super fixed configuration? 
Simply construct your environment such that the data is separate from 
the OS and also make sure any OS customisation is scriptable and now you 
would have a way to occasionally build a new image which you rolled out 
to all virtual machines




1) Build your base images without a toolchain or portage and use a
minimal package installer to install pre-built binary packages.
This seems fraught with issues long term though...

How can this be done w/ Gentoo ?
AFAIK it always has a system set containing things like portage,
toolchain, etc, etc.


System is just a statement of some defaults. At the level you are 
working at you can largely ignore it.


Build what you like into the target build



Would be still too much for my goals: I don't wanna have anything in
the final system that's not necessary for runtime. Building and even
installation should be driven from outside (actually, the final system
could even have /bin, /usr, etc mounted ro and /var w/o noexec)


OK, so sounds like a simple build script like the one above would suit you?

For more advanced use consider learning to use catalyst or Funtoos build 
system? These are full on scriptable ways to repeatably build non 
trivial target builds. (remember they can build as little as single 
package targets or more complicated builds)




Note that Gentoo's binary packages (which are practically snapshots
of what will be copied in by merge stage) are alway bound to specific
domains, eg. the full configuration (make.conf, useflags, etc, etc),
and there may be a lot of interdependencies between installed packages
(and versions), so it takes some care to use it in stable ways.

This might be very fine for you (eg. if you have dozens of really
equal systems), but you'll have to be aware of the limitations.


Well, your requirements suggest that you don't need a package manager 
and you will instead build a new read-only OS build every so often and 
deploy that. However, in the interests of exploring the idea further:


- By definition binary packages only work where you have equal systems? 
This is true of gentoo AND rpm/deb.
- If you need binary packages for non equal systems then you need some 
kind of namespacing method to separate the binaries. Gentoo has no 
automated way to do this, but it's reasonably workable to have binary 
package repos and simply tell your guest machine to use the appropriate 
binary repo
- Gentoo's binaries are slightly cleverer than some notice in that they 
have some awareness of when they don't match the target and the operator 
has the option to either ignore that and continue the install, OR to 
ignore the binary package and generate it's own binaries using the local 
compiler, etc.


So, I don't really see the problem, in fact Gentoo binaries can be seen 
as slightly more flexible than RPMs to some extent if you allow them to 
be optional.




As long as you have the same 

Re: [gentoo-dev] avoiding urgent stabilizations

[Ed W]

Hi


But, for me, even a trimmed-down Gentoo is still too large
(has to contain the whole base packages, from portage to
toolchain, includes, etc). I'd prefer having only the essential
runtime stuff within the containers.


I'm just building some embedded devices on the side using gentoo and my 
minimal builds are only a few MB? Curious why you feel you need to move 
from Gentoo to get the size smaller?


Seems like your complaint is that you have gentoo installs which are 
full featured with a toolchain and portage, which you are comparing to 
an installation you built with a different tool that doesn't have a 
toolchain installed?  However, you can do the same using gentoo if you 
wish? (you just need a lightweight package installer to avoid installing 
portage)


I think your main options are:

1) Build your base images without a toolchain or portage and use a 
minimal package installer to install pre-built binary packages.  This 
seems fraught with issues long term though...


2) Build your base images without a toolchain, but with portage (and 
perhaps a very minimal python). This gives you full dependency tracking 
and obviously bind mount/nfs mount the actual portage tree to avoid 
space used there. This seems workable and minimal?


3) If we are talking virtual machines then who cares if your containers 
are individually quite large, if the files in them are duplicated across 
all containers?  Simply use an appropriate de-duplication strategy to 
coalesce the space and most of the disadvantages disappear?  eg 
linux-vserver you can simply hardlink all the common files across your 
installations and allow the COW patch to break hardlinks if anyone 
alters a file in a single instance. Or you could use aufs to mount a 
writeable layer over your common base VM instance?  Or you could use one 
of the filesystems which de-duplicates files in the background (some 
caveats apply here to avoid memory still being used multiple times in 
each VM).  Or under KVM there is the memory coalescing feature which 
merges similar code pages (forget it's name?)


Personally I think option 3) is quite interesting from the medium number 
of virtual machines, ie in the 10s to hundreds, ie simply don't worry 
about it, let the OS do the work.  In the hundreds to thousands plus 
level I guess you have unique challenges and I would be wrong to try and 
suggest a solution from the comfort of a laptop without having that 
responsibility, but I would have thought there was some advantage in a 
very rigidly deployed base OS generated and updated very precisely?




For this we need a different approach (strictly separating build
and production environments). Binary distros (eg. Debian) might
be one option, but they're lacking the configurability and mostly
are still too large. So I'm going a different route using my own
buildsystem - called Briegel - which originally was designed for
embedded/small-device targets.

For now I didn't have the spare time to port all the packages
required for complete server systems (most of it is making
them all cleanly crosscompile'able, as this is a fundamental
concept of Briegel). But maybe you'd like to join in and try it :)


Sounds like an interesting challenge, but I'm unconvinced you can't 
solve 90% of your problem within the constraints of Gentoo? This saves 
you a bunch of time that could be invested in the last 10% through more 
traditional means?




It does appear like managing large numbers of virtual machines is one
are that gentoo could score very well?  Interested to see any chatter on
how others solve this problem, or any general advocacy?  Probably we
should start a new thread though...

I'm not sure if Gentoo really is the right distro for that purpose,
as it's targeted to very different systems (i.g. Gentoo boxes are
expected to be quite unique, beginning with different per-package
useflags, even down to cflags, etc). But it might still be a good
basis for building specific system images (let's call them stage5 ;-))


I won't disagree on your where it's targeted, but just to re-iterate 
why I think Gentoo works well is that it does have a very workable 
binary package feature!


My way of working is to use (several) shared binary package repos and 
the guests largely pull from those shared package directories.  In fact 
what I do is have a minimal number of shared /usr/portage/package 
directories and I mount an appropriate one to the guest type at boot 
time.  At the moment my main two options are 32bit and 64bit for the 
package mounts, but I recently introduced a new machine type which is 
held back to perl 5.8 and that guest gets it's own package mount since 
it's obviously linking a lot of binaries differently


So, my process is to test an update on a small number of guests, either 
dedicated test guests or less important live guests.  If this looks good 
then I run the upgrade against all other Vms of the same type and they 
will update quickly from package binaries

Re: [gentoo-dev] avoiding urgent stabilizations

[Enrico Weigelt]

* Ed W li...@wildgooses.com schrieb:

 I'm just building some embedded devices on the side using
 gentoo and my minimal builds are only a few MB? 

How to do you get out all the buildtime stuff (portage, toolchain, etc) ?

 Seems like your complaint is that you have gentoo installs which are 
 full featured with a toolchain and portage, which you are comparing to 
 an installation you built with a different tool that doesn't have a 
 toolchain installed? 

Well, with Briegel, I'm strictly differenciating between the build
host and the target image. Everything's crosscompiled by design.
So I can explicitly define which packages need to be in the final
image (and only those are built). The build host can by any fairly
recent GNU/Linux system (happens to be Gentoo at my site)

 1) Build your base images without a toolchain or portage and use a 
 minimal package installer to install pre-built binary packages.
 This seems fraught with issues long term though...

How can this be done w/ Gentoo ?
AFAIK it always has a system set containing things like portage,
toolchain, etc, etc.

 2) Build your base images without a toolchain, but with portage (and 
 perhaps a very minimal python). This gives you full dependency tracking 
 and obviously bind mount/nfs mount the actual portage tree to avoid 
 space used there. This seems workable and minimal?

Would be still too much for my goals: I don't wanna have anything in 
the final system that's not necessary for runtime. Building and even
installation should be driven from outside (actually, the final system
could even have /bin, /usr, etc mounted ro and /var w/o noexec)

 3) If we are talking virtual machines then who cares if your containers 
 are individually quite large, if the files in them are duplicated across 
 all containers?  Simply use an appropriate de-duplication strategy to 
 coalesce the space and most of the disadvantages disappear?

Well, that might be an option, if the *is* enough duplication
(within an storage enclousure). But storage space is not the only
reason. My personal background is from medical and industrial
appliances, so I have some more rigid rules ;-o

 Personally I think option 3) is quite interesting from the medium number 
 of virtual machines, ie in the 10s to hundreds, ie simply don't worry 
 about it, let the OS do the work.  In the hundreds to thousands plus 
 level I guess you have unique challenges and I would be wrong to try and 
 suggest a solution from the comfort of a laptop without having that 
 responsibility, but I would have thought there was some advantage in a 
 very rigidly deployed base OS generated and updated very precisely?

I'm thinking of several thousands of vm's for dedicated purposes,
which fall under similar requirements as industrial embedded
systems (eg. there will be practically no human operator).

 For this we need a different approach (strictly separating build
 and production environments). Binary distros (eg. Debian) might
 be one option, but they're lacking the configurability and mostly
 are still too large. So I'm going a different route using my own
 buildsystem - called Briegel - which originally was designed for
 embedded/small-device targets.
 
 For now I didn't have the spare time to port all the packages
 required for complete server systems (most of it is making
 them all cleanly crosscompile'able, as this is a fundamental
 concept of Briegel). But maybe you'd like to join in and try it :)
 
 Sounds like an interesting challenge, but I'm unconvinced you can't 
 solve 90% of your problem within the constraints of Gentoo? This saves 
 you a bunch of time that could be invested in the last 10% through more 
 traditional means?

Maybe. But I need 100%. (cant explain all the reasons here right now).
I'm well aware that this all is far out of Gentoo's scope.

 I'm not sure if Gentoo really is the right distro for that purpose,
 as it's targeted to very different systems (i.g. Gentoo boxes are
 expected to be quite unique, beginning with different per-package
 useflags, even down to cflags, etc). But it might still be a good
 basis for building specific system images (let's call them stage5 ;-))
 
 I won't disagree on your where it's targeted, but just to re-iterate 
 why I think Gentoo works well is that it does have a very workable 
 binary package feature!

Note that Gentoo's binary packages (which are practically snapshots
of what will be copied in by merge stage) are alway bound to specific
domains, eg. the full configuration (make.conf, useflags, etc, etc),
and there may be a lot of interdependencies between installed packages
(and versions), so it takes some care to use it in stable ways.

This might be very fine for you (eg. if you have dozens of really
equal systems), but you'll have to be aware of the limitations.

 My way of working is to use (several) shared binary package repos and 
 the guests largely pull from those shared package directories.  In fact 
 what I do is have a minimal 

Re: [gentoo-dev] avoiding urgent stabilizations

[Ed W]

On 21/02/2011 00:11, Enrico Weigelt wrote:

* Markos Chandrashwoar...@gentoo.org  schrieb:


My suggestion, as I said to fosdem, is to freeze, or take a
snapshot if you like, of the current tree, stabilize what you
need to stabilize, test the whole tree ( at least compile wise )
for a couple of weeks and then replace the existing stable tree.

hmm, would it make sense to add a new masking for the testing
tree, so users could decide which stability grade vs they wish ?
or perhaps use overlays for that ?

For example, I'd like to have the critical packages (everything
that's needed to bootup and do basic remote maintenance) from
the new frozen-stable tree, but other things should stay as
they are.



Perhaps this is an argument for a git based portage tree?  Master can 
stay as the current status quo and anyone who wants to can maintain a 
branch or fork which points to a slightly different subset of the tree?


I doubt we actually have the capacity to make this work, but it would at 
least in theory be cool to have a (weekly/monthly) branch which gets 
cut, run through a tinderbox in various forms and then pushed?  Or if 
someone wants to maintain a redhat style antique set of packages where 
the tree is largely held back to 2005 state with only bug fixes and 
essential packages bumped?


Just thinking...

Ed W

Re: [gentoo-dev] avoiding urgent stabilizations

[Matthew Marlowe]

All,

 Perhaps this is an argument for a git based portage tree?  Master can
 stay as the current status quo and anyone who wants to can maintain a
 branch or fork which points to a slightly different subset of the tree?
 

I'm starting to put together a portage/stable server configuration for a large 
number of gentoo VM's that will eventually be hosted on a VMware ESX 4.1U1 
cluster - with the goal of limiting major changes to once/year and otherwise 
only applying security/minimum necessary updates.  I doubt it will be easy but 
I'm doing my best at it :)

As part of that I'm maintaining on github several related repositories, 
including portage mask/use/config files, cluster management utilities, etc.

 https://github.com/deploylinux

I've also started to document work on my blog:

 http://www.deploylinux.net/matt

I'm not currently planning to utilize a separate overlay for packages/ebuilds, 
but it's not out of the question.

I'd be happy to work with any other devs w/ similiar interests or production 
networks to manage.github makes group development relatively easy.

MattM
-- 
Matthew Marlowe/  858-400-7430  /DeployLinux Consulting, Inc
  Professional Linux Hosting and Systems Administration Services
  www.deploylinux.net   *   m...@deploylinux.net
 'MattM' @ irc.freenode.net
   

Re: [gentoo-dev] avoiding urgent stabilizations

[Ed W]

Hi


I'm starting to put together a portage/stable server configuration for a large
number of gentoo VM's that will eventually be hosted on a VMware ESX 4.1U1
cluster - with the goal of limiting major changes to once/year and otherwise
only applying security/minimum necessary updates.  I doubt it will be easy but
I'm doing my best at it :)


This sounds very interesting.  I haven't yet plugged through your blog, 
but just to chime in:


I maintain a, likely much smaller, number of VMs using linux vservers.  
The approach here is to almost cut each machine down to a chroot that 
runs only one (or thereabouts) interesting service.  To do this I have 
found customised portage profiles to be the under-plugged secret since 
they allow you to basically push a set of packages which should be 
installed and control per type of vm use flags and package keywords 
(eg I have www-nginx, www-apache, mail, proxy, mysql, ftp, etc 
profiles).  Additionally I have a small overlay of local ebuilds that 
sit in the same tree


Up until now I haven't really made any effort to sync this whole tree 
across multiple physical machines and it's a bit of an ad-hoc process.  
Using something like git would probably be perfect


The still missing step is configuration management across the machine 
types, eg I want to upgrade all my Apache-WWW class machines and merge 
in all changes in /etc in a certain way... At the moment I just run 
dispatch-conf across all machines, but it can be quite boring merging 20 
instances of sshd.conf...  Seems like Puppet/Chef could be a solution 
here, but the step up and investment to make it work seems pretty large?




It does appear like managing large numbers of virtual machines is one 
are that gentoo could score very well?  Interested to see any chatter on 
how others solve this problem, or any general advocacy?  Probably we 
should start a new thread though...


Regards

Ed W

Re: [gentoo-dev] avoiding urgent stabilizations

[Enrico Weigelt]

* Ed W li...@wildgooses.com schrieb:

 I maintain a, likely much smaller, number of VMs using linux vservers.  
 The approach here is to almost cut each machine down to a chroot that 
 runs only one (or thereabouts) interesting service.

I'm working in a similar way: my dedicated boxes are VM hosts
(currently ovz, but later lguest or lxc), each of them running
only specific services (eg. one for nginx-based front proxy,
several other for backend webservers, RDBMs'es, mailservers, etc).

But, for me, even a trimmed-down Gentoo is still too large
(has to contain the whole base packages, from portage to
toolchain, includes, etc). I'd prefer having only the essential
runtime stuff within the containers.

For this we need a different approach (strictly separating build
and production environments). Binary distros (eg. Debian) might
be one option, but they're lacking the configurability and mostly
are still too large. So I'm going a different route using my own
buildsystem - called Briegel - which originally was designed for
embedded/small-device targets.

For now I didn't have the spare time to port all the packages
required for complete server systems (most of it is making
them all cleanly crosscompile'able, as this is a fundamental
concept of Briegel). But maybe you'd like to join in and try it :)

 Using something like git would probably be perfect

I'm using git'ed portage trees in production for several month now
(I sometimes had to touch some eclasses, which _IMHO_ doesn't via
overlays). The repos are configured to rebase on pull and everything
runs automatically :)

 The still missing step is configuration management across the machine 
 types, eg I want to upgrade all my Apache-WWW class machines and merge 
 in all changes in /etc in a certain way... At the moment I just run 
 dispatch-conf across all machines, but it can be quite boring merging 20 
 instances of sshd.conf...  Seems like Puppet/Chef could be a solution 
 here, but the step up and investment to make it work seems pretty large?

I haven't used puppet yet, but several collegues have made good
experiences with it. If you're maintaining dozens of very similar
systems (mine tend to be very different from each other), it likely
worth investigating.

 It does appear like managing large numbers of virtual machines is one 
 are that gentoo could score very well?  Interested to see any chatter on 
 how others solve this problem, or any general advocacy?  Probably we 
 should start a new thread though...

I'm not sure if Gentoo really is the right distro for that purpose,
as it's targeted to very different systems (i.g. Gentoo boxes are
expected to be quite unique, beginning with different per-package
useflags, even down to cflags, etc). But it might still be a good
basis for building specific system images (let's call them stage5 ;-))

An setup for 100 equal webserver vm's could look like this:

* run a normal Gentoo vm (tailored for the webserver appliance),
  where do you do regular updates (emerge, revdep-rebuild, etc, etc)
* from time to time take a snapshot, strip off the buildtime-only
  stuff (hmm, could turn out to be a bit tricky ;-o)
* this stripped snapshot now goes into testing vm's
* when approved, the individual production vm's are switched over
  to the new image (maybe using some mount magic, unionfs, etc)


At this point I've got a question for to the other folks here:

emerge has an --root option which allows to (un)merge in a separate
system image. So it should be possible to unmerge a lot of system
packages which are just required for updating/building (even
portage itself), but this still will be manual - what about 
dependency handling ?

Is there some way to drop at least parts of the standard system set,
so eg. portage, python, gcc, etc, etc get unmerged by --depclean
if nobody else (in world set) doesn't explicitly require them ?


cu
-- 
--
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weig...@metux.de
 mobile: +49 151 27565287  icq:   210169427 skype: nekrad666
--
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
--

Re: [gentoo-dev] avoiding urgent stabilizations

[Enrico Weigelt]

* Markos Chandras hwoar...@gentoo.org schrieb:

 My suggestion, as I said to fosdem, is to freeze, or take a
 snapshot if you like, of the current tree, stabilize what you
 need to stabilize, test the whole tree ( at least compile wise )
 for a couple of weeks and then replace the existing stable tree. 

hmm, would it make sense to add a new masking for the testing
tree, so users could decide which stability grade vs they wish ?
or perhaps use overlays for that ?

For example, I'd like to have the critical packages (everything
that's needed to bootup and do basic remote maintenance) from
the new frozen-stable tree, but other things should stay as
they are.


cu
-- 
--
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weig...@metux.de
 mobile: +49 151 27565287  icq:   210169427 skype: nekrad666
--
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
--

Re: [gentoo-dev] avoiding urgent stabilizations

[Enrico Weigelt]

* Fabian Groffen grob...@gentoo.org schrieb:

 Hmmm, odd.  I experience amd64 (stable) as being pretty stable on my
 servers.  Last breakage which really got me upset was php, but that's
 already some time ago.

the ini file issue ?

 With Gentoo you should update on fairly regular intervals, and have the
 time inbetween as short as possible, but 2 or 3 weeks appears to be
 fine.  I myself have a cronjob that syncs every night, and mails me the
 output of emerge -Dupv world.  When this list gets too large, it's
 typically about time to do some updating.

I've automatized even a bit more: my cron script also automatically
rebuilds the packages I have explicitly whitelisted in my
/etc/portage/package.autoupdate file.

Let me know if anyone likes to have it.

 I have masked new major releases of PostgreSQL and MySQL for instance,
 and of course Python 3. 

/me too.

Perhaps we can find a way to make the update safe (so eg. new
postmaster is installed along the old one) and provide some
migration tool ?


cu
-- 
--
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weig...@metux.de
 mobile: +49 151 27565287  icq:   210169427 skype: nekrad666
--
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
--

Re: [gentoo-dev] avoiding urgent stabilizations

[Enrico Weigelt]

* Pawe?? Hajdan, Jr. phajdan...@gentoo.org schrieb:

 By the way, to turn this thread into some action: what testing do we
 currently perform for auto-generated stages? It'd be cool to at least
 compile-test that the stage can emerge -e world itself, and emerge
 some common packages (with FEATURES=test so that we run at least some
 of the produced code).

If we have enough VM's, we could also collect configs and perhaps
even example workloads from users which run their updates with
~arch automatically and look whether they still run properly.


cu
-- 
--
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weig...@metux.de
 mobile: +49 151 27565287  icq:   210169427 skype: nekrad666
--
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
--

Re: [gentoo-dev] avoiding urgent stabilizations

[Rich Freeman]

On Tue, Feb 8, 2011 at 12:57 PM, Fabian Groffen grob...@gentoo.org wrote:
 On 08-02-2011 18:46:32 +0100, Andreas K. Huettel wrote:
  Other than monitoring bugzilla, how does a Gentoo user even know that they
  have a package pending a security update?  It seems like glsa's lag
  stabilization by a considerable timeframe.

 Yep. GLSA is something that seems to happen roughly one year after no 
 affected package is in tree anymore.

 Well, it's not too bad lately:
 http://archives.gentoo.org/gentoo-announce/

So I'll agree that it is better now in the sense that we're actually
publishing them at all.

However, it still seems non-ideal.  Take this bug for example:
http://bugs.gentoo.org/show_bug.cgi?id=351920

amd64/x86 were stable weeks ago, but the GLSA still isn't published
because we're waiting on one arch.  That means that anybody who does
updates once a quarter or whatever except for security updates will be
vulnerable, because they don't know they still have a vulnerability.

Even after the last arch is updated it often takes a little time to
get the GLSA published.

About the only thing glsa-checking tools do for me is bug me about
having libpng-1.2.44   installed (bug 340261 - most likely glsa is
incorrect).  I almost never catch vulnerabilities on my live system
that way since even if I'm slow I get the updates installed before the
glsa comes out anyway.  However, I do get noise sometimes.

Perhaps we should target having glsas published within a certain
amount of time after a vulnerability is disclosed, whether corrected
or not.  We could re-publish a final notice once all is well.  We
really shouldn't consider users safe from a security vulnerability
until the vulnerability is patched in the tree AND the notice to
update has been sent out.

Rich

Re: [gentoo-dev] avoiding urgent stabilizations

[Paweł Hajdan, Jr.]

On 2/9/11 2:57 PM, Rich Freeman wrote:
 Perhaps we should target having glsas published within a certain
 amount of time after a vulnerability is disclosed, whether corrected
 or not.  We could re-publish a final notice once all is well.  We
 really shouldn't consider users safe from a security vulnerability
 until the vulnerability is patched in the tree AND the notice to
 update has been sent out.

I think http://www.gentoo.org/security/en/vulnerability-policy.xml
specifies the target delay, and also mentions temporary GLSAs.
Unfortunately, that process does not seem to be followed due to general
difficulty of drafting GLSAs (I don't even know what is the problem, as
GLSAmaker is only available to security team members).



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Rich Freeman]

On Wed, Feb 9, 2011 at 9:08 AM, Paweł Hajdan, Jr.
phajdan...@gentoo.org wrote:
 I think http://www.gentoo.org/security/en/vulnerability-policy.xml
 specifies the target delay, and also mentions temporary GLSAs.
 Unfortunately, that process does not seem to be followed due to general
 difficulty of drafting GLSAs (I don't even know what is the problem, as
 GLSAmaker is only available to security team members).


I think the policy itself is completely appropriate, and of course
publishing it makes the process transparent to the users.

I think our problem is more with complying with that policy.

I have heard similar complaints about GLSAmaker.  I half-wonder if it
would make more sense to just edit the xml files directly and validate
them with a tool, and send out an email, if the tool really is that
bad.

Could the security team use a staff position of some kind that an
interested user could take on that handled some of the more
administrative aspects of security bugs?  Maybe we aren't that bad at
fixing our code, but nobody wants to sit around tinkering with
notices/etc.  Perhaps we might have interested users who wouldn't mind
sending out notices and closing bugs who otherwise might not want to
or be able to maintain ebuilds/etc?

Rich

Re: [gentoo-dev] avoiding urgent stabilizations

[Donnie Berkholz]

On 10:26 Wed 09 Feb , Rich Freeman wrote:
 I have heard similar complaints about GLSAmaker.  I half-wonder if it 
 would make more sense to just edit the xml files directly and validate 
 them with a tool, and send out an email, if the tool really is that 
 bad.

If this is really the problem, please propose a GSoC project:

http://en.gentoo-wiki.com/wiki/Google_Summer_of_Code_2011_ideas

Sure, it's a ways off, so maybe we'd need an interim workaround. But at 
least we'd be likely to end up with a fix in the end, and more GSoC 
ideas are always welcome (we had 19 students last year!).

-- 
Thanks,
Donnie

Donnie Berkholz
Sr. Developer, Gentoo Linux
Blog: http://dberkholz.com


pgpMyQryx46Ud.pgp
Description: PGP signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Robin H. Johnson]

On Wed, Feb 09, 2011 at 10:26:19AM -0500, Rich Freeman wrote:
 I have heard similar complaints about GLSAmaker.  I half-wonder if it
 would make more sense to just edit the xml files directly and validate
 them with a tool, and send out an email, if the tool really is that
 bad.
a3li has been working on GLSAmaker2. I don't know the status.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee  Infrastructure Lead
E-Mail : robb...@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

Re: [gentoo-dev] avoiding urgent stabilizations

[Markos Chandras]

On Mon, Feb 07, 2011 at 10:02:36PM +0100, Paweł Hajdan, Jr. wrote:
 On 2/7/11 9:50 PM, Markos Chandras wrote:
  My suggestion, as I said to fosdem, is to freeze, or take a
  snapshot if you like, of the current tree, stabilize what you need to
  stabilize, test the whole tree ( at least compile wise ) for a couple of
  weeks and then replace the existing stable tree. Of course this requires
  automated script testing, hardware facilities etc etc that we don't have
  so claiming that stable tree is stable is quite wrong.
 
 This more thorough testing sounds really interesting. But do we really
 lack hardware resources?

Yes!
 There are machines available for various arches at
 http://www.gentoo.org/proj/en/infrastructure/dev-machines.xml. I have
 at least a few chromium-related chroots on miranda, and I've never heard
 complaints, so it seems a few more chroots for arch testing wouldn't hurt.
 
No. Miranda, in particular, can go down anytime soon. This is what infra
told me twice. 

 Of course for testing bootability and whether X11 starts up correctly,
 etc we'd probably have to host some virtual machines, but better compile
 testing (for example for toolchain updates) would be a good start.
 
 Paweł
 
Lets move this conversation on the thread that Tim started earlier
today. Seems like there is some hardware available for us on OSUOSL

Regards,
-- 
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2


pgpKH5be3N1th.pgp
Description: PGP signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Paweł Hajdan, Jr.]

On 2/8/11 9:24 AM, Markos Chandras wrote:
 On Mon, Feb 07, 2011 at 10:02:36PM +0100, Paweł Hajdan, Jr. wrote:
 There are machines available for various arches at
 http://www.gentoo.org/proj/en/infrastructure/dev-machines.xml. I have
 at least a few chromium-related chroots on miranda, and I've never heard
 complaints, so it seems a few more chroots for arch testing wouldn't hurt.

 No. Miranda, in particular, can go down anytime soon. This is what infra
 told me twice. 

Oh, they didn't tell me that. What's the problem with miranda?



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Roy Bamford]

On 2011.02.07 20:50, Markos Chandras wrote:
[snip]

 My suggestion, as I said to fosdem, is to freeze, or take a
 snapshot if you like, of the current tree, stabilize what you need to
 stabilize, test the whole tree ( at least compile wise ) for a couple
 of weeks and then replace the existing stable tree. Of course this
 requires automated script testing, hardware facilities etc etc that 
 we don't have so claiming that stable tree is stable is quite 
 wrong.
 
 Regards,
 -- 
 Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
 

Markos,

This is exactly what releng used to do for installer CDs.  This was 
last used for 2008.0 CD/DVD. A snapshot of the stable tree was taken in 
February 200.8 and the release hit the mirrors in September.

The seven month test/fix/retest that it took meant that the CD would 
not boot on new hardware as the kernel lacked drivers.

You will find similar lags when releng used this approach for other 
earlier releases.

We would need to move to a release cycle like the kernel. Calling a 
feature freeze at the start of the test cycle. As we can't everything, 
we might as well distribute binaries of what was tested - just as 
releng used to do. 

To me thats not Gentoo as we would loose the rolling updates.
 
There are degrees of stable.  I believe most Gentoo users 
realise this fairly early on and stick with Gentoo because they like 
the balance it strikes between Debian stable and bleeding edge.
There is a price to pay for being more up to date and it a trade off 
Gentoo users are aware off. Of course, that does not prevent them 
bringing breakages to our attention. 

-- 
Regards,

Roy Bamford
(Neddyseagoon) a member of
gentoo-ops
forum-mods
trustees


pgpXWHwIwNyRh.pgp
Description: PGP signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Markos Chandras]

On Tue, Feb 08, 2011 at 11:43:33AM +, Roy Bamford wrote:
 On 2011.02.07 20:50, Markos Chandras wrote:
 [snip]
 
  My suggestion, as I said to fosdem, is to freeze, or take a
  snapshot if you like, of the current tree, stabilize what you need to
  stabilize, test the whole tree ( at least compile wise ) for a couple
  of weeks and then replace the existing stable tree. Of course this
  requires automated script testing, hardware facilities etc etc that 
  we don't have so claiming that stable tree is stable is quite 
  wrong.
  
  Regards,
  -- 
  Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
  
 
 Markos,
 
 This is exactly what releng used to do for installer CDs.  This was 
 last used for 2008.0 CD/DVD. A snapshot of the stable tree was taken in 
 February 200.8 and the release hit the mirrors in September.
 
 The seven month test/fix/retest that it took meant that the CD would 
 not boot on new hardware as the kernel lacked drivers.
 
 You will find similar lags when releng used this approach for other 
 earlier releases.
 
 We would need to move to a release cycle like the kernel. Calling a 
 feature freeze at the start of the test cycle. As we can't everything, 
 we might as well distribute binaries of what was tested - just as 
 releng used to do. 
 
 To me thats not Gentoo as we would loose the rolling updates.
  
 There are degrees of stable.  I believe most Gentoo users 
 realise this fairly early on and stick with Gentoo because they like 
 the balance it strikes between Debian stable and bleeding edge.
 There is a price to pay for being more up to date and it a trade off 
 Gentoo users are aware off. Of course, that does not prevent them 
 bringing breakages to our attention. 
 
 -- 
 Regards,
 
 Roy Bamford
 (Neddyseagoon) a member of
 gentoo-ops
 forum-mods
 trustees

Roy, 

I see what you are saying. However, the 6 months testing is far from
what I have in mind. My only intention is to bring a more stable
experience to our users. Or, stop claiming that our stable tree rocks
and Gentoo is perfect for servers because it is not. Ye ye ye I know
that many many of you have Gentoo on servers but do not forget that you
are developers and you know your way around during breakages. Yes,
stable tree breaks FAR TOO often. I blame myself for my arch testing of
course however I can't do much about that. Arch teams cannot afford any
slacking (at least the two major arches) because we have 100 new bugs
every three days. This is extremely dangers to demotivate people who
will burn out sooner or later. You may think that this is an extreme
case but I can guarantee you that amd64 stable tree can become really
obsolete within one month. . 

Our stable tree is definitely not suitable for server usage unless 
you have plenty of free time to
deal with stupid upgrades because nobody, for example, cared to write a
proper elog or news item. You are probably not aware of that, since 99%
of you run testing tree however if you visit forums and stuff you will
see many many users complaining about stable tree. If we keep going down
this road we will end up sooner or later to be similar to Exherbo where
only really power users and developers will know their way around. 

Either you like it or not, arch teams are understaffed. All of them.
Therefore we cannot afford a updated stable tree with high QA around
it. We need to find a more efficient way to test packages on testing
tree so we can mark them stable with minimal time and cpu cost. We need
dedicated build boxes, like Diego's tinderbox, to test the testing tree
over and over against critical/common/trivial QA problems. If we manage
that, moving packages from testing-stable will be much more time
efficient and we can guarantee a high quality stable tree.

ps1: Personally I have stopped suggesting gentoo stable for server usage
and I always suggest testing to new users.

ps2: Roy, this is not a personal attack. Do not misinterpret my tone :)

Regards,
-- 
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2


pgp2plyh7a28v.pgp
Description: PGP signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Fabian Groffen]

On 08-02-2011 12:03:48 +, Markos Chandras wrote:
 I see what you are saying. However, the 6 months testing is far from
 what I have in mind. My only intention is to bring a more stable
 experience to our users. Or, stop claiming that our stable tree rocks
 and Gentoo is perfect for servers because it is not. Ye ye ye I know
 that many many of you have Gentoo on servers but do not forget that you
 are developers and you know your way around during breakages. Yes,
 stable tree breaks FAR TOO often. I blame myself for my arch testing of

Hmmm, odd.  I experience amd64 (stable) as being pretty stable on my
servers.  Last breakage which really got me upset was php, but that's
already some time ago.

 Our stable tree is definitely not suitable for server usage unless 
 you have plenty of free time to
 deal with stupid upgrades because nobody, for example, cared to write a
 proper elog or news item. You are probably not aware of that, since 99%
 of you run testing tree however if you visit forums and stuff you will
 see many many users complaining about stable tree. If we keep going down

With Gentoo you should update on fairly regular intervals, and have the
time inbetween as short as possible, but 2 or 3 weeks appears to be
fine.  I myself have a cronjob that syncs every night, and mails me the
output of emerge -Dupv world.  When this list gets too large, it's
typically about time to do some updating.

I think it is just regular sysadmin work to evaluate the list of
packages that's going in, and whether you deem that necessary.  I have
masked new major releases of PostgreSQL and MySQL for instance, and of
course Python 3.  I keep the rest updated, and in this way, I can only
say that I experience Gentoo to be definitely suitable for server usage,
with only little time caring about them.


-- 
Fabian Groffen
Gentoo on a different level

Re: [gentoo-dev] avoiding urgent stabilizations

[Paweł Hajdan, Jr.]

tl;dr - can we add more automated tests to auto-generated stages?

On 2/8/11 1:22 PM, Fabian Groffen wrote:
 Hmmm, odd.  I experience amd64 (stable) as being pretty stable on my 
 servers.  Last breakage which really got me upset was php, but
 that's already some time ago.

Makes sense. Most of the breakages I had in mind when starting this
thread were related to consolekit/policykit/login manager/de breakages,
or some desktop software not compiling with an updated gcc. Those issues
generally don't affect servers.

By the way, to turn this thread into some action: what testing do we
currently perform for auto-generated stages? It'd be cool to at least
compile-test that the stage can emerge -e world itself, and emerge
some common packages (with FEATURES=test so that we run at least some
of the produced code).

 With Gentoo you should update on fairly regular intervals, and have
 the time inbetween as short as possible, but 2 or 3 weeks appears to
 be fine.  I myself have a cronjob that syncs every night, and mails
 me the output of emerge -Dupv world.  When this list gets too large,
 it's typically about time to do some updating.

Sounds like a good idea. I think I'll start using it.

 I have masked new major releases of PostgreSQL and MySQL for
 instance, and of course Python 3.

Yeah, this ease of masking is a great strength of Gentoo.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Roy Bamford]

Markos,

A few thoughts inlined.

On 2011.02.08 12:03, Markos Chandras wrote:
 
My main point was that as you move from an old dated set of packages to 
newer packages which by definition are less well tested, stability 
decreases. Users pick somewhere between the two extremes that they are 
happy with. Gentoo stable lies somewhere between Debian stable and LFS 
built live from all the repositories.

 I see what you are saying. However, the 6 months testing is far from
 what I have in mind. 
Thats what releng used to take.

 My only intention is to bring a more stable
 experience to our users. Or, stop claiming that our stable tree rocks
 and Gentoo is perfect for servers because it is not. Ye ye ye I know
 that many many of you have Gentoo on servers but do not forget that
 you
 are developers and you know your way around during breakages. Yes,
 stable tree breaks FAR TOO often. I blame myself for my arch testing
 of
 course however I can't do much about that. 
[snip]

For servers I can point you at the stillborn Gentoo-LAMP project. I 
don't remember much more than its name. Google seems to have forgotten 
it too.

A big part of the problem comes from being a meta-distro. Everyones 
Gentoo is different and we we cannot test all combinations to ensure 
everyone is ok.

More testing will not eliminate the issue but would catch some 
problems. There would be less breakage but not zero.  There is a trade 
off to be made there by both the developers doing the testing and 
the users experiencing the breakage.

I agree that given more resources, the tree could be improved but 
before we move in that direction, I would like to ask is that the best 
use of resources?

As I said above, users are aware of the trade offs involved in choosing 
Gentoo. Are our users really unhappy, or are they just looking for help 
to fix issues when they occur?
Most users do not expect a zero issue upgrade path.

[snip]
 
 Our stable tree is definitely not suitable for server usage unless 
 you have plenty of free time to
 deal with stupid upgrades because nobody, for example, cared to write
 a
 proper elog or news item. 
[snip]
 
 Either you like it or not, arch teams are understaffed. All of them.
All of Gentoo is understaffed.

 Therefore we cannot afford a updated stable tree with high QA around
 it. We need to find a more efficient way to test packages on testing
 tree so we can mark them stable with minimal time and cpu cost. We
 need
 dedicated build boxes, like Diego's tinderbox, to test the testing
 tree
 over and over against critical/common/trivial QA problems. If we
 manage
 that, moving packages from testing-stable will be much more time
 efficient and we can guarantee a high quality stable tree.
If this means a more up to date stable tree, that has to be good as the 
stable tree will move closer to testing and there will be fewer 
packages to maintain.  (Counting different versions as packages)
 
 ps1: Personally I have stopped suggesting gentoo stable for server
 usage
 and I always suggest testing to new users.

I don't quite agree about not recommending Gentoo for servers. Gentoo 
is fine on servers but you need to run a testing environment for your 
updates so you know when you do do an update, exactly what in involved 
and what will happen.  Without your own testing, your server will go 
down from time to time. If you cannot do your own testing, either 
tolerate the downtime or don't use Gentoo.

 
 ps2: Roy, this is not a personal attack. Do not misinterpret my tone
 :)

I see no personal attack in your words. 
 
 Regards,
 -- 
 Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
 
I'll buy you a insert_refreshment_of_your_choice next time we meet.

-- 
Regards,

Roy Bamford
(Neddyseagoon) a member of
gentoo-ops
forum-mods
trustees


pgp1kdwTwgef8.pgp
Description: PGP signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Rich Freeman]

On Tue, Feb 8, 2011 at 7:03 AM, Markos Chandras hwoar...@gentoo.org wrote:
 I see what you are saying. However, the 6 months testing is far from
 what I have in mind.

I could see there being room for something in-between, but I share the
concerns of others that rolling releases are part of what makes
Gentoo, well, Gentoo.

The problem with snapshots is that there is almost always SOMETHING
wrong with them, and if you don't release until they're near-perfect
then you're pursing 99.999% quality and most devs don't care enough to
work hard towards that.  As a result you end up with very long release
cycles.

I could see room for a system where every week a portage snapshot is
created, and then run through automated testing.  The test results are
then posted, and the release tarball is made available for download.
Then people can update to it if they think it is good enough.  Serious
issues would of course be spotted and immediately fixed in-tree so
that the next weekly release is better, and the typical user
experience would still be to use the live tree so that they get an
experience similar to what they have.

Honestly, I don't even know that this would really work well.  It
might be better to just have a tinderbox that does automated full-tree
testing weekly and just post the results and let devs look at them and
fix things.

However, I don't think any system is likely to work (except on Debian
timelines) if it involves a release-when-its-ready approach unless
ready is something really minimal like system set compiles and
boots.

Time vs quality vs cost - pick two.  Oh, for Gentoo we've pretty-much
picked cost as being about as close to zero as you can get, so make
that pick one.  Debian stable favors quality, and there are definitely
things I'd use debian for that I'd never use Gentoo for.  That isn't
knocking Gentoo - it is just a reflection of the fact that the distros
have different philosophies.

Re: [gentoo-dev] avoiding urgent stabilizations

[Donnie Berkholz]

On 13:22 Tue 08 Feb , Fabian Groffen wrote:
 With Gentoo you should update on fairly regular intervals, and have 
 the time inbetween as short as possible, but 2 or 3 weeks appears to 
 be fine.  I myself have a cronjob that syncs every night, and mails me 
 the output of emerge -Dupv world.  When this list gets too large, it's 
 typically about time to do some updating.

FWIW, in experience from my past two positions, updates to critical 
systems at least quarterly, and no more than monthly, seem to produce 
good results that balance effort and stability.

(With exceptions for security issues.)

-- 
Thanks,
Donnie

Donnie Berkholz
Sr. Developer, Gentoo Linux
Blog: http://dberkholz.com


pgp1Jg4PAhd9O.pgp
Description: PGP signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Rich Freeman]

On Feb 8, 2011 11:44 AM, Donnie Berkholz dberkh...@gentoo.org wrote:

 (With exceptions for security issues.)

Other than monitoring bugzilla, how does a Gentoo user even know that they
have a package pending a security update?  It seems like glsa's lag
stabilization by a considerable timeframe.

I get the impression that half the reason for this is due to the difficulty
of generating the xml files.

Re: [gentoo-dev] avoiding urgent stabilizations

[Andreas K. Huettel]

 Other than monitoring bugzilla, how does a Gentoo user even know that they
 have a package pending a security update?  It seems like glsa's lag
 stabilization by a considerable timeframe.

Yep. GLSA is something that seems to happen roughly one year after no affected 
package is in tree anymore. 

[Sorry, no offense intended. I know we're all understaffed...]

-- 
Andreas K. Huettel
Gentoo Linux developer - kde, sci, arm, tex
dilfri...@gentoo.org
http://www.akhuettel.de/


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] avoiding urgent stabilizations

[Fabian Groffen]

On 08-02-2011 18:46:32 +0100, Andreas K. Huettel wrote:
  Other than monitoring bugzilla, how does a Gentoo user even know that they
  have a package pending a security update?  It seems like glsa's lag
  stabilization by a considerable timeframe.
 
 Yep. GLSA is something that seems to happen roughly one year after no 
 affected package is in tree anymore. 

Well, it's not too bad lately:
http://archives.gentoo.org/gentoo-announce/


-- 
Fabian Groffen
Gentoo on a different level

Re: [gentoo-dev] avoiding urgent stabilizations

[Andreas K. Huettel]

On Tuesday 08 February 2011 18:57:20 Fabian Groffen wrote:
 On 08-02-2011 18:46:32 +0100, Andreas K. Huettel wrote:
   Other than monitoring bugzilla, how does a Gentoo user even know that they
   have a package pending a security update?  It seems like glsa's lag
   stabilization by a considerable timeframe.
  
  Yep. GLSA is something that seems to happen roughly one year after no 
  affected package is in tree anymore. 
 
 Well, it's not too bad lately:
 http://archives.gentoo.org/gentoo-announce/
 

Hmm. Maybe I should just subscribe to gentoo-announce again... :]

-- 
Andreas K. Huettel
Gentoo Linux developer - kde, sci, arm, tex
dilfri...@gentoo.org
http://www.akhuettel.de/


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] avoiding urgent stabilizations

[Donnie Berkholz]

On 12:37 Tue 08 Feb , Rich Freeman wrote:
 On Feb 8, 2011 11:44 AM, Donnie Berkholz dberkh...@gentoo.org wrote:
  (With exceptions for security issues.)
 
 Other than monitoring bugzilla, how does a Gentoo user even know that 
 they have a package pending a security update?  It seems like glsa's 
 lag stabilization by a considerable timeframe.

I read LWN's security page and ensure updates happen, regardless of 
whether the ebuild exists yet.

But if I didn't, I would rely on GLSAs — either via the mailing list or 
with a daily sync and glsa-check.

-- 
Thanks,
Donnie

Donnie Berkholz
Sr. Developer, Gentoo Linux
Blog: http://dberkholz.com


pgpKFWCifmVrx.pgp
Description: PGP signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Samuli Suominen]

On 02/07/2011 06:19 PM, Paweł Hajdan, Jr. wrote:
 From time to time there are stabilization bugs where the current stable
 is broken. For example, https://bugs.gentoo.org/show_bug.cgi?id=353487
 
 However, in theory that should not happen, because presumably the
 current stable has been tested in the past and considered not broken.
 

In this case, xdm has been broken for more than a year because it has
been ignoring pambase by not using system-local-login.

Only the problem came to light after ConsoleKit really started to
require this functionality from pambase.

Tricky...

But it was still tested, unfortunately the test environment that was
used leaked some environment variables that made XDM appear to work, so
this was catched too late.

Sorry.

- Samuli

Re: [gentoo-dev] avoiding urgent stabilizations

[Pacho Ramos]

El lun, 07-02-2011 a las 18:43 +0200, Samuli Suominen escribió:
 On 02/07/2011 06:19 PM, Paweł Hajdan, Jr. wrote:
  From time to time there are stabilization bugs where the current stable
  is broken. For example, https://bugs.gentoo.org/show_bug.cgi?id=353487
  
  However, in theory that should not happen, because presumably the
  current stable has been tested in the past and considered not broken.
  
 
 In this case, xdm has been broken for more than a year because it has
 been ignoring pambase by not using system-local-login.
 
 Only the problem came to light after ConsoleKit really started to
 require this functionality from pambase.
 
 Tricky...
 
 But it was still tested, unfortunately the test environment that was
 used leaked some environment variables that made XDM appear to work, so
 this was catched too late.
 
 Sorry.
 
 - Samuli
 
 

I think that maybe we could have an even higher priority field called,
for example, Broken Stable that should be *only* used for that cases
and that arch teams should try fix sooner. What do you think?


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-dev] avoiding urgent stabilizations

[Andreas K. Huettel]

We've been discussing this @FOSDEM too. My suggestion was that any bug that 
visibly hurts stable users should always be considered at least MAJOR in 
bugzilla. 

To expand on this a bit more
* a stable update that makes the computer nonfunctional is definitely BLOCKER 
(and should be reverted in CVS immediately when it becomes known, at latest 
when it is understood, by anyone who is around at the time and can do so)
* a non-functional stable package in the system set should be CRITICAL.

Just my 2ct, but it is really important not to hurt stable users. This is how 
we lose most people.


 I think that maybe we could have an even higher priority field called,
 for example, Broken Stable that should be *only* used for that cases
 and that arch teams should try fix sooner. What do you think?

-- 

Andreas K. Huettel
Gentoo Linux developer 
dilfri...@gentoo.org
http://www.akhuettel.de/



signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-dev] avoiding urgent stabilizations

[Markos Chandras]

On Mon, Feb 07, 2011 at 06:45:10PM +0100, Andreas K. Huettel wrote:
 
 We've been discussing this @FOSDEM too. My suggestion was that any bug that 
 visibly hurts stable users should always be considered at least MAJOR in 
 bugzilla. 
 
 To expand on this a bit more
 * a stable update that makes the computer nonfunctional is definitely BLOCKER 
 (and should be reverted in CVS immediately when it becomes known, at latest 
 when it is understood, by anyone who is around at the time and can do so)
 * a non-functional stable package in the system set should be CRITICAL.
 
 Just my 2ct, but it is really important not to hurt stable users. This is how 
 we lose most people.

The rolling way we stabilize the packages makes the stable tree pretty
much fragile to breakages and stuff. This is because you cannot predict
what is going to happen to the rest of the tree if you stabilize a newer
package. It may have unpredictable consequences to the rest of the
packages. My suggestion, as I said to fosdem, is to freeze, or take a
snapshot if you like, of the current tree, stabilize what you need to
stabilize, test the whole tree ( at least compile wise ) for a couple of
weeks and then replace the existing stable tree. Of course this requires
automated script testing, hardware facilities etc etc that we don't have
so claiming that stable tree is stable is quite wrong.

Regards,
-- 
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2


pgpF2aIntpgO2.pgp
Description: PGP signature

Re: [gentoo-dev] avoiding urgent stabilizations

[Paweł Hajdan, Jr.]

On 2/7/11 9:50 PM, Markos Chandras wrote:
 My suggestion, as I said to fosdem, is to freeze, or take a
 snapshot if you like, of the current tree, stabilize what you need to
 stabilize, test the whole tree ( at least compile wise ) for a couple of
 weeks and then replace the existing stable tree. Of course this requires
 automated script testing, hardware facilities etc etc that we don't have
 so claiming that stable tree is stable is quite wrong.

This more thorough testing sounds really interesting. But do we really
lack hardware resources?

There are machines available for various arches at
http://www.gentoo.org/proj/en/infrastructure/dev-machines.xml. I have
at least a few chromium-related chroots on miranda, and I've never heard
complaints, so it seems a few more chroots for arch testing wouldn't hurt.

Of course for testing bootability and whether X11 starts up correctly,
etc we'd probably have to host some virtual machines, but better compile
testing (for example for toolchain updates) would be a good start.

Paweł



signature.asc
Description: OpenPGP digital signature