Re: [gentoo-dev] The fallacies of GLEP55

2009-05-21 Thread Robert R. Russell
On Saturday 16 May 2009 20:17:14 Nick Fortino wrote:
 Ciaran McCreesh wrote:
  On Sat, 16 May 2009 16:39:40 -0700
  Nick Fortino wrote:
  Given the above, it should be clear that any argument which states
  some future improvement to the ebuild format  will be impossible based
  upon making the wrong choice between proposal 1 and proposal 2 must be
  invalid, as they have the same expressive power. Note that allowable
  algorithms for which the proof works includes caching and version
  ordering as well as the simple execution of the ebuild.
  Unfortunately, your argument is entirely wrong, as can be illustrated
  by a simple counter-example that you would already know about, had you
  read the GLEP or the thread.
  With EAPI in a fixed format, it is impossible to allow extensions to the
  version format in future EAPIs. Even given a fixed format and a constant
  extension, adding foo-1.23-rc1.ebuild will cause breakage, but adding
  foo-1.23-rc1.ebuild-4 will not.
  This has already been covered at length, and is explained in the GLEP.
  Why did you disregard this when posting your 'proof'?

 I didn't intentionally disregard that case, but I see your point. I made
 the assumption that package mangers wouldn't try to source ebuilds with
 a sourcing EAPI they didn't understand. I concede this is a terrible
 assumption, unless such a thing is specified in the PMS itself. It is
 still fixed by a single extension change, as opposed to a whole set.
 Once this is done, simply state that all package managers should ignore
 EAPIs they don't understand (a requirement of GLEP-55 as well).

 Your point still does not dispute that specifying the EAPI within the
 ebuild and outside the ebuild convey identical information (this is all
 I was trying to prove in the first place). For the case you bring up:
 If foo-1.23-rc1.ebuild is added, it must not be in any of the currently
 existing EAPIs, for if it were, it would be illegal. Thus, a package
 manager would open this file, get the sourcing EAPI in an EAPI
 independent way, realize it doesn't understand, and abort the sourcing
 of that ebuild. Changing the extension once insures current package
 managers don't try to do things they aren't capable of (I apologize for
 not putting this in my first mailing). Given this change, however, I
 still assert the statement of the two schemes have identical expressive

 For versioning, it has been pointed out (by you and others) that getting
 the latest version would require, under any implementation, opening N
 files in case 1, and reading N file names in case 2. I do not dispute
 this in any way. Instead, I would like to point out that moving the
 argument from features which are possible to support (which I still
 contend are essentially identical), to efficiency vs. a perceived
 prettiness would be significant progress. Indeed, at this point it would
 be possible to make a decision based on reference implementations for
 known common use cases, and an executive council decision about whether
 timing or extension consistency is more important. If it turns out that
 using a solution of type 1 takes minutes to resolve versions, than by
 all means, GLEP-55 is by far the best proposed solution. If, instead,
 the runtime difference in real use cases is negligible, then the pure
 philosophical arguments for using a single extension holds true (in my

 Nick Fortino

And we could probably switch between types if forced to do so.

Re: [gentoo-dev] The fallacies of GLEP55

2009-05-15 Thread Robert R. Russell
On Friday 15 May 2009 05:44:47 Richard Freeman wrote:
 Ciaran McCreesh wrote:
  On Thu, 14 May 2009 20:06:51 +0200
  Patrick Lauer wrote:
  Let EAPI be defined as (the part behind the = of) the first line of
  the ebuild starting with EAPI=
  Uh, so horribly utterly and obviously wrong.
  inherit foo
  where foo is both a global and a non-global eclass that sets metadata.

 This seems to come up from time to time but I don't see how this is a
 problem that GLEP 55 solves.  If the rule is first line of the ebuild
 starting with EAPI= and the ebuild is as you suggest above, then the
 EAPI is 4 (without any regard whatsoever to what might be in foo).

 The counterargument seems to be that eclasses should be able to modify
 EAPI behavior.  However, if you want to do this then you DEFINITELY
 don't want to put the EAPI in the filename - unless you want eclasses to
 start renaming the ebuilds to change their EAPIs and then trigger a
 metadata regen.

 This seems to be a case where a problem is proposed, with a solution.
 Somebody proposes an alternate solution and the complaint is raised that
 it doesn't handle situation X.  However, the original proposed solution
 doesn't handle situation X either, so that can hardly be grounds for
 accepting it over the alternate.

 I'm actually more in favor of an approach like putting the EAPI in a
 comment line or some other place that is more out-of-band.  Almost all
 modern file formats incorporate a version number into a fixed position
 in the file header so that it is trivial for a program to figure out
 whether or not it knows how to handle the file.  Another common approach
 is to put a header-length field and add extensions to the end of a
 header, so that as long as you don't break past behavior you could
 create a file that is readable by older program versions (perhaps with
 the loss of some metadata that the older version doesn't understand).
 Just look up the UStar tar file format or the gzip file format for
 examples.   Of course, such file formats generally aren't designed to be
 human-readable or created with a text editor.

 The same applies to executables.  It is impossible from the filename to
 tell if /bin/bash is in a.out or ELF format, or if it is a shell script.
   Instead a simple standard is defined that allows the OS to figure it
 out and handle it appropriately.  If you try to run an ELF on some
 ancient version of linux it doesn't crash or perform erratic behavior -
 it will simply tell you that it doesn't understand the file format
 (invalid magic number).

 In any case, I'm going to try to restrain myself from replying further
 in this thread unless something genuinely new comes up.  When I see 26
 new messages in my gentoo-dev folder I should know by now that somebody
 has managed to bring up GLEP 55 again...  :)

If I understand the problem GLEP 55 is trying to solve correctly, it stems 
from portage's assumption that an unknown EAPI is equal to EAPI 0. Could that 
assumption be changed to an unknown EAPI is equal to the latest supported 
EAPI. Now I understand that this change would have to wait until all the 
ebuilds in the portage tree correctly define their EAPI, but would the idea be 
technically feasible at least excluding EAPI0 ebuilds? I think it would be if 
all EAPIs are forward compatible up until the EAPI declaration in the ebuild.

Re: [gentoo-dev] RFC: Deprecating EAPI0

2009-03-22 Thread Robert R. Russell
On Saturday 21 March 2009 19:03:45 AllenJB wrote:
 Patrick Lauer wrote:
  Hi all,
  with the discussion about EAPI3 we have now 4 (or 7, depending on how you
  count them ;) ) EAPIs available or almost available. This is getting
  quite confusing.
  To make our lives easier I would suggest deprecating EAPI0 and migrating
  existing ebuilds over some time to EAPI1 or higher until EAPI0 can be
  obsoleted at some point in the future.
  I would set the start of deprecation warnings about 3 months from now and
  the obsoletion quite a time later, maybe 12 months from now, when a
  sufficient amount of ebuilds has been migrated.
  Deprecating EAPI1 at the same time would reduce the amount of EAPIs we
  have to think about, but since it has some changes like adding
  src_prepare migration would not be as trivial. So I'd prefer keeping it
  around a bit longer.

 While there definitely arguments for deprecating EAPIs, I would suggest

 A low number of active EAPIs can make life easier for developers, but it
 can also make life harder for some users. There are still users coming
 to the forums upgrading systems which only understand EAPI 0. I accept
 that Gentoo is certainly a relatively fast moving distro, but I think
 that developers also do need to consider users who have systems that are
 currently just working and may not upgrade often (once a year or even

 As such, I would suggest that forcing a move to the most recent stable
 EAPI is possibly unwise.


Note, this just my opinion. It may not be entirely practical or cover all the 
issues regarding backwards compatibility. I intend it to provoke questions 
and thought as to what a reasonable policy for backwards compatibility might 

Waiting a year or longer to upgrade a gentoo system carries a good risk of 
having something explode into a near unrecoverable mess.

I definitely do think that some major focus needs to be placed on how far 
behind a gentoo system could be and should still be expected to catch up.

An oversimplified example. Assume that I can find all required patches and 
source code to do the initial builds, and that all normal configuration file 
checks/updates are done after the current tree is installed.

I create three different virtual machines from cvs snapshots of the portage 
tree. The first is dated 6 months ago. The second is dated 12 months ago. The 
third is dated 18 months ago.

Which of these should be reasonably updateable to the current portage tree?

My suggestion is that policy should allow machine 1 to be updated through 
regular update procedures to the current tree, unless major changes dictate 
otherwise. Major changes being incompatible ebuild formats, toolchains, and 
other here is the line sorry kind of changes.

A careful operator should probably be able get machine 2 updated to the 
current tree, again unless major changes dictate otherwise. We should not 
make go out of our way to make upgrading from this point out hard for the 
operator, but, new growth should be favored over the ease of upgrading.

Machine 3 is just out of luck. Here is the new install cd and handbook, have 

Regular update procedures would be:
emerge --sync; emerge -uND world -f; emerge -uND world; revdep-rebuild; 
emerge --depclean;

The careful operator might update the toolchain first and emerge -e world or 
something similar.

Re: [gentoo-dev] Re: what happened to /etc/init.d/hal{d,daemon,whatever} script ?

2008-12-23 Thread Robert R. Russell
On Monday 22 December 2008 11:40:32 pm Branko Badrljica wrote:
 Duncan wrote:
  Branko Badrljica posted, excerpted below, on  Mon, 22 Dec 2008
  05:18:32 +0100:
  Maybe I should have filed this as a bug, but don't have a clue to which
  package should I assign it, if any.
  FWIW, this would have been a perfect question for the gentoo-desktop
  list, but doesn't really belong on the -dev list.  There's also the
  gentoo-user list, altho that one has very heavy volume so you might not
  wish to subscribe there.

 Well, regarding the actual error, i think it might interest someone
 here, also.
 Although I mentioned just baselayout and openrc, I did check ( end
 reemerged etc) hal also, and  it indeed emerged  _without_

 I tracked it down to root cause: Although I don't use it, I have
 compiled-in selinux support ( and selinux=0 as kernel start parameter).
 When I was makeconfiging my kernel, I saw also SMACK support, read info
 and thought  what the heck, it can't hurt me, but I might want to play
 with it, so I compiled-in  that, too.

 Then after some time I realised that I never got to actually used all
 that and changed my config file by cutting out that all that security
 stuff. And recompiled all my kernels accordingly.
 Around that time I saw people recommending using tmpfs for /var/tmp as
 this would speed-up emerges etc, so I did that.

 I didn't know that while I was on SMACK (pun intended) , machine would
 add extended attr to every file machine would write. ( It was SMACK64 in
 security domain ).

 After cleaning my system, even though those attributes were still on all
 files, everything was fine until I actually tried to copy something from
 that FS to some other FS.
 /bin/cp would realise that there are extra security attrs on a file and
 would try to duplicate them on a copy. And since new kernel was without
 SMACK support, it would fail.

 When emerging stuff  with /var/tmp on tmpfs, /bin/cp seems to get rarely
 used in such way when copying stuff into /var/tmp or maybe it was
 because distfiles were without SMACK attrs- so most ebuilds would
 seemingly sucseed. Most errors seem tho have been made when ebuild
 needed some local data, usually in /etc that had SMACK64 attr. If
 /bin/cp was used to get that data, it would fail, but this would not
 stop the ebuild. It would usually finished its work just as if nothing

 Once I unmounted /var/tmp, ebuild could finish normally. Also, after
 removing security attr from all files, ebuild has started working
 normally from tmpfs partition again.

  It is also interesting that on 2.6.27* kernel ebuild fails sometimes
 and when it fails, it does so silently most of the time. With newest
 2.6.28-rc9 i couldn't emerge a thing...

 Since I might not be the only tinkerer on Gentoo to try stuff like that
 and since it took me a day to find this, maybe it wouldn't hurt to check
 for this kind of thing in portage ?
 At the very least failed cp should stop emerge...

Very nice edge case and great work tracking down the cause.

Re: [gentoo-dev] Re: Last Rites: app-portage/udept

2008-12-16 Thread Robert R. Russell
On Monday 15 December 2008 05:47:47 pm Duncan wrote:
 Paul Varner posted, excerpted below, on

 Mon, 15 Dec 2008 14:10:17 -0600:
  # Paul Varner (14 Dec 2008) # Dead upstream,
  masked for removal in ~30 to 60 days. app-portage/udept
  Additionally, it doesn't play well with EAPI's greater than zero.
  The removal bug is Bug #250839.  If upstream comes back alive or someone
  forks and actively maintains, I will unmask or re-add to the tree.

 Ouch!  This one hurts!

 The main thing I use it for, and therefore the question I have about any
 useful substitute, is quickly getting a changelog when portage won't spit
 one out.  In particular, when there's a USE flag change in an existing
 package, or a downgrade, emerge --log won't output anything, because it's
 not an upgrade.  However, I find the info in such logs often useful!

 Of course I can check the package category and type in the whole long
 path to the changelog and edit/head/view it by hand, but a quick dep -j
 pkg is a lot faster and very useful!

 While I'm at it, is there anything useful to display metadata.xml?  In
 particular, the long descriptions and use flags can be useful.  With
 use.desc and especially the local version thereof going deprecated, and
 with additional info about global flags sometimes in the metadata...

Does anyone have a substitute for udept's clean word file and 
clean /etc/portage options?

Re: [gentoo-dev] EAPI 2 policy for portage tree

2008-12-10 Thread Robert R. Russell
On Tuesday 09 December 2008 12:13:40 pm Petteri R├Ąty wrote:
 Robert R. Russell wrote:
  My personal opinion on this matter is pick one of the following:
  1)  perform the bugfix without a version bump and remain at the current
  EAPI version
  2)  perform the bugfix with a version bump and remain at the current EAPI
  3)  perform the bugfix with a version bump and upgrade to the latest EAPI
  Options 1 and 2 are how most updates are done, the user can mask the
  latest version or upgrade. Option 3 allows the user to continue using the
  previous version while they decide to update to a portage version that
  supports the new EAPI.

 The current policy states that ebuilds should only be bumped if the
 installed files change. Changing EAPI from 1 to 2 has no effect outside
 the vdb so the current policy means developers should use option 3.
 There was a bug in stable Portage when EAPI 2 went in the tree that made
 Portage stack trace but that's a problem with Portage not with the
 policy in general.

  I would prefer that option 3 be made policy because I run several ~arch
  packages that either don't have a stable version (kradio) or have a
  feature that I need (gentoo-sources), and will not be pushed to stable
  immediately for various reasons from lack of maintainer time to everybody
  says it conflicts with major pieces of the system (Firefox 3, 64 bit
  netscape-flash, and xorg).

 Why should we prefer making it a little bit easier for stable users over
 making ~arch users needlessly recompile stuff?


My answer is a simple example from my own system. My current system uses a 
motherboard that is around 6 months old and is only correctly supported by 
the latest ~arch gentoo-sources. The add on video card, a 1 to 2 year old 
nvidia card, works great with x11-drivers/xf86-video-nv-2.1.12 as long as I 
am using the latest ~arch xorg-x11. The internal video card isn't even 
recognized by the xf86-video-intel drivers except the ~arch versions. Even 
some of the packages I use for school work such as kile have bugfixes and 
other improvements between the versions in stable and ~arch that are 
important to getting work done. The ability to selectively upgrade only the 
specific packages needed to get a working system is a major strength for 
Gentoo. Why should I have to run more packages from ~arch than I absolutely 
need to? We all know that upgrading more software than absolutely necessary 
will result in bad things happening to a computer.

The easiest solution to the problem with ~arch having the only working 
versions of some packages is to get more of those packages stabilized. But, 
we all know that the manpower required simply doesn't exist.

Re: [gentoo-dev] EAPI 2 policy for portage tree

2008-12-08 Thread Robert R. Russell
On Monday 08 December 2008 06:00:10 pm Jean-Marc Hengen wrote:
 This mail is about EAPI usage in the portage tree. Let me describe it,
 with what happened today: I'm running a mostly stable system (91 of 1255
 installed packages are unstable), but I test here and there some
 packages. On of the packages, which I installed and is currently masked
 unstable, is dev-util/cmake-2.6.2. I use it on a daily basis and happy
 with it so far. Today, while updating, portage wanted to downgrade cmake
 to the stable release, due to all cmake 2.6.x version masked by EAPI 2.
 The cmake-2.6.2 ebuild was updated to use EAPI 2 (along with fixing a
 bug). My rule of thumb is to only use unstable on none system packages


 With kind regards,
 Jean-Marc Hengen, a happy gentoo user

The problem is not that an EAPI 2 supporting portage is unstable or that he is 
using a ~arch version of one particular package, but the during a bugfix the 
maintainer moved the ebuild to EAPI 2 without a version bump forcing 
Jean-Marc to downgrade to the stable version. The question on policy is, can 
a maintainer upgrade an ebuild to the latest EAPI while doing some other 
bugfixing without a version bump?

My personal opinion on this matter is pick one of the following:
1)  perform the bugfix without a version bump and remain at the current EAPI 
2)  perform the bugfix with a version bump and remain at the current EAPI 
3)  perform the bugfix with a version bump and upgrade to the latest EAPI
Options 1 and 2 are how most updates are done, the user can mask the latest 
version or upgrade. Option 3 allows the user to continue using the previous 
version while they decide to update to a portage version that supports the 
new EAPI.

I would prefer that option 3 be made policy because I run several ~arch 
packages that either don't have a stable version (kradio) or have a feature 
that I need (gentoo-sources), and will not be pushed to stable immediately 
for various reasons from lack of maintainer time to everybody says it 
conflicts with major pieces of the system (Firefox 3, 64 bit netscape-flash, 
and xorg).

[gentoo-dev] Questions about stabilization requests

2008-09-05 Thread Robert R. Russell
Some of the packages that I use are ~arch keyworded right now and I was 
wanting to ask some questions about when to file stabilization requests and 
about how much information needs to be provided when filling them.

1)  How long do you recommend using a package before filing a stabilization 

2)  Should I file stabilization requests on software that works mostly as 
everything that I use it for normally but I can force it to fail if I feed it 
some really strange input or contrive a specific set of options that will 
cause invalid or incorrect output.

3)  Should I provide a patch that removes the ~arch keyword when I file the 
stabilization request?