Re: [gentoo-dev] mtime preservation
On Wed, 25 Nov 2009, Zac Medico wrote: Not non-preservation. Partial and inconsistent corruption. Wouldn't loss of precision be a more accurate description? Yes. Or even rounding. Of the known packages which require timestamp preservation, do any of them use sub-second precision in their timestamp comparisons? I can speak for Emacs only, where the comparison code (in fileio.c) is as follows: if (stat (SDATA (absname1), st) 0) return Qnil; mtime1 = st.st_mtime; if (stat (SDATA (absname2), st) 0) return Qt; return (mtime1 st.st_mtime) ? Qt : Qnil; It uses stat(2), therefore nanoseconds are ignored. Ulrich
Re: [gentoo-dev] Gentoo Prefix: on EPREFIX, ED and EROOT inside ebuilds
On 25-11-2009 16:43:32 -0700, Denis Dupeyron wrote:
Things seem to be progressing nicely on this front. We have answers to
the questions people had and they look satisfactory to me.
One thing that I think would be valuable is a document that explains
the average dev how to make his/her ebuilds prefix compliant with
links to more details when necessary. I understand that there's the
trivial situations and the less trivial ones. In the latter case it
would be nice to explain why the case isn't trivial and how to fix it.
Using python as an example could be one way to do it. I'm thinking of
something practical that could possibly be patched into devmanual. If
such a document already exists then please just point us to it.
I'm working on a PMS patch at this very moment. I hope to finish a
first version today.
The Python ebuild is a very bad example, IMO. Not that the style there
is horrendous or something, but more that a couple of things are mixed
up in there, which are not directly a Prefix issue, but a derivative of
that -- support for arches which cannot be supported without Prefix.
We can explain some cases though. The *blas-* family comes to mind as
we recently had to alter them for Prefix.
Next to that, it is part of the Prefix team's job to make sure that
whatever is installed, does not reference the host system when this is
not absolutely necessary.
Could you give some examples of when it is absolutely necessary?
Simple example is perl. If you install a script, for instance ekeyword,
then it is important that this script doesn't say '#!/usr/bin/perl' in
its shebang. /usr/bin/perl may simply not exist, but more importantly
it is not the perl that Portage has installed and also installed all
required dependencies for. Hence, ekeyword should be installed such
that it references the perl from the offset installation, e.g.
/home/joe/gentoo/usr/bin/perl.
/bin/sh is another nice one. Technically this should work fine, as
almost every system has this shell available, however in practice it
won't work that well, because many people assume that /bin/sh is a
POSIX shell, which is not true on every UNIX that exists, as far as I
know. People may also rely on things from the Gentoo environment in
such scripts, which of course are not present in the host provided
/bin/sh, hence also here we need to call /home/joe/gentoo/bin/sh
(for the chosen offset-prefix /home/joe/gentoo).
Outside scripts, also libraries have this same principle. An
application linking against libexpat should do so against the Prefix
installed version of expat. If the host system provides
/usr/lib/libexpat.so and the upstream configure script is hardwired to
look in /usr/local/lib /usr/lib64 /usr/lib32 /usr/lib /usr/sfw/lib /sw
it will never find libexpat from /home/joe/gentoo/usr/lib. In such
case either an --with-expat=${EPREFIX}/usr option needs to be passed
to configure (if it exists), or if nothing possible configure be hacked
not to use this hardwired list of paths, but instead fall back on the
default, which for a Prefix toolchain includes the offset library search
paths before the host system's. In any case, it must compile and link
against the Prefix installed libexpat, since the one from the host
system usually is very archaic and not under Portage's control.
--
Fabian Groffen
Gentoo on a different level
Re: [gentoo-dev] Gentoo Prefix: on EPREFIX, ED and EROOT inside ebuilds
On 25-11-2009 17:01:19 -0700, Denis Dupeyron wrote: It looks like this question is still unanswered: On Thu, Nov 19, 2009 at 5:26 PM, Denis Dupeyron calc...@gentoo.org wrote: How are dynamically linked set*id programs going to work? Depends on how the host OS/libc handles this :) If you're root, you can install a program that any user can call and run as if that user were root. If you're a normal user, you just install a program that any user can call and run as if that user were you. If you refer to GNU glibc ignoring or greatly limiting variables like LD_LIBRARY_PATH, then there is no problem in that for Prefix, since these variables are considered harmful anyway. We don't use them. Prefix uses RUNPATH (RPATH) in ELF objects to have the loader find the correct libraries at runtime without any tricks. This is for instance handy when someone calls a binary from a Prefix without having properly initialised the Prefix environment. While this is not what we advise people to do, it can work fine for applications like subversion. -- Fabian Groffen Gentoo on a different level
Re: [gentoo-dev] mtime preservation
On Thu, 26 Nov 2009, Ulrich Mueller wrote: On Wed, 25 Nov 2009, Zac Medico wrote: Of the known packages which require timestamp preservation, do any of them use sub-second precision in their timestamp comparisons? I can speak for Emacs only, where the comparison code (in fileio.c) is as follows: [...] It uses stat(2), therefore nanoseconds are ignored. I've also checked with dev-lisp/sbcl and dev-lisp/cmucl. They use stat(2) or fstat(2), i.e. compare timestamps on the seconds level. Ulrich
[gentoo-dev] Re: gentoo-x86 commit in media-sound/squeezeboxserver: ChangeLog squeezeboxserver-7.4.1.ebuild metadata.xml
* Joe Peterson (lavajoe) lava...@gentoo.org:
1.1
media-sound/squeezeboxserver/squeezeboxserver-7.4.1.ebuild
file :
http://sources.gentoo.org/viewcvs.py/gentoo-x86/media-sound/squeezeboxserver/squeezeboxserver-7.4.1.ebuild?rev=1.1view=markup
plain:
http://sources.gentoo.org/viewcvs.py/gentoo-x86/media-sound/squeezeboxserver/squeezeboxserver-7.4.1.ebuild?rev=1.1content-type=text/plain
Index: squeezeboxserver-7.4.1.ebuild
===
RDEPEND=
=dev-perl/Class-XSAccessor-1.03
=dev-perl/Class-XSAccessor-Array-1.04
Class-XSAccessor-Array was merge in dev-perl/Class-XSAccessor-1.05 (#275520)
Please depend on =dev-perl/Class-XSAccessor-1.05 and drop -Array.
# Selected contents of SqueezeCenter's local CPAN collection that we include
# in the installation. This removes duplication of CPAN modules. (See Gentoo
# bug #251494).
Hm, I've added a bunch of these modules as requested in
https://bugs.gentoo.org/showdependencytree.cgi?id=251494
Why don't you use them now?
CPANKEEP=
Class/XSAccessor/Array.pm
JSON/XS/VersionOneAndTwo.pm
Class/Accessor/
Class/Accessor.pm
MRO/Compat.pm
Algorithm/C3.pm
Data/
DBIx/
File/BOM.pm
Net/UPnP/
Net/UPnP.pm
Proc/Background/
Proc/Background.pm
Text/Unidecode/
Text/Unidecode.pm
Tie/Cache/LRU/
Tie/Cache/LRU.pm
Tie/LLHash.pm
Tie/RegexpHash.pm
UUID/Tiny.pm
URI/Find.pm
PAR/
PAR.pm
enum.pm
LIBDIR=/usr/lib/squeezeboxserver
get_libdir ?
pkg_setup() {
# Sox has optional OGG and FLAC support, so make sure it has that
included
# if required
if use ogg; then
if ! built_with_use media-sound/sox ogg; then
eerror media-sound/sox not built with USE=ogg
die Squeezebox Server needs media-sound/sox to be
built with USE=ogg
fi
fi
if use flac; then
if ! built_with_use media-sound/sox flac; then
eerror media-sound/sox not built with USE=flac
die Squeezebox Server needs media-sound/sox to be
built with USE=flac
fi
fi
Use EAPI=2 and USE Dependencies
http://devmanual.gentoo.org/ebuild-writing/eapi/index.html
src_install() {
# The main Perl executables
exeinto /usr/sbin
newexe slimserver.pl squeezeboxserver
newexe scanner.pl squeezeboxserver-scanner
newexe cleanup.pl squeezeboxserver-cleanup
|| die
# Get the Perl package name and version
eval `perl '-V:package'`
eval `perl '-V:version'`
# The custom OS module for Gentoo - provides OS-specific path details
cp ${FILESDIR}/gentoo-filepaths.pm Slim/Utils/OS/Custom.pm || die
Unable to install Gentoo custom OS module
# The server Perl modules
dodir /usr/lib/${package}/vendor_perl/${version}
cp -r Slim ${D}/usr/lib/${package}/vendor_perl/${version} || die
Unable to install server Perl modules
You can make use of:
perl -V:installvendorlib
# Compiled CPAN module go under lib as they are arch-specific
dodir /usr/lib/squeezeboxserver/CPAN
cp -r CPAN/arch ${D}/usr/lib/squeezeboxserver/CPAN || die Unable to
install compiled CPAN modules
# Preseve some of the Squeezebox Server-packaged CPAN modules that
Gentoo
# doesn't provide ebuilds for.
for ITEM in ${CPANKEEP}; do
dodir /usr/share/squeezeboxserver/CPAN/$(dirname ${ITEM})
cp -r CPAN/${ITEM}
${D}/usr/share/squeezeboxserver/CPAN/${ITEM} || die Unable to preserve
CPAN item ${ITEM}
done
For CPANKEEP, see above.
# Install preferences
insinto ${PREFSDIR}
if [ ! -f ${PREFSDIR}/squeezeboxserver.prefs ]; then
This test in src_test is wrong.
newins ${FILESDIR}/squeezeboxserver.prefs
squeezeboxserver.prefs
fi
fowners squeezeboxserver:squeezeboxserver ${PREFSDIR}
fperms 770 ${PREFSDIR}
pkg_postinst() {
# Album art requires PNG and JPEG support from GD, so if it's not there
# then warn the user. It's not mandatory as the user may not be using
# album art.
if ! built_with_use dev-perl/GD jpeg || \
! built_with_use dev-perl/GD png || \
! built_with_use media-libs/gd jpeg || \
! built_with_use media-libs/gd png; then
EAPI=2 and if ! has_version dev-perl/GD[jpeg] || \ ... is prefered.
Regards
Re: [gentoo-dev] mtime preservation
On 08:59 2009-11-26 +0100, Ulrich Mueller wrote: I can speak for Emacs only, where the comparison code (in fileio.c) is as follows: [snip] It uses stat(2), therefore nanoseconds are ignored. So, when that is updated to require nanosecond resolution where supported, we're off to having this discussion again? -- Pozdrawiam, Łukasz P. Michalik pgpLhT1VNsYev.pgp Description: PGP signature
[gentoo-dev] Re: Gentoo Prefix: on EPREFIX, ED and EROOT inside ebuilds
Fabian Groffen posted on Thu, 26 Nov 2009 09:53:04 +0100 as excerpted: Next to that, it is part of the Prefix team's job to make sure that whatever is installed, does not reference the host system when this is not absolutely necessary. Could you give some examples of when it is absolutely necessary? Simple example is perl. If you install a script, for instance ekeyword, then it is important that this script doesn't say '#!/usr/bin/perl' in its shebang. /usr/bin/perl may simply not exist, but more importantly it is not the perl that Portage has installed and also installed all required dependencies for. Hence, ekeyword should be installed such that it references the perl from the offset installation, e.g. /home/joe/gentoo/usr/bin/perl. /bin/sh is another nice one. At least here, that it would ordinarily be best to reference the prefix system was taken for granted, so when it's absolutely necessary to reference the host system is the interesting case, and how I parsed the request. You provided examples of just the opposite, the case I (and evidently Denis, if I parsed the request correctly) assumed to be normal, where referencing the prefix is strongly desirable or absolutely necessary. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-dev] Re: Gentoo Prefix: on EPREFIX, ED and EROOT inside ebuilds
On 26-11-2009 10:01:24 +, Duncan wrote: required dependencies for. Hence, ekeyword should be installed such that it references the perl from the offset installation, e.g. /home/joe/gentoo/usr/bin/perl. /bin/sh is another nice one. At least here, that it would ordinarily be best to reference the prefix system was taken for granted, so when it's absolutely necessary to reference the host system is the interesting case, and how I parsed the request. You provided examples of just the opposite, the case I (and evidently Denis, if I parsed the request correctly) assumed to be normal, where referencing the prefix is strongly desirable or absolutely necessary. I see, thanks for bringing that up. To clarify (I hope this is): Gentoo Prefix tries to be as much self-sufficient as possible, and hence applications *must* not reference the host system, unless absolutely necessary, such as for e.g. /lib/libc.so. If Gentoo Prefix would not do this, Portage's dependencies would become useless, and many uncontrollable and unpredictable errors may arise both at compile time as well as at run time. -- Fabian Groffen Gentoo on a different level
[gentoo-dev] Re: Gentoo Prefix: on EPREFIX, ED and EROOT inside ebuilds
Fabian Groffen posted on Thu, 26 Nov 2009 11:10:09 +0100 as excerpted: Gentoo Prefix tries to be as much self-sufficient as possible, and hence applications *must* not reference the host system, unless absolutely necessary, such as for e.g. /lib/libc.so. Thanks. Host libc /does/ make sense as absolutely necessary. Are there any less obvious ones, say of the type that might reach out and grab an unsuspecting dev trying to make his ebuilds prefix compliant? It seems to me that enumerating all (or all non-corner) cases where referencing the host is desired/mandatory, with a blanket rule saying reference prefix unless it's a known exception, should be by /far/ the easiest alternative, here. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-dev] Re: Gentoo Prefix: on EPREFIX, ED and EROOT inside ebuilds
On 26-11-2009 10:37:10 +, Duncan wrote: Fabian Groffen posted on Thu, 26 Nov 2009 11:10:09 +0100 as excerpted: Gentoo Prefix tries to be as much self-sufficient as possible, and hence applications *must* not reference the host system, unless absolutely necessary, such as for e.g. /lib/libc.so. Thanks. Host libc /does/ make sense as absolutely necessary. Are there any less obvious ones, say of the type that might reach out and Some that you may find are: /lib/libm.so /lib/libsocket.so /lib/libpthread.so /lib/libnsl.so On a side note, we have a question about this in our prefix-ebuild-quiz[1] (question 5 from the second section). grab an unsuspecting dev trying to make his ebuilds prefix compliant? It seems to me that enumerating all (or all non-corner) cases where referencing the host is desired/mandatory, with a blanket rule saying reference prefix unless it's a known exception, should be by /far/ the easiest alternative, here. I think there's unfortunately no simple way to tell what should be in and what unfortunately has to be out. It depends a lot on the host system. I feel -- but I can't back this up with hard evidence -- that it are usually the libs that are not in *DEPEND that can only be available in the host system. Basically because they usually are part of the libc, which we assume to be installed. [1] http://dev.gentoo.org/~grobian/prefix-quiz -- Fabian Groffen Gentoo on a different level
Re: [gentoo-dev] mtime preservation
On Thu, 26 Nov 2009, Łukasz Michalik wrote: I can speak for Emacs only, where the comparison code (in fileio.c) is as follows: [snip] It uses stat(2), therefore nanoseconds are ignored. So, when that is updated to require nanosecond resolution where supported, we're off to having this discussion again? No, then my second argument still holds, since *.el and *.elc are installed into the same directory. Ulrich
Re: [gentoo-dev] mtime preservation
2009/11/26 Brian Harring ferri...@gmail.com: This discussion in generall is daft. No package can rely on nanonsecond resolution for installation because the most common FS out there (ext3) does *second* level resolution only. As such, I can pretty much gurantee there is *zero* packages out there that require nanosecond resolution for installation. It's possible that a package might assume that *if* nanosecond timestamps were available on the build filesystem, then they can do nanosecond comparisons on the installed filesystem. That would be a rather questionable assumption, and I'm not sure what we could do about packages that do require that, but that's why we discuss things, right? To find solutions? It's an academic discussion, and pointless. We don't mandate the filesystems PMS implementations are run on- as such we cannot make a gurantee to ebuilds that nanosecond resolution is available. It's daft to encode in the spec NS resolution when it's essentially impossible to gurantee it If we're not going to insist on preserving nanoseconds as far as possible, then package managers should be required to explcitly clear the nanoseconds part. Otherwise we'll get situations where a package works on the maintainer's machine, because it happens to use a package manager, filesystem setup, configuration etc that happens to cause nanoseconds to be preserved accurately, but it randomly and mysteriously fails for users. This is especially a concern in this case because a maintainer can easily have no idea that he's accidentally relying on nanoseconds being preserved - it's not something the ebuild requests explicitly, and if the timestamps happen to be set on his machine as the package expects, it'll just work without any indication that there was a potential issue. I'm honestly not sure why we're having this discussion beyond the python sucks angle. Because some of us want to find a correct solution, not just one that no-one's complained about yet in the context of the few filetypes that are currently known to benefit from preservation. Shocking, I know.
[gentoo-dev] Re: Gentoo Prefix: on EPREFIX, ED and EROOT inside ebuilds
Fabian Groffen posted on Thu, 26 Nov 2009 11:51:06 +0100 as excerpted: Are there any less obvious ones Some that you may find are: /lib/libm.so /lib/libsocket.so /lib/libpthread.so /lib/libnsl.so On a side note, we have a question about this in our prefix-ebuild-quiz[1] (question 5 from the second section). I think there's unfortunately no simple way to tell what should be in and what unfortunately has to be out. It depends a lot on the host system. I feel -- but I can't back this up with hard evidence -- that it are usually the libs that are not in *DEPEND that can only be available in the host system. Basically because they usually are part of the libc, which we assume to be installed. [1] http://dev.gentoo.org/~grobian/prefix-quiz Thanks again. That quiz was particularly enlightening (to this gentoo, not prefix, user) on the type of stuff you think about, as a part of what prefix /does/. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-dev] mtime preservation
2009/11/26 Brian Harring ferri...@gmail.com: Why is this one special? Two out of three do this already, and it works. You mean two out of three blatently ignored long-standing behaviour and added a new feature without discussion or an EAPI bump. Paludis doesn't preserve mtime You mean Paludis carefully emulated Portage behaviour, and is now somehow being blamed for the whole matter, to the extent that people are trying to use threats (to the effect of 'I'm going to deliberately break packages for Paludis users) to try to get their way in the discussion. and it's approach to randomly resetting mtimes There's nothing random about it. Files' mtimes are reset to the current time while being merged, just as Portage did for years.
[gentoo-dev] Re: mtime preservation
David Leverton posted on Thu, 26 Nov 2009 12:35:53 + as excerpted: If we're not going to insist on preserving nanoseconds as far as possible, then package managers should be required to explcitly clear the nanoseconds part. While I'm not sure what it's going to do to install-times and thus whether it's practical, this /does/ at least in theory make quite some sense to me. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-dev] mtime preservation
Potentially just being a tool and taking the bait.. On Thu, Nov 26, 2009 at 12:41:55PM +, David Leverton wrote: 2009/11/26 Brian Harring ferri...@gmail.com: Why is this one special? Two out of three do this already, and it works. You mean two out of three blatently ignored long-standing behaviour and added a new feature without discussion or an EAPI bump. It was always on the todo to convert portage over to preserving mtime- this long predates PMS and even EAPI. This stretches all the way back to '03/'04. I'll also note that pkgcore from the getgo preserved mtime, back when it was called 'portage'. Ahh, the good old days before svn pissed me off enough to move it outside of gentoo. Beyond that, I presume your intention is to stir things up- PMS went out of it's way to explicitly leave it up to the manager if they preserve mtimes. Meaning portage hasn't done anything wrong in changing it's behaviour. It's a bit ironic really. Y'all didn't want mtime in there so it was left unspecified. Now you're complaining that portage changed it's behaviour (2+ years after the fact) as an arguement against adding mtime preservation into the next eapi. Paludis doesn't preserve mtime You mean Paludis carefully emulated Portage behaviour, and is now somehow being blamed for the whole matter, to the extent that people are trying to use threats (to the effect of 'I'm going to deliberately break packages for Paludis users) to try to get their way in the discussion. I mean paludis doesn't preserve mtimes. People aren't going out of their way to break paludis (and claiming so is just trolling). Breakages happen because the common sense assumption devs have (preservation of mtime) aren't actually encoded as a format standard. Further, there are quite a few postinst hacks (and quite a few that don't work all that well) working around the lack of mtime preservation. and it's approach to randomly resetting mtimes There's nothing random about it. Files' mtimes are reset to the current time while being merged, just as Portage did for years. Just because portage did something for a few years, does not make it right (this is something the PMS folk have been claiming since day one). So... that arguement is invalidated by your own statements. I label it as 'randomly' due to the fact it's contrary to what ebuild developers expect- it was a flaw when portage was doing it, it's a flaw that paludis does it due to it unnecessarily creating a gotcha for ebuild developers. Most importantly, no one has brought up a single instance of mtime preservation *breaking* things- the only criticism leveled has been ciaranms aesthetic complaints about ca-certificates and older timestamps making their way to the FS. Hence labeling it unnecessarily. Basically, y'all need to provide actual examples of this causing issues- right now the responses are either quite flammable, or lacking in technical arguements. More obstructionist then useful for a technical discourse. ~harring pgpCWlYoZPal0.pgp Description: PGP signature
Re: [gentoo-dev] mtime preservation
On Thu, Nov 26, 2009 at 12:35:53PM +, David Leverton wrote: 2009/11/26 Brian Harring ferri...@gmail.com: It's an academic discussion, and pointless. We don't mandate the filesystems PMS implementations are run on- as such we cannot make a gurantee to ebuilds that nanosecond resolution is available. It's daft to encode in the spec NS resolution when it's essentially impossible to gurantee it If we're not going to insist on preserving nanoseconds as far as possible, then package managers should be required to explcitly clear the nanoseconds part. I'd had the same thought actually. If folks really consider it to be needed, then jam it in as far as I'm concerned. It has the potential to remove a heisenbug or two. Presuming I'm reading posixmodule correctly from the python source, pkgcore already does second truncation (reliance on utime) and portage does it whenever movefile is invoked- any direct mv invocations obviously do not however. Basically, if stat_float_mtimes is False (the case for all portage context and most of pkgcore context), os.utime auto truncates to second precision. ~harring pgppAPyf5JrFX.pgp Description: PGP signature
Re: [gentoo-dev] Gentoo Prefix: on EPREFIX, ED and EROOT inside ebuilds
On 25-11-2009 16:43:32 -0700, Denis Dupeyron wrote:
Yes, I agreed coming up with some patch. I admit I haven't yet even
looked into it.
Great, thanks. If you can have it ready some time before the meeting
so that all devs can get a chance to review it before the council
members vote on it that will be even better. If you need help don't
hesitate to contact me. I'll try and look for the right people to help
you depending on what you need.
Here is the patch. I tried to cover all places.
--
Fabian Groffen
Gentoo on a different level
diff --git a/eapi-differences.tex b/eapi-differences.tex
index 7e792b8..2fdf763 100644
--- a/eapi-differences.tex
+++ b/eapi-differences.tex
@@ -176,7 +176,8 @@ Use dependencies \compactfeatureref{use-deps} No No
\IFANYKDEBUILDELSE{\IFKDEBUILDCOLOUR{None} }{}
\parbox[t]{1in}{\t{pkg\_nofetch}, \t{src\_unpack}, \t{src\_prepare},
\t{src\_configure}, \t{src\_compile}, \t{src\_test}}
\parbox[t]{1in}{\t{pkg\_nofetch}, \t{src\_unpack}, \t{src\_prepare},
\t{src\_configure},
-\t{src\_compile}, \t{src\_install}, \t{src\_test}} \\
+\t{src\_compile}, \t{src\_install}, \t{src\_test},
+\t{EPREFIX}, \t{ED}, \t{EROOT}} \\
\t{AA} \compactfeatureref{aa} Yes Yes
\IFANYKDEBUILDELSE{\IFKDEBUILDCOLOUR{Yes} }{} Yes No \\
@@ -335,6 +336,8 @@ EAPI 3 is EAPI 2 with the following changes:
\item \t{RDEPEND=DEPEND} no longer done, \featureref{rdepend-depend}.
\item Utilities now die on failure, \featureref{die-on-failure}, unless called
under \t{nonfatal},
\featureref{nonfatal}
+\item offset-prefix support by definition of \t{EPREFIX}, \t{ED} and
+\t{EROOT}, \featureref{offset-prefix-vars}
\end{compactitem}
% vim: set filetype=tex fileencoding=utf8 et tw=100 spell spelllang=en :
diff --git a/ebuild-env-vars.tex b/ebuild-env-vars.tex
index e093218..3f9d25c 100644
--- a/ebuild-env-vars.tex
+++ b/ebuild-env-vars.tex
@@ -13,7 +13,7 @@ variable.
\begin{landscape}
\begin{longtable}{l p{0.15\textwidth} l p{0.5\textwidth}}
-\caption{Defined variables}\\
+\caption{Defined variables\label{tab:defined_vars}}\\
\toprule
\multicolumn{1}{c}{\b{Variable}}
\multicolumn{1}{c}{\b{Legal in}}
@@ -104,6 +104,11 @@ variable.
\t{roo...@. Also of note is that in a cross-compiling environment, binaries
inside of \t{ROOT}
will not be executable on the build machine, so ebuilds must not call them.
\t{ROOT} must be
non-empty and end in a trailing slash. \\
+\t{EROOT}
+\t{pkg\_*}
+ No
+ Like \t{ROOT}, but with \t{EPREFIX} appended. This is a convenience
+ variable. See also the \t{EPREFIX} variable. \\
\t{T}
All
Partially\footnote{Consistent and preserved across a single connected
sequence of install or
@@ -120,6 +125,17 @@ variable.
Ditto
The full path to an appropriate temporary directory for use by any
programs invoked by the
ebuild that may read or modify the home directory. \\
+\t{EPREFIX}
+All
+ Yes
+ The normalised offset-prefix path of an offset installation. When
+ \t{EPREFIX} is not set in the calling environment, \t{EPREFIX}
+ defaults to the built in offset-prefix that was set during
+ installation of the package manager. When a different \t{EPREFIX}
+ value than the built in value is set in the environment, a
+ cross-prefix build is performed where using the existing utilities,
+ a package is build for the given \t{EPREFIX}, akin to \t{ROOT}. See
+ also~\ref{sec:offset-vars}. \\
\t{D}
\t{src\_install}
No
@@ -130,6 +146,11 @@ variable.
Yes
Contains the full path to the image that is about to be or has just been
merged. Must be
non-empty and end in a trailing slash. \\
+\t{ED}
+\t{src\_install}
+ See \t{D}
+ Like \t{D}, but with \t{EPREFIX} appended. This is a convenience
+ variable. See also the \t{EPREFIX} variable. \\
\t{IMAGE}\footnote{Deprecated in favour of \t{D}.}
\t{pkg\_preinst}, \t{pkg\_postinst}
Yes
@@ -216,6 +237,22 @@ variable.
}
\end{centertable}
+\begin{centertable}{EAPIs supporting offset-prefix env variables}
\label{tab:offset-env-vars-table}
+\begin{tabular}{ l l l l }
+ \toprule
+ \multicolumn{1}{c}{\textbf{EAPI}}
+ \multicolumn{1}{c}{\textbf{\t{EPREFIX}?}}
+ \multicolumn{1}{c}{\textbf{\t{EROOT}?}}
+ \multicolumn{1}{c}{\textbf{\t{ED}?}} \\
+ \midrule
+\t{0} No No No \\
+\t{1} No No No \\
+\t{2} No No No \\
+\t{3} Yes Yes Yes \\
+\bottomrule
+\end{tabular}
+\end{centertable}
+
Except where otherwise noted, all variables set in the active profiles'
\t{make.defaults} files must
be exported to the ebuild environment. \t{CHOST}, \t{CBUILD} and \t{CTARGET},
if not set by
profiles, must contain either an appropriate machine tuple (the definition of
appropriate is beyond
@@ -309,6 +346,18 @@ installing \t{foo-2:2} to replace \t{foo-2:1} and
\t{foo-3:2}.
Re: [gentoo-dev] mtime preservation
On Wed, 25 Nov 2009 17:28:33 -0800
Brian Harring ferri...@gmail.com wrote:
It's not in the least bit nasty. It's requiring people to be
explicit about special requirements.
I honestly wish that explicitness you're pushing for mtime were
applied to all parts of mtime.
Why is this one special? Two out of three do this already, and it
works. Paludis doesn't preserve mtime- and it's approach to randomly
resetting mtimes is known to cause issues (bug 263387, python eclass
having to do pyc/pyo compilation in postinst, etc).
No, two out of three do a half-arsed job about it. If we're going to
make it behaviour upon which packages can rely, we should be doing so
properly to avoid the whole mess cropping up again as people start
following the newer POSIX standard.
You can't have been listening very hard, then. The complaint is
that it results in files with dumb mtimes being merged to /.
As far as I can tell, no one but you is complaining about the
mtimes.
Your complaints primarily have been that tarball'd mtimes can pass
through to the fs- I've not seen any comments from you in bug 264130
that actually enumerate spots were this is anything but an aesthetic
complaint. Specifically your aesthetics- the pkgs in question have
functioned fine w/ odd mtimes passing through.
Yes, again, it's about doing it properly.
Wording such as that just isn't suitable for a specification. It
requires implementers to guess what future ebuilds are going to
rely upon (and ebuilds regularly do rely upon weird quirks), and
makes it impossible to produce a package manager that can be shown
to be compliant.
That wording is explicit that PMS cannot lock down all potentials.
Ok, so PMS would effectively say ebuilds must not rely upon any
particular behaviour, which in turn means any ebuild that breaks with
older Portage / Paludis is broken. That's not what you're after, is it?
This is no different then PMS's stance on VDB (and similar in
reasoning)- ebuilds are reliant on vdb internals yet PMS explicitly
ducked documenting it. Double standards suck.
That one's being fixed, slowly...
Further, drop the specificity BS. You've intentionally left parts of
PMS vague- trailing '/' for ebuild vars is a good example (ssmtp
breaks if ${D} vs ${D}/, perl likely has the same breakage). These
lead to actual incompatibilities, and because PMS isn't explicit,
there is no 'right' implementation for it.
These are entirely unrelated. Any ebuild that relies upon the slash
either way is broken. But we can't realistically do that for mtimes,
since it would involve modifying *the package that gets installed*, and
doing so by patching sources to use things POSIX considers deprecated.
It's not just an ebuild tweak.
The daft thing about you picking at this wording is that the
scenarios you can come up with are all hypothetical. Yes, a manager
could randomize mtimes when it's doing splitdebug.
The manager wouldn't however because the only reason to do things
like that is to intentionally cause issues.
It's about doing it right, so we don't get bitten by it later on when
everyone's using nanosecond-resolution timestamps like POSIX suggests.
--
Ciaran McCreesh
signature.asc
Description: PGP signature
Re: [gentoo-dev] mtime preservation
On Wed, 25 Nov 2009 16:49:17 -0800 Zac Medico zmed...@gentoo.org wrote: Not non-preservation. Partial and inconsistent corruption. Wouldn't loss of precision be a more accurate description? Of the known packages which require timestamp preservation, do any of them use sub-second precision in their timestamp comparisons? No. When you're dealing with decimals, floating point rounding isn't a simple truncation: 1234567890.9 1234567891.0 nor is it a rounding: 1234567890.1 1234567890.112 The former is particularly bad, since POSIX deals with legacy functions by ignoring the nanosecond part, not by rounding it. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] mtime preservation
On Wed, 25 Nov 2009 17:14:27 -0800 Brian Harring ferri...@gmail.com wrote: This discussion in generall is daft. No package can rely on nanonsecond resolution for installation because the most common FS out there (ext3) does *second* level resolution only. As such, I can pretty much gurantee there is *zero* packages out there that require nanosecond resolution for installation. We aren't discussing requiring nanosecond resolution. We're discussing not getting corrupted timestamps, which is what happens. If you're building on a filesystem which does support nanosecond resolution timestamps, Portage will screw up the *second* part of the timestamp as well as the nanosecond part. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] mtime preservation
On Thu, 26 Nov 2009 08:59:44 +0100 Ulrich Mueller u...@gentoo.org wrote: Not non-preservation. Partial and inconsistent corruption. Wouldn't loss of precision be a more accurate description? Yes. Or even rounding. No, corruption, including of the seconds part, is the right way to describe it. Of the known packages which require timestamp preservation, do any of them use sub-second precision in their timestamp comparisons? I can speak for Emacs only, where the comparison code (in fileio.c) is as follows: if (stat (SDATA (absname1), st) 0) return Qnil; mtime1 = st.st_mtime; if (stat (SDATA (absname2), st) 0) return Qt; return (mtime1 st.st_mtime) ? Qt : Qnil; It uses stat(2), therefore nanoseconds are ignored. Uh, two things. First, Portage sometimes corrupts the second part too. Second, [1]: For compatibility with earlier versions of this standard, the st_atime macro shall be defined with the value st_atim.tv_sec. Similarly, st_ctime and st_mtime shall be defined as macros with the values st_ctim.tv_sec and st_mtim.tv_sec, respectively. Note that POSIX considers st_mtime to be legacy, and thinks you should be using st_mtim instead. Do you want this to break when Emacs starts following the newer standard, or do you want to get this right now so we won't have to have the discussion again a year later? [1]: http://www.opengroup.org/onlinepubs/9699919799/basedefs/sys_stat.h.html -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Next council meeting on 7 Dec 2009 at 1900UTC
On Wed, 25 Nov 2009 17:34:38 -0800 Brian Harring ferri...@gmail.com wrote: I'd like http://archives.gentoo.org/gentoo-dev/msg_6b3e00049a1bf35fbf7a5e66d1449553.xml to be discussed, specifically zacs form of forced mtime updating of /var/db/pkg on vdb modifications I've still not had an answer to: Provide proof that all existing and future caches that would rely upon this validation mechanism are functions purely and exclusively dependent upon the VDB content, and I shall be happy to make the change. Without that proof, all introducing that change would do is provide another unreliable mechanism, so it's of no use at all. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Next council meeting on 7 Dec 2009 at 1900UTC
On Thu, Nov 26, 2009 at 03:31:17PM +, Ciaran McCreesh wrote: On Wed, 25 Nov 2009 17:34:38 -0800 Brian Harring ferri...@gmail.com wrote: I'd like http://archives.gentoo.org/gentoo-dev/msg_6b3e00049a1bf35fbf7a5e66d1449553.xml to be discussed, specifically zacs form of forced mtime updating of /var/db/pkg on vdb modifications I've still not had an answer to: Provide proof that all existing and future caches that would rely upon this validation mechanism are functions purely and exclusively dependent upon the VDB content, and I shall be happy to make the change. First I've seen this question actually or at least this particular interesting phrasing. That said, no comes to mind, since the requirement you set is daft. The timestamp updating is for whenever the vdb content (addition of a pkg, pkgmoves being applied, removal of a pkg, modification of metadata, etc) is changed. That's all that timestamp is for. Vdb content. In light of what the timestamp is, your demand for proof is pretty off the mark. If you still consider it to be a valid question, please rephrase it and clarify why exactly proof must be provided that people reading that timestamp (which is for vdb content only) will only be using that timestamp for vdb content. Your request is akin to demanding proof that a hammer only be used as a hammer. It's a fricking hammer- it has one use, one way of being used. If someone goes out of their way to be an idiot, they're an idiot, not the specs problem. Seriously, if you're actually worrying about some specific usage case, state it- on the face of it, your request for proof right now makes zero sense. Kindly provide a scenario or elucidation. ~harring pgpLDUFFcYk7T.pgp Description: PGP signature
Re: [gentoo-dev] Next council meeting on 7 Dec 2009 at 1900UTC
On Thu, 26 Nov 2009 08:33:03 -0800 Brian Harring ferri...@gmail.com wrote: Provide proof that all existing and future caches that would rely upon this validation mechanism are functions purely and exclusively dependent upon the VDB content, and I shall be happy to make the change. First I've seen this question actually or at least this particular interesting phrasing. That said, no comes to mind, since the requirement you set is daft. It's clipboarded from the bug. The timestamp updating is for whenever the vdb content (addition of a pkg, pkgmoves being applied, removal of a pkg, modification of metadata, etc) is changed. That's all that timestamp is for. Vdb content. In light of what the timestamp is, your demand for proof is pretty off the mark. If you still consider it to be a valid question, please rephrase it and clarify why exactly proof must be provided that people reading that timestamp (which is for vdb content only) will only be using that timestamp for vdb content. Your request is akin to demanding proof that a hammer only be used as a hammer. It's a fricking hammer- it has one use, one way of being used. If someone goes out of their way to be an idiot, they're an idiot, not the specs problem. Seriously, if you're actually worrying about some specific usage case, state it- on the face of it, your request for proof right now makes zero sense. Kindly provide a scenario or elucidation. You're asking for a cache validation mechanism that's based not upon what it logically invalidates, but upon something you assume to be equivalent. Specifically, you assume that VDB has changed and the installed packages have changed are exactly the same thing, and you're wanting to build a cache based upon that highly questionable assumption. There are at least two logically different sets of 'changes' that might apply to VDB (metadata changes, and package version changes), and you're attempting to confuse the two, along with any others that people come up with later on. What's wrong with, instead, having cache files for something has changed, the set of installed packages has potentially changed, the metadata for installed packages has potentially changed and the set of installable packages has potentially changed? That way you can write your cache checks to depend explicitly upon the thing upon which they depend (along with a global catch-all to make future new validation mechanisms easier), not upon something you assume is equivalent but probably isn't. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] mtime preservation
On Thursday 26 November 2009 13:21:43 Brian Harring wrote: It was always on the todo to convert portage over to preserving mtime- this long predates PMS and even EAPI. Like, for example, use deps? Yet somehow we managed to introduce those in a new EAPI, instead of retroactively adding them to all EAPIs. Why should mtimes be different? Beyond that, I presume your intention is to stir things up I suppose you have the right to presume whatever you want. It's a bit ironic really. Y'all didn't want mtime in there so it was left unspecified. Now you're complaining that portage changed it's behaviour (2+ years after the fact) as an arguement against adding mtime preservation into the next eapi. I'm certainly not arguing against adding it, I just want it to be done properly, and I'm expressing distaste at people trying to blame Paludis for the fact that it's not as easy as some people want it to be. I mean paludis doesn't preserve mtimes. People aren't going out of their way to break paludis (and claiming so is just trolling). I don't think anyone's talking about changing packages purely for the sake of break Paludis and for no other reason, but people have been talking about making changes that they know will break Paludis. (Whether they've actually done so is a different question, but the talk, and people blaming Paludis both when it behaves differently from Portage and when we've taken care to make it behave the same as Portage only for Portage to randomly change, are quite irritating.) Just because portage did something for a few years, does not make it right (this is something the PMS folk have been claiming since day one). So... that arguement is invalidated by your own statements. PMS tries to document Portage behaviour as long as it's not clearly unreasonable and unspecifiable. Discarding mtimes is suboptimal behaviour, yes, but it's coherent enough that it can't be considered a blatent bug. Much like the lack of use deps in older EAPIs.
Re: [gentoo-dev] Re: mtime preservation
Duncan wrote: David Leverton posted on Thu, 26 Nov 2009 12:35:53 + as excerpted: If we're not going to insist on preserving nanoseconds as far as possible, then package managers should be required to explcitly clear the nanoseconds part. While I'm not sure what it's going to do to install-times and thus whether it's practical, this /does/ at least in theory make quite some sense to me. This seems like a pretty reasonable (and easy) thing to do. All that we have to do is round the mtime value before it's passed into the os.utime() call. -- Thanks, Zac
