Re: thoughts on architectures
On Mon, Feb 11, 2002 at 01:12:28PM -0500, [EMAIL PROTECTED] wrote:
Of course you have to specify this if you want to use a flexible
architecture system. But it should not be part of the design of the
architecture system (eg nothing should depend on the selection of such
interface names).
Actually, I think you'd want to standardize the naming. I was just trying
to establish a concrete example. I didn't follow through well enough, though.
Of course you want to standardise it as soon as you start to use it. But only
for yourself (your universe). There is no way you can devise a scheme that
fits everyone.
So, for Debian, you would want to standardise it. And for Debian, it would
be a mistake to make radical change. It would be much better to make a copy
of the existing scheme in the new interface, adn then add to it over time.
Why do you want to replace one evil with another? The existing Provides:
and Depends: can cope with non-existing package names (called virtual
packages). Those are exactly what you need for the interfaces like the CPU
etc.
True, but that makes it difficult to distinguish between between regular
dependacies and these architecture-type dependacies in software.
Which is the right thing to do because all such differences are artificial.
That is exactly the core point of my idea. As soon as I accept that there
are fundamental differences between such environmental dependencies and normal
dependencies (to stick with your naming), I have to reconsider the whole idea.
In other words, you are not consequent. Either there are differences, then you
should explain them, or there are not, and then I don't understand why you
want different fields.
Simple example: you want to be able to quickly find all possible packages that
would work on your system. At that point, it'd be very useful to have a file
that looks like:
To find all possible installable packages, you have to take all dependencies
into account. This should be done by the distributor as a part of the
archive management. As you have to do this for the normal dependencies anyway
(it is done currently for normal Debian packages), the same procedures will
work if you add the arch interfaces to the normal list.
Maybe it is worthwhile to split up the checks into two (for example,
it might help error diagnostics or increase performance). I have not thought
this through. But I don't see conceptional advantages.
4. Drop keeping metadata in package filenames. Make them just a unique
string, assigned when the package is installed into the archive.
I don't understand at all what you are aiming at here.
That gets rid of the need to add something to the filename to
distinguish i386 from sparc packages.
Can you cite an example?
Sure: apt_0.5.4_freebsd-i386.deb vs. apt_0.5.4_i386.deb
Oh that, well. I don't care about the name. Using a human readable name
has advantages, and should bbe preserved if possible. See my other mail,
too.
It'd be unnecessary. If you have the dependacies in the Packages file, you
can trivially determine which packages are available to your system in dselect
or apt.
Well, my arch-handling text describes a different strategy. I don't think
this should be part of dpkg or apt, and I don't think it is trivial in the
framework I think of.
Thanks,
Marcus
re: thoughts on architectures
There's a field in the ELF header called OS/ABI. Readelf -h finds it, and it looks like this: Normal binaries: OS/ABI:UNIX - System V FreeBSD binaries: OS/ABI:UNIX - FreeBSD I need to look into it a bit more, and figure out exactly what FreeBSD does and doesn't do with this. basically, when you set OSABI, you are declaring this is not standard ELF. tools can no longer assume anything about how it works. actually, given that freebsd's ELF *is* not the SYSV ABI, this is probably the right thing for them to do.. (they have a different (slower -- see -fpcc-struct-return) structure return convention than SYSV.) but all the rest of the world besides HPUX use ELF as it is and shouldn't set ELF_OSABI. this field has a bad name, but there you have it :-)
Re: thoughts on architectures
On Mon, Feb 11, 2002 at 01:25:51AM -0500, [EMAIL PROTECTED] wrote: 4. Drop keeping metadata in package filenames. Make them just a unique string, assigned when the package is installed into the archive. That gets rid of the need to add something to the filename to distinguish i386 from sparc packages. Just use the Packages files, and the control fields from the packages themselves. I'd be in favour of going further: use one Packages file, and determine available packages based on the tags your kernel and libc support. Which would lead to the problem, that in the pool/ there would be stuff like: pool/main/p/package/: package_Version-1_abcde.deb package_Version-1_lsahd.deb package_Version-1_iorzq.deb package_Version-1_mbmnb.deb package_Version-1_poiuz.deb package_Version-1_mjuhb.deb Essentially rendering manual download impossible. This is a question to Marcus Brinkmann: How can one distinguish package_1.0-1.deb (i686,glibc,mmx) and package_1.0-1.deb (sparc,netbsd4) in the pool (i.e. filesystem)? I know, in your doc, you don't explicitly specify an 'encoding' for this dependency information, but people (as simpleminded as I am) would think about some extra entry in the Depends: field, which wouldn't help much with filesystemlayout in the mirrors as Philip Charles mentioned in regard to partial mirrors and things as pointet out above. Regards, David -- Signaturen sind wie Frauen. Man findet selten eine Vernuenftige -- gesehen in at.linux pgpj0DaiWgzvF.pgp Description: PGP signature
Re: thoughts on architectures
On Mon, Feb 11, 2002 at 01:25:51AM -0500, [EMAIL PROTECTED] wrote: I think you could say that a binary's environment is made up these things: Whatever. The exact set of virtual package names that make up the architecture depends on your needs (I called this the universe). Of course you have to specify this if you want to use a flexible architecture system. But it should not be part of the design of the architecture system (eg nothing should depend on the selection of such interface names). 1. Drop the Architecture field. 2. Add two new multi-value fields. I'll call them Env-Provides, and Env-Depends. Probably someone else can think of better names. Why do you want to replace one evil with another? The existing Provides: and Depends: can cope with non-existing package names (called virtual packages). Those are exactly what you need for the interfaces like the CPU etc. 3. Make those fields provide and depend on tags which are completely separate from package names. This makes it easy to distinguish what packages a package depends on from the environment it's been built for. I don't understand what you mean here. Can you elaborate? It also makes it easier to make an index of packages for a particular architecture combination, Why? and doesn't pollute the package namespace with a lot of virtual packages. Beauty is in the eye of the beholder. I think having two fields duplicated without any real technical advantage is adding to ugliness. 4. Drop keeping metadata in package filenames. Make them just a unique string, assigned when the package is installed into the archive. I don't understand at all what you are aiming at here. That gets rid of the need to add something to the filename to distinguish i386 from sparc packages. Can you cite an example? favour of going further: use one Packages file, and determine available packages based on the tags your kernel and libc support. I think you misunderstand the purpose of the packages file. The packages file determines what is in the distribution for a specific architecture. So dinstall (katie, whatever) would decide if a new package in the pool would enter a packages file or not based on the dependencies in the package. I am probably misunderstanding you, but without an idea want you want to achieve with 4 above I have trouble to see why you would want to change that. Yes, exactly. And you need some differences in the algorithm that decides when to compile a package from source for a given architecture (the pool might contain a compatible but inferior binary package). But that's not a problem for the guy who makes the CD's. No, of course not. Thanks, Marcus -- `Rhubarb is no Egyptian god.' Debian http://www.debian.org [EMAIL PROTECTED] Marcus Brinkmann GNUhttp://www.gnu.org[EMAIL PROTECTED] [EMAIL PROTECTED] http://www.marcus-brinkmann.de
Re: thoughts on architectures
On Mon, Feb 11, 2002 at 05:18:51PM +1300, Philip Charles wrote: Maybe there should be several layers of Packages files. I don't think the distributor should try to anticipate the needs of individual users of the distribution at this level. If you want to get rid of all emulated packages, just remove the emulator providing package (or just drop its Provides:) and remove everything that is not installable now because of dependencies. This is a fully automated process. (I think you already have such a process fro CD building, so this is nothing new for you). There might be other metadata in the package that shows if a package is a special optimization, or is only available if emulated. Such metadata might be available implicit and only be valid in the universe the package is distributed in (eg depends on the selection of virtual names chosen). I think this is something that could be added to the design at any time, when it is more clear how an implementation would look like. This is indeed an implementation detail. Special casing the selection rules for the distribution, so that some packages would be installed in different Packages files (based on the criteria they were enabled for the distribution) certainly be possible. Thanks, Marcus -- `Rhubarb is no Egyptian god.' Debian http://www.debian.org [EMAIL PROTECTED] Marcus Brinkmann GNUhttp://www.gnu.org[EMAIL PROTECTED] [EMAIL PROTECTED] http://www.marcus-brinkmann.de
Re: thoughts on architectures
On Mon, Feb 11, 2002 at 11:07:18AM +0100, David Schmitt wrote: Which would lead to the problem, that in the pool/ there would be stuff like: pool/main/p/package/: package_Version-1_abcde.deb package_Version-1_lsahd.deb [...] Essentially rendering manual download impossible. Which is indeed a problem. This is a question to Marcus Brinkmann: How can one distinguish package_1.0-1.deb (i686,glibc,mmx) and package_1.0-1.deb (sparc,netbsd4) in the pool (i.e. filesystem)? I know, in your doc, you don't explicitly specify an 'encoding' for this dependency information, but people (as simpleminded as I am) would think about some extra entry in the Depends: field, which wouldn't help much with filesystemlayout in the mirrors as Philip Charles mentioned in regard to partial mirrors and things as pointet out above. This is a good question. I think the best answer I have is that if you want to keep it transparent and usable by a normal user (eg allow manual download without a lot of tinkering in the packages file), then your best bet is to limit your universe to only allow very constrained overlaps between architectures. In other words, you would normally use the traditional encoding and use special encodings for special cases of overlaps. You don't even need a special case for overlap if one package for GNU/Linux works on the Hurd, for example. Of course, the package name wouldn't tell you that it runs on the Hurd, but the dependencies would. The other possibility is to build symlink trees based on the packages file. We got rid of symlink trees in Debian in favor of the packages pool, but the pool has a similar problem in that it is less transparent for users (they have to know the source name of the package). So this problem would only be hardened. These are just some examples. Please don't think I am doing a fallacy here. Although my proposal is evry broad and flexible, I don't think you should really exploit it in that property. In an actual implementation, you would carefully choose the overlaps where they are useful. An actual implementation would hard code some of the generic parts of the design and limit flexibility in favor for transparency and pragmatics. Thanks, Marcus -- `Rhubarb is no Egyptian god.' Debian http://www.debian.org [EMAIL PROTECTED] Marcus Brinkmann GNUhttp://www.gnu.org[EMAIL PROTECTED] [EMAIL PROTECTED] http://www.marcus-brinkmann.de
Re: thoughts on architectures
On Sun, Feb 10, 2002 at 09:28:35PM -0500, [EMAIL PROTECTED] wrote: Like libc.so.4 on FreeBSD, soon to be libc.so.5? Not compatible with libc5 on Linux. It's confusing, but I don't know any good way around it. Well, then you have to include the OS-ABI field into the dependency name for all library dependencies. As the information is easily available from the binary with objdump, this can be as automatic as the soname. You have to make such decisions based on the universe you live in. If you build a distribution that only ever has binaries from a single ABI tag, you don't need to include it. But if you have different ABI tags you want to support, you mangle it into the dependency name. Bear in mind that quite a few systems already support multiple kinds of binaries, My text was written with that in mind. Also, FreeBSD (and possibly NetBSD as well) uses the ELF OSABI field to mark it's binaries. The GNU/Hurd does it as well. Of course, dpkg doesn't know that... dpkg doesn't know a lot of things ;) That should be correct. A distribution already is just a Package file with references to files in the pool. No real change there. The difference would seem to be in the generation of the Packages file. Yes, exactly. And you need some differences in the algorithm that decides when to compile a package from source for a given architecture (the pool might contain a compatible but inferior binary package). Thanks, Marcus -- `Rhubarb is no Egyptian god.' Debian http://www.debian.org [EMAIL PROTECTED] Marcus Brinkmann GNUhttp://www.gnu.org[EMAIL PROTECTED] [EMAIL PROTECTED] http://www.marcus-brinkmann.de
Re: thoughts on architectures
On Mon, Feb 11, 2002 at 02:35:50PM +1100, matthew green wrote: Also, FreeBSD (and possibly NetBSD as well) uses the ELF OSABI field to mark it's binaries. The GNU/Hurd does it as well. hmm, last i looked hurd used an ELF note, like NetBSD. Yes, sorry, I thought that was he meant (and he did :). I did not even know about a special OSABI field. ulysses:/tmp/x# objdump -s -j .note.ABI-tag /bin/bash /bin/bash: file format elf32-i386 Contents of section .note.ABI-tag: 8048108 0400 1000 0100 474e5500 GNU. 8048118 0200 1e00 ulysses:/tmp/x# objdump -s -j .note.ABI-tag /gnu/bin/bash /gnu/bin/bash: file format elf32-i386 Contents of section .note.ABI-tag: 8048100 0400 1000 0100 474e5500 GNU. 8048110 0100 Thanks, Marcus -- `Rhubarb is no Egyptian god.' Debian http://www.debian.org [EMAIL PROTECTED] Marcus Brinkmann GNUhttp://www.gnu.org[EMAIL PROTECTED] [EMAIL PROTECTED] http://www.marcus-brinkmann.de
Re: thoughts on architectures
On Mon, Feb 11, 2002 at 04:41:07AM +0100, Marcus Brinkmann wrote: On Mon, Feb 11, 2002 at 02:35:50PM +1100, matthew green wrote: Also, FreeBSD (and possibly NetBSD as well) uses the ELF OSABI field to mark it's binaries. The GNU/Hurd does it as well. hmm, last i looked hurd used an ELF note, like NetBSD. Yes, sorry, I thought that was he meant (and he did :). I did not even know about a special OSABI field. ulysses:/tmp/x# objdump -s -j .note.ABI-tag /bin/bash /bin/bash: file format elf32-i386 Contents of section .note.ABI-tag: 8048108 0400 1000 0100 474e5500 GNU. 8048118 0200 1e00 ulysses:/tmp/x# objdump -s -j .note.ABI-tag /gnu/bin/bash /gnu/bin/bash: file format elf32-i386 Contents of section .note.ABI-tag: 8048100 0400 1000 0100 474e5500 GNU. 8048110 0100 There's a field in the ELF header called OS/ABI. Readelf -h finds it, and it looks like this: Normal binaries: OS/ABI:UNIX - System V FreeBSD binaries: OS/ABI:UNIX - FreeBSD I need to look into it a bit more, and figure out exactly what FreeBSD does and doesn't do with this.
Re: thoughts on architectures
On Sun, Feb 03, 2002 at 05:59:28PM +0100, Andreas Schuldei wrote: Markus Brinkmanm (of HURD fame) has written this http://master.debian.org/~brinkmd/arch-handling.txt which is mandatory reading for us. You're right, that's a very good document. He's apparently been thinking along similar lines that I have. There are other problems, like sparc64/32, and ia64 or AMD's 64-bit arch supporting i386, that could be helped by this. The major oversight is that he completely fails to mention libc. And for us, that remains a big issue. Right now, and for immediately forseeable future, I'm not going to be using glibc. Another one is branding binaries. Linux emulation could be quite useful on *BSD and maybe hurd, as well. But it would be really helpful if binutils were altered to do branding. That would help make it possible to install i386 binary packages directly onto freebsd-i386, given the appropriate libraries in /compat/linux. There'd be some other headaches, but it could be done. Might help the Hurd developers, too, if they had it available. they (the hurd people) are very interested in working closely together with us, simply since we together might have enough weight to make debian make more architecture aware. they are both highly cooperativ and competent, so we better are nice to them. (c: please give some feedback on that document. Excellent. I think we're getting to the point where it makes sense to have those conversations. Perhaps we can all agree to strangle the person who wrote the docs for dpkg in SGML... ;-) ---Nathan
