reference implementation
So what do people think of a reference implementation distribution around rpm5? I recognize that the politics could quickly overshadow the utility, but my idea was nothing fancy, no patches, just a minimalist Linux system that reflects how to use rpm and what ways rpm provides to solve different packaging problems (like lua for glibc scriptlets as an example). The rationale is that I'm one of those people that learns by example, and seeing how this team of rpm veterans pieces together a base system I think could greatly benefit the rpm using community at large and perhaps even become a base for distros to work with. Also I think having clear examples of how to use new RPM features like lua, YAML, runtime probes, etc. will help with eventual adoption. As Jeff points out, new rpm features take a few years to reach the masses so maybe this is one way to help speed up that process. Thoughts? Jason __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: reference implementation
On Jun 4, 2007, at 10:50 AM, Jason Corley wrote: So what do people think of a reference implementation distribution around rpm5? I recognize that the politics could quickly overshadow the utility, but my idea was nothing fancy, no patches, just a minimalist Linux system that reflects how to use rpm and what ways rpm provides to solve different packaging problems (like lua for glibc scriptlets as an example). The rationale is that I'm one of those people that learns by example, and seeing how this team of rpm veterans pieces together a base system I think could greatly benefit the rpm using community at large and perhaps even become a base for distros to work with. Also I think having clear examples of how to use new RPM features like lua, YAML, runtime probes, etc. will help with eventual adoption. As Jeff points out, new rpm features take a few years to reach the masses so maybe this is one way to help speed up that process. Thoughts? I too am one who learns by example, and there are many reasons why a small set of known quality packaging would help rpm development, not the least of which is in starting down the rugged path of --rollback QA. (aside) There's a test-harness in rpm cvs written by James for --rollback. The test-harness has many appealing features, simple but extensible, documented, many useful bells and whistles that I can appreciate, etc. Tying reference packaging into the test-harness would permit automatic QA for installs and the features used. The alternative is what I used to use, /usr/lib/rpm/trpm, an absolutely grotty piece of scripting that I used for assembling and reproducing rpm install/ upgrade/erase problems from package collections. Think: implicit dependency solving using globbing. How about using the packages in the inner circle of dependency hell as a starting reference implementation? That basically means that the package set is enough to get /bin/sh functional in a chroot and nothing more. That already includes glibc-common, which can/will benefit from lua scriptlets, provides a testbed for bootstrapping from an empty chroot, where there are many hard packaging problems to solve, and does not include too many packages that we will lose our focus or get distracted by politics. If up to me, I'd suggest we choose some embedded architecture useful for Mark @work or, alternatively) swipe packages from Fedora/ARM just starting up. The goal is to find something useful for someone, perhaps Ralf and openpkg has useful work like bootstrapping Mac OS X instead. (aside) I helped Ralph Siemsen with the last RHL9 ARM distro a couple summers back while waiting to be fired from RedHat. I have 2 functional netwinders and a fair amount of ARM experience if you want to try jump-starting Fedora-ARM. I'll likely be talkin g to Ralph SIemsen in Ottawa at OLS this month too. We (or at least I) do have to focus on rpm-5.0 and XAR and depsolving and Mac OS X by, say, mid-July at the latest or rpm-5.0 won't exist in time for Leopard. 73 de Jeff __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: reference implementation
How about using the packages in the inner circle of dependency hell as a starting reference implementation? That basically means that the package set is enough to get /bin/sh functional in a chroot and nothing more. This is exactly what I was thinking, everything up to a shell with perhaps one daemon (like ssh) to serve as an example for any more complicated packaging features (although just to /bin/sh is a clearer delineation and perhaps less political goal). Jason __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: reference implementation
On Jun 4, 2007, at 2:12 PM, Andy Green wrote: Hatle, Mark wrote: I'm more of the mindset for this we need to be Fedora based, or some other common point of reference (both cross compiled and not) so that it's easy to see what changes we made to another frame of reference. I'm familiar with making Fedora packages cross compile myself. And we have many of these packaging cross compiling already. Fair enough... cross compiled Fedora would be valuable alright. Since they don't seem to be interested in it then all the better it's done externally... If you'ld like, I'll turn my current remote fetch example using rpm-4.4.9 into a distribution point for Fedora CVS. Here are the quick instructions, the top level URI is ftp://wraptastic.org/F 0) Create an empty /L directory, insure that builder can mkdir and write to /L 1) Download frpm and macros files and examine. There's not much going on. 2) Download one of the spec files in the subdirs. The subdirs are all directly from Fedora CVS, with BuildRequires: digest(...) and BuildRequires: gnupg(...) dependencies to verify downloads. 3) Invoke frpm. E.g. to build the ed package, type frpm -ba ed.spec The elements to build will be fetched into /L/ed, and the build element digests and signatures will be verified before the build starts. I'll happily add other Fedora packages from CVS to the mix, and will add patches and whatever else is necessary, if you'ld like to try rpmbuild from remote build elements. That way all the changes necessary to Fedora packages can bee seen, and diffs can be sent along if/when the time comes. Note digest() and gnupg() is rpm-4.4.9 functionality, earlier versions of rpm do not have the ability to verify build elements. And apologies if off-topic, I'm shopping Guinea pigs for my remote rpm-4.4.9 fetch as much as I am trying to help establish the contents of a reference install. 73 de Jeff __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
RE: reference implementation
(apologies in advance for wonderful email formatting.. outlook strikes again...) On Jun 4, 2007, at 3:22 PM, Hatle, Mark wrote: QEMU, can't emulate EABI and/or NPTL on ARM.. (or couldn't last I looked...) Are EABI and/or NPTL needed for *building*? Sure they are needed for installing on the target. NPTL is probably a no. But EABI is a yes. You can't mix OABI and EABI binaries. All of the embedded ARM development has moved to EABI binaries. (As an aside, the purpose of EABI is to allow for a fixed calling convention so that regardless of the optimizations embedded in the binaries, they will still work with each other.. no problems w/ hard vs soft float vs vfp float.. etc. OABI had to change everytime someone changed the floating point or register sets available to an app!) So you end up having to boot into an ARM environment in QEMU for it to work. (This is why I keep saying QEMU isn't a reasonable environment for cross scriptlets.. replacing w/ lua or some other mechanism is the only reasonable way I can see to do this.) Well, the other component is that the QEMU environment should match the environment installed onto the target. So again, NPTL probably not a huge issue.. but EABI is a big deal. At MV we used to run the scriptlets in the host context during an install.. here at WR, we just disable the scriptlets and have some external things do many of those actions. (We'd obviously like to get rid of the external things, and replace w/ internal Lua!) The main issue for cross-arch installs is that /bin/sh may not be functional if the target file system includes binaries from two arches. Correct. That's not true if the install is into a qemu sysimage, one should have a functional /bin/sh. OTOH, one does have the clunkiness of a qemu sysimage to drag around, which is likely why you'ld like lua scriptlets instead. That assumes QEMU can run the binaries in the image. I see that useful for many cases, but not every case. (QEMU is improving every day. It's now capable of emulating a few 64-bit environments... even those are full system environments and not simple application emulation.) Which brings the next question: Is EABI and NPTL necessary for install scriptlets? Most install scriptlets I've seen do very little, but I'm always amazed at what people try to accomplish with rpm scriptlets. E.g. I have grub installs wired into triggers @work. Clever, but Oh my gawd!. Same here.. user/group add/remove... shell setup and initscript setup cover probably 75+% of the script usages. I figure that another 20% cover misc file manipulations.. the last 5% cover the case where they want to actually run an executable to do something. Always got to be a better way to do that (or don't do that if necessary). --Mark
Re: reference implementation
Andy Green wrote:
Jeff Johnson wrote:
Are EABI and/or NPTL needed for *building*? Sure they are needed
for installing on the target.
NPTL is probably a no. But EABI is a yes. You can't mix OABI and
EABI binaries. All of the embedded ARM development has moved to EABI
binaries. (As an aside, the purpose of EABI is to allow for a fixed
calling convention so that regardless of the optimizations embedded in
the binaries, they will still work with each other.. no problems w/
hard vs soft float vs vfp float.. etc. OABI had to change
everytime someone changed the floating point or register sets
available to an app!)
Apologies for my lack of knowledge on ARM.
So why not sparate OABI and EABI qemu sysimages?
(the lights come on, EABI doesn't work with qemu)
OK, so EABI has to be cross-compiled unitil qemu can emulate.
OABI - sysimage still seems viable.
My lights must still be off, I don't see why you need qemu to build the
cross packages at all. It's like I might want to make a PowerPC binary
rpm for some package, but I don't have a PowerPC. That's okay if I am
using this proposed Cross-Fedoraish thing, I can tell my i386 box to
cross build my Ultrafoo package as a PowerPC binary RPM. Or if all I
have is a PowerPC, I can make i386 binary rpms of my package.
Where does EABI come into it? That you need a crosscompiled EABI uclibc
lib on the target to link to? Well that's easily done without needing
any emulation on the build host.
Just to explain this a bit further, what I did with Octotux was to add a
dir devel-filesystem-%{_target_cpu} under %_topdir, so for example there
was %_topdir/devel-filesystem-arm.
After I built a crosscompiled package that I wanted to let other
packages be built against, say I built libdb4 for ARM and now I wanted
to build postfix for arm that wants libdb4, I used a script to install
my arm libdb4 and libdb4-devel rpms into the *build* host using
--root=%_topdir/devel-filesystem-arm. Then when I cross-built postfix
for ARM, I took care to tell postfix's configure to have CFLAGS with
-I%_topdir/devel-filesystem-%{_target_cpu}, so it picks up the right libs.
Basically .../rpm/devel-filesystem-arm is a little island of ARM
libraries and include files on the build host that other ARM builds can
use, and they have all come out of the actual rpms that will later be
installed on the real target.
I already build for arm and avr32 on the same build host -- from the
same spec file -- using this system and it works fine, with no emulation
needed anywhere.
The existing Fedora spec files %build stuff won't work like that as it
is, they only support the build host arch being the target arch.
-Andy
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org
RE: reference implementation
We're devolving a bit into implementation, but if you use the sysroot
facility of binutils and gcc cross compilers many of the issues go away, and
you can use much of Fedora packaging directly.
The biggest issue is to make sure that CC, LD, AS, etc are all set correctly
prior to calling configure and attempting to cross compile.
Give me a bit of time, and I'll show some examples.. I have to get up to speed
w/ rpm 4_5 development first.
-Original Message-
From: [EMAIL PROTECTED] on behalf of Andy Green
Sent: Mon 6/4/2007 4:28 PM
To: rpm-devel@rpm5.org
Subject: Re: reference implementation
Andy Green wrote:
Jeff Johnson wrote:
Are EABI and/or NPTL needed for *building*? Sure they are needed
for installing on the target.
NPTL is probably a no. But EABI is a yes. You can't mix OABI and
EABI binaries. All of the embedded ARM development has moved to EABI
binaries. (As an aside, the purpose of EABI is to allow for a fixed
calling convention so that regardless of the optimizations embedded in
the binaries, they will still work with each other.. no problems w/
hard vs soft float vs vfp float.. etc. OABI had to change
everytime someone changed the floating point or register sets
available to an app!)
Apologies for my lack of knowledge on ARM.
So why not sparate OABI and EABI qemu sysimages?
(the lights come on, EABI doesn't work with qemu)
OK, so EABI has to be cross-compiled unitil qemu can emulate.
OABI - sysimage still seems viable.
My lights must still be off, I don't see why you need qemu to build the
cross packages at all. It's like I might want to make a PowerPC binary
rpm for some package, but I don't have a PowerPC. That's okay if I am
using this proposed Cross-Fedoraish thing, I can tell my i386 box to
cross build my Ultrafoo package as a PowerPC binary RPM. Or if all I
have is a PowerPC, I can make i386 binary rpms of my package.
Where does EABI come into it? That you need a crosscompiled EABI uclibc
lib on the target to link to? Well that's easily done without needing
any emulation on the build host.
Just to explain this a bit further, what I did with Octotux was to add a
dir devel-filesystem-%{_target_cpu} under %_topdir, so for example there
was %_topdir/devel-filesystem-arm.
After I built a crosscompiled package that I wanted to let other
packages be built against, say I built libdb4 for ARM and now I wanted
to build postfix for arm that wants libdb4, I used a script to install
my arm libdb4 and libdb4-devel rpms into the *build* host using
--root=%_topdir/devel-filesystem-arm. Then when I cross-built postfix
for ARM, I took care to tell postfix's configure to have CFLAGS with
-I%_topdir/devel-filesystem-%{_target_cpu}, so it picks up the right libs.
Basically .../rpm/devel-filesystem-arm is a little island of ARM
libraries and include files on the build host that other ARM builds can
use, and they have all come out of the actual rpms that will later be
installed on the real target.
I already build for arm and avr32 on the same build host -- from the
same spec file -- using this system and it works fine, with no emulation
needed anywhere.
The existing Fedora spec files %build stuff won't work like that as it
is, they only support the build host arch being the target arch.
-Andy
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org
