[Cooker] Re: Requires & Provides on -devel packages (Continued)
>> On Topic: >> Is there a way to exclude files from a %files list? Quite some of >> these duplicate because for instance the files for the icewm-light >> theme is also included in icewm. Would be nice if these files can >> excluded from a job. > > How about %exclude ? That one is new to me. > And wouldn't the packages conflict if they have the same files ? I think > I haven't seen this behaviour. With the "icewm" example the files are identical. so when you install icewm over icewm-light then rpm doesn't complain. I only noticed that the same files were in both packages after running the script. Stefan
Re: [Cooker] Re: Requires & Provides on -devel packages (Continued)
>> Need to setup a SF project for the scripts... > May i suggest joining youri project instead (see http://youri.zarb.org), > and make automatic and safe package building one of the objectives of > the project ? Good one. I will. I subscribed to the mailing list. Continue the slbd related discussion there? Stefan
Re: [Cooker] Re: Requires & Provides on -devel packages (Continued)
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Stefan van der Eijk wrote:
> On Topic:
> Is there a way to exclude files from a %files list? Quite some of these
> duplicate because for instance the files for the icewm-light theme is also
> included in icewm. Would be nice if these files can excluded from a job.
>
> %files -n package1
> %{_bindir}/* !%{_binddir}/dont_want_this_here
>
> %files -n package1
> %{_binddir}/dont_want_this_here
>
> to avoid duplication = save ISO space.
>
%exclude?
%files -n package1
%{_bindir}/*
%exclude %{_binddir}/dont_want_this_here
Regards,
Buchan
- --
|--Another happy Mandrake Club member--|
Buchan MilneMechanical Engineer, Network Manager
Cellphone * Work+27 82 472 2231 * +27 21 8828820x121
Stellenbosch Automotive Engineering http://www.cae.co.za
GPG Key http://ranger.dnsalias.com/bgmilne.asc
1024D/60D204A7 2919 E232 5610 A038 87B1 72D6 AC92 BA50 60D2 04A7
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.1 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQE+gGuCrJK6UGDSBKcRArDVAJwMs5noPhe/ythVd2SjBBD75FN8bwCgx1Dx
xTBFBpB6/QeX5quvcMjbcww=
=Gszr
-END PGP SIGNATURE-
Re: [Cooker] Re: Requires & Provides on -devel packages (Continued)
Ainsi parlait Stefan van der Eijk : > >> I'm using a dedicated machine (in a VMware) to do this at the moment. > >> A normal user performs the rebuilding, so damage to the system should > >> be limited. > >> > >> When Jeff Garzik was still employed at Mandrakesoft we discussed using > >> a chroot() environment to build packages. The idea was to install a > >> stripped system (basesystem + rpm-build + BuildRequires) for every > >> package to be rebuilt in a chroot environment. In the end we didn't go > >> through with the idea. > > > > Creating a slim clean build environment for each package seems to be > > overkill (to me). I simply don't have the resources. I have one fat > > clean build environment for each distribution. > > Let me rephraise that. The script will create a build environment, build > the package, remove the build environment, and restarts. Keeping the build > environment intact after rebuilding for each package is insane. > > > Building as a user helps you not to pollute your build environment, but > > for each violation the build is halted. Compare this to rpmbuild telling > > you only the first file from %files it cannot find ;) > > Yes. My scripts (mentioned in a earlier post --> can be found at: > http://eijk.homelinux.org/~stefan/slbd.html) do this. After each build a > cleanup is done (temp & build directories are emptied, BuildRequires are > de-installed). > > Need to setup a SF project for the scripts... May i suggest joining youri project instead (see http://youri.zarb.org), and make automatic and safe package building one of the objectives of the project ? I've been in touch with Matthias, the guy maintaining freshrpm.net, at FOSDEM, he would be very interested by such topic. He already started such project two years ago on sf.net, without interesting anyone else as him. -- All components become obsolete. -- Murphy's Computer Laws n°8
[Cooker] Re: Requires & Provides on -devel packages (Continued)
>> Before something like this is implemented, the current FUNDEMENTAL
>> breakage in the dependancies of the -devel packages needs to be
>> resolved. My experience with finding the right BuildRequires is that
>> due to the fact that the dependancies for -devel packages are not
>> automatically found it's difficult (if not impossible) to maintain.
>
> I see large potential in Dag's method. He is currently using it to
> see/avoid writes outside the buildroot, but it could be used to also
> register all the reads and to deduce the BuildRequires packages from
> that. Possibly one would get too many dependencies, as configure
> sometimes `overchecks' the environment, but it would give the packager a
> maximum set to choose from.
One thing you shouldn't forget is that many files and DIRECTORIES (the
directories are the worst --> rpmlint doesn't bork about them) exist in
multiple files. These packages will show up in the list, eventhough the
file wasn't installed.
This has everything todo with developper discipline.
On a urpmi enabled system this script will lookup all duplicate files &
directories:
#!/usr/bin/perl
my %count;
my %packages;
open F, "parsehdlist --files --quiet /var/lib/urpmi/hdlist.* |";
while () {
/([^:]*):(\S*)/ or next;
$count{$2} = $count{$2} + 1;
$package{$2} = "${1}" . " " . "$package{$2}";
}
foreach $a (sort keys %count) {
if ( $count{$a} >= 2 ) {
print "$a $package{$a}\n";
}
}
Off Topic: I don't know perl, but I'd like the script to accept a media
switch (like urpmi) so you can choose which urpmi media you want to check
against. Right now it just takes everything it can find
--> /var/lib/urpmi/hdlist.*
for Mandrake cooker, Mandrake contrib and PLF this adds up to 8670
duplicate files currently.
On Topic:
Is there a way to exclude files from a %files list? Quite some of these
duplicate because for instance the files for the icewm-light theme is also
included in icewm. Would be nice if these files can excluded from a job.
%files -n package1
%{_bindir}/* !%{_binddir}/dont_want_this_here
%files -n package1
%{_binddir}/dont_want_this_here
to avoid duplication = save ISO space.
with kind regards,
Stefan van der Eijk
[Cooker] Re: Requires & Provides on -devel packages (Continued)
>> I'm using a dedicated machine (in a VMware) to do this at the moment. >> A normal user performs the rebuilding, so damage to the system should >> be limited. >> When Jeff Garzik was still employed at Mandrakesoft we discussed using >> a chroot() environment to build packages. The idea was to install a >> stripped system (basesystem + rpm-build + BuildRequires) for every >> package to be rebuilt in a chroot environment. In the end we didn't go >> through with the idea. > Creating a slim clean build environment for each package seems to be > overkill (to me). I simply don't have the resources. I have one fat > clean build environment for each distribution. Let me rephraise that. The script will create a build environment, build the package, remove the build environment, and restarts. Keeping the build environment intact after rebuilding for each package is insane. > Building as a user helps you not to pollute your build environment, but > for each violation the build is halted. Compare this to rpmbuild telling > you only the first file from %files it cannot find ;) Yes. My scripts (mentioned in a earlier post --> can be found at: http://eijk.homelinux.org/~stefan/slbd.html) do this. After each build a cleanup is done (temp & build directories are emptied, BuildRequires are de-installed). Need to setup a SF project for the scripts... >> > I'd like to help get this thing inside rpmbuild (and in debugging it >> more). As far as I can see, what rpmbuild needs is easier for >> collecting build requirements, than what I'm doing in Soapbox. But >> reality bites... >> It would be cool if rpmbuild would be able to build in a clean >> environment, but do you think this should be mandatory for rpmbuild, >> or should such a mode be turned on by an option? > Hmmm, Soapbox is not about creating a clean environment for building. > IMO rpmbuild should never have to write outside its builddir and > buildroot. You could use Soapbox to do this. OK, but isn't this what happens when you let the rpm's be rebuilt as user (not root)? >> Before something like this is implemented, the current FUNDEMENTAL >> breakage in the dependancies of the -devel packages needs to be >> resolved. My experience with finding the right BuildRequires is that >> due to the fact that the dependancies for -devel packages are not >> automatically found it's difficult (if not impossible) to maintain. > Well, you could use the same techniques (overloading open()) to make a > list of all the requirements a build-process has. And list them for > inclusion or add them to the source-package automatically. The strace mode of slbd.sh does this... OK. With this you will have a list of all the files that were needed for rebuilding the package. Translating that to packages is a peice of cake. But that list of packages is quite exhausting. For the BuildRequires you only need to "top" packages in the rpm hierarchy. When libpng3-devel is required, there is no need to list libpng3 and zlib1 --> these are redundant (Build)Requires. > And that was the original intent of my mail ;) Please try out slbd. It might suite your needs quite well. Perhaps only the chroot() part needs to be added. with kind regards, Stefan > «Any errors in spelling, tact or fact are transmission errors» cool .sig!
Re: [Cooker] Re: Requires & Provides on -devel packages (Continued)
>> True. What I've noticed (at least at MDK) that people aren't >> interested in a clean build environment. They just want their >> packages to build and not to worry about (useless?) stuff like >> BuildRequires. > no. we're many that're happy with better buildrequires/requires. > but this is less important than having fixed bugs or development. > always the same ressources problem. >> due to the fact that the dependancies for -devel packages are not >> automatically found it's difficult (if not impossible) to maintain. > yes >> recently the libart_lgpl2-devel has been created. I beleive that the >> files in this package used to be part of the kdelibs-devel or >> kdebase-devel packages. Now the package is split off. Now quite some >> packages that BuildRequire kdebase-devel or kdelibs-devel are not >> compiling due to the fact that the dependancies have been broken. > this is the problem of kde being not splited at first Well... It's true that this issue showed up when these packages were split up --> it broke the dependancies. IMHO what is needed is a dependancy system that defeats these things from occuring. The BuildRequires of a package should still point to for instance libpng-devel. But when libpng3-devel is built, it should be able to find the dependancies the files it carries require (and provide). just grep for "include" in one of the header files in the package. And the dependancies become clear. $ fgrep include /usr/include/png.h * We must not include leading zeros. /* include the compression library's header */ #include "zlib.h" /* include all user configurable info, including optional assembler routines */ #include "pngconf.h" /* Finish a chunk started with png_write_chunk_start() (includes CRC). */ #include This shows that libpng-devel requires zlib.h and pngconf.h. Where zlib.h is outside of the package --> a Requires. png.h can be put down as a Provides. If something like this is put in place (this is just an example) then the dependancies between -devel packages won't need to be maintained manually. And problems when splitting packages won't have an impact on other parts of the distro. perhaps this perl module http://search.cpan.org/author/AMICHAUER/C-Include-1.40/Include.pm can help to implement this. What do you think? Stefan
Re: [Cooker] Re: Requires & Provides on -devel packages (Continued)
"Stefan van der Eijk" <[EMAIL PROTECTED]> writes: > True. What I've noticed (at least at MDK) that people aren't > interested in a clean build environment. They just want their > packages to build and not to worry about (useless?) stuff like > BuildRequires. no. we're many that're happy with better buildrequires/requires. but this is less important than having fixed bugs or development. always the same ressources problem. > due to the fact that the dependancies for -devel packages are not > automatically found it's difficult (if not impossible) to maintain. yes > recently the libart_lgpl2-devel has been created. I beleive that the > files in this package used to be part of the kdelibs-devel or > kdebase-devel packages. Now the package is split off. Now quite some > packages that BuildRequire kdebase-devel or kdelibs-devel are not > compiling due to the fact that the dependancies have been broken. this is the problem of kde being not splited at first
[Cooker] Re: Requires & Provides on -devel packages (Continued)
Dag, Sorry for the late reply. > Axel Thimm forwarded me here about something I'm implementing in my > buildsystem. I was fed up with Makefiles writing outside the buildroot > and I wrote a library yesterday that overloads the glibc functions > (systemcalls) and checks some conditions before executing. It's called > Soapbox. http://dag.wieers.com/home-made/soapbox/ I'm using a dedicated machine (in a VMware) to do this at the moment. A normal user performs the rebuilding, so damage to the system should be limited. When Jeff Garzik was still employed at Mandrakesoft we discussed using a chroot() environment to build packages. The idea was to install a stripped system (basesystem + rpm-build + BuildRequires) for every package to be rebuilt in a chroot environment. In the end we didn't go through with the idea. > If a process is doing something outside the buildroot, it can fake > succesful completion without messing up my (clean) build environment. True. > Axel told me that this (overloading glibc functions) would be something > that should be included in rpmbuild for generating build requirements, > which I agree with. If it can be done safeproof ofc. If we could use it > also to prevent builds to pollute the build environment, I would be even > more happy. True. What I've noticed (at least at MDK) that people aren't interested in a clean build environment. They just want their packages to build and not to worry about (useless?) stuff like BuildRequires. > I'm not a real C programmer, ie. it's been a couple of years and I never > did anything useful with my C education ;) I think this might be the > first thing worth mentioning and I'm really interested in people's > opinion of Soapbox and the implementation. Would such an application really need to be a C application? The build scripts I've made are bash script, with some perl bits and they work fine. > I'd like to help get this thing inside rpmbuild (and in debugging it > more). As far as I can see, what rpmbuild needs is easier for collecting > build requirements, than what I'm doing in Soapbox. But reality bites... It would be cool if rpmbuild would be able to build in a clean environment, but do you think this should be mandatory for rpmbuild, or should such a mode be turned on by an option? Before something like this is implemented, the current FUNDEMENTAL breakage in the dependancies of the -devel packages needs to be resolved. My experience with finding the right BuildRequires is that due to the fact that the dependancies for -devel packages are not automatically found it's difficult (if not impossible) to maintain. Example: recently the libart_lgpl2-devel has been created. I beleive that the files in this package used to be part of the kdelibs-devel or kdebase-devel packages. Now the package is split off. Now quite some packages that BuildRequire kdebase-devel or kdelibs-devel are not compiling due to the fact that the dependancies have been broken. See: http://eijk.homelinux.org/build/cooker/i586/problem/kdeartwork-3.1-3mdk.src.rpm.txt with kind regards, Stefan van der Eijk
