One should note that apt (and also dpkg, though with less power) already has a dependency engine which covers this case, as it deals with all the kinds of dependencies fink needs (minus build deps of course). Hence it might be a good idea to try to adapt its engine for fink (why reinvent the wheel?). Of course we'd have to interface perl and c(++) for this. We also would have to figure out how to hook into the apt-get code. We'd need to marry it with our code that deals with build deps, too (but that's covered in the next section). We also need to override apt's way of choosing which package to use to fulfill a multiple choice dependency (apt just use the first possible combination, it seems, while Fink tries to ask the user for his wishes). This all adds up to some complication. So while I still think we should at least consider it, it might not be worth the effort after all. In any case we can try to look at their sources and learn from them (how they solved various problems etc.).Yuck. We have already determined problems with this method, and it seems crude, especially when better alternatives are available.
[... here I never finished what I wanted to write.]
Dealing with build time =======================
Several different approaches are possible... all have their issues and advantages
The dumb way
------------
This is mostly how we do it now. But if we combine this with pkg-order, it might still be better than the current code...
Roughly, what you do is this: when hooking into pkg-order, when it asks us for the deps of package "foo", we check if a .deb is available for it. If yes, we return just the deps; if no we return the deps merged with the build deps (actually this is a bit more complicated for splitoffs, but we already have the code for all this anyway).
Pro: this is straight forward to implement (we already do it this way afterall).
Con: Is not very flexible; e.g. if package "foo" needs "bar" as a build dependency, and another pkg "qux" later conflicts with "bar", we can't proceed, rather the user has to manually issues several commands (first building "foo", then removing "bar", then installing "quy").
Or possibly take code from pkg-order, add additional features, etc, and put it all into a perlmod. This would be the hardest, but I think that with a little thought we can come up with some kind of colored graph algorithm that makes the process a little easier.The proper way --------------
Proper would be to (as I mentioned in the past) have a full fledged package orderer that is aware of the fact that build dependencies can be removed after they have been used. My hope is that it's possible to add support for this to pkg-order, maybe one could even get support from the pkg-order author(s) for this project. However, I consider it a hard piece of work to be done properly.
Also, when in doubt about multiple possible ways to fulfill a dependency, it should be possible to either let fink make a good (automatic) guess at it, *or* let fink ask the user what he prefers. Not sure how hard it would be to implement that atop pkg-order.Rather than using an external perl program, why don't we just adjust the code into a module and use the functions straight from fink.
Pro: This is very powerful, and flexible, and gives the best user experienceYep, unfortunately.
Con: Quite hard to implement in my estimation
[Note: I wanted to go more into detail here, but never finished it]
Whee!!! Fun!!! I would recommend against hard links to directories, though. If we do this, we should make it optional, and add something like the following:Chroot jail -----------
The idea is to build packages as we do it now, with the only exception that we first chroot into a custom root directory. For a chroot jail to work (i.e. in order to be able to run program in it), all necessary libraries (including system libraries) have to be existent inside the jail. This usually means copying these libraries (hard links might work, but only if the build dir is on the same drive as the system, which is not necessarily true for fink). And of course if you want to compile stuff, the header files and tools (compiler, linker, ...) are needed, too).
A new info option 'Chroot'
Packages with this tag defined are
limited in whether they can be put
into a Chroot or not. If it is not defined,
then it doesn't matter.
Package 'chroot-image-free':
Should be available from apt for Fink.
Chroot: Yes
Essential: Yes
Package 'chroot-image-apple'
Must be built from a Mac OS X install for
anything not-redistributable from Apple.
Chroot: Yes
Essential: Yes
bootstrap:
Installs packages with 'Essential: Yes'
and without 'Chroot: Yes'
'fink --chroot install foo' would do the following:
Make $builddir/chroot-foo-1.2.3-4
Install any packages with 'Essential: Yes'
that do not have 'Chroot: No' into
the created chroot.
Chroot into the chroot-foo-1.2.3-4 dir.
Run the 'fink' installed in the chroot.
Install any dependencies and sub-
dependencies into the chroot.
Build the package in the chroot.
Exit the fink in the chroot.
Copy the .deb file to the correct deb dir.
Delete the chroot.
Install the package in the root directory.
Then fink verify would do the build in the chroot to
ensure that files are installed correctly. Also, it
should then remove each package and print a
list of any extra files present in the chroot afterwards.
This immediately shows a big limitation: such a jail has to be built, which takes time, and disk space (potentially a lot). The advantages, though, are:Yay!!!
* the /sw we put into the jail can be custom made: that is, it would only contain the minimal dependencies of the package being built, thus avoiding hidden dependencies much better
* build dependencies of the pkg are never installed in the real /sw, thus avoiding any problems they can cause, dependency wise (like conflicting with the build deps of another package)Yay!!!!!!!
* detecting files that get installed directly into /sw during build is easier (because they will be missing from the real /sw)
* multiple build runs are easily possible (although they would take a lot of disk space), even if the builds have conflicting dependencies
* no more problem caused by users running "fink remove FOO" while something is building that depends on foo
Essentially, since with this approach build dependency/conflicts only have to be resolved in the context of the chroot jail, we can indeed completely ignore them for our "normal" dependency engine.
Overall, the chroot approach solves a lot of annoying issues very neatly. However, the additional disk space for the chroot jail(s) (and the CPU time to set them up) are quite severe limitation. Of course modern machines have plenty of disk space and CPU power, but e.g. my iBook G3/500 only has 1 GB free on its HD. For my daily fink usage that's fine, but if I suddenly need several 100 MBs to setup chroot jails, that would be annoying. We might be able to severely cut this down if we can use hardlinks when building on the same volume as the system is on. This definitely requires research should we decide to consider using it :-)
Hardlinks would break the 'jail' nature of chroot, and installing files into a hardlinked /sw would effectively install them into the real /sw
Another tricky thing about the chroot jail approach is to make it work efficiently when used "recursively". To understand what I mean, consider this: The .deb for pkg "foo" (which we want to install) is missing. We already ensured all its dependencies are fulfilled. Now "foo" BuildDepends on "bar", for which the .deb is also missing! Hence we first have to recurse to get "bar" built (with its own chroot jail)... and of course "bar" might depend on more things which need to be built first.So maybe only the maintainer and bindist builder need to operate with chroots, everybody else might just need to build normally.
Pro: Relatively easy to implement; could be implemented atop pkg-order; also allows to detect hidden dependenciesSo add this as an option to everything else, for the maintainers, on top of a new dependency engine.
Con: Takes up additional disk space & CPU time; might have unforseen problems
Cheers, Kyle Moffett
-----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCM/CS/IT/U d- s++:- a16 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$ L+++(++) E W++(+) N+++(++) o? K? w---(-) O? M++ V? PS+() PE+(-) Y+ PGP? t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h! !r-- !y? ------END GEEK CODE BLOCK------
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Fink-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/fink-devel
