On Thu, 15 Apr 2010 13:41:31 +0300, Lauri T. Aarnio wrote:
Hi,
>> I've been wondering for some time if it would be possible create
>> pbuilder-like infrastructure on top of scratchbox2 (for building
>> .debs).
>> Namely, what I would like to have is tools (consisting of say squeeze +
>> cross toolchain) and target (say armel). I can setup these with
>> debootstrap, no problem here.
>
> You didn't say if you intention is to compile squeeze for armel. I hope
> it is, if your tools are based on that.
Well, partially yes. My company tries to build its own customized
distribution on top of Debian. Changes relative to Debian are twofold -
replacing few packages with its lighter versions (coreutils with busybox,
etc.) and stripping out unneeded bits (docs, device support, ...).
Ah, we would also like to supply our own software. ;)
We would like to have only our modified packages in target's rootstrap
and everything else in tools. Well, that was our original plan...
> Our experiments with various configurations have *proved* that the
> safest way is to use the same base system for tools as what you are
> compiling. I.e. 100% same sources, just different binary architectures.
> Well, the "devel" mode of sb2 was originally based on an idea that we
> could use tools from debian (first from etch, later from lenny) to
> compile something else (Maemo's sources). But it isn't really a good
> idea;
Yeesss... We're trying to do exactly that. Unfortunately, to my
knowledge there are few (if any) other options in such scenarios.
At least if one wants speed, that is. qemu{-system,}-arm seems good
enough now (qemu works rather good) but it's slow. Too slow for us.
However, it seems that meegoo has gone for qemu approach (if I understand
correctly how OBS works internally). I guess that is due to scratchbox
and cross compilation related issues.
> usually works, but it isn't a generic solution. Note that
> scratchbox1 suffers from the same problems; devkits are usually not too
> well in sync with the distro that you are compiling, and it causes pain
> there, too.
This is why we would like to use pristine Debian for tools
(+emdebianized, lightly customized toolchain) and our, probably not-so-
fast-to-update target's rootstrap.
> One thing that causes much trouble is that there appears to be lots of
> silent dependencies between the source packages and the tools. Many OSS
> tools are not at all forward- or backward compatible with itself => my
> recommendation is that please try to use a consistent starting point,
> and don't mix tools from another distribution with something else.
> Unless you really have to (for bootstrapping an own distro, or something
> like that?)
We're evaluating if it's possible to have something like:
- most (if not all) tools from Debian,
- rootstrap with libs and includes,
We aren't trying to compile full Debian archive, hopefully. However, we
would like to make it easy to import original packages and compile these
without altering scratchbox configuration.
> Latest versions of sb2 have also another development mode, "accel",
> which can be used when your tools and target distributions are the same.
> Use it if possible, it is simply better than what "devel" can ever do.
Yes, I know. accel mode requires installing much, much more packages to
target. We would like to minimize this amount as much as possible. This
is because we have to maintain target's packages to some extent and we
have (surprise!) rather limited resources ...
>> However, problem starts with introduction of non-trivial Build-Depends
>> (other than 'build-essential'). Having basic native/target
>> environments
>> I would like to automatically resolve and install dependencies. devel's
>> mode dpkg-checkbuilddeps wrapper is quite capable of handling pre-
>> setup environment, however I have hard time finding how to
>> automatically
>> decide which packages would be needed to be installed to tools, target
>> or
>> tools+target.
>
> The solution is simple: You should install all missing dependencies to
> the target ("rootstrap"). If that brings in some additional tools as
> armel binaries, that only means that they will be slow. But still safe
> to use.
>
> There is no way to automatically decide what can be installed to tools.
> And the reason for this is quite simple, too:
>
> Some tools are not safe to be executed with a "wrong" cpu architecture.
> Most are safe, but not all => the contents of "tools" must be selected
> manually.
What we were trying to do (well, still trying, or not ... ;) was to avoid
duplication of tools-provided programs in target environment. Doing so
would mean less maintenance work for us and probably more weird
compilation problems...
Doing as you suggest would mean that we would need to have Debian proper
in tools, and our bastardized packages and Debian ones in target.
I guess this is how our system will end up eventually (after some pain
trying to do things in different way ;)
> Also, you probably don't need to modify the dpkg-checkbuilddeps wrapper.
> Some people have just parsed output of it, and used a separate tool to
> get the dependencies.
Yes, this does work but installing what dpkg-checkbuilddeps suggests via
apt-get brings lot of packages that are already in tools (gcc, perl,
python, ...) Disk is cheap, I do wonder however what problems will arise
trying to combine our modified packages with original Debian ones.
> I might have something that could be useful, as examples about how sb2
> can be used: The scripts that I have used for building massive amounts
> of packages. Unfortunately those are currently in that kind of state
> that they should stay under the carpet ;-) ...but if you are interested,
> I can try find some time to clean up those.
Thanks, at this time I will continue my pai^H investigation without
these. ;)
> Lauri
Thanks a lot for you very helpful reply Lauri.
Best regards,
Karol Lewandowski
_______________________________________________
Scratchbox-devel mailing list
[email protected]
http://lists.scratchbox.org/cgi-bin/mailman/listinfo/scratchbox-devel
