On Oct 21, 2011, at 9:33 PM, ext Carsten Munk wrote:
> 2011/10/21 Lauri T. Aarnio <lauri.t.aar...@nokia.com>:
>>
>> On Oct 21, 2011, at 4:05 PM, ext Carsten Munk wrote:
>>
>> That is where your assumptions go wrong.
>>
>> It isn't done like that. Or should't be, x86 perl must not used for running
>> scripts from target packages - either from the source package or from
>> target's binary packages. It simply isn't safe, and there are packages where
>> it fails. Badly.
>
> Ah! That part I didn't know indeed. So, let's say I have a random
> foo.pl in my target, i run perl, you don't attempt to accelerate it at
> all? (definition of accelerate: use a x86 binary in place of perl).
> That is, let's say we provide autom4te in SB2 tools set, it'll be run
> with the 'x86' perl.
Yes. And then whatever perl is running, it will see compatible files in e.g.
/usr/lib/perl5; so mapping of such directories depends on the active policy
(which is, naturally, process-specific so that you can have both perls running
at the same time...)
> OK, that is a fairly clever approach. I wonder if
> we could do something similar with the approach we currently use for
> cross compilation.
Probably not, without risking something else.
Perl is an extremely tricky tool, which makes already itself use of far too
many "clever" features that are available.
One example is that built-in variable $^X is implemented by reading where
/proc/self/exe symlink points to.
And many scripts actually record that to something what they produce [this was
the original reason why SB2 got a special handler for the /proc FS!]
This means that you should not install perl anywhere else than to
/usr/bin/perl. So something like a simple redirector at /usr/bin, which would
determine what version to use and then exec a perl from somewhere else, would
just create more problems that what it would solve.
The same applies to the locations of the libraries, etc. => In theory it is
possible to change the default locations, but in practice it it just a stupid
thing to try.
>> SB2's tools collection is based on two ideas
>>
>> 1) the contents must be built from the same sources as what goes to the
>> target FS
>> 2) it doesn't have to be updated so often, because it doesn't contain as
>> much stuff as what goes to the target side. And updating it could be
>> automated.
> I wonder if we somehow can make that automated in OBS through the fact
> we utilize same sources. That could be interesting.
Should be easy to automate. Downloading and installing the tools should be
similar operation as setting up the build root.
Since the tools collection is never modified during builds, it is possible to
reuse it without reinstalling, as long as the selected tools are not modified
in the repository.
>> The big question is that what should go to the tools collection (what can be
>> put there, what can't). Some tools are easy to handle, some require more
>> work (like perl & python). But this is not something that would create an
>> endless maintenance loop.
> I think it could be useful, for all of us, to do some analysis on
> where exactly the pain points of compilation is these days.
SB2 makes it quite easy to see where time is spent. It can combine information
collected by acct(2) with SB2's logs, and draw call graphs between processes.
I have used it to see how a fully emulated (everything running with qemu-arm)
build can be accelerated. For those RPM builds, the biggest pain point was the
toolchain (>50% of CPU cycles).
After the toolchain was replaced with a native one (replaced it by SB2's
configuration => absolutely no changes to the package or to the buildroot), the
pressure shifted mostly to bash, make and m4, and to some other smaller but
usual utilities.
Re-mapping those to native tools means that it was easy to get 2.5x - 3x the
original speed. And I didn't even try to do anything for perl in this case.
> I know for
> a fact I'd like to speed up autoconf and friends, that's
> second-biggest painpoint I see.
Yes, but it is only the automonsters here... I'd like to say that the real
building-with-an-emulated-perl -pain lies in the debian build system;
Having package management in native code (rpm) or as perl script (dpkg-*) makes
a huge difference.
>> I have been prototyping just that... it isn't integrated to OBS yet, but
>> rpmbuilding with SB2 is possible. And easy.
>>
> Well, if you want a nice 312 public source package distribution to try
> it out on and get feedback on issues, I'd be interested in helping you
> out to test it on Mer. As I said in one of the earliest posts, it's
> always useful to evaluate technologies and well, there's no holy cows
> - we're open to ideas and innovation.
Thanks. I'll try to find time for cleaning up and releasing the things that I
have in the queue for the next SB2 development version. That includes what I
currently have for RPM builds. Nothing special, just minor fixes.
Lauri
_______________________________________________
MeeGo-dev mailing list
MeeGo-dev@meego.com
http://lists.meego.com/listinfo/meego-dev
http://wiki.meego.com/Mailing_list_guidelines
