On 8/16/19 4:50 PM, richard.pur...@linuxfoundation.org wrote:
On Fri, 2019-08-16 at 14:29 -0700, Khem Raj wrote:
On Fri, Aug 16, 2019 at 2:25 PM Richard Purdie
<richard.pur...@linuxfoundation.org> wrote:
This was discussed on IRC. We really really want to avoid that as a
dependency if we care about build performance. Its really easy to
just
add that in but the implications are significant :(
A captured version of this utility would also mean that we have to
keep this in sync for bug fixes and security fixes etc. which could
also be considered as expense, so how much do we lose if util-linux-
native is brought early into dependency chain?
The tool is being used for a specific purpose at build time. Its not
exposed on target and its use is extremely constrained to a specific
set of files where its behaviour should be deterministic.
As such, yes there are questions about bug fixes and about security but
I don't think its going to be a serious burden. We'd need to carry this
until such times as the host systems we support have hardlink which is
probably a few years out but will arrive.
The impact of having util-linux-native in the dependency graph at this
point is significant. I say that as someone who has done a lot of work
in trimming down the times of our bootstrap process. This is exactly
the kind of dependency I've removed and avoided.
glibc is special in that its the worst choke point on the dependency
graph. Nothing to do with target binaries happens until it is ready and
then gcc-runtime can build. Whilst the locales are split from glibc,
they are still connected in the task graph for packaging purposes so
this is important.
Its been pointed out there are other places where util-linux-native has
crept in, in particular python3-native. My preference would be to
minimise these dependencies rather than use them as a way to justify
adding it here too.
So what is the consensus here? Can we require the host system to have
hardlink? It is available as a stand alone binary on Ubuntu 14.04 and up and
on Fedora systems, but isn't installed by default, and we'd have to check the
version because the arguments sadly are slightly different. We could
certainly make util-linux a dependency, but that is why this patch existed, in
order to avoid the dependency.
I did find a python version at
https://raw.githubusercontent.com/akaihola/hardlinkpy/master/hardlink.py. I
made a one line modification to it and it works, but it isn't nearly as
efficient. The proof is below, using the same locale-tree each time. It is
important to look at both the real and user time. The difference gets more
significant when you run it under pseudo, where we expect it to run 100% of the
time. It is roughly 10x slower to do this in python.
time recipe-sysroot-native/usr/bin/cross-localedef-hardlink -c -v locale-tree
Directories: 982
Objects: 7441
Regular files: 6441
Comparisons: 3331
Linked: 3109
Saved: 754675712
real 0m0.641s
user 0m0.120s
sys 0m0.518s
time python3 hardlink.py -v 0 -c locale-tree
Hard linking Statistics:
Directories : 982
Regular files : 6441
Comparisons : 31884
Hardlinked this run : 3109
Total hardlinks : 3109
Bytes saved this run : 744089843 (709.619 mebibytes)
Total bytes saved : 744089843 (709.619 mebibytes)
Total run time : 2.124422073364258 seconds
real 0m2.180s
user 0m1.220s
sys 0m0.953s
============Now lets run it under pseudo ================
time recipe-sysroot-native/usr/bin/cross-localedef-hardlink -c -v locale-tree
Directories: 982
Objects: 7441
Regular files: 6441
Comparisons: 3331
Linked: 3109
Saved: 754675712
real 0m2.505s
user 0m0.373s
sys 0m1.286s
time python3 hardlink.py -v 0 -c locale-tree
Hard linking Statistics:
Directories : 982
Regular files : 6441
Comparisons : 31884
Hardlinked this run : 3109
Total hardlinks : 3109
Bytes saved this run : 744089843 (709.619 mebibytes)
Total bytes saved : 744089843 (709.619 mebibytes)
Total run time : 12.971824884414673 seconds
real 0m13.046s
user 0m3.633s
sys 0m5.192s
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core