Re: [gentoo-dev] Developers, please work on underlinking issues!

[Michał Górny]

On Thu, 18 Aug 2016 15:21:16 +0200
Alexis Ballier  wrote:

> On Thu, 18 Aug 2016 08:13:14 -0400
> Rich Freeman  wrote:
> 
> > If you just check your packages occassionally to make sure they build
> > with gold it completely achieves the goal, and it will actually result
> > in fewer bugs using the non-gold linker as well.  
> 
> That's what a tinderbox is for. The only QA problem I see here is that
> QA doesn't automate that kind of checks anymore since Diego left. Maybe
> QA should ask Toralf to run a ld.gold tinderbox and avoid asking people
> to randomly test random packages ?

Yes, tinderboxing makes a lot of sense if the bugs are afterwards
ignored by package maintainers. Or in the best case, the maintainer
tells reporter (Toralf) to file the bug upstream.

-- 
Best regards,
Michał Górny



pgpk3uzO9JfPO.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] New project: LLVM

[C Bergström]

I think you're getting a bit confused

libsupc++ is the default now, from GNU

libcxxabi is the bloated runtime from Apple

libcxxrt is the faster c++ runtime, PathScale+David Chisnall, which
PathScale and FreeBSD use by default. We don't need a version number
because it's pretty much rock solid stable for a while.
I'd encourage you to consider libcxxrt for at least the code size and
performance reasons. Build it and you'll see. Locally my unoptimized
libcxxrt.so is like 88K. How much is your libcxxabi (static and
shared)

88K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.so
140K/opt/enzo-2016-06-26/lib/6.0.983/x8664/64/libcxxrt.a
// This seems larger than I remember and I need to check why.

https://github.com/pathscale/libcxxrt

On Fri, Aug 19, 2016 at 10:40 AM, Lei Zhang  wrote:
> 2016-08-19 10:07 GMT+08:00  :
>> That seems a lot like what we've already done. I guess a GSOC student is 
>> working on the libcxxabi piece.
>
> I am that GSoC student :)
>
> I'm currently trying to push libc++abi to replace libcxxrt as the
> default runtime: https://github.com/gentoo/gentoo/pull/2048
>
> The reason is I think libc++abi blends in more naturally with other
> LLVM components, and it has a clear version number as opposed to
> libcxxrt.
>
>> The only advantage to using our runtime, libcxxrt, is performance and code 
>> size.‎
>
> Honestly I don't know what essential difference these two libs have; I
> can't find any decent comparison of them on the internet. Do you have
> some real numbers to show the difference in performance and code size?
>
>
> Lei
>

Re: [gentoo-dev] New project: LLVM

[Lei Zhang]

2016-08-19 10:07 GMT+08:00  :
> That seems a lot like what we've already done. I guess a GSOC student is 
> working on the libcxxabi piece.

I am that GSoC student :)

I'm currently trying to push libc++abi to replace libcxxrt as the
default runtime: https://github.com/gentoo/gentoo/pull/2048

The reason is I think libc++abi blends in more naturally with other
LLVM components, and it has a clear version number as opposed to
libcxxrt.

> The only advantage to using our runtime, libcxxrt, is performance and code 
> size.‎

Honestly I don't know what essential difference these two libs have; I
can't find any decent comparison of them on the internet. Do you have
some real numbers to show the difference in performance and code size?


Lei

Re: [gentoo-dev] New project: LLVM

[cbergstrom]

That seems a lot like what we've already done. I guess a GSOC student is 
working on the libcxxabi piece. The only advantage to using our runtime, 
libcxxrt, is performance and code size.‎ 

  Original Message  
From: Lei Zhang
Sent: Friday, August 19, 2016 06:34
To: gentoo-dev@lists.gentoo.org
Reply To: gentoo-dev@lists.gentoo.org
Cc: gentoo-dev-annou...@lists.gentoo.org
Subject: Re: [gentoo-dev] New project: LLVM

2016-08-18 21:56 GMT+08:00 C Bergström :
> @mgorny may be able to help with some of this and has quite a bit of
> experience building clang/llvm. Where I work we use a "wrapper" that
> helps coordinate a lot of the moving pieces.
>
> https://github.com/pathscale/llvm-suite/
>
> This may not be the perfect "gentoo" way to handle it, but the
> approach would produce a clean and correct compiler. With llvm
> dependencies getting more and more complicated, I'm not sure if it
> would be possible to have both a gnu-free and also perfect
> 1-project-source-repo:1-ebuild ratio.

Currently the ebuilds for libunwind, libc++abi and libc++ all involve
some hackery to support standalone build. And the package clang is
just a placeholder, while it's actually built in llvm.

Maybe we can put them all into the llvm package, and control what
components to build via USE flags.


Lei

Re: [gentoo-dev] Developers, please work on underlinking issues!

[Mart Raudsepp]

Ühel kenal päeval, N, 18.08.2016 kell 23:56, kirjutas Alexis Ballier:
> On Thu, 18 Aug 2016 14:20:41 -0700
> Daniel Campbell  wrote:
> 
> > On 08/18/2016 06:21 AM, Alexis Ballier wrote:
> > > On Thu, 18 Aug 2016 08:13:14 -0400
> > > Rich Freeman  wrote:
> > >   
> > > > If you just check your packages occassionally to make sure they
> > > > build with gold it completely achieves the goal, and it will
> > > > actually result in fewer bugs using the non-gold linker as
> > > > well.  
> > > 
> > > 
> > > That's what a tinderbox is for. The only QA problem I see here is
> > > that QA doesn't automate that kind of checks anymore since Diego
> > > left. Maybe QA should ask Toralf to run a ld.gold tinderbox and
> > > avoid asking people to randomly test random packages ?
> > >   
> > I dunno, if testing packages that one maintains is as simple as
> > reconfiguring a package, testing, and switching back then I don't
> > think it's unreasonable to ask us to test our own packages. We're
> > supposed to do that already, and for packages whose dependencies
> > aren't 100% hashed out, it can help us figure out what the real
> > deps
> > are.
> 
> 
> test against... all linkers, all compilers, all libcs, all kernels,
> all
> userlannds, all useflags, ... ? :)
> 
> 
> by all means, please do it, but there are things machines are better
> at, like ensuring all packages have been tested against gold linker
> and
> every failure has been reported

The tl;dr did say to switch to ld.gold, but the main point was to
actually fix the bugs reported against your packages about it by other
developers, users and any future tinderbox runs, instead of ignoring
them as something that isn't supposed to matter and very low priority.
I think that should be sufficient, and we don't need to all rush to
switching to it, as long as we take care of the bugs about it when
others notice and file a bug.
Though it gives some good benefits when you are able to use it, afaik,
so hey, why not use it when you can :)


Mart

Re: [gentoo-dev] New project: LLVM

[Lei Zhang]

2016-08-18 21:56 GMT+08:00 C Bergström :
> @mgorny may be able to help with some of this and has quite a bit of
> experience building clang/llvm.  Where I work we use a "wrapper" that
> helps coordinate a lot of the moving pieces.
>
> https://github.com/pathscale/llvm-suite/
>
> This may not be the perfect "gentoo" way to handle it, but the
> approach would produce a clean and correct compiler. With llvm
> dependencies getting more and more complicated, I'm not sure if it
> would be possible to have both a gnu-free and also perfect
> 1-project-source-repo:1-ebuild ratio.

Currently the ebuilds for libunwind, libc++abi and libc++ all involve
some hackery to support standalone build. And the package clang is
just a placeholder, while it's actually built in llvm.

Maybe we can put them all into the llvm package, and control what
components to build via USE flags.


Lei

Re: [gentoo-dev] Developers, please work on underlinking issues!

[Alexis Ballier]

On Thu, 18 Aug 2016 14:20:41 -0700
Daniel Campbell  wrote:

> On 08/18/2016 06:21 AM, Alexis Ballier wrote:
> > On Thu, 18 Aug 2016 08:13:14 -0400
> > Rich Freeman  wrote:
> >   
> >> If you just check your packages occassionally to make sure they
> >> build with gold it completely achieves the goal, and it will
> >> actually result in fewer bugs using the non-gold linker as well.  
> > 
> > 
> > That's what a tinderbox is for. The only QA problem I see here is
> > that QA doesn't automate that kind of checks anymore since Diego
> > left. Maybe QA should ask Toralf to run a ld.gold tinderbox and
> > avoid asking people to randomly test random packages ?
> >   
> I dunno, if testing packages that one maintains is as simple as
> reconfiguring a package, testing, and switching back then I don't
> think it's unreasonable to ask us to test our own packages. We're
> supposed to do that already, and for packages whose dependencies
> aren't 100% hashed out, it can help us figure out what the real deps
> are.


test against... all linkers, all compilers, all libcs, all kernels, all
userlannds, all useflags, ... ? :)


by all means, please do it, but there are things machines are better
at, like ensuring all packages have been tested against gold linker and
every failure has been reported

Re: [gentoo-dev] Developers, please work on underlinking issues!

[Daniel Campbell]

On 08/18/2016 06:21 AM, Alexis Ballier wrote:
> On Thu, 18 Aug 2016 08:13:14 -0400
> Rich Freeman  wrote:
> 
>> If you just check your packages occassionally to make sure they build
>> with gold it completely achieves the goal, and it will actually result
>> in fewer bugs using the non-gold linker as well.
> 
> 
> That's what a tinderbox is for. The only QA problem I see here is that
> QA doesn't automate that kind of checks anymore since Diego left. Maybe
> QA should ask Toralf to run a ld.gold tinderbox and avoid asking people
> to randomly test random packages ?
> 
I dunno, if testing packages that one maintains is as simple as
reconfiguring a package, testing, and switching back then I don't think
it's unreasonable to ask us to test our own packages. We're supposed to
do that already, and for packages whose dependencies aren't 100% hashed
out, it can help us figure out what the real deps are.

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] rfc: openrc using modprobe directly to load kernel modules

[William Hubbs]

On Wed, Aug 17, 2016 at 11:23:13PM -0700, Daniel Campbell wrote:
> Is there a reliable way to test for kernel functionality _before_
> calling modprobe? I think if a service needs certain kernel
> functionality then it should complain -- loudly, if needed -- so the
> admin knows what to do, be it building the feature into the kernel or
> facilitating a module. But I don't think modules should be required. I
> generally don't enable things with M unless some technical situation
> requires it. The only real module I have is the nvidia-drivers module
> which takes care of itself. Everything else is built into my kernels.
 
Yes, and we do that testing. What we do right now is, if that testing
fails, we run modprobe to attempt to load the modules. That causes
"modprobe: command not found" errors for systems that do not have kmod
installed.

I want to change what happens if the testing fails so that it loads the
module, then complains to the admin letting them know that the module
needs to be built in or loaded in /etc/conf.d/modules.

In a future release, I will remove the module loading assuming that you
have everything built in, configured in /etc/conf.d/modules, or your
device manager is taking care of loading the modules.

> lsmod and modprobe can handle modules -- what can be used to target
> kernels that have functionality built-in? Not every system will have
> /proc/config.gz support (though honestly I don't know why you wouldn't
> want that).
> 
> Hiding error or warning messages seems irresponsible to me and could
> lead to confusion. So I guess I'm in favor overall, but don't want to
> see lightweight installs lose anything or become forced to install
> things as modules, as it complicates the kernel configuring process
> needlessly.

There's nothing to do if the kernel has the functionality built in, I
just want to deprecate the behind-the-scenesloading of modules you don't
know we are loading.

William




signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: /etc/init.d/modules loading modules defined in files

[William Hubbs]

On Thu, Aug 18, 2016 at 10:02:35AM -0400, NP-Hardass wrote:
> Just a side note on how I currently take advantage of the modules
> initscript:
> 
> I have several symlinks, one for each set of modules that I want to
> control.  Then I create a corresponding conf.d based off of the modules
> conf.d.  So, for example, /etc/init.d/vbox-modules->/etc/init.d/modules
> just controls the vbox modules.  This gives me fine control over module
> sets without having to worry about all of them at once.  I can have
> certain sets of modules start in different runlevels, and I can
> independently load and unload sets of modules when necessary.
> 
> If you do end up switching to some system that reads a directory, I'd
> like to see something that if the name of the initscript isn't modules,
> it sources just the corresponding file, rather than the whole dir.

I'm not planning to kill the current system, just extend it, so what you
are doing should still work.

That gives me an idea to consider writing a separate service to handle
this, modules-load-d or something similar.

Thanks,

William

> 
> -- 
> NP-Hardass
> 





signature.asc
Description: Digital signature

Re: [gentoo-dev] rfc: /etc/init.d/modules loading modules defined in files

[NP-Hardass]

On 08/16/2016 07:20 PM, William Hubbs wrote:
> All,
> 
> I have received a request to implement a feature in OpenRC to allow
> multiple software packages to drop files in a directory, /etc/modules.d
> for example, which would define modules the /etc/init.d/modules script
> would load.
> 
> The design I'm thinking of would not change the use of
> /etc/conf.d/modules, but the entries from the files would be added to
> the appropriate module lists after the ones listed there.
> 
> I'll write more about the design as I get closer to formulating the
> details, but at this point I just want to know what others thinkabout
> this feature.
> 
> Thanks,
> 
> William
> 

Just a side note on how I currently take advantage of the modules
initscript:

I have several symlinks, one for each set of modules that I want to
control.  Then I create a corresponding conf.d based off of the modules
conf.d.  So, for example, /etc/init.d/vbox-modules->/etc/init.d/modules
just controls the vbox modules.  This gives me fine control over module
sets without having to worry about all of them at once.  I can have
certain sets of modules start in different runlevels, and I can
independently load and unload sets of modules when necessary.

If you do end up switching to some system that reads a directory, I'd
like to see something that if the name of the initscript isn't modules,
it sources just the corresponding file, rather than the whole dir.

-- 
NP-Hardass



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] New project: LLVM

[C Bergström]

@mgorny may be able to help with some of this and has quite a bit of
experience building clang/llvm.  Where I work we use a "wrapper" that
helps coordinate a lot of the moving pieces.

https://github.com/pathscale/llvm-suite/

This may not be the perfect "gentoo" way to handle it, but the
approach would produce a clean and correct compiler. With llvm
dependencies getting more and more complicated, I'm not sure if it
would be possible to have both a gnu-free and also perfect
1-project-source-repo:1-ebuild ratio.

I'm sure there's llvm/clang ebuilds already and curious what others think..



On Thu, Aug 18, 2016 at 9:48 PM, Ian Bloss  wrote:
> Woot! Don't tell Stallman lol.
>
>
> On Tue, Aug 16, 2016, 09:22 Michał Górny  wrote:
>>
>> Hello, everyone.
>>
>> I have the pleasure of announcing that after the long period of split
>> maintenance, we are forming an united LLVM project [1] to maintain all
>> of LLVM packages in Gentoo and work on establishing improved support for
>> a healthy, gcc-free ecosystem.
>>
>> [1]:https://wiki.gentoo.org/wiki/Project:LLVM
>>
>> --
>> Best regards,
>> Michał Górny
>> 

Re: [gentoo-dev] New project: LLVM

[Ian Bloss]

Woot! Don't tell Stallman lol.

On Tue, Aug 16, 2016, 09:22 Michał Górny  wrote:

> Hello, everyone.
>
> I have the pleasure of announcing that after the long period of split
> maintenance, we are forming an united LLVM project [1] to maintain all
> of LLVM packages in Gentoo and work on establishing improved support for
> a healthy, gcc-free ecosystem.
>
> [1]:https://wiki.gentoo.org/wiki/Project:LLVM
>
> --
> Best regards,
> Michał Górny
> 
>

Re: [gentoo-dev] Developers, please work on underlinking issues!

[Alexis Ballier]

On Thu, 18 Aug 2016 08:13:14 -0400
Rich Freeman  wrote:

> If you just check your packages occassionally to make sure they build
> with gold it completely achieves the goal, and it will actually result
> in fewer bugs using the non-gold linker as well.


That's what a tinderbox is for. The only QA problem I see here is that
QA doesn't automate that kind of checks anymore since Diego left. Maybe
QA should ask Toralf to run a ld.gold tinderbox and avoid asking people
to randomly test random packages ?

Re: [gentoo-dev] Developers, please work on underlinking issues!

[Rich Freeman]

On Thu, Aug 18, 2016 at 7:47 AM, Lars Wendler  wrote:
>
> And as long as I keep reading such statements I won't use ld.gold
> anywhere on my (dev-)systems. A linker IMHO is a far too crucial
> toolchain component to blindly play around with.
>

There really isn't any need to use it as a daily driver.

If you just check your packages occassionally to make sure they build
with gold it completely achieves the goal, and it will actually result
in fewer bugs using the non-gold linker as well.

-- 
Rich

Re: [gentoo-dev] Developers, please work on underlinking issues!

[Rich Freeman]

On Thu, Aug 18, 2016 at 1:39 AM, Daniel Campbell  wrote:
>
> Is it as simple as switching the linker and re-merging packages that one
> maintains? Is gold supposed to be a big deal? Does it do the job of
> linking better? I read the blog post and all but nobody's explaining
> what gold does better than standard ld.
>

There are a bunch of reasons why people prefer it for the long term.
However, one of the biggest drives to get devs to use it is that it
serves as a good canary for underlinking.

If you underlink with the bfd linker the package will build and run
fine typically, at least initially.  However, down the road as
libraries are updated you can run into problems.  Since you didn't
specify all your dependencies portage won't do rebuilds when they're
necessary.  My understanding is that if the needed libraries aren't
even linked then even preserve-rebuild will fail.  The package is
basically relying on unspecified behavior to work.  If you need a
symbol you're supposed to link to the providing library, not just
assume that it will happen to be around because some other library
you're linked to happens to pull it in.

The gold linker generates errors if you attempt to underlink, which
turns this into a build-time error, and not a
maybe-you-see-it-maybe-you-don't runtime issue for random users in six
months.

Correctly specifying dependencies and ensuring they're linked is of
benefit to all users, and it will prevent subtle problems for users of
the default bfd linker.

At least, that is my understanding of the issue.  It is entirely
possible I missed something; I don't profess to be an ELF expert.  :)

-- 
Rich

Re: [gentoo-dev] Developers, please work on underlinking issues!

[Lars Wendler]

On Thu, 18 Aug 2016 13:43:42 +0300 Andrew Savchenko wrote:

>Hi,
>
>On Wed, 17 Aug 2016 22:37:42 +0200 Michał Górny wrote:
>[...]
>> That said, I'd really like for Gentoo to be able to finally switch to
>> ld.gold by default.  
>
>It still has problems with kernel or kernel modules here and there.
>
>Also from my experience gold demands more memory on linking large
>binaries, thus it it unsuitable for low-memory setups.
>
>While gold indeed has its benefits, it is still too immature to
>become system default.

And as long as I keep reading such statements I won't use ld.gold
anywhere on my (dev-)systems. A linker IMHO is a far too crucial
toolchain component to blindly play around with.

>Best regards,
>Andrew Savchenko

Cheers
Lars

-- 
Lars Wendler
Gentoo package maintainer
GPG: 21CC CF02 4586 0A07 ED93  9F68 498F E765 960E 9B39

Attention! New gpg key! See
https://www.gentoofan.org/blog/index.php?/archives/9-New-gpg-keys.html


pgpQ0Dl2uKltV.pgp
Description: Digitale Signatur von OpenPGP

Re: [gentoo-dev] Developers, please work on underlinking issues!

[Andrew Savchenko]

Hi,

On Wed, 17 Aug 2016 22:37:42 +0200 Michał Górny wrote:
[...]
> That said, I'd really like for Gentoo to be able to finally switch to
> ld.gold by default.

It still has problems with kernel or kernel modules here and there.

Also from my experience gold demands more memory on linking large
binaries, thus it it unsuitable for low-memory setups.

While gold indeed has its benefits, it is still too immature to
become system default.

Best regards,
Andrew Savchenko


pgp6NAXB4caLn.pgp
Description: PGP signature

Re: [gentoo-dev] Developers, please work on underlinking issues!

[Alexis Ballier]

On Wed, 17 Aug 2016 22:39:37 -0700
Daniel Campbell  wrote:

> Is it as simple as switching the linker and re-merging packages that
> one maintains? Is gold supposed to be a big deal? Does it do the job
> of linking better? I read the blog post and all but nobody's
> explaining what gold does better than standard ld.

I think gold is faster and doesnt eat 8+gb of ram when linking chromium
with debug symbols :)

> That said, if it's that simple and then just requires some DEPEND
> updating, doesn't sound all that bad to me.

Nah, it's adding proper -lfoo when foo is used; more like asneeded
fixes.

[gentoo-dev] Package up for grabs

[Pacho Ramos]

This package is up for grabs: app-office/gtimelog

Re: [gentoo-dev] rfc: openrc using modprobe directly to load kernel modules

[Consus]

On 23:23 Wed 17 Aug, Daniel Campbell wrote:
> lsmod and modprobe can handle modules -- what can be used to target
> kernels that have functionality built-in? Not every system will have
> /proc/config.gz support (though honestly I don't know why you wouldn't
> want that).

Well,

$ ls /sys/module

if we are talking about modules. Built-in modules are also listed there.

Re: [gentoo-dev] New Working Group established to evaluate the stable tree

[Raymond Jennings]

Strict compliance with the handbook would seem to forbid having a stable
package depend on an unstable package, and if you have to downgrade a
dependency and it causes a cascade, I would opine, that, perhaps, the
package in question should not have been stabilized to begin with.

That said, I as a user have noticed that packages tend to stall in
stabilization for awhile.

What about a script that can rank ~arch keyworded packages by some sort of
priority?

Maybe point out which one has the most reverse dependencies?  Or which one
has been stuck in ~arch the longest?  Or some sort of scoring mechanism
that can flag the most urgently needed stabilizations?

Come to think of it, I think debian has a system that flags the most
popular packages.  Does gentoo have a way to note what packages are most
important?

On Wed, Aug 17, 2016 at 1:50 AM, Pacho Ramos  wrote:

> El lun, 15-08-2016 a las 15:27 -0400, Rich Freeman escribió:
> > [...]
> > Well, I wasn't suggesting that breaking the depgraph is great.  Just
> > that I think it is better than calling things stable which aren't.
> >
> > A better approach is a script that does the keyword cleanup.
> >
> > So, if you want to reap an ebuild you run "destabilize
> > foo-1.2.ebuild".  It searches the tree for all reverse deps and
> > removes stable keywords from those.  Then you commit all of that in
> > one commit.
> >
> > If you want to be extra nice you stick it in a pull request in github
> > and point it out to the arch team and ask them if they're sure they
> > don't want to stabilize your package...  :)
> >
>
> Well, the reason I was suggesting to allow maintainers to stabilize
> after the 90 days timeout over *current* policy of allowing the
> dekeywording is that the dekeywording is completely unrealistic to do
> as some packages have a huge amount of reverse deps. Even with the
> script (and, well, I would like to see that script existing... because
> we are having this issue for ages, and that is the reason that nobody
> is moving things to testing actively), you will find many many cases of
> packages having so many reverse dependencies that if you try to move it
> to testing it becomes soon a hell.
>
> The main issue is that, once you start dekeywording one package, you
> jump to, for example, dekeywording another 3 reverse deps, then you
> need to continue with the reverse deps of that reverse deps... and at
> the end, it's completely impossible to manage it (I still remember how
> hard was to move to testing most of Gnome... and we even were lucky as
> we were able to do that with the jump to Gnome3).
>
> Then, my point it to allow the maintainer to keep stabilizing it
> *after* the 90 days timeout. If after that time, the arch team is
> unable to even reply, nobody has reported any build/runtime issues
> related with that arch, I would go ahead. Otherwise, it looks pretty
> evident to me that that arch is near to be used by nobody and maybe it
> should be moved completely to testing (or most of their packages moved
> to testing and only the core apps in stable).
>
>
>
>