[Differential] [Accepted] D1327: Do not strip all when stripping an explicit symbol

2014-12-16 Thread imp (Warner Losh)
imp added a subscriber: imp.
imp accepted this revision.
imp added a reviewer: imp.
imp added a comment.
This revision is now accepted and ready to land.

Looks good to me.

REVISION DETAIL
  https://reviews.freebsd.org/D1327

To: emaste, imp
Cc: imp, freebsd-toolchain
___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"


[Differential] [Request, 3 lines] D1327: Do not strip all when stripping an explicit symbol

2014-12-16 Thread emaste (Ed Maste)
emaste created this revision.
emaste added a subscriber: freebsd-toolchain.

REVISION SUMMARY
  When asked to strip a specific symbol (-N) strip still defaulted to 
STRIP_ALL. In this case binutils defaults to stripping nothing (other than the 
requested symbol(s), of course), which makes a lot more sense.
  
  PR 196038
  https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196038

TEST PLAN
- reduced test case from PR
- build ffmpeg port

REVISION DETAIL
  https://reviews.freebsd.org/D1327

AFFECTED FILES
  contrib/elftoolchain/elfcopy/main.c

To: emaste
Cc: freebsd-toolchain
___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"


[Differential] [Updated] D1327: Do not strip all when stripping an explicit symbol

2014-12-16 Thread emaste (Ed Maste)
emaste updated the summary for this revision.

REVISION DETAIL
  https://reviews.freebsd.org/D1327

To: emaste
Cc: freebsd-toolchain
___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"


Re: RFT: Please help testing the llvm/clang 3.5.0 import

2014-12-16 Thread Jung-uk Kim
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 12/16/2014 14:36, Dimitry Andric wrote:
> * The second exp-run had much better results: the failure with the 
> highest number of dependencies is devel/mingw32-gcc, but this
> seems to be due to a problem with makeinfo, not clang.  The next
> highest on the list is java/openjdk6, for which ports r374780 [4]
> was very recently committed.

Unfortunately, r374780 was not enough.  Instead, I just turned off
"-Werror" for now (r374824).

https://svnweb.freebsd.org/changeset/ports/374824

Jung-uk Kim
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJUkKgGAAoJEHyflib82/FGGtAH/jyK3fVhWeXlgID5MKov0+vq
34BwE98ppJWreu4LdkXGqCUZeciyMmcw4ROfEPo6IthIxcHsRleh+O+BnmA5wFce
gMczWBO1R+uEzcSH75UhyaVJVMKy8BJ2vRU2s90GANUnMhcMvNjN0Y89+8PdCHWF
zaR8oy/GlVpJ13RTbyeaMf8K0T6MyQp58VQYP1gmlhjafEjVOLO9IVZyLWVx/nsI
+DtjLj1DdNrPKrV1jrVRmZ+bJqOLaLgL4FUV/vruSduA1U8E1BZgnklXqRPowXqN
jmFbLYE4kiygcEmUnpVbLQeB2EWXbQq7g4pijh90qDrhCSX1rUN3gz2DxY/Mub4=
=reYk
-END PGP SIGNATURE-
___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"


Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread Ed Maste
On 16 December 2014 at 14:47, Warner Losh  wrote:
>
> My comment was
> more to Ed’s notion that these numbers will be dropping any time soon.

To be clear, I don't expect we'll be dropping knobs soon. We should
keep that as a goal to aim for.
___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"

Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread Warner Losh

> On Dec 16, 2014, at 12:01 PM, Dimitry Andric  wrote:
> 
> On 16 Dec 2014, at 18:54, Warner Losh  wrote:
>> 
>>> On Dec 16, 2014, at 10:44 AM, Ed Maste  wrote:
>>> 
>>> Fair enough, I'd definitely like to see fewer build-time knobs over
>>> time, not more.
>> 
>> Until we stop using build-time knobs to control what’s in the final image
>> as a poor man’s packaging scheme, I expect the number of knobs to
>> continue to grow.
> 
> How does a packaging scheme solve the problem of not compiling in
> dependencies, or linking everything static?  You cannot pre-build all
> possible combinations of selectable options.
> 
> As for knobs that just say "build foo" or "don't build bar", those would
> indeed be fine for a packaging system, as long as packages aren't too
> dependent on each other.

Right now we mix build options for building things or not (e.g. WITH_SENDMAIL)
with build options for things like this (WITH_SHARED_TOOLCHAIN). The number
of the former is increasing all the time (with a big increase when Ngie’s work 
hits
the tree). So in many senses, it is an orthogonal issue. My comment was
more to Ed’s notion that these numbers will be dropping any time soon.

packing the base is actually a hard problem because the phrase “as long as
packages aren’t dependent on each other” turns out to not be the case as
much as we might like. The base is fairly interdependent of all that since we
have it as one big ball of stuff right now. Most of the issues revolve around
dependence on the libraries and such. With patience and diligence, I think
we can package the base, but it isn’t going to be a trivial slam dunk if it is
going to be useful.

Warner


signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: RFT: Please help testing the llvm/clang 3.5.0 import

2014-12-16 Thread Dimitry Andric
On 28 Nov 2014, at 22:03, Dimitry Andric  wrote:
> 
> We're working on updating llvm, clang and lldb to 3.5.0 in head.  This
> is quite a big update again, and any help with testing is appreciated.
> 
> To try this out, ensure you have good backups or snapshots, then build
> world and kernel from the projects/clang350-import branch [1].  Please
> use a Subversion mirror [2], if you are able to.

Here are some updates about the status of the 3.5.0 import.

* i386 and amd64 have been tested through make universe, and everything
  should compile and run.
* Little-endian ARM builds should now compile and run, thanks to Andrew
  Turner for putting in lots of work.
* Big-endian ARM is apparently supposed to work, but I'm not sure if
  Andrew managed to test it on real hardware.
* PowerPC64 should mostly work, thanks to Justin Hibbits.
* PowerPC32 might start working soon; it really needs some backporting
  of fixes to clang 3.4.1, which is now in head, so there is an easier
  upgrade path for PowerPC users.
* Sparc64 still does not work, and I don't see any quick solutions to it
  for now.  It should probably stay with gcc.
* Mips will only have a chance with the upcoming clang 3.6.0, but that
  is way too late for this import.  It will probably require external
  toolchain support to get it working.
* Another ports exp-run was done [3], after fixing the problem with
  lang/gcc, which lead to many skipped dependent ports.
* The second exp-run had much better results: the failure with the
  highest number of dependencies is devel/mingw32-gcc, but this seems
  to be due to a problem with makeinfo, not clang.  The next highest on
  the list is java/openjdk6, for which ports r374780 [4] was very
  recently committed.

I would really like to merge this branch to head in about a week,
pending portmgr approvall; I don't expect the base system (outside of
llvm/clang) to need any further updates.

Lastly, to clear things up about the requirements for this branch (and
thus for head, in a while); to build it, you need to have:
* A C++11 capable "host" compiler, e.g. clang >= 3.3 or later, or gcc
  >= 4.8 (I'm not 100% sure if gcc 4.7 will work, reports welcome)
* A C++11 standard library, e.g. libc++, or libstdc++ from gcc >= 4.8.

So from any earlier standard 10.x or 11.x installation, you should be
good, unless you explicitly disabled clang or libc++.  In that case,
you must build and install both of those first.

On a 9.x installation, you will have clang by default, but not libc++,
so libc++ should be built and installed first, before attempting to
build the clang350-import branch.

On 8.x an earlier, you need to upgrade to at least 9.x first, follow
the previous instruction.

As for MFC'ing, I plan on merging clang 3.5.x to 10.x in a while
(roughly a month), but this will cause upgrades from 9.x to 10.x to
start requiring the build of libc++, as described above.  I don't think
we can merge clang 3.5.x to 9.x, unless clang becomes the default
compiler there (but that is very unlikely).

-Dimitry

[1] svn://svn.freebsd.org/base/projects/clang350-import
[2] 
https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/svn.html#svn-mirrors
[3] 
http://pb2.nyi.freebsd.org/build.html?mastername=head-amd64-PR195480-default&build=2014-12-12_23h17m02s
[4] https://svnweb.freebsd.org/changeset/ports/374780



signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread Dimitry Andric
On 16 Dec 2014, at 18:54, Warner Losh  wrote:
> 
>> On Dec 16, 2014, at 10:44 AM, Ed Maste  wrote:
>> 
>> Fair enough, I'd definitely like to see fewer build-time knobs over
>> time, not more.
> 
> Until we stop using build-time knobs to control what’s in the final image
> as a poor man’s packaging scheme, I expect the number of knobs to
> continue to grow.

How does a packaging scheme solve the problem of not compiling in
dependencies, or linking everything static?  You cannot pre-build all
possible combinations of selectable options.

As for knobs that just say "build foo" or "don't build bar", those would
indeed be fine for a packaging system, as long as packages aren't too
dependent on each other.

-Dimitry



signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread Warner Losh

> On Dec 16, 2014, at 10:44 AM, Ed Maste  wrote:
> 
> Fair enough, I'd definitely like to see fewer build-time knobs over
> time, not more.

Until we stop using build-time knobs to control what’s in the final image
as a poor man’s packaging scheme, I expect the number of knobs to
continue to grow.

In this case, however, it should be covered under the existing knobs
though (or we should eliminate the knob :).

Warner




signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread Ed Maste
On 16 December 2014 at 12:39, Warner Losh  wrote:
>
> We should defer testing until after that import :)

Indeed. I have an early WIP patch in a local branch, but any new work
before 3.5.x lands will be wasted, so it will be in the new year that
I'll have something to start testing.
___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"


Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread Ed Maste
On 16 December 2014 at 11:15, David Chisnall  wrote:
>
> Upstream doesn't call it libclang (that's the name of the library with a 
> stable C ABI, which is why I'd like us not to confuse it with something with 
> a library with an unstable C++ API). They do produce a libLLVM.so from the 
> LLVM builds and work is underway to have shared library builds for clang.

Agreed. Even if there's no potential issue since it will be in
/usr/lib/private it would be confusing. David's suggestion of libcfe
sounds reasonable.

>> That said, I agree with the general idea, but one of the first things
>> we should decide is whether this will be optional or not.  Having to
>> maintain yet another WITH_CLANG_FOO option is burdensome...

Fair enough, I'd definitely like to see fewer build-time knobs over
time, not more.

> I agree.  I'd quite like to see performance numbers for the compiler.  I 
> think I saw about a 10% overhead for buildworld last time I tried, but that 
> was a couple of years ago.

Ok. I will put this together in a branch and we can compare results
and decide which way to go when we have actual numbers.
___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"


Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread Warner Losh

> On Dec 16, 2014, at 9:32 AM, Dimitry Andric  wrote:
> 
> On 16 Dec 2014, at 17:15, David Chisnall  wrote:
>> 
>> On 16 Dec 2014, at 16:04, Dimitry Andric  wrote:
>>> This is precisely why the libs should go into /usr/lib/private, so as to
>>> avoid collisions with any upstream libraries installed by e.g. ports (or
>>> when you manually run "make install" after building).
>> 
>> That's still potentially an issue if we add local tools that use libclang 
>> APIs (which we may well do).
>> 
>>> I'm not sure we want to go the 'libbsdfoo.so' route again, as Baptiste
>>> tried this before, and seems to have reversed it again. :)
>> 
>> Upstream doesn't call it libclang (that's the name of the library with a 
>> stable C ABI, which is why I'd like us not to confuse it with something with 
>> a library with an unstable C++ API).  They do produce a libLLVM.so from the 
>> LLVM builds and work is underway to have shared library builds for clang.
>> 
>> libLLVM.so could potentially be in /usr/lib in 11 if we have a packaged base 
>> system, as it would allow us to have different .so versions installed if 
>> things demanded them.  The point releases guarantee backwards ABI 
>> compatibility, so we can still upgrade to them if required.
> 
> Unfortunately we already imported quite a lot of ABI-breaking bug fixes.
> I would prefer only our own tools to be linked against the "FreeBSD"
> version of libllvm.so/libwhatever.so.

:(

>> That said, I agree with the general idea, but one of the first things
>>> we should decide is whether this will be optional or not.  Having to
>>> maintain yet another WITH_CLANG_FOO option is burdensome...
>> 
>> I agree.  I'd quite like to see performance numbers for the compiler.  I 
>> think I saw about a 10% overhead for buildworld last time I tried, but that 
>> was a couple of years ago.
> 
> There is already a WITH_SHARED_TOOLCHAIN option, that defaults to off,
> but I have had it on since approximately the time Kostik added it.  I
> might just have gotten used to the overhead, if any…

The 10% figure has been relatively constant over the lifetime of shared
libraries in FreeBSD. This is the average hit of using shared libraries
and everybody accepts that. I doubt time has changed this much at all.

> I would like to do a bit of testing with that, but my TODO list is
> rather full at this point, working on the 3.5.0 import. :)

We should defer testing until after that import :)

Warner


signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread Dimitry Andric
On 16 Dec 2014, at 17:15, David Chisnall  wrote:
> 
> On 16 Dec 2014, at 16:04, Dimitry Andric  wrote:
>> This is precisely why the libs should go into /usr/lib/private, so as to
>> avoid collisions with any upstream libraries installed by e.g. ports (or
>> when you manually run "make install" after building).
> 
> That's still potentially an issue if we add local tools that use libclang 
> APIs (which we may well do).
> 
>> I'm not sure we want to go the 'libbsdfoo.so' route again, as Baptiste
>> tried this before, and seems to have reversed it again. :)
> 
> Upstream doesn't call it libclang (that's the name of the library with a 
> stable C ABI, which is why I'd like us not to confuse it with something with 
> a library with an unstable C++ API).  They do produce a libLLVM.so from the 
> LLVM builds and work is underway to have shared library builds for clang.
> 
> libLLVM.so could potentially be in /usr/lib in 11 if we have a packaged base 
> system, as it would allow us to have different .so versions installed if 
> things demanded them.  The point releases guarantee backwards ABI 
> compatibility, so we can still upgrade to them if required.

Unfortunately we already imported quite a lot of ABI-breaking bug fixes.
I would prefer only our own tools to be linked against the "FreeBSD"
version of libllvm.so/libwhatever.so.


> That said, I agree with the general idea, but one of the first things
>> we should decide is whether this will be optional or not.  Having to
>> maintain yet another WITH_CLANG_FOO option is burdensome...
> 
> I agree.  I'd quite like to see performance numbers for the compiler.  I 
> think I saw about a 10% overhead for buildworld last time I tried, but that 
> was a couple of years ago.

There is already a WITH_SHARED_TOOLCHAIN option, that defaults to off,
but I have had it on since approximately the time Kostik added it.  I
might just have gotten used to the overhead, if any...

I would like to do a bit of testing with that, but my TODO list is
rather full at this point, working on the 3.5.0 import. :)

-Dimitry



signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread David Chisnall
On 16 Dec 2014, at 16:04, Dimitry Andric  wrote:

> This is precisely why the libs should go into /usr/lib/private, so as to
> avoid collisions with any upstream libraries installed by e.g. ports (or
> when you manually run "make install" after building).

That's still potentially an issue if we add local tools that use libclang APIs 
(which we may well do).

> I'm not sure we want to go the 'libbsdfoo.so' route again, as Baptiste
> tried this before, and seems to have reversed it again. :)

Upstream doesn't call it libclang (that's the name of the library with a stable 
C ABI, which is why I'd like us not to confuse it with something with a library 
with an unstable C++ API).  They do produce a libLLVM.so from the LLVM builds 
and work is underway to have shared library builds for clang.

libLLVM.so could potentially be in /usr/lib in 11 if we have a packaged base 
system, as it would allow us to have different .so versions installed if things 
demanded them.  The point releases guarantee backwards ABI compatibility, so we 
can still upgrade to them if required.

> That said, I agree with the general idea, but one of the first things
> we should decide is whether this will be optional or not.  Having to
> maintain yet another WITH_CLANG_FOO option is burdensome...

I agree.  I'd quite like to see performance numbers for the compiler.  I think 
I saw about a 10% overhead for buildworld last time I tried, but that was a 
couple of years ago.

David

___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"


Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread Dimitry Andric
On 16 Dec 2014, at 16:58, David Chisnall  wrote:
> On 16 Dec 2014, at 15:46, Ed Maste  wrote:
>> One of goals for the toolchain prior to the FreeBSD 11 branch is to
>> create a libllvm.so and libclang.so for use by all of the LLVM family
>> tools installed in the base system. This message is just a heads-up in
>> case anyone has questions or comments on the idea.
>> 
>> We currently build a large number of static libs for the llvm and
>> clang components, which are reused in a number of tools in the LLVM
>> family. The resulting binaries end up quite large, and as a group
>> require a lot of disk space. For example, LLDB includes a copy of
>> Clang, used as its expression parser.  As a result, on my desktop
>> /usr/bin/clang and /usr/bin/lldb are both 27MB.
>> 
>> Over time we may add additional LLVM family tools (e.g.,
>> llvm-objdump), and this will help avoid excessive bloat as we do so.
>> 
>> I expect libllvm.so and libclang.so will go in /usr/lib/private.
> Hi Ed,
> 
> Please can we have a name other than libclang, to avoid name collisions and 
> confusion with, uh, libclang?  libcfe maybe?

This is precisely why the libs should go into /usr/lib/private, so as to
avoid collisions with any upstream libraries installed by e.g. ports (or
when you manually run "make install" after building).

I'm not sure we want to go the 'libbsdfoo.so' route again, as Baptiste
tried this before, and seems to have reversed it again. :)

That said, I agree with the general idea, but one of the first things
we should decide is whether this will be optional or not.  Having to
maintain yet another WITH_CLANG_FOO option is burdensome...

-Dimitry



signature.asc
Description: Message signed with OpenPGP using GPGMail


Re: Migration to dynamic libs for llvm and clang

2014-12-16 Thread David Chisnall
Hi Ed,

Please can we have a name other than libclang, to avoid name collisions and 
confusion with, uh, libclang?  libcfe maybe?

David

On 16 Dec 2014, at 15:46, Ed Maste  wrote:

> One of goals for the toolchain prior to the FreeBSD 11 branch is to
> create a libllvm.so and libclang.so for use by all of the LLVM family
> tools installed in the base system. This message is just a heads-up in
> case anyone has questions or comments on the idea.
> 
> We currently build a large number of static libs for the llvm and
> clang components, which are reused in a number of tools in the LLVM
> family. The resulting binaries end up quite large, and as a group
> require a lot of disk space. For example, LLDB includes a copy of
> Clang, used as its expression parser.  As a result, on my desktop
> /usr/bin/clang and /usr/bin/lldb are both 27MB.
> 
> Over time we may add additional LLVM family tools (e.g.,
> llvm-objdump), and this will help avoid excessive bloat as we do so.
> 
> I expect libllvm.so and libclang.so will go in /usr/lib/private.
> ___
> freebsd-toolchain@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
> To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"

___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"


Migration to dynamic libs for llvm and clang

2014-12-16 Thread Ed Maste
One of goals for the toolchain prior to the FreeBSD 11 branch is to
create a libllvm.so and libclang.so for use by all of the LLVM family
tools installed in the base system. This message is just a heads-up in
case anyone has questions or comments on the idea.

We currently build a large number of static libs for the llvm and
clang components, which are reused in a number of tools in the LLVM
family. The resulting binaries end up quite large, and as a group
require a lot of disk space. For example, LLDB includes a copy of
Clang, used as its expression parser.  As a result, on my desktop
/usr/bin/clang and /usr/bin/lldb are both 27MB.

Over time we may add additional LLVM family tools (e.g.,
llvm-objdump), and this will help avoid excessive bloat as we do so.

I expect libllvm.so and libclang.so will go in /usr/lib/private.
___
freebsd-toolchain@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain
To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"