GCC 12.3 ICE: format(printf...) failing in C++ with virtual inheritance

[Paul Smith via Gcc]

I've tried this with both older versions as well as GCC 12.3 (latest I
have access to).  This is on GNU/Linux on x86_64.


I have the following code:

  #include 

  class Exception : public std::exception
  {
  public:
  Exception(const char* text, ...)
__attribute__((format(printf, 2, 3)));
  };

  void foo()
  {
  throw Exception("something bad");
  }

this works fine.

However, due to some diamond inheritance I need to switch to virtual
inheritance here:

  #include 

  class Exception : public /**/ virtual /**/ std::exception
  {
  public:
  Exception(const char* text, ...)
  __attribute__((format(printf, 2, 3)));
  };

  void foo()
  {
  throw Exception("something bad");
  }

this does not work AT ALL.

First, the extra virtual infrastructure means that the offsets we use
(note we need to add one to the format parameters anyway due to the
this pointer I assume) are wrong:

  /tmp/virt.cpp:7:45: error: 'format' attribute argument 2 value '2'  
refers to parameter type 'int'
  7 | __attribute__((format(printf, 2, 3)));
| ^

Just adding 1 to this, for 3, 4, doesn't help:

  /tmp/virt.cpp:7:45: error: 'format' attribute argument 2 value '3'  
refers to parameter type 'const void**'
  7 | __attribute__((format(printf, 3, 4)));
| ^

And if I add 2 instead so it's 4,5 instead, I get an ICE:

  /tmp/virt.cpp: In function 'void foo()':
  /tmp/virt.cpp:12:36: error: 'format' attribute argument 2 value '4'   exceeds 
the number of function parameters 2 [-Werror=attributes]
 12 | throw Exception("something bad");
|^
  /tmp/virt.cpp:12:36: internal compiler error: in get_constant, at 
c-family/c-format.cc:323
  0x7fba675a3082 __libc_start_main
  ../csu/libc-start.c:308
  Please submit a full bug report, with preprocessed source (by using 
-freport-bug).
  Please include the complete backtrace with any bug report.
  See  for instructions.

I'm assuming a bug should be filed for this ICE (can anyone repro it in
the current release?), but does anyone know if there's any way to make
it work in GCC 12.3?

Re: Please see the following post. Some of you may know the answer.

[Paul Smith]

On Wed, 2023-08-09 at 11:14 +0200, Luís Carlos Carneiro Gonçalves via
Gcc wrote:
> Without -O3 and -O2 the OpenCL gave consistently good results without
> returning any error.
> 
> I think my code is Ok but I open to criticism.

By "has a bug" we don't necessarily mean something that is obviously
wrong and fails every time like trying to dereference a null pointer.

We mean, you are making an assumption about how the compiler will
compile your code, that is not guaranteed by the standard.  Because
it's not guaranteed, when you enable optimizations the compiler is free
to rearrange things in ways you maybe don't expect, but are allowed by
the language definition.

> If you are open to help me I can send all the sources.

I doubt anyone here has the time or energy to read and debug "all the
sources" for you.

If you reduce your code to a small example that shows the error, then
post that to the correct mailing list as Jonathan suggested.  Then
someone may be able to point out your mistake.

Or, maybe it really is a bug in the optimizer.  Such things are very
rare but have been known to happen.  In my long experience as a
programmer, however, "the compiler has a bug" has NEVER been a winning
bet.

Re: Using _POSIX_SEM_VALUE_MAX not _SC_SEM_VALUE_MAX?

[Paul Smith]

On Sat, 2023-07-08 at 12:33 -0400, Paul Smith wrote:
> On Sat, 2023-07-08 at 17:30 +0200, Andreas Schwab wrote:
> > That needs to be sysconf (_SC_SEM_VALUE_MAX), and thus is not
> > suitable for a constexpr.
> 
> Oh right, obviously.
> 
> Well, I guess I'll have to try to figure out why it's not defined. 
> Sigh.

I figured it out.  I'm sure it's such a weird problem no one else will
hit it but just in case:

In order to use clangd without errors I need to use the clangd
intrinsics: clang can't parse the GCC intrinsics correctly.  The clang
limits.h uses #include_next  which happens to find GCC's
include-fixed/limits.h and includes it in such a way that it doesn't
try to include the full system limits.h, so I only have the basics and
not all the POSIX extensions etc.

If I convince clangd to ignore BOTH the GCC intrinsics directory AND
the GCC include-fixed directory, then everything works as expected.

Re: Using _POSIX_SEM_VALUE_MAX not _SC_SEM_VALUE_MAX?

[Paul Smith]

On Sat, 2023-07-08 at 17:30 +0200, Andreas Schwab wrote:
> That needs to be sysconf (_SC_SEM_VALUE_MAX), and thus is not
> suitable for a constexpr.

Oh right, obviously.

Well, I guess I'll have to try to figure out why it's not defined. 
Sigh.

Using _POSIX_SEM_VALUE_MAX not _SC_SEM_VALUE_MAX?

[Paul Smith]

I have been trying to make clangd work with my GCC-based builds (it's a
long story don't ask) and after a ton of effort (and, unfortunately,
some hacks due to clangd limitations) there is one thing left that's
causing a lot of spurious errors:

In gcc/libstdc++-v3/include/bits/semaphore_base.h we find:

56  #ifdef SEM_VALUE_MAX
57  static constexpr ptrdiff_t _S_max = SEM_VALUE_MAX;
58  #else
59  static constexpr ptrdiff_t _S_max = _POSIX_SEM_VALUE_MAX;
60  #endif

Unfortunately because I am using limits.h from the clang intrinsics /
resource directory (else many other things break), _POSIX_SEM_VALUE_MAX
is not defined.  I can't quite figure out why, and my attempts to
generate preprocessor output to track it down through the #include /
#define maze have failed.  But the error I get (from clangd) suggests I
want to be using _SC_SEM_VALUE_MAX and indeed if I make that change it
works.

Should GCC be using the _SC version, since that's what's defined in
POSIX?

Or at least expanding the above to use it as a last resort, as in:

  #if defined(SEM_VALUE_MAX)
  static constexpr ptrdiff_t _S_max = SEM_VALUE_MAX;
  #elif defined(_POSIX_SEM_VALUE_MAX)
  static constexpr ptrdiff_t _S_max = _POSIX_SEM_VALUE_MAX;
  #else
  static constexpr ptrdiff_t _S_max = _SC_SEM_VALUE_MAX;
  #endif

??

Re: Will GCC eventually learn to use BSR or even TZCNT on AMD/Intel processors?

[Paul Smith]

On Tue, 2023-06-06 at 16:36 +0800, Julian Waters via Gcc wrote:
> Sorry for my outburst, to the rest of this list. I can no longer stay
> silent and watch these little shits bully people who are too kind to
> fire back with the same kind of venom in their words.

Many of us have had Dave in our killfiles for a long time already.  I
recommend you (and everyone else) do the same.  You won't miss out on
any information of any use to anyone: he apparently just enjoys making
other people angry.

I'm quite serious: it's so not worth the mental energy to even read his
messages, much less to reply to him.  Arguing with "people who are
wrong on the internet" can be cathartic but this is not arguing, it's
just stabbing yourself in the eye with a pencil.  Don't play.

Re: LSP based on GCC

[Paul Smith]

On Mon, 2023-05-29 at 17:16 -0300, Alexandre Oliva via Gcc wrote:
> On May 17, 2023, Arsen Arsenović  wrote:
> 
> > ISTR Alexandre Oliva (CC added) mentioning leveraging GDB to
> > implement various bits of LSP functionality, such as handling
> > multiple TUs.
> 
> I recall advancing that suggestion, reasoning that GDB was capable of
> combining information from multiple translation units and of
> reloading debug information, which GCC doesn't.

I'm not sure this will work well.  The information LSP servers need is
quite a bit more detailed (I believe) than what is found in the object
file debug sections.

Typically LSP servers generate their own index files, which are
completely separate from any compiler-generated output.  When the
server starts it will proceed to parse and index the entire codebase
(which is unfortunately slow) then they try to keep that index updated
over time as code changes.  So the LSP server will use the compiler
front-end to parse the code and keep information about symbols and
locations separately.

LSP servers are not intended to be limited to dealing with the code
that has already been compiled: not only do you want to be able to edit
code before it's been compiled, but you want to be able to query new
code as it's written in the editor, before it's even saved to disk.

I recommend that anyone who is interested in this project, examine the
LSP spec to understand what information the server needs to deal with:

https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/

Re: LSP based on GCC

[Paul Smith]

On Thu, 2023-05-18 at 10:52 -0400, Ben Boeckel wrote:
> PCH files can "be ignored" in some sense because they can be
> recalculated from `#include` files pretty easily. Module files,
> however, cannot.

This makes it even more important that there be a GCC-based LSP server,
if a Clang-based one can't read GCC module files AND module files are
required to exist.  Without a GCC-based LSP server, it means you have
to build your modules for _both_ GCC (for compilation) _and_ Clang (for
LSP) which would be extremely frustrating.

It would also likely make Eli's initial contention that you need to
install a full Clang compiler to use a Clang-based LSP server, into a
reality.

Re: LSP based on GCC

[Paul Smith]

On Wed, 2023-05-17 at 18:38 -0400, Ben Boeckel wrote:
> > egregious example I'm aware of is that they look for GCC-named
> > precompiled headers (.gch), even though the Clang PCH format is
> > completely different.  So if Clang (and the LSP servers built on
> > it) find a .gch header file they will try to read it, fail, and
> > give an error.  I filed a bug about this in 2019 but it's been
> > ignored.
> > 
> > This means you need to modify your LSP server arguments to omit any
> > PCH compiler command line arguments; for environments based on
> > auto-generated definitions like compile_commands.json this is
> > frustrating.
> 
> FWIW, this is only going to get worse with C++ modules.

There's no reason it should.  Of course the right answer is to tell
people to fix their build systems and if they want to use a different
compiler AND use PCH, they use the appropriate suffix for that
compiler.

But even if you don't want to do that the fix in this case is trivial.
I even sent a patch (although since I don't know the clang code there's
no doubt that it was not done "the right way" and needed to be
massaged), they just never cared about it.

The GCC PCH files use a special 4-byte prefix in every file; all you
have to do in clang is, if you find a .gch file open the file and read
the first 4 bytes and if it's a real GCC PCH file you ignore it and if
it's actually a Clang PCH with a malformed name you complain bitterly
and dump core er, I mean, you read it silently as if it had the
right name.

One would hope that, if the GCC module files have a similar compiler-
specific format (I'm not too familiar with modules) they also use a
similar magic number at the beginning of the file.

But anyway this is losing the thread of Eli's hopeful request.

Re: LSP based on GCC

[Paul Smith]

On Wed, 2023-05-17 at 17:28 +0300, Eli Zaretskii via Gcc wrote:
> If there are no current plans for implementing LSP, I hope someone
> will work on that soon, given that Emacs can now use it, and because
> having a GCC-based LSP implementation will allow people to use their
> installed GCC as the basis for LSP features, instead of having to
> install yet another compiler.

Just to note, existing LSP servers are stand-alone binaries: you don't
need to install the compiler.  The two main C/C++ LSP servers I'm aware
of are clangd and ccls.  Both of them are built from libclang, as you
suggest, but you don't need a full clang installation to get them.

You do need to install some Clang "resource" header files for
intrinsics etc. but not the entire compiler (for example you don't need
the STL or anything like that: they do work fine with the GCC STL).

Nevertheless I wholeheartedly agree with your hopes Eli because using
Clang-based LSP is annoying for people developing with GCC:

First, when you're viewing code that is using #ifdefs to choose between
compilers you always see the code for Clang as "active", even though
you're using GCC as the compiler, since it's using the Clang built-in
macros.

More frustratingly, Clang has made some poor decisions around
"compatibility": they tried to leverage the GNU ecosystem by emulating
GCC features and arguments but sometimes break things.  The most
egregious example I'm aware of is that they look for GCC-named
precompiled headers (.gch), even though the Clang PCH format is
completely different.  So if Clang (and the LSP servers built on it)
find a .gch header file they will try to read it, fail, and give an
error.  I filed a bug about this in 2019 but it's been ignored.

This means you need to modify your LSP server arguments to omit any PCH
compiler command line arguments; for environments based on auto-
generated definitions like compile_commands.json this is frustrating.

Re: GCC 12.3.0 regression, ccache 4.8.0 build fails witch ICE, upstream fix identified

[Paul Smith]

FYI I reported this crash to the GCC bugzilla yesterday:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109850

and it was closed as a dup of this:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109241

If people want to watch it.


On Sun, 2023-05-14 at 10:50 +, Thomas Backlund via Gcc wrote:
> ccache 4.8.0 build fails with ICE when building with gcc 12.3.0:
> https://bugs.mageia.org/show_bug.cgi?id=31893#c0
> regression introduced in gcc-12-20230421 snapshot by commit:
> 
> From 890d711a2477119a34cf435f6159b6253b124374 Mon Sep 17 00:00:00
> 2001
> From: Jason Merrill <
> ja...@redhat.com
> > 
> Date: Thu, 16 Mar 2023 15:11:25 -0400
> Subject: [PATCH 35/61] c++: generic lambda, local class, __func__
> [PR108242]
> 
> I fixed  gcc ICE by backporting from upstream master branch:
> 
> From 4872e46e080c6695dfe1f9dc9db26b4703bc348c Mon Sep 17 00:00:00
> 2001
> From: Jason Merrill <
> ja...@redhat.com
> > 
> Date: Wed, 22 Mar 2023 16:11:47 -0400
> Subject: [PATCH] c++: local class in nested generic lambda [PR109241]
> 
> --
> Thomas

GCC ASAN breaks glob()?

[Paul Smith]

OK here's something super-strange I discovered:

Enabling -faddress=sanitize in GCC, causes the glob(3) function to
misbehave.

I'm using GCC 11.3 / glibc 2.35 (x86_64 native).  I have this simple
program:

$ cat /tmp/tstglob.c
#include 
#include 

int main(int argc, char *argv[])
{
glob_t gl = {0};
int res = glob(argv[1], 0, NULL, &gl);

switch (res)
{
case 0: printf("success\n"); break;
case GLOB_NOMATCH: printf("no match\n"); break;
default: printf("unknown: %d\n", res); break;
}

return 0;
}

Now I create a symlink that doesn't point to anything:

  $ ln -s nosuchfile /tmp/badlink
  $ ls -al /tmp/badlink
  lrwxrwxrwx 1 pds pds 10 Mar 26 14:52 /tmp/badlink -> nosuchfile

Now I compile the above program normally and run it:

  $ gcc -o /tmp/tstglob /tmp/tstglob.c
  $ /tmp/tstglob /tmp/badlink
  success

This is what I expect: the symlink does exist even though it doesn't
point to anything so glob() should return it.

But now if I compile with ASAN:

  $ gcc -fsanitize=address -o /tmp/tstglob /tmp/tstglob.c
  $ /tmp/tstglob /tmp/badlink
  no match

...?!?!?!

Is there something in the ASAN library that takes over glob(3) and
installs a different version (there have been plenty of versions of
glob(3) over the years in glibc which behave incorrectly when faced
with broken symlinks, heavens knows...) that overrides the glibc
version?

Or...??

Re: Debugging C++ frontend using CLion IDE

[Paul Smith]

On Wed, 2023-03-01 at 20:59 +0300, Berke Yavas via Gcc wrote:
> One thing I haven't figured out yet, how can I debug C++ frontend(or
> any other language frontend) with CLion. If anybody managed to do
> this (or using another IDE), could you please share your settings
> with me?

Since CLion is (a) not free (by either definition), and (b) requires
projects to use CMake (at least it did last time I checked), it's
highly unlikely you'll find too many GNU projects that have any
information about how to configure it.

I'm not sure what IDEs other devs use: probably vim or Emacs mostly.

Re: Why does filing a bug report have to be so damn hard?

[Paul Smith]

What is going on out there these days?  I've added more addresses from
the GCC mailing list to my killfile in the last week than in the
previous two years combined.

Yeesh.

Re: Contribution

[Paul Smith]

On Wed, 2022-12-14 at 17:32 +0200, Mohamed Atef via Gcc wrote:
> Hello all,
>    As I mentioned before, I don't have much time to contribute.
> So is there any way I can contribute some medium project I can work
> on but myself in the long run? I loved this community, I contributed
> before so can you suggest some work for me?

I don't know if it's on the EasyHacks list but there's this bug with a
patch I filed years ago, that has been asked for multiple times since,
but I've never had the bandwidth to polish and submit it and it looks
like I never will.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78147

Re: Good grief Charlie Brown

[Paul Smith]

On Tue, 2022-12-13 at 11:35 -0600, Dave Blanchard wrote:
> *angry, grumpy, pissed off, GNU-hating distro maintainer enters the
> chat*
> nobody on the mailing list has any idea what it could possibly be I
> guess, since nobody responded to my email.

Yes, I can't imagine any other reason no one has responded to your
emails.

Re: Good grief Charlie Brown

[Paul Smith]

On Tue, 2022-12-13 at 11:28 -0500, Jim Anderson via Gcc wrote:
> 1) I could not find any place to download the man page. I lost my 
> internet connect and I have not man page :(

You didn't say what platform you're working on, nor how you obtained
the GCC that you are using, but recall that the GCC team doesn't
publish already-built binary packages.  If you built your own version
of GCC from source, then all the documentation was contained with the
source code.

If you didn't build it yourself and instead installed a pre-built
version, then you need to discuss this with the group that provided
that pre-built version: the GCC development team can't help.  Every
binary package of GCC I've ever heard of comes with at least the man
page (some distributions bundle the full documentation separately from
the compiler but they all include the man page with the compiler).

Did you try just running "man gcc" (assuming you are on a POSIX-type
system)?

> 2) How over pervasive gcc has become. It appears to be everything for
> everybody for every environment possible.
> 
> I guess this is a good think, but I personally think not!

I guess if you were one of the "everybody" needing "everything" on
"every" environment, instead of you needing what you need in your
environment, you might think differently!

Re: GNU = Junkware

[Paul Smith]

On Sat, 2022-11-26 at 03:20 -0600, Dave Blanchard wrote:
> GNU GCC/GLIBC IS THE MOST PAINFUL PIECE OF SHIT FUCKING SOFTWARE ON
> THE PLANET TO BOOTSTRAP.

Seems like a good time to remember the old adage, if you think everyone
else is the asshole then you should consider that it's really you who's
the asshole.  If it's taken a full year and you're not done yet, maybe
you're just not suited to this type of work.  I'm not trolling either:
it's not for everyone and that's fine.  There's lots of things to do
and you'd probably be happier doing something else.

> Seemingly the only way that will ever finally get this junk working
> is to just sit there randomly trying different stuff in the hope that
> it somehow moves one closer toward getting something that works
> right.

Yes, I believe you've proved my guess correct here.

Re: Toolchain Infrastructure project statement of support

[Paul Smith]

On Tue, 2022-10-18 at 14:14 -0400, Siddhesh Poyarekar wrote:
> On 2022-10-18 14:13, Siddhesh Poyarekar wrote:
> > only Job Corbet's questions to Carlos/David are pending an answer;
> 
> s/Job/Jon/ sorry about misspelling your name.

I thought it was great!  We all have known for years that Jon has the
requisite patience for that role...

[PATCH 2/2] libstdc++: Use template form for pretty-printing tuple elements

[Paul Smith via Gcc]

std::tuple elements are retrieved via std::get<> (template) not
[] (array); have the generated output string match this.

libstdc++-v3/ChangeLog:

* python/libstdcxx/v6/printers.py (StdTuplePrinter): Use <> not [].
---
The previous patch seems uncontroversial to me.  I don't know about this one:
I'm not sure if there's any precedent for this type of output although to me
it looks better since tuples cannot be retrieved via array indexing.

 libstdc++-v3/python/libstdcxx/v6/printers.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libstdc++-v3/python/libstdcxx/v6/printers.py 
b/libstdc++-v3/python/libstdcxx/v6/printers.py
index 14a6d998690..0063a3185a6 100644
--- a/libstdc++-v3/python/libstdcxx/v6/printers.py
+++ b/libstdc++-v3/python/libstdcxx/v6/printers.py
@@ -567,7 +567,7 @@ class StdTuplePrinter:
 if len (fields) > 0 and fields[0].name == "_M_head_impl":
 impl = impl['_M_head_impl']

-out = '[%d]' % self.count
+out = '<%d>' % self.count
 self.count = self.count + 1

 return (out, impl)
--
2.28.0

[PATCH 1/2] libstdc++: Count pretty-printed tuple elements from 0 not 1

[Paul Smith via Gcc]

Show 0-based offsets for std::tuple members, to match with std::get.

libstdc++-v3/ChangeLog:

* python/libstdcxx/v6/printers.py (StdTuplePrinter): don't increment
self.count until after generating the result string.
---
 libstdc++-v3/python/libstdcxx/v6/printers.py | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/libstdc++-v3/python/libstdcxx/v6/printers.py 
b/libstdc++-v3/python/libstdcxx/v6/printers.py
index 550e0ecdd22..14a6d998690 100644
--- a/libstdc++-v3/python/libstdcxx/v6/printers.py
+++ b/libstdc++-v3/python/libstdcxx/v6/printers.py
@@ -560,16 +560,17 @@ class StdTuplePrinter:
 # Process left node and set it as head.
 self.head  = self.head.cast (nodes[0].type)

-self.count = self.count + 1
-
 # Finally, check the implementation.  If it is
 # wrapped in _M_head_impl return that, otherwise return
 # the value "as is".
 fields = impl.type.fields ()
-if len (fields) < 1 or fields[0].name != "_M_head_impl":
-return ('[%d]' % self.count, impl)
-else:
-return ('[%d]' % self.count, impl['_M_head_impl'])
+if len (fields) > 0 and fields[0].name == "_M_head_impl":
+impl = impl['_M_head_impl']
+
+out = '[%d]' % self.count
+self.count = self.count + 1
+
+return (out, impl)

 def __init__ (self, typename, val):
 self.typename = strip_versioned_namespace(typename)
--
2.28.0

Re: GCC 10.2: undefined reference to vtable: missing its key function

[Paul Smith]

On Sun, 2021-06-06 at 17:41 -0400, Paul Smith wrote:
> How can I figure out why the compiler decides, while compiling
> foo.cxx, that MyClass is virtual and needs a vtable when as far as I
> can tell it's not (if I use nm on MyClass.o I see no hints of vtable
> etc.)

Err... I think this might have been a problem caused by ccache :(.

Not sure how but cannot reproduce after I deleted my cache.  Sigh.

GCC 10.2: undefined reference to vtable: missing its key function

[Paul Smith]

I have a class which is NOT, as far as I can see, polymorphic.

It doesn't inherit from any other class and none of its methods are
declared virtual.  The class implementation and all its callers all
compile just fine.

Is there some other way that a class can be thought to be virtual,
stealthily (say by its usage in some other class or something)?

The problem is that when I link this class I get a linker error from
gold:

  MyClass.h:25: error: undefined reference to 'vtable for MyClass'
ld: the vtable symbol may be undefined because the class is missing its
key function

Line 25 is the first method in the class (a constructor).

Sure enough if I use nm -C on the other object files I'm linking, I see
things like this:

   foo.o:  U vtable for MyClass

How can I figure out why the compiler decides, while compiling foo.cxx,
that MyClass is virtual and needs a vtable when as far as I can tell
it's not (if I use nm on MyClass.o I see no hints of vtable etc.)

This is with GCC 10.2 / binutils 2.35.1 (x86_64)

Re: Update to GCC copyright assignment policy

[Paul Smith]

On Tue, 2021-06-01 at 11:50 -0400, David Edelsohn via Gcc wrote:
> The current, active license in GPL v3.0.  This is not an announcement
> of any change in license.
> 
> Quoting Jason Merrill:
> 
> "GCC's license is "GPL version 3 or later", so if there ever needed
> to be a GPL v4, we could move to it without needing permission from
> anyone."

It depends on what you mean by "move to it".

It's true that anyone could redistribute it under GPLv4.

What's not true is that you can *change the license*.  For example you
can't change the current wording of the license from "GPL version 3 or
later" to "GPL version 4 or later".  Or make any other changes to the
license, without collecting approval from all copyright holders.

So, if there should be some issue with GPLv3 so that you really want to
stop using it (maybe a court case is decided which negates a
significant element of GPLv3 and GPLv4 is released to address the
issue), it won't be possible to do that easily.

Someone else mentioned that new code could be released only under that
license so that in effect the entirety of the codebase becomes GPLv4+.

I'm not sure about that.  Doing that for brand new files that were
created solely by one person who wanted to use GPLv4 or later only
would work I suppose.  But adding changes to an existing file that was
GPLv3+ and saying that these new changes are GPLv4+ would be pretty
gross.  You might have to list both licenses in these files, since you
can't change the previously-in-use license unless you get agreement
from the FSF, who currently holds the license, plus anyone else who
changed the file since the assignment was relaxed.

Personally I think that while assignment is a PITA and I wish it were
easier, it is extremely valuable and provides a lot of flexibility, and
shouldn't be abandoned without very, VERY careful consideration.

And, that decision and those considerations should be documented and
the responses to the issues raised here published for everyone to see.

Re: Has FSF stopped processing copyright paperwork

[Paul Smith]

On Mon, 2021-04-26 at 21:01 +, Romain GEISSLER via Gcc wrote:
> So anyone has some insider information about the FSF copright
> assignment process ?

I have no insider information but you can try asking on
gnu-prog-disc...@gnu.org (I think that list is open)

I assume you're communicating with copyright-cl...@fsf.org ?

Re: static inline giving multiple definition errors with "unity" builds

[Paul Smith]

On Sun, 2020-10-04 at 03:36 -0400, Paul Smith wrote:
> I have a templated class C that required its type T to have operator
> bool() defined in order to work properly.

Never mind, I think there was some local error where things were not
being recompiled when they should be.  I don't know why that might be
but a full clean/rebuild fixed it.  I've never had this problem
before... so odd.

Sorry for the noise!

Re: static inline giving multiple definition errors with "unity" builds

[Paul Smith]

On Sun, 2020-10-04 at 03:36 -0400, Paul Smith wrote:
> But!!  When I nm the individual cxx.o files none of these operator
> symbols appear, at all!

As you might have expected, this turns out to be wrong: the symbols
were just in a set of object files that I did not expect.  I'll see if there's 
anything "unusual" about those source files although I'm hard-pressed to think 
of what it might be.

Re: static inline giving multiple definition errors with "unity" builds

[Paul Smith]

On Sun, 2020-10-04 at 03:36 -0400, Paul Smith wrote:
> I made it static inline like this:
> 
>   static inline bool operator!(T t) { return t != T::EMPTY; }

Obviously in my real code I used t == T::EMPTY here... :-/

static inline giving multiple definition errors with "unity" builds

[Paul Smith]

I'm using GCC 8.1.0 / binutils 2.30 (I plan to upgrade to latest stuff
in the next few weeks or so...), on GNU/Linux Intel 64bit.

I have a templated class C that required its type T to have operator
bool() defined in order to work properly.

I want to use that with a enum class so clearly that doesn't work for
me.  However I had an idea: I changed the implementation of the
templated class so that instead of "if (t)" it uses "if (!!t)" then I
created an operator!(T).

I made it static inline like this:

  static inline bool operator!(T t) { return t != T::EMPTY; }

When I compile "normally" this all works fantastically.

But, when I compile using a "unity" build, I'm getting multiple
definition errors:

  ld: error: libFoo.a(unity_2.cxx.o): multiple definition of 'operator!(T)'
  ld: libFoo.a(unity_12.cxx.o): previous definition here

If you're not familiar with "unity" builds basically the build system
will generate a meta-source file (above called things like unity_2.cxx
or unity_12.cxx etc.) and compile that: the only thing in the meta-
source file are includes of other source files:

  #include "File1.cpp"
  #include "File2.cpp"
...

The goal is to improve compile times (at the expense of compiling more
stuff when something changes obviously).

I don't understand why I'm getting multiple definition errors.

And it gets even stranger: when I nm my library, sure enough I see:

  libFoo.a:unity_12.cxx.o:3ef0 T operator!(T)
  libFoo.a:unity_9.cxx.o:3190 T operator!(T)
  libFoo.a:unity_8.cxx.o:3640 T operator!(T)
  libFoo.a:unity_2.cxx.o:5d80 T operator!(T)

But!!  When I nm the individual cxx.o files none of these operator
symbols appear, at all!

It's like they were created by the ar command, which is just:

  ar qc libFoo.a 

Which seems bizarre to me.  Is this a bug in ... something ... either
the compiler or binutils?  Or, is my trick above not valid for some
reason?

Re: Problem with 64-bit only compiler build

[Paul Smith]

On Wed, 2020-08-12 at 15:05 -0500, Segher Boessenkool wrote:
> > As usual I've built my own version of GCC, and then I check it into
> > Git so that all builds can use this one canonical compiler
> > regardless of operating system, etc.
> 
> There's your problem.  Git is not an archiver.  Git does not track 
> directories at all, including empty directories.  Git is a "stupid 
> content tracker", and empty directories are contentless ;-)

Yes, I'm aware of Git's shortcomings in this area but... the advantages
of using it over other alternatives are simply too great to overcome,
at least in my environment.  It provides branching (which I need, since
older versions of our software need to build with older versions of
these tools), it allows for tracking of changes, reversion, it provides
a single interface across all platforms including multiple GNU/Linux
distributions, Windows, and MacOS (and possibly others in the future),
without users having to learn new tools.  And it's available both
online and offline for developers working remotely, which is critical.

Yes, it's not as efficient as it could be for binary files, and it has
this "issue" with empty directories (which is not really a big deal,
most of the time).  But it supports symbolic links, tracks executable
permissions, handles moved files and created/deleted directories, etc.
 Also, an installation of a package like GCC has some binary files,
yes, but it also has a LOT of text files.

Combined with worktrees so we can share the same repo across different
builds on the same system, to reduce the overhead of cloning, it works
very well for the most part.

At some point this may become too unwieldy and I'll have to look more
carefully into git-lfs or similar.  But that day has not arrived.

> > Of course, I can force the lib directory to exist by adding a dummy
> > file there and I will do so.  However, it seems to me that it would
> > be better if GCC would apply a simplifying function to pathnames to
> > reduce all the "/.." elements, before it starts looking for
> > directories, to avoid failing if some unnecessary element doesn't
> > exist.
> 
> It would be nice if this could be done (where possible -- see Jakub's
> reply) before install already, so people looking at your install do
> not have to look at ../../../../.. and remove five path components by
> hand. There shouldn't be a problem doing it at file open time as you
> say (unless we wand to support users putting .. in the paths
> themselves, instead of just our make system doing that :-) )

I confess I didn't really follow all of that :)

I'll file a bug tonight/tomorrow, then possible solutions can be
discussed there.

Re: Problem with 64-bit only compiler build

[Paul Smith]

On Wed, 2020-08-12 at 16:53 +0100, Jonathan Wakely wrote:
> On Wed, 12 Aug 2020 at 14:33, Paul Smith 
> wrote:
> 
> > I'm not talking about PREFIX/lib, though.  As can be seen from my
> > question I'm talking about PREFIX///lib.  This is
> > where GCC keeps its own internal libraries,
> 
> Not by default, it isn't. I'm not sure what directory that is, but
> none of my builds have it.
> 
> Is this a cross-compiler? Mine have PREFIX//lib instead, and
> it's not empty (for a 64-bit --disable-multlib build)

Sorry, you're right, the  above is part of the PREFIX I
provided; my mistake.  The path is indeed PREFIX//lib.

However, the lib directory is empty in my build.  What lives in your
version of lib?

  $ cd 

  $ ls -1
  bin/
  include/
  lib/
  lib64/
  libexec/
  share/
  x86_64-unknown-linux-gnu/

  $ ls -1 x86_64-unknown-linux-gnu/
  bin/
  include/
  lib/
  lib64/

  $ ls -1 x86_64-unknown-linux-gnu/lib/

  $

I am invoking configure like this (with some flags etc. removed for
simplicity):

  .../gcc-10.2.0/configure \
  --prefix= --with-local-prefix= \
  --target=x86_64-unknown-linux-gnu --with-sysroot= \
  --enable-languages=c,c++ --disable-nls --disable-werror \
  --disable-multilib --disable-libgomp --disable-libvtv \
  --disable-libstdcxx-dual-abi --with-build-config=bootstrap-lto \
  --enable-bootstrap --with-gnu-as --with-gnu-ld --enable-gold

so yes, it's a "cross compiler" because I'm setting --target even
though the target is really the same architecture.  I want to ensure
that the entire build both of GCC, and also the binaries built with
this GCC, uses only the sysroot and not any local content.

Maybe that's overkill, I dunno... it works! :)

Re: Problem with 64-bit only compiler build

[Paul Smith]

On Wed, 2020-08-12 at 15:37 +0200, Jakub Jelinek wrote:
> The important thing is that GCC wants to be relocatable, so most
> paths are not hardcoded into the compiler, but depend on where the
> gcc driver actually is.  One can then just move the whole gcc tree
> somewhere else and it should still work.

For sure!  I take full advantage of this: as mentioned I check GCC into
Git and users are free to clone that repo into whatever directory they
want, anywhere on their filesystem, and it works.

If we consider the internal paths GCC computes to be constructed in two
parts: / then the part that I'm
talking about is the .

> > It appears that what happens is that the frontend computes the path
> > to the "lib" (32bit) directory, then when it wants to look for
> > 64bit libraries instead of re-computing the path from scratch with
> > "lib64", it just adds on a "../lib64" to the 32bit path.
> 
> That is because that is the way OS multilib directories are
> specified, it is defined by adding `gcc -print-multi-os-directory
> $CFLAGS` to the normal paths.

Well, perhaps the right thing to do is simplify the results of this
without requiring that the original exists.

Re: Problem with 64-bit only compiler build

[Paul Smith]

On Wed, 2020-08-12 at 15:08 +0200, Jakub Jelinek wrote:
> > I think it's worth adding this to bugzilla. Depending on the
> > existence of empty directories seems less than ideal.
> 
> But canonicalizing the paths without taking the filesystem state into
> account will significantly change the behavior in some setups (e.g.
> in presence of symlinks) and I'm not sure it is desirable.

You're right, about paths that the user provides on the command line.
 Those should not be modified.  However, I'm talking about paths that
GCC computes on its own.  These are not paths taken from GCC's command
line, they are paths computed by the front-end from scratch to be
passed to collect2.  I agree it would need to be considered carefully.

As someone who takes a lot of advantage of the flexibility that tools
like GCC provide I'm wary of reducing that flexibility.  On the other
hand I'm not sure that breaking up the internal structure of GCC's
installation via symlinks in random ways is something that needs to be
supported.  Anyway, the user can always replace the "lib64" directory
with a symlink as well.

> The lib paths usually are not empty, the typical installation is
> together with other programs/libraries.  And even the PREFIX/lib path
> should have gcc subdirectory with it with lots of files in there.

I'm not talking about PREFIX/lib, though.

As can be seen from my question I'm talking about
PREFIX///lib.  This is where GCC keeps its own
internal libraries, and that directory will always be empty if you
don't build a 32bit compiler.  I don't think it's common at all for
people to be dropping non-GCC libraries into these internal GCC
directories (just from a packaging standpoint it would be a nightmare
to maintain).

It appears that what happens is that the frontend computes the path to
the "lib" (32bit) directory, then when it wants to look for 64bit
libraries instead of re-computing the path from scratch with "lib64",
it just adds on a "../lib64" to the 32bit path.

So maybe the right answer is to build the lib64 path using the same
algorithm used to build the "lib" path, except with "lib64", rather
than adding a "../lib64" to the end of the "lib" path.

I'll be the first to admit I've not looked at the frontend code at all
so I could be completely wrong about how it works.

Problem with 64-bit only compiler build

[Paul Smith]

This is a kind of esoteric problem, but all the more annoying for that.

As usual I've built my own version of GCC, and then I check it into Git
so that all builds can use this one canonical compiler regardless of
operating system, etc.

After being checked into Git, the compiler started failing to link with
an error that it could not find libgcc_s.so, even though it exists in
exactly the same place (relatively) as it was before.

After some debugging and strace-ing, I discovered that in the failure
mode the front-end didn't provide this flag to collect2, which is where
libgcc_s.so lives:

-L/.../cc/unknown/bin/../lib/gcc/x86_64-unknown-linux-gnu/10.2.0/../../../../x86_64-unknown-linux-gnu/lib/../lib64/.

The problem is this: when I built this compiler I only built the 64bit
version of it, not the 32bit version.  As a result, the lib/.
directories are empty: only the lib64 directories have any content.

When I check this into Git, of course all empty directories are
ignored.  So while the /.../cc/unknown/x86_64-unknown-linux-gnu/lib64
directory, where 64bit libgcc_s.so lives, does exist, the
/.../cc/unknown/x86_64-unknown-linux-gnu/lib directory doesn't exist,
which means the above path cannot be resolved.

Of course, I can force the lib directory to exist by adding a dummy
file there and I will do so.  However, it seems to me that it would be
better if GCC would apply a simplifying function to pathnames to reduce
all the "/.." elements, before it starts looking for directories, to
avoid failing if some unnecessary element doesn't exist.  This
shouldn't be too difficult since the path will be either the same
length as, or shorter than, the original so there's no need to
reallocate memory etc.: it can be updated in place.

Oddly, I looked through the gnulib library and didn't find any
appropriate module for this.  It seems like there should be one.


Anyway, not sure what people think and whether this is worth an
enhancement request in bugzilla.

Re: GCC 10.2: Spurious(?) stringop-overflow warning (not strlen/strcpy/etc.)

[Paul Smith]

On Mon, 2020-08-03 at 17:02 -0600, Martin Sebor wrote:
> If the code is designed to treat Node sort of like a struct with
> a flexible array member I would suggest to make that explicit by
> adding a zero-element array member to Node and using it to access
> other memory.  E.g., add:
> 
>  unsigned char data[0];
> 
> as the last member of Node and change getPayload to:
> 
>  PAYLOAD& getPayload() {
>  return *(reinterpret_cast(data) + 
> Node::actualSize(_keyLength, alignof(PAYLOAD)));
>  }

Thanks Martin; I suspected it was something like that.  However, I
haven't been able to get a workaround to work.

My understanding is that you suggest replacing:

struct {} key;

with:

unsigned char key[0];

in Node.  And I think you have a paren in the wrong place in the
getPayLoad() example above; shouldn't it be:

return *(reinterpret_cast(data + 
Node::actualSize(_keyLength, alignof(PAYLOAD;

(that is, the size should be added to data then the whole thing cast).

However, even trying this I still get the same failures:

  stringop.cpp: In function 'void applyTo(LeafNode*)':
  stringop.cpp:37:33: error: writing 1 byte into a region of size 0 
[-Werror=stringop-overflow=]
 37 | void markUpdate() { flags() |= UPDATE; }
| ^
  stringop.cpp:15:19: note: at offset 0 to object 'Node::key' with size 0 
declared here
 15 | unsigned char key[0];
|   ^~~
  cc1plus: all warnings being treated as errors

Were you able to get this working and I just am not understanding what
you mean?

Cheers!

GCC 10.2: Spurious(?) stringop-overflow warning (not strlen/strcpy/etc.)

[Paul Smith]

I'm testing upgrading from GCC 9.3 to 10.2 and I'm seeing this new
warning:

  $ g++ --version
  x86_64-unknown-linux-gnu-g++ (GCC) 10.2.0
  ...

  $ g++ -Wall -Werror -O2 -c -o stringop.o stringop.cpp

  In member function 'void LeafNode::markUpdate()',
  inlined from 'void applyTo(LeafNode*)' at stringop.cpp:45:21:
  stringop.cpp:37:33: error: writing 1 byte into a region of size 0 
[-Werror=stringop-overflow=]
 37 | void markUpdate() { flags() |= UPDATE; }
| ^
  stringop.cpp: In function 'void applyTo(LeafNode*)':
  stringop.cpp:34:7: note: at offset 0 to object 'LeafNode::' with 
size 4 declared here
 34 | class LeafNode : public NodeWithPayload
|   ^~~~
  cc1plus: all warnings being treated as errors

It could well be that this code is dodgy; I take no responsibility for
it :).  It's part of a very tricky and performance- and memory-
sensitive area of the software.

I've stripped out tons of stuff and gotten it down to this, which is
still probably not the minimal test case but is pretty small.  If, for
example, I take out the _sequence field in LeafNodePayload, it doesn't
give any warnings.

The repro case is provided below.  If people think this is actually a
bug I can file an issue.

-
nclude 
#include 

constexpr unsigned char UPDATE = 0x4;

class Node
{
protected:
static size_t actualSize(uint16_t keyLength, size_t alignment) {
return alignment * (((offsetof(Node, key) + keyLength - 1) / alignment) 
+ 1);
}

public:
const uint16_t _keyLength;
struct {} key;
};

template
class NodeWithPayload : public Node
{
protected:
PAYLOAD& getPayload() {
return *(reinterpret_cast(reinterpret_cast(this) + Node::actualSize(_keyLength, alignof(PAYLOAD;
}
};

class LeafNodePayload
{
public:
uint64_t _sequence;
unsigned char _flags;
};

class LeafNode : public NodeWithPayload
{
public:
void markUpdate() { flags() |= UPDATE; }

private:
unsigned char& flags() { return getPayload()._flags; }
};

void applyTo(LeafNode* node)
{
node->markUpdate();
}

Compilation speed notes

[Paul Smith]

Hi all; thanks for the help recently with sysroot compilers.  I was
able to get things building following advice from Joseph and Jonathan.

I did want to mention that I noticed that GCC got non-trivially slower
between my previous version (8.1) and this new version (9.3).

I have a decently-sized C++14 or so code base, configured to use pre-
compiled headers and unity builds (we pass 20 .cpp files to each
invocation of the compiler).  The system is GNU/Linux with 8 cores and
32G memory, on an older but still usable Intel Xeon E3-1270 3.5GHz and
I use "make -j8" to build it.

I built it 5 times in a row (although I can't believe caching would be
an issue with code this size) with a git clean before each build, and
no ccache or anything.

The GCC 8.1 builds took about 14m35s.

The GCC 9.3 builds took about 16m40s, so that's a drop of about 13%.

I used the same binutils for both.  Both compiles used the same compile
options of course; in particular they used -ggdb3 -O2.

I did discover that if I build GCC 9.3 with profiledbootstrap-lean, I
gained back that loss and got the compile times back down to almost the
same as GCC 8.1... but I assume that if I'd built GCC 8.1 with
profiledbootstrap then that would have been even faster.

Note I haven't tried to compare the performance of the resulting code
(yet...)  This is just looking at compile speed.


Anyway I know that detail-free messages like this are more annoying
than helpful and I don't mean to be annoying.  If anyone wants to ask
for more details or for me to do more testing I'm amenable.  Otherwise
this is just one data point to consider, or not.

Thanks for the new release!

Re: Compiling GCC using an older sysroot

[Paul Smith]

On Wed, 2020-03-11 at 17:15 +, Jonathan Wakely wrote:
> My thinking was to build GCC inside the container, then package up the
> results and install that on your dev machines (outside a container).
> But actually that won't work because you'd also need to package the
> glibc from the container, and at that point you're basically back to
> using a sysroot.

Yes.  It's actually much simpler to use RPMs for this, and far more
accurate.  I have a vague memory of trying to extract the necessary files
from a live system, sometime in the past.  Hopefully I only attempted this
for a few minutes before realizing that getting the RPMs was a far more
reliable and efficient way to generate a sysroot :).

Thanks!

Re: Compiling GCC using an older sysroot

[Paul Smith]

On Wed, 2020-03-11 at 14:17 +, Jonathan Wakely wrote:
> On Mon, 9 Mar 2020 at 21:56, Paul Smith wrote:
> > The tricky bit is that although both the host and target are always
> > x86_64/i686 GNU/Linux systems, I need the generated compiler to run on
> > much older systems than the one I build it on.
> 
> I suggest using containers for this. Once you've got a working
> Dockerfile it makes life much easier in my experience.
> 
> The podman and buildah utilities are great alternatives to docker itself.
> 
> By building in a container using something like "FROM centos:6" you
> know you're using those packages, not trying to convince GCC to use a
> cobbled-together sysroot based on old RPMs.

That's a good idea, certainly for building the compiler: it would save me
having to build the initial compiler ("step 1" in Joseph's email): I can
skip that step and just build the target compiler in a container.

Unfortunately currently I target RHEL/CentOS 6.5 and there's no official
docker image for that, but that's not a major problem.

However I don't think, at least for now, I can switch to having developers
build in a docker container.  Getting that integrated with all the
tooling/IDE environments everyone uses, etc. will be a lot of change. 
Since I want to use a much newer version of GCC than was provided in these
older releases, a docker image won't save me having to build the compiler
anyway; it would only change the way the compiler is distributed and used
(inside a docker image/container vs. stand-alone with a sysroot).

Luckily for me our product only relies on a handful of system libraries and
I have a simple script that will extract them from RPMs via cpio.  It's
actually quite easy and robust to create the sysroot and the entire thing
is only 16M.

Re: Compiling GCC using an older sysroot

[Paul Smith]

On Mon, 2020-03-09 at 22:01 +, Joseph Myers wrote:
> On Mon, 9 Mar 2020, Paul Smith wrote:
> > I have a sysroot I've created (downloading RPMs from older systems and
> > unpacking them) which is sufficient to build GCC (and binutils etc.)  I
> > need the GCC binaries I create to be compiled using this sysroot so
> > that they can run on older systems.
> > 
> > It seems that options like --with-sysroot and --with-build-sysroot are
> > more about how the compiler builds other things and less about how the
> > compiler itself is built.
> 
> Yes.  The natural way to do this sort of thing is:
> 
> 1. Build a cross compiler (GCC and binutils) that's configured using 
> --with-sysroot to make use of the old-system sysroot.
> 
> 2. Use the compiler you built in the previous step whenever you want to 
> build binaries (of GCC or anything else) that will run on older systems.

Heh.  That's sort of what my makefile was already doing (except for
some reason I was building GCC 3 times not twice... I can't remember
why.  Some kind of canadian-cross; probably I just got lost in the
weeds somewhere all those years ago... or maybe back then the
sysroot options were not as complete/capable as they are now).

Reducing the build to two steps would help me but I only do those once
a year or so, so that's not so important.  Mainly I want to build a
profiled GCC, to reduce compile times.

OK, I'll give this a whirl and see how it goes; thanks Joseph!

As you astutely surmise, the resulting output needs to be relocatable
but not just once: the entire toolchain is checked into a Git
repository and I have no control over the path it's checked out into on
a given system.  So I don't think --with-build-sysroot suffices.

What I do is use shell script wrappers around the compiler which sets
up the correct sysroot flags etc. based on the invocation path then
calls the real binaries.  I'm happy to continue doing that.

I do also use -static-libgcc / -static-libstdc++ as you suggest.

Compiling GCC using an older sysroot

[Paul Smith]

I have a somewhat complex makefile that I've been using for many years
to build GCC: it builds tools needed (make, bison, flex, m4, binutils),
downloads the source prerequisites and links them, etc.

I'd like some advice, hopefully an easy answer, that allows me to
simplify that system, which currently does a psuedo-cross-compile,
involving rebuilding GCC a few times.  Also I'd like to build GCC
profiled, which doesn't seem to work with my makefile system.

The tricky bit is that although both the host and target are always
x86_64/i686 GNU/Linux systems, I need the generated compiler to run on
much older systems than the one I build it on.

I have a sysroot I've created (downloading RPMs from older systems and
unpacking them) which is sufficient to build GCC (and binutils etc.)  I
need the GCC binaries I create to be compiled using this sysroot so
that they can run on older systems.

It seems that options like --with-sysroot and --with-build-sysroot are
more about how the compiler builds other things and less about how the
compiler itself is built.

Should it be sufficient to invoke configure with --sysroot options for
the compiler used to build GCC, like this?

   $ cd /obj

   $ ../gcc-9.2.0/configure CFLAGS=--sysroot=/sysroot \
CXXFLAGS=--sysroot=/sysroot \
LDFLAGS=--sysroot=/sysroot \
--disable-multilib --enable-gold \
--with-build-config=bootstrap-lto

When I try this I get a failure trying to create libstdc++.  I describe
it below, but I wonder if I'm just chasing down the wrong path
altogether here?

The failure:

/bin/bash ../libtool --tag CXX   --mode=link /obj/./gcc/xgcc -shared-
libgcc -B/obj/./gcc -nostdinc++ -L/obj/x86_64-pc-linux-gnu/libstdc++-
v3/src -L/obj/x86_64-pc-linux-gnu/libstdc++-v3/src/.libs -L/obj/x86_64-
pc-linux-gnu/libstdc++-v3/libsupc++/.libs -B/dist/x86_64-pc-linux-
gnu/bin/ -B/dist/x86_64-pc-linux-gnu/lib/ -isystem /dist/x86_64-pc-
linux-gnu/include -isystem /dist/x86_64-pc-linux-gnu/sys-include   -
fno-checking  -Wl,-O1 -Wl,-z,relro -Wl,--gc-sections  -std=gnu++98
-fPIC -DPIC -fno-implicit-templates  -Wall -Wextra -Wwrite-strings
-Wcast-qual -Wabi=2  -fdiagnostics-show-location=once   -ffunction-
sections -fdata-sections  -frandom-seed=libstdc++.la  -o libstdc++.la
-version-info 6:27:0 -Wl,--version-script=libstdc++-symbols.ver -lm
-rpath /dist/lib/../lib64 compatibility.lo compatibility-debug_list.lo
compatibility-debug_list-2.lo  compatibility-c++0x.lo compatibility-
atomic-c++0x.lo compatibility-thread-c++0x.lo compatibility-chrono.lo
compatibility-condvar.lo  ../libsupc++/libsupc++convenience.la
../src/c++98/libc++98convenience.la ../src/c++11/libc++11convenience.la
../src/c++17/libc++17convenience.la

Interestingly although my sysroot doesn't appear on this command, the
output generated by libtool does show it:

libtool: link:  /obj/./gcc/xgcc -shared-libgcc -B/obj/./gcc -nostdinc++
-L/obj/x86_64-pc-linux-gnu/libstdc++-v3/src -L/obj/x86_64-pc-linux-
gnu/libstdc++-v3/src/.libs -L/obj/x86_64-pc-linux-gnu/libstdc++-
v3/libsupc++/.libs -B/dist/x86_64-pc-linux-gnu/bin/ -B/dist/x86_64-pc-
linux-gnu/lib/ -isystem /dist/x86_64-pc-linux-gnu/include -isystem
/dist/x86_64-pc-linux-gnu/sys-include   -fno-checking  -fPIC -DPIC
-D_GLIBCXX_SHARED -shared -nostdlib
/sysroot/tools/usr/lib/../lib64/crti.o
/obj/./gcc/crtbeginS.o  .libs/compatibility.o .libs/compatibility-
debug_list.o .libs/compatibility-debug_list-2.o .libs/compatibility-
c++0x.o .libs/compatibility-atomic-c++0x.o .libs/compatibility-thread-
c++0x.o .libs/compatibility-chrono.o .libs/compatibility-condvar.o  -
Wl,--whole-archive ../libsupc++/.libs/libsupc++convenience.a
../src/c++98/.libs/libc++98convenience.a
../src/c++11/.libs/libc++11convenience.a
../src/c++17/.libs/libc++17convenience.a -Wl,--no-whole-archive  -
L/obj/x86_64-pc-linux-gnu/libstdc++-v3/libsupc++/.libs -L/obj/x86_64-
pc-linux-gnu/libstdc++-v3/src -L/obj/x86_64-pc-linux-gnu/libstdc++-
v3/src/.libs -lm -L/obj/./gcc -L/sysroot/tools/lib/../lib64
-L/sysroot/tools/usr/lib/../lib64 -L/sysroot/tools/lib
-L/sysroot/tools/usr/lib -lc -lgcc_s /obj/./gcc/crtendS.o
/sysroot/tools/usr/lib/../lib64/crtn.o  -Wl,-O1 -Wl,-z -Wl,relro -Wl,
--gc-sections -Wl,--version-script=libstdc++-symbols.ver   -Wl,-soname
-Wl,libstdc++.so.6 -o .libs/libstdc++.so.6.0.27

you can see the -L/sysroot/tools/... here.

However this command fails:

/tools/bin/ld: cannot find /lib64/libc.so.6
/tools/bin/ld: cannot find /usr/lib64/libc_nonshared.a
collect2: error: ld returned 1 exit status
make[6]: *** [Makefile:697: libstdc++.la] Error 1
make[5]: *** [Makefile:730: all-recursive] Error 1
make[4]: *** [Makefile:562: all-recursive] Error 1
make[3]: *** [Makefile:487: all] Error 2
make[2]: *** [Makefile:19557: all-stage1-target-libstdc++-v3] Error 2
make[1]: *** [Makefile:27618: stage1-bubble] Error 2
make: *** [Makefile:28760: profiledbootstrap-lean] Error 2

Well, I mean I don't want

Re: Make LTO Patch for Job Server Thread Detection Agnostic

[Paul Smith]

On Fri, 2020-02-28 at 17:37 +0100, Martin Jambor wrote:
> > Recently Honza, me and others discussed LTO's interaction with
> > build systems, and that perhaps the module mapper could be
> > generalized for other purposes.  (Yes, still need to resurrect my
> > Make PoC)
> 
> see also the "Create a general jobserver client/server library"
> library suggested GSoC project at 
> https://gcc.gnu.org/wiki/SummerOfCode.

If you get someone to work on that please touch base with
bug-m...@gnu.org (or talk to me directly, at psm...@gnu.org) as it
would be great if the implementation could be used in GNU make itself,
as well.

One caveat: GNU make still requires C90.

Cheers!

Re: Make LTO Patch for Job Server Thread Detection Agnostic

[Paul Smith]

On Thu, 2020-02-27 at 16:58 +, Jonathan Wakely wrote:
> > That's a problem then as were assuming a user's build system for this
> > to work. I mean for now its fine but in the future wouldn't it de a
> > good ideal to not assume this?
> 
> It works fine for everybody. There's just an optimisation for people
> with a GNU make jobserver available. I don't see a problem.
> 
> If somebody wants to add an optimisation for their preferred build
> system they can propose a patch.

And/or they can suggest to other build systems that they also add support
for this service.

I'm not aware of any service like this which is supported by all build
tools, so it's not like we're choosing this over something else that's more
widely available.  Actually as far as I know other build tools don't
provide anything like it, portable or not.

Re: issue with GDB python pretty-printers for libstdc++

[Paul Smith]

On Wed, 2020-02-26 at 17:23 +, Jonathan Wakely wrote:
> I compiled this program:
> 
> #include 
> 
> int main()
> {
>   std::unordered_map m;
>   m[3] = 5;
>   return m[1];
> }
> 
> And with Fedora 31's GDB 8.3.50 I get this behaviour:

I can't reproduce with a simple test either.  But in my full system I
have something like this:

  class Obj {
typedef uint64_t GroupId;
typedef uint64_t ObjectId;
typedef std::unordered_map GroupIdToObjectIdMap;

GroupIdToObjectIdMap objMap;
  };

then in GDB I do this:

  (gdb) ptype $v->objMap
  type = std::unordered_map

Fine, but then when I do this:

  (gdb) py v = gdb.parse_and_eval('$v')
  (gdb) py print str(v.type)
  Obj::GroupIdToObjectIdMap

OK, but then when I do this:

  (gdb) py print str(v.type.strip_typedefs())
  class std::unordered_map, std::equal_to, std::allocator > >

Note the "class " prefix here.  And:

  (gdb) py t = gdb.lookup_type(str(v.type.strip_typedefs())
Traceback (most recent call last):
File "", line 1, in 
  gdb.error: No type named class std::unordered_map, std::equal_to, 
std::allocator > >::__node_type.
  Error while executing Python code.

If I use v.type.strip_typedefs().name it works.

I haven't been able to reproduce this in a small separate test case.  I
have no idea what the difference is :(


Thanks!!

issue with GDB python pretty-printers for libstdc++

[Paul Smith]

Hi all.  I was seeing a strange error in GDB (8.2.1) debugging some C++
code while trying to print a value.  The pretty printer was throwing Python
exceptions.

Debugging it I discovered the problem, which is here (from GCC 9.2):

libstdc++-v3/python/libstdcxx/v6/printers.py:
  # Starting with the type ORIG, search for the member type NAME.  This
  # handles searching upward through superclasses.  This is needed to
  # work around http://sourceware.org/bugzilla/show_bug.cgi?id=13615.
  def find_type(orig, name):
  typ = orig.strip_typedefs()
  while True:
  # Strip cv-qualifiers.  PR 67440.
-->   search = '%s::%s' % (typ.unqualified(), name)
  try:
  return gdb.lookup_type(search)
  except RuntimeError:
  pass
  # The type was not found, so try the superclass.  We only need
  # to check the first superclass, so we don't bother with
  # anything fancier here.
  field = typ.fields()[0]

(First that GDB bug was fixed in 2012 so I'm not sure if we still need this
method, but anyway...)

The issue is on the marked line above.  Here we are using the __str__()
method on a gdb.Type to obtain the string name of the type.  However, I've
discovered that (at least on my system) the __str__() representation of a
gdb.Type prepends the keyword "class " or "struct " (as appropriate) to the
output.  So the above will result in a string like:

  search = 'class std::unordered_map...::...'

The prepended "class " causes the code to break: it doesn't find the type,
then we try to use typ.fields()[0] which doesn't work as follows:

  Traceback (most recent call last):
  ...
File "/cc/python/libstdcxx/v6/printers.py", line 97, in find_type
  field = typ.fields()[0]
  IndexError: list index out of range

I think that it's not correct for the Python macros here to be using the
gdb.Type.__str__() method to obtain the type name for anything other than
displaying to users.  They should always be using the gdb.Type.name data
member instead.  If I change the marked line above to be:

  search = '%s::%s' % (typ.unqualified().name, name)

then it all works as expected.

However, I took a quick look through the code and it _appears_ to me that
this type of thing (using the implied, or explicit, gdb.Type.__str__() to
obtain the type name) is done in a number of places.

This makes me wonder whether (a) for some reason no one noticed this
before, or (b) there's something bizarre about my GDB which is prepending
this "class" or "struct" where other peoples' don't do that?

Re: Git ChangeLog policy for GCC Testsuite inquiry

[Paul Smith]

On Fri, 2020-01-24 at 22:45 +0100, Jakub Jelinek wrote:
> > > In my experience the output of git log is a total mess so cannot
> > > replace ChangeLogs.  But we can well decide to drop ChangeLog for
> > > the testsuite.
> > 
> > Well, glibc has moved to extracting them from git, building
> > policies and scripts around that.  I'm pretty sure other
> > significant projecs are also extracting their ChangeLogs from git.
> > 
> > We could do the same, selecting some magic date as the cutover
> > point after which future ChangeLogs are extracted from GIT.  In
> > fact, that's precisely what I'd like to see us do.
> 
> We don't have a tool that can do it, not even get the boilerplate
> right. Yes, mklog helps, but it very often gets stuff wrong.  Not to
> mention that the text what actually changed can't be generated very
> easily.

I don't know if it counts as a significant project, but GNU make has
been doing this for years.

What I did was take the existing ChangeLogs and rename them to
ChangeLog.1 or whatever, then started with a new ChangeLog generated
from scratch from Git messages.

I use the gnulib build-aux/gitlog-to-changelog script to do it.  It
requires a little bit of discipline to get right; in particular you
have to remember that the Git commit message will be indented 8 spaces
in the ChangeLog, so you have to be careful that your commit messages
wrap at char 70 (or less) in your Git commit.

If you have Git hooks you could enforce a bit of formatting; for
example any line not indented by space must be <=70 chars long; this
allows people to use long lines for formatted content if they indent it
with a space or two.

Otherwise, it's the same as writing the ChangeLog and you only have to
do it once.

Just to note, the above script simply transcribes the commit message
into ChangeLog format.  It does NOT try to auto-generate ChangeLog-
style content (files that changed, functions, etc.) from the Git diff
or whatever.

There are a few special tokens you can add to your Git commit message
that get reformated to special changelog tokens like "(tiny change)"
etc.

As mentioned previously, it's very important that the commit message be
provided as part of the code review, and it is very much fair game for
review comments.  This is common practice, and a good idea because bad
commit messages are always a bummer, ChangeLog or not.

Re: 1-800-GIT-HELP

[Paul Smith]

On Mon, 2020-01-13 at 11:33 +, Jonathan Wakely wrote:
> I imagine a lot of people are going to feel lost in the first few
> weeks of using Git.

I don't do IRC (and I'm not a GCC dev :)) but I'm happy to help via email.

One thing I'll say though: if you're an Emacs user then IMO you should run,
not walk, to your nearest MELPA site and install the Magit package.

https://magit.vc/

https://magit.vc/manual/magit/Installing-from-Melpa.html#Installing-from-Melpa

It will make your day-to-day Git use much more pleasant.  It provides a
single "status" buffer that shows you the current state of your workspace
and allows you to do everything from the very simple "commit all modified
files", to the more complex "pick and choose individual diff hunks to
commit".  It provides Emacs-based interfaces to everything from push/pull
to merging to interactive rebasing.

The Emacs VC integration with Git is pretty nice, but Magit takes things to
a whole different level.

Re: Status of C++11 support

[Paul Smith]

On Mon, 2019-12-09 at 18:09 +, Andrew Haley wrote:
> On 12/9/19 5:03 PM, Nicholas Krause wrote:
> > > https://gcc.gnu.org/projects/cxx-status.html#cxx11
> > 
> > I'm asking of what support exists in the gcc codebase itself not
> > for other projects using gcc.
> 
> That is what you got.

IIUC Nicholas is not asking what language features GCC _can compile_. 
He's asking which language version the compiler itself is _written in_.

If he were to write a new GCC feature, can he write it in C++11?

I'm pretty sure the link above shows the former, not the latter. 
Unless you're trying to imply, indirectly, that this is the language
GCC is written in as well as what it can compile.

Re: ICE with precompiled headers (GCC 8.1)

[Paul Smith]

I was able to reproduce this with GCC 9.0.1 20190430 as well.

It appears that adding -MD to the PCH build sometimes causes the "deps"
data in the PCH file to be empty, but this is never noticed unless you
use the -fpch-deps option on the source compile line: if you don't do
this then nothing in GCC will try to access that pointer.

I feel like I've hit the end of what I can do with tracking this down,
so I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90306

Although I couldn't create a repro case, I can reproduce it here on
some systems so if someone wants me to investigate further I'm happy to
do that: please provide some pointers/advice for debugging.


On Wed, 2019-05-01 at 10:56 -0400, Paul Smith wrote:
> On Wed, 2019-05-01 at 09:35 -0400, Paul Smith wrote:
> > > Unfortunately my GCC is heavily optimized and stripped so backtraces
> > > are useless.  I will generate a debuggable GCC and see if I can get
> > > more info on the ICE.
> > 
> > I have created a GCC with debug enabled so I'll see what I find.
> 
> I was able to reproduce this ICE quite readily with my debuggable GCC
> 8.1.0.  Here's the failure:

Re: ICE with precompiled headers (GCC 8.1)

[Paul Smith]

On Wed, 2019-05-01 at 09:35 -0400, Paul Smith wrote:
> > Unfortunately my GCC is heavily optimized and stripped so backtraces
> > are useless.  I will generate a debuggable GCC and see if I can get
> > more info on the ICE.
> 
> I have created a GCC with debug enabled so I'll see what I find.

I was able to reproduce this ICE quite readily with my debuggable GCC
8.1.0.  Here's the failure:

: internal compiler error: Segmentation fault
0x9cae61 crash_signal
/work/src/cc/gcc-8.1.0/gcc/toplev.c:325
0x1293778 apply_vpath
/work/src/cc/gcc-8.1.0/libcpp/mkdeps.c:127
0x1293acc deps_add_dep(deps*, char const*)
/work/src/cc/gcc-8.1.0/libcpp/mkdeps.c:258
0x1293fe3 deps_restore(deps*, _IO_FILE*, char const*)
/work/src/cc/gcc-8.1.0/libcpp/mkdeps.c:432
0x129535b cpp_read_state(cpp_reader*, char const*, _IO_FILE*, save_macro_data*)
/work/src/cc/gcc-8.1.0/libcpp/pch.c:884
0x596d59 c_common_read_pch(cpp_reader*, char const*, int, char const*)
/work/src/cc/gcc-8.1.0/gcc/c-family/c-pch.c:373
0x12872fe should_stack_file
/work/src/cc/gcc-8.1.0/libcpp/files.c:814
0x12874f1 _cpp_stack_file
/work/src/cc/gcc-8.1.0/libcpp/files.c:900
0x12876a7 _cpp_stack_include
/work/src/cc/gcc-8.1.0/libcpp/files.c:1049
0x1287b22 cpp_push_include(cpp_reader*, char const*)
/work/src/cc/gcc-8.1.0/libcpp/files.c:1484
0x5943ec push_command_line_include
/work/src/cc/gcc-8.1.0/gcc/c-family/c-opts.c:1483
0x594615 c_finish_options
/work/src/cc/gcc-8.1.0/gcc/c-family/c-opts.c:1452
0x5963a2 c_common_parse_file()
/work/src/cc/gcc-8.1.0/gcc/c-family/c-opts.c:1126

Unsurprisingly the problem is that the "deps" data member in
cpp_reader* is null:

#0  apply_vpath (d=d@entry=0x0,
t=t@entry=0x2174860 "/src/foo_pch.h") at 
/work/src/cc/gcc-8.1.0/libcpp/mkdeps.c:127
#1  0x01293acd in deps_add_dep (d=d@entry=0x0,
t=t@entry=0x2174860 "/src/foo_pch.h") at 
/work/src/cc/gcc-8.1.0/libcpp/mkdeps.c:258
#2  0x01293fe4 in deps_restore (deps=0x0, fd=fd@entry=0x2171750,
self=self@entry=0x2106810 "/src/obj/foo_pch.h.gch")
at /work/src/cc/gcc-8.1.0/libcpp/mkdeps.c:432
#3  0x0129535c in cpp_read_state (r=r@entry=0x20f4d60,
name=name@entry=0x2106810 "/src/obj/foo_pch.h.gch", f=f@entry=0x2171750, 
data=0x210bce0)
at /work/src/cc/gcc-8.1.0/libcpp/pch.c:884

I have no idea whether the problem is that it should never be possible
for "deps" to be null, or whether the problem is that we're not
checking for that possibility when we should be.

I'm building the current GCC 9.0.1 prerelease to see if I can reproduce
it there.

Once again removing -fpch-deps solves the problem.  I can only assume
that without that flag we never bother to walk the deps data member at
all.

Re: ICE with precompiled headers (GCC 8.1)

[Paul Smith]

On Tue, 2019-04-30 at 09:43 -0400, Paul Smith wrote:
> Here's the thing: I added the "-MD -MF foo.d" options to the PCH file
> compile (I wanted to get a manifest of which files were included in the
> PCH file).  Without those options I cannot reproduce this at all (in a
> number of tries).  With those options it happens most of the time on
> some systems (but again, never on my local system).

I have made more discoveries.

In the compiler invocation that is ICEing, if I remove the -fpch-deps
option then it no longer ICEs (with the same PCH file that ICEd
before!)  I can remove/add that option to my .cpp compile line, and it
ICEs with it and works without it, no other changes.

But again, it works fine on my system always.  So there's _sometimes_
something odd inside the PCH file which is triggered by the source file
compilation adding -fpch-deps.

FWIW, here's the compile line I use to generate the PCH:

  g++ ...opts... -MD -MF dir/foo.h.d -o dir/foo.h.gch -x c++-header foo.h

And here's the compile line I use to compile source code:

  g++ ...opts... -fpch-deps -fpch-preprocess -Winvalid-pch \
  -iquote"dir" --include=foo.h

Adding/removing -fpch-preprocess doesn't seem to matter.  It's only the
-fpch-deps option during the compile, combined with the -MD/-MF options
when creating the PCH file, that seems to trigger the ICE.

> Unfortunately my GCC is heavily optimized and stripped so backtraces
> are useless.  I will generate a debuggable GCC and see if I can get
> more info on the ICE.

I have created a GCC with debug enabled so I'll see what I find.

Re: ICE with precompiled headers (GCC 8.1)

[Paul Smith]

On Tue, 2019-04-30 at 09:34 -0600, Zan Lynx wrote:
> > In the meantime, does this remind anyone of an existing bug,
> > hopefully one that was fixed in 8.2 or 8.3?
> 
> It does remind me of a race condition bug with a Makefile I wrote
> years ago.
> 
> One or two build tasks did not properly depend on the precompiled
> headers but used them anyway, and sometimes during make -j8 would get
> unlucky and use a partial header file.

Unfortunately once one of these corrupted PCH files is created every
compile fails, not just the first few.  Also after the build is fully
complete (failed) and I log into the build server and run the compiler
by hand, I continue to get the failure every time with that PCH file. 
So, it's not just a temporary use of a partially-created file.

I've checked the build logs and as far as I can see each precompiled
header is built only one time, so it's also not a matter of the same
PCH file being built multiple times and the commands stomping on each
other.

Thx for the note...

ICE with precompiled headers (GCC 8.1)

[Paul Smith]

I have GCC 8.1.0 / binutils 2.30 on GNU/Linux 64bit (built locally). 
My codebase is almost all C++.

I'm implementing precompiled headers and it was going well.  However,
sometimes a PCH file is generated that causes the compiler to ICE when
trying to use it during a source file compilation:

  : internal compiler error: Segmentation fault
  Please submit a full bug report,
  with preprocessed source if appropriate.
  See  for instructions.

If I rename the .gch file and compile the source it works; if I put
back the .gch file and compile the source it will ICE again.  So
clearly there's something corrupt about the PCH file itself.

Also this does not happen every time, even using the identical code. 
Sometimes the PCH file is fine and the compile is fine.  It has never
happened on my local system but it happens quite often on some of our
build servers (typically these are a bit older/slower with slower
disks).  And, it doesn't happen to all PCH files in the same build (I
create different PCH files for different libraries), even ones that
include the same files!

Since it's not reproducible and the code is proprietary I haven't tried
to produce a publishable case so far.

Here's the thing: I added the "-MD -MF foo.d" options to the PCH file
compile (I wanted to get a manifest of which files were included in the
PCH file).  Without those options I cannot reproduce this at all (in a
number of tries).  With those options it happens most of the time on
some systems (but again, never on my local system).

Unfortunately my GCC is heavily optimized and stripped so backtraces
are useless.  I will generate a debuggable GCC and see if I can get
more info on the ICE.

In the meantime, does this remind anyone of an existing bug, hopefully
one that was fixed in 8.2 or 8.3?

Re: GCC 4.8.1 unable to compile a .c File

[Paul Smith]

On Sat, 2019-03-23 at 22:54 +0530, Vinaya Dandur wrote:
> My mistake. Thanks, but even if you include  the issue still
> exists. Yes TRAP_BRKPT is not included in signal.h, however gcc somehow
> finds the definition for TRAP_BRKPT and I don't know how. This works in
> other hosts.

A peruse of the sigaction man page for GNU/Linux might be enlightening:

http://man7.org/linux/man-pages/man2/sigaction.2.html

> Since glibc 2.20, the definitions of most of these symbols are
> obtained from  by defining feature test macros (before
> including any header file) as follows:
> 
>   *  _XOPEN_SOURCE with the value 500 or greater;
>   *  _XOPEN_SOURCE and _XOPEN_SOURCE_EXTENDED; or
>   *  _POSIX_C_SOURCE with the value 200809L or greater.
> 
> For the TRAP_* constants, the symbol definitions are provided only
> in the first two cases.  Before glibc 2.20, no feature test macros
> were required to obtain these symbols.

So if you add #define _XOPEN_SOURCE 500 or both #define _XOPEN_SOURCE
and #define _XOPEN_SOURCE_EXTENDED before your first #include, or if
you add -D_XOPEN_SOURCE=500 or both -D_XOPEN_SOURCE and
-D_XOPEN_SOURCE_EXTENDED to your compile line, then things should work.

This is a question related to the GNU libc project, not GCC.

Re: GCC 4.8.1 unable to compile a .c File

[Paul Smith]

On Sat, 2019-03-23 at 21:55 +0530, Vinaya Dandur wrote:
> Thank you. Yes it is not an issue with the GCC but the TRAP_BRKPT is
> defined in signal.h which the GCC could include but can't find the
> constant mentioned.
> Is there any way this problem could be resolved ? any pointers I can
> get ?

If the macro is defined in signal.h then you should #include signal.h. 
Is there some issue here we're not seeing?

> > > Program:
> > > ++
> > > #include 
> > > #include 
> > > 
> > > int main () {
> > > 
> > > printf("TRAP_BRKPT is: %d\n", TRAP_BRKPT);
> > > 
> > > return 0;
> > > }
> > > +++

Questions about how to write correct C are definitely not on-topic for
this mailing list.  Maybe try posting a question on StackOverflow?

Cheers!

Incorrect change to generic_morestack.c:__morestack_load_mmap()?

[Paul Smith]

I was looking through the diffs between GCC 8.1.0 and GCC 8.2.0 and saw
this change, which doesn't look right to me: it changes munmap() to use
the new static_pagesize variable, but that variable is not assigned
until after it's used.  Since it's global it's always 0, so this code
runs munmap(0, 0) which doesn't seem too useful to me.

I checked and the current SVN trunk still has this change as-is.



@@ -825,7 +811,12 @@
  TLS accessor function is resolved.  */
   mmap (__morestack_current_segment, 0, PROT_READ, MAP_ANONYMOUS, -1, 0);
   mprotect (NULL, 0, 0);
-  munmap (0, getpagesize ());
+  munmap (0, static_pagesize);
+
+  /* Initialize these values here, so as to avoid dynamic linker
+ activity as part of a __morestack call. */
+  static_pagesize = getpagesize();
+  use_guard_page = getenv ("SPLIT_STACK_GUARD") != 0;
 }

 /* This function may be used to iterate over the stack segments.

Re: [RFC] Adding Python as a possible language and it's usage

[Paul Smith]

On Fri, 2018-07-27 at 14:53 +, Michael Matz wrote:
> perl is currently included in the bootstrap set.  There's no reason
> why python couldn't be included as well,

If Perl is already in the bootstrap set and the awk scripts are hard to
maintain then why can't the awk scripts be rewritten in Perl instead of
Python?  That would avoid adding more prerequisites and surely Perl is
sufficiently expressive that it can perform these translations just as
well as Python.

I understand some people have an issue with Perl's maintainability but
just because you CAN write difficult to maintain code in Perl doesn't
mean you HAVE to.

I've seen plenty of difficult to understand and maintain Python
scripting... just saying "use Python" is not a panacea for
supportability problems.

Re: Good news, bad news on the repository conversion

[Paul Smith]

On Mon, 2018-07-09 at 10:57 -0600, Jeff Law wrote:
> On 07/09/2018 10:53 AM, Janus Weil wrote:
> > 2018-07-09 18:35 GMT+02:00 Eric S. Raymond :
> > > David Edelsohn :
> > > > > The truth is we're near the bleeding edge of what conventional tools
> > > > > and hardware can handle gracefully.  Most jobs with working sets as
> > > > > big as this one's do only comparatively dumb operations that can be
> > > > > parallellized and thrown on a GPU or supercomputer.  Most jobs with
> > > > > the algorithmic complexity of repository surgery have *much* smaller
> > > > > working sets.  The combination of both extrema is hard.
> > > > 
> > > > If you come to the conclusion that the GCC Community could help with
> > > > resources, such as the GNU Compile Farm or paying for more RAM, let us
> > > > know.
> > > 
> > > 128GB of DDR4 registered RAM would allow me to run conversions with my
> > > browser up, but be eye-wateringly expensive.  Thanks, but I'm not
> > > going to yell for that help
> > 
> > I for one would certainly be happy to donate some spare bucks towards
> > beastie RAM if it helps to get the GCC repo converted to git in a
> > timely manner, and I'm sure there are other GCC
> > developers/users/sympathizers who'd be willing to join in. So, where
> > do we throw those bucks?
> 
> I'd be willing to throw some $$$ at this as well.

I may be misreading between the lines but I suspect Eric is more hoping
to get everyone to focus on moving this through before the GCC commit
count gets even more out of control, than he is asking for a hardware
handout :).

Maybe the question should rather be, what does the dev community need
to do to help push this conversion through soonest?

Re: gcc 7.3: Replacing global operator new/delete in shared libraries

[Paul Smith]

On Wed, 2018-02-07 at 19:26 -0500, Paul Smith wrote:
> Thanks for the conversation.  I'm moving forward with a real global
> operator new/delete and working out the magic needed to ensure those
> symbols are not global in our shared library.

I remember one annoying thing I ran into: through compiler "magic" the
-fvisibility=hidden compile-time attribute is ignored when it comes to
global operator new/delete.  Similarly, it's a compiler error to force
visibility("hidden") in the declaration via __attribute__.

The only way to get operator new/delete to be hidden inside a shared
library is to really force it using a linker script that declares them
"local:" specifically.

Re: gdb 8.x - g++ 7.x compatibility

[Paul Smith]

On Thu, 2018-02-08 at 11:26 +, Michael Matz wrote:
> As I said upthread, the mangled name of a type (sans _Z prefix) is what is 
> stored as the type name for RTTI purposes (i.e. std::type_info::name()).
> 
> It's just that the debug info currently doesn't have any reference to that 
> definitely-unique string, only to the "human-friendly" name, which 
> somewhat resembles the demangled name (and that's exactly the crux, it 
> really isn't the demangled one right now, as you found out the painful 
> way).

Isn't the problem with the mangled name, which otherwise would be just
what we wanted since it's easy to match and is unique in just the way
we want it to be, that mangling is not standardized?  If GDB relied on
the mangled name it would need to incorporate demanglers for any
compiler that it wanted to be able to debug and figure out which
demangler to use when it was trying to debug a program.  This goes
against the concept of a common debug format like DWARF I would expect.

Re: gcc 7.3: Replacing global operator new/delete in shared libraries

[Paul Smith]

On Thu, 2018-02-08 at 01:17 +0100, Marc Glisse wrote:
> On Wed, 7 Feb 2018, Paul Smith wrote:
> > > > My question is, what do I need to do to ensure this behavior
> > > > persists if I create a global operator new/delete?
> > > > 
> > > > Is it sufficient to ensure that the symbol for our shared
> > > > library global new/delete symbols are hidden and not global,
> > > > using a linker map or -fvisibility=hidden?
> > > 
> > > I think so (hidden implies not-interposable, so locally bound),
> > > but I don't have much experience there.
> > 
> > OK I'll pursue this for now.
> 
> I answered too fast. It isn't just new/delete that need to be hidden.
> It is also anything that uses them and might be used in both
> contexts.  For instance, std::allocator::allocate is an inline
> function that calls operator new. You get one version that calls
> new1, and one version that calls new2. If you don't do anything
> special, the linker keeps only one (more or less arbitrarily). So I
> believe you need -fvisibility=hidden to hide everything but a few
> carefully chosen interfaces.

At the moment I'm compiling all my code with -fvisibility=hidden, and
marking out specific symbols to be public in the source code, with
__attribute__((visibility("default"))).

This is nice andeasy, but I've grown frustrated with it.  In order to
run our unit tests we must statically link them with the code; we can't
create a shared library because all the internal symbols that the unit
tests want to refer to are hidden.  Since we have about 150 individual
unit test programs and each one statically links all the code it uses
(although we're using GTest framework and I call these "unit tests",
they're not really unit tests in the classical sense that they mock out
a single class for test), that uses a lot of disk space and takes a lot
of link time during the builds... it's already aggravating and we're
only adding more unit tests over time.

So in the near future I intend to reset this and compile all the code
without -fvisibility=hidden, then when I create our shared library I'll
generate a linker map to make only specific symbols visible and hide
everything else.

What would be ideal is if I could continue to use the visibility
attributes in the source code to mark out symbols I wanted to publish. 
If that information were preserved in the ELF output in a way I could
extract with a script running objdump or readelf on the object files,
for example, then I could automate the generation of the proper linker
map for my shared library.

However the last time I checked this, visibility("default") didn't
leave a trace in the object file unless -fvisibility=hidden was used to
make the default visibility hidden.

Too bad.

Re: gcc 7.3: Replacing global operator new/delete in shared libraries

[Paul Smith]

On Wed, 2018-02-07 at 16:38 -0700, Martin Sebor wrote:
> I'm not sure I see how defining operator new inline can work
> unless you recompile the world (i.e., all the DSOs used by
> the program, including libstdc++). As Marc already hinted,
> if libstdc++ dynamically allocates an object using the default
> operator new and returns that object to the program to destroy,
> it will end up causing a mismatch.  The same can happen in
> the opposite direction.  To avoid such mismatches the entire
> program needs to use a single operator new (each of
> the required forms), and the only safe way to do that
> is to define each out-of-line.

I'm aware of these issues, and I know it's a dangerous game.  As I
mentioned I wasn't surprised, really, that eventually it caught us out.

However, it did work.  We don't use huge amounts of the STL but we use
basics like vector, string, a bit of unordered_map, etc., and those
worked fine.  All that's required for it to work is that either both
the new and delete were both performed inside libstdc++, or both the
new and delete were performed in STL header file implementations.  In
the former case that memory is coming from the system alloc, not
jemalloc, but the amount is small enough that it's not worrisome.  In
the latter case it will use jemalloc via the inline.

The problem comes in where you do the new in an STL header and the
delete in the compiled libstdc++, or vice versa... that's what I ran
into in GCC 7.3.

On GNU/Linux you can just replace malloc/free using non-global symbols
in the shared library, to ensure even libstdc++.a implementations use
jemalloc.  Unfortunately this is not possible in Windows.


On Wed, 7 Feb 2018 Jonathan Wakely  writes:
> The code to read a string from an istream is instantiated in the
> library, so the string gets resized (allocating using malloc) by code
> inside libstdc++.so.6 and then the destructor is run (deallocating
> using your inline operator delete) in main.

We don't use C++ iostream in our code.  Although, I'm sure GTest does
use it in THEIR code.  Of course, for unit tests we don't really care
which allocator is used as long as there's no mismatch.  I haven't
investigated exactly how it all worked in 6.2, I can only tell you that
for our use, it did :).

Thanks for the conversation.  I'm moving forward with a real global
operator new/delete and working out the magic needed to ensure those
symbols are not global in our shared library.

Re: gcc 7.3: Replacing global operator new/delete in shared libraries

[Paul Smith]

On Wed, 2018-02-07 at 11:32 +0100, Marc Glisse wrote:
> On Tue, 6 Feb 2018, Paul Smith wrote:
> 
> > My environment has been using GCC 6.2 (locally compiled) on
> > GNU/Linux systems.  We use a separate heap management library
> > (jemalloc) rather than the libc allocator.  The way we did this in
> > the past was to declare operator new/delete (all forms) as inline
> > functions in a header
> 
> Are you sure you still have all forms? The aligned versions were
> added in gcc-7 IIRC.

Hm.  I didn't realize there were new forms in C++17; I had the throw
and no-throw versions and also sized delete.  No, I didn't create the
new ones.  I'll do that.

> > and ensure that this header was always the very first thing in
> > every source file, before even any standard header files.  I know
> > that inline operator new/delete isn't OK in the C++ standard, but
> > in fact it has worked for us on the systems we care about.
> 
> Inline usually works, but violating the ODR is harder... I would at
> least  use the always_inline attribute to improve chances (I assume
> that static  (or anonymous namespace) versions wouldn't work)

I definitely do that already; for example:

  inline __attribute__((__always_inline__)) void* operator new(size_t size)  ...

> > Now when I run our code, I get a core on exit.  It appears an STL
> > container delete is invoking libc free() with a pointer to memory
> > allocated by jemalloc.
> 
> An example would help the discussion.

Good point.  Here's an example (from a unit test crash, in GTest):

*** Error in `EncodedStreamTests.gtest': free(): invalid pointer:
0x7481d0a0 ***

Program received signal SIGABRT, Aborted.
0x74c62c37 in raise () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0  0x74c62c37 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x74c66028 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x74c9f2a4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x74cab82e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#4  0x00539b89 in __gnu_cxx::new_allocator::deallocate
(this=, __p=) at /work/src/build/x86_64-
linux/cc/generic/x86_64-generic-linux-
gnu/include/c++/7.3.0/ext/new_allocator.h:125
#5  std::allocator_traits >::deallocate (__a=...,
__n=, __p=) at /work/src/build/x86_64-
linux/cc/generic/x86_64-generic-linux-
gnu/include/c++/7.3.0/bits/alloc_traits.h:462
#6  std::__cxx11::basic_string,
std::allocator >::_M_destroy (__size=,
this=0x7fffe4e0) at /work/src/build/x86_64-linux/cc/generic/x86_64-
generic-linux-gnu/include/c++/7.3.0/bits/basic_string.h:226
#7  std::__cxx11::basic_string,
std::allocator >::_M_dispose (this=0x7fffe4e0) at
/work/src/build/x86_64-linux/cc/generic/x86_64-generic-linux-
gnu/include/c++/7.3.0/bits/basic_string.h:221
#8  std::__cxx11::basic_string,
std::allocator >::~basic_string (this=0x7fffe4e0,
__in_chrg=) at /work/src/build/x86_64-
linux/cc/generic/x86_64-generic-linux-
gnu/include/c++/7.3.0/bits/basic_string.h:647
#9  testing::internal::CodeLocation::~CodeLocation
(this=0x7fffe4e0, __in_chrg=) at
Tests/GTest/include/gtest/internal/gtest-internal.h:504
#10 testing::internal::MakeAndRegisterTestInfo (test_case_name=test_cas
e_name@entry=0x5d8ed7 "ESTests", name=name@entry=0x5d8ec9
"bytesForCount", type_param=type_param@entry=0x0, value_param=value_par
am@entry=0x0, code_location=..., fixture_class_id=fixture_class_id@entr
y=0x65dd5c ::dummy_>,
set_up_tc=0x48e010 ,
tear_down_tc=0x48e020 ,
factory=0x7481c000) at Tests/GTest/src/gtest.cc:2580
#11 0x00475286 in __static_initialization_and_destruction_0
(__priority=65535, __initialize_p=1) at EncodedStreamTests.cpp:351
#12 0x005d8dcd in __libc_csu_init ()
#13 0x74c4ded5 in __libc_start_main () from /lib/x86_64-linux-
gnu/libc.so.6
#14 0x00477375 in _start ()

> > My question is, what do I need to do to ensure this behavior
> > persists if I create a global operator new/delete?
> > 
> > Is it sufficient to ensure that the symbol for our shared library
> > global new/delete symbols are hidden and not global, using a linker
> > map or -fvisibility=hidden?
> 
> I think so (hidden implies not-interposable, so locally bound), but
> I don't have much experience there.

OK I'll pursue this for now.

Thanks!

gcc 7.3: Replacing global operator new/delete in shared libraries

[Paul Smith]

Hi all.

Hopefully this isn't too annoying a question :).

My environment has been using GCC 6.2 (locally compiled) on GNU/Linux
systems.  We use a separate heap management library (jemalloc) rather
than the libc allocator.  The way we did this in the past was to
declare operator new/delete (all forms) as inline functions in a header
and ensure that this header was always the very first thing in every
source file, before even any standard header files.  I know that inline
operator new/delete isn't OK in the C++ standard, but in fact it has
worked for us on the systems we care about.

I'm attempting a toolchain upgrade which is switching to GCC 7.3 /
binutils 2.30 (along with many other updates).

Now when I run our code, I get a core on exit.  It appears an STL
container delete is invoking libc free() with a pointer to memory
allocated by jemalloc.

I suspect that between 6.2 and 7.3 something in the STL has been
modified to call new in a header file, so it's using our inline
operator new, but call the matching delete from inside libstdc++.a (we
link with static libstdc++ for portability), so it doesn't use our
inline operator delete.

While it's unfortunate for us, obviously that's a perfectly legal
implementation choice.  I don't know whether this is something the GCC
folks care about.  If so I can do more to track down the specifics.

If I create a real global operator new/delete, even keeping the inlined
versions, then the problem goes away (lending more weight to my guess
above).

I should point out that we don't use much STL memory so having some
compiled (not header-based) STL use the libc allocator is not a big
deal to us... it's just the mismatch which is a problem.

This leads to my question:

One of the things we provide is a shared library including much of our
code, and also jemalloc.  Users link this shared library with their
code and we do not want them to use our allocator.  By having all our
operator new/delete inlined we are sure that all our requests go to our
allocator and their requests do not.  It's a bit grungy, perhaps, but
it's worked well until now.

My question is, what do I need to do to ensure this behavior persists
if I create a global operator new/delete?

Is it sufficient to ensure that the symbol for our shared library
global new/delete symbols are hidden and not global, using a linker map
or -fvisibility=hidden?

Re: gdb 8.x - g++ 7.x compatibility

[Paul Smith]

On Fri, 2018-02-02 at 23:54 -0500, Simon Marchi wrote:
> Your problem is probably linked to these issues, if you want to follow 
> them:
> 
> gcc: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81932
> gdb: https://sourceware.org/bugzilla/show_bug.cgi?id=22013
> 
> As Carl said, it's a good idea to try with the latest version of both 
> tools, but I think the issue will still be present.
> 
> GCC changed how it outputs unsigned template parameters in the debug 
> info (from 2u to just 2), and it doesn't look like it's going to change 
> it back.  So I suppose we'll have to find a way to make GDB deal with 
> it.

I also tried a couple of times [1][2][3] to get a discussion started on
the mailing lists for how to resolve this but didn't get any replies,
and I got busy with other things.

We really need to find someone who is knowlegeable on type lookup in
GDB.  That person needs to engage with the experts on the GCC side and
hash out the right answer to this problem.  In my experience, Jonathan
Wakely on the GCC side is extremely responsive, I'm just too much of a
tyro to be able to discuss it with him at the level needed to find a
solution.

I think it's an extremely serious issue, if GDB can't resolve some very
common (IME) types, but so far it hasn't risen to the level of getting
attention from those who have sufficient expertise to solve it.


[1] https://gcc.gnu.org/ml/gcc-help/2017-08/msg00120.html
[2] https://sourceware.org/ml/gdb/2017-08/msg00069.html
[3] https://sourceware.org/ml/gdb/2017-09/msg00042.html

GCC 7.3 timeline?

[Paul Smith]

Hi all; are we on track to have a GCC 7.3 sometime in the next few
weeks, as per usual for the last few years?  Not looking for a date,
just a feeling.

I'm not sure why my toolchain rollouts always seem to fall right around
the time of a new fix release for GCC...

Thanks!

Re: Support Library Requirements for GCC 7.1

[Paul Smith]

On Tue, 2017-05-02 at 18:17 -0500, Joel Sherrill wrote:
> With gcc 6.3.0, we have this in our build recipe:
> 
> %define mpfr_version   2.4.2
> %define mpc_version    0.8.1
> %define gmp_version    4.3.2

Best thing to do is look at the contrib/download_prerequisites script
that comes with the GCC source code.

The 7.1 version says:

gmp='gmp-6.1.0.tar.bz2'
mpfr='mpfr-3.1.4.tar.bz2'
mpc='mpc-1.0.3.tar.gz'
isl='isl-0.16.1.tar.bz2'

In general, it's better to just run that script, if possible.

Re: Why does 2nd loop only print values 10-19 and not 0-19?

[Paul Smith]

On Sat, 2017-01-21 at 13:25 -0800, L A Walsh wrote:
> It may be this should go to "gcc-help", due to my
> not seeing my error, but

Probably better to start there... if it's really a GCC bug you'll be
quickly redirected here.

>     for (i==0; i < sizeof(source)/(2*sizeof(source[0])); ++i) {
^^

This should be "for (i=0; ...".

Re: Throwing exceptions from a .so linked with -static-lib* ?

[Paul Smith]

On Thu, 2017-01-12 at 21:49 +, Yuri Gribov wrote:
> Note that documentation for -static-libgcc explicitly mentions that
>    There are several situations in which an application should
> use the shared libgcc instead of the static version.  The most
>    common of these is when the application wishes to throw and
> catch exceptions across different shared libraries.  In that case,
>    each of the libraries as well as the application itself
> should use the shared libgcc.
> Removing -static-libgcc fixes problem with your reprocase.

I could have sworn I tried all different combinations of the different
static flags, but sure enough if I don't add -static-libgcc on either
the .so or the executable things work OK.

Thanks!

Throwing exceptions from a .so linked with -static-lib* ?

[Paul Smith]

TL;DR:
I have an issue where if I have a .so linked with -static-lib* making
all STL symbols private, and if I throw an exception out of that .so to
be caught by the caller, then I get a SIGABRT from a gcc_assert() down
in the guts of the signal handling:

#0  0x7773a428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x7773c02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x0040e938 in _Unwind_SetGR (context=, 
index=, val=) at 
/usr/src/cc/gcc-6.2.0/libgcc/unwind-dw2.c:271
271   gcc_assert (index < (int) sizeof(dwarf_reg_size_table));

Should it be possible to do this successfully or am I doomed to failure?
More details and a test case below.


More detail:

I'm trying to distribute a shared library built with the latest version
of C++ (well, GCC 6.2 with C++14) on GNU/Linux.  I compile it with an
older sysroot, taken from RHEL 6.3 (glibc 2.12) so it will run on older
systems.

My .so is written in C++ and programs that link it will also be written
in C++ although they may be compiled and linked with potentially much
older versions of GCC (like, 4.9 etc.)  I'm not worried about programs
compiled with clang or whatever at this point.

Because I want to use new C++ but want users to be able to use my .so on
older systems, I link with -static-libgcc -static-libstdc++.  Because I
don't want to worry about security issues etc. in system libraries, I
don't link anything else statically.

I also use a linker script to force all symbols (even libstdc++ symbols)
to be private to my shared library except the ones I want to publish.

Using "nm | grep ' [A-TV-Z] '" I can see that no other symbols besides
mine are public.

However, if my library throws an exception which I expect to be handled
by the program linking my library, then I get a SIGABRT, as above; the
full backtrace is:

#0  0x7773a428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x7773c02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x0040e938 in _Unwind_SetGR (context=, 
index=, val=) at 
/usr/src/cc/gcc-6.2.0/libgcc/unwind-dw2.c:271
#3  0x004012a2 in __gxx_personality_v0 ()
#4  0x77feb903 in _Unwind_RaiseException_Phase2 
(exc=exc@entry=0x43b890, context=context@entry=0x7fffe330) at 
/usr/src/cc/gcc-6.2.0/libgcc/unwind.inc:62
#5  0x77febf8a in _Unwind_RaiseException (exc=0x43b890) at 
/usr/src/cc/gcc-6.2.0/libgcc/unwind.inc:131
#6  0x77fde84b in __cxa_throw () from 
/home/psmith/src/static-eh/libmylib.so
#7  0x77fddecb in MyLib::create () at mylib.cpp:2
#8  0x00400da4 in main () at myprog.cpp:2

I should note that if I use the GCC 5.4 that comes standard on my OS
rather than my locally-built version I get identical behavior and
backtrace (except not as much debuggability of course).  So I don't
think it's an incorrect build.

If I don't use -static-libstdc++ with my .so then it doesn't fail.  Also
if I don't use a linker script to hide all the C++ symbols it doesn't
fail (but of course anyone who links with my shared library will use my
copy of the STL).


Here's a repro case (this shows the problem on my Ubuntu GNOME 16.04
GNU/Linux system with GCC 5.4 and binutils 2.26.1):

~$ cat mylib.h
class MyLib { public: static void create(); };

~$ cat mylib.cpp
#include "mylib.h"
void MyLib::create() { throw 42; }

~$ cat myprog.cpp
#include "mylib.h"
int main() { try { MyLib::create(); } catch (...) { return 0; } return 1; }

~$ cat ver.map
{ global: _ZN5MyLib6createEv; local: *; };

~$ g++ -I. -g -fPIC -static-libgcc -static-libstdc++ \
-Wl,--version-script=ver.map -Wl,-soname=libmylib.so \
-shared -o libmylib.so mylib.cpp

~$ g++ -I. -g -fPIC  -L. -Wl,-rpath="\$ORIGIN" -o myprog myprog.cpp \
-lmylib

~$ ./myprog
Aborted (core dumped)

Now if I rebuild without the --version-script argument or without
-static-libstdc++, I get success as expected:

~$ ./myprog 

~$ echo $?
0

Re: Question on DSO and visibility

[Paul Smith]

On Wed, 2016-09-14 at 10:13 +0100, Jonathan Wakely wrote:
> On 11 September 2016 at 22:38, Paul Smith wrote:
> > I wonder if someone can comment on this situation: I'll do some testing
> > but I likely can't test everything.
> > 
> > I'm creating DSO's for GNU/Linux with GCC 4.9.2 right now.  I want to
> > upgrade to GCC 6.2.0.  My code is written in C++.  I'm aware of the C++
> > STL ABI break in GCC 5.x.
> 
> Based on the solution you outlined, I'm not sure you've fully
> understood the ABI change in GCC 5.

There's little doubt you're correct about that :)

> See https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html

Thanks.  That is relatively clear.

> >3. Use a linker map to make all symbols in my DSO hidden except the
> >   specific ones I want to be public.  I use a linker map and not just
> >   -fvisibility=hidden, so that all the symbols I've statically linked
> >   from libstdc++.a will also be marked hidden (since libstdc++.a was
> >   not compiled with -fvisibility=hidden).
> 
> The different ABIs coexist automatically. Affected symbols mangle
> differently so they don't collide.

Aha.  Yes, this is the critical point I was glossing over.

> > In other words, I can use std::basic_string and std::list in my library
> > and get the C++11 ABI from GCC 6.2 that I've statically linked, and
> > users can use std::basic_string and std::list in their code and get
> > their version of the libstdc++.so (presumably that is provided by their
> > GNU/Linux distribution) and all will work properly.
> 
> You don't necessarily need two version of libstdc++ in the process.
> The newer libstdc++.so is compatible with the users' code.
> 
> If all you're worried about is the ABI change then just build your
> library with _GLIBCXX_USE_CXX11_ABI defined to 0 (or build your gcc
> with --with-default-libstdcxx-abi=gcc4-compatible)

I could do that.  The problem is: I actually create two different
binaries from my code: one is the shared library which users can link
and the other is a standalone application.  They contain almost all the
same code (the idea is that users may want to embed the application or
run it stand-alone).  I really don't want to have to compile the entire
project twice.

And all things being equal, I'd like to use the newer ABI (since 99% of
my users actually use the stand-alone binary).

But maybe the complexity of both having my cake and eating it is not
worth it.

> The real problem is that your library will depend on a newer libstdc++
> but that's orthogonal to the ABI changes. Statically linking it is one
> solution, deploying the newer libstdc++.so with your library is
> another.

Yes, again good point.  So, as long as I don't pass objects of the
problematic types across the ABI all is fine assuming a sufficient
libstd++ library is available.

There's one question: what about exceptions?  std::exception::what()
returns a const char*; I don't know if that's stored internally as a
std::string which might have problematic types.  If I throw an exception
from my code (new GCC/ABI) and it's caught by the caller's code (old
GCC/ABI) is there a problem?  I'm assuming this is not an issue.

One reason all this came up was that a user was linking with my shared
library and got a crash in their own code, and saw my version of
std::string in the stacktrace (because I linked libstdc++ statically)
even though my code was not involved.  Even though it all was working
fine, seeing this stack trace concerned them.  I was thinking that if I
ensured that my static libstdc++ was marked hidden, they would get their
own dynamically linked libstdc++ and use that from their code and I
could avoid this "appearance of impropriety".

Thanks for the response Jonathan!

Question on DSO and visibility

[Paul Smith]

I wonder if someone can comment on this situation: I'll do some testing
but I likely can't test everything.

I'm creating DSO's for GNU/Linux with GCC 4.9.2 right now.  I want to
upgrade to GCC 6.2.0.  My code is written in C++.  I'm aware of the C++
STL ABI break in GCC 5.x.

I have users who will be using my library who are also writing C++ code
and they will be using older versions of GCC (I build my own GCC and I
use a specific sysroot for an older version of libc etc. so I know my
code will run properly on their system: they'll use their distribution's
version of GCC).

What I was thinking of doing was this:
   1. Link my DSO with -static-libstdc++ and -static-libgcc
   2. Ensure that no STL typed objects are passed across the ABI between my
  library and its callers; also that no memory I allocate is freed by
  the user and no memory the user allocates is freed by me (my library
  also runs on Windows as a DLL so I already have this restriction).
   3. Use a linker map to make all symbols in my DSO hidden except the
  specific ones I want to be public.  I use a linker map and not just
  -fvisibility=hidden, so that all the symbols I've statically linked
  from libstdc++.a will also be marked hidden (since libstdc++.a was
  not compiled with -fvisibility=hidden).

Is this plan sufficient to allow people to link with my library and not
have their version of GCC's libstdc++.so interfere with my library's
version, so the different ABI's can coexist in the same program without
interfering with each other?

In other words, I can use std::basic_string and std::list in my library
and get the C++11 ABI from GCC 6.2 that I've statically linked, and
users can use std::basic_string and std::list in their code and get
their version of the libstdc++.so (presumably that is provided by their
GNU/Linux distribution) and all will work properly.

Re: GCC 6.2?

[Paul Smith]

On Wed, 2016-07-27 at 16:06 +0200, Richard Biener wrote:
> I'm doing 4.9.4 now and 6.2 only afterwards so you can expect 6.2 earliest
> in about three weeks.

OK, thanks for the info.  I'll have to discuss with folks and see if
they can wait that long, and take a look at the issues in 6.1 which are
resolved in 6.2.

Cheers!

GCC 6.2?

[Paul Smith]

Hi all.  Don't want to be a noodge but is there any info on a timeline
for the 6.2 release?

I'm planning a major tools upgrade (from GCC 4.9.2) and I've been kind
of putting it off until 6.2 is out so I can jump to that... but the
natives are getting restless as they want some C++ features that aren't
available in 4.9.2.

Usually it seems like the "first patch release" for a new major release
is out right around now (3 months after the initial release).  Just
wondering if there is any info on this or if things are going to be
very different for 6.2.

Cheers!

Re: GCC 4.9.2 -O3 gives a seg fault / GCC 4.8.2 -O3 works

[Paul Smith]

On Tue, 2015-01-06 at 09:43 +0100, Jakub Jelinek wrote:
> GCC trunk -fsanitize=undefined (in particular
> -fsanitize=nonnull-attribute)
> diagnoses it:
> /tmp/mystring.cpp:103:26: runtime error: null pointer passed as
> argument 2, which is declared to never be null

Unfortunately adding -fsanitize=undefined in GCC 4.9.2 doesn't notice
this (in fact it actually causes the segfault to go away).

I can try to build a trunk version for this test, I suppose.

> LD_PRELOAD=libmemstomp.so detects it too.
> 
> Calling memcpy (p, NULL, 0); is invalid according to C and C++
> standards, you need to guard it, e.g. with if (data) memcpy (p, data,
> len1);
> or if (len1) memcpy (p, data, len1);

I'm on a Debian-based system and can't find a memstomp package so I
grabbed git://fedorapeople.org/home/fedora/wcohen/public_git/memstomp
and built it myself, but for some reason it doesn't fire in my
environment:

$ LD_PRELOAD=/home/psmith/src/memstomp/.libs/libmemstomp.so ./tst
memstomp: 0.1.4 sucessfully initialized for process tst (pid 26438).
Segmentation fault (core dumped)

Even if I rebuild without -O3 it passes with no warnings.  My GCC
installation uses --sysroot to build against an older glibc, etc. so
maybe that's causing some sort of issue...

Seems like I have some work to do here to come up with a way to detect
other failure situations like this.

Re: GCC 4.9.2 -O3 gives a seg fault / GCC 4.8.2 -O3 works

[Paul Smith]

On Tue, 2015-01-06 at 09:43 +0100, Jakub Jelinek wrote:
> On Tue, Jan 06, 2015 at 03:18:48AM -0500, Paul Smith wrote:
> > Hi all.  It's possible my code is doing something illegal, but it's also
> > possible I've found a problem with -O3 optimization in GCC 4.9.2.  I've
> > built this same code with GCC 4.8.2 -O3 on GNU/Linux and it works fine.
> > It also works with GCC 4.9.2 with lower -O (-O2 for example).
> 
> Your testcase is invalid.
> GCC trunk -fsanitize=undefined (in particular -fsanitize=nonnull-attribute)
> diagnoses it:
> /tmp/mystring.cpp:103:26: runtime error: null pointer passed as argument 2, 
> which is declared to never be null
> LD_PRELOAD=libmemstomp.so detects it too.
> 
> Calling memcpy (p, NULL, 0); is invalid according to C and C++
> standards, you need to guard it, e.g. with if (data) memcpy (p, data, len1);
> or if (len1) memcpy (p, data, len1);

Ah interesting.  You're right, this is definitely not correct.  But
since len1 is 0 in this case, no implementation of memcpy() actually
tried to dereference the data pointer and so there was no failure (we
build and test with clang on OSX and MSVC on Windows, and run with
valgrind and ASAN (clang)).

I'll have to look at other possible failure situations.

GCC 4.9.2 -O3 gives a seg fault / GCC 4.8.2 -O3 works

[Paul Smith]

Hi all.  It's possible my code is doing something illegal, but it's also
possible I've found a problem with -O3 optimization in GCC 4.9.2.  I've
built this same code with GCC 4.8.2 -O3 on GNU/Linux and it works fine.
It also works with GCC 4.9.2 with lower -O (-O2 for example).

When I try a build with GCC 4.9.2 -O3 I'm seeing a segfault, because we
get confused and invoke code that we should be skipping.

I've compressed the test down to a self-contained sample file that I've
attached here.  Save it and run:

  g++-4.9.2 -g -O3 -o mystring mystring.cpp

Then:

  ./mystring
  Segmentation fault

You can also add -fno-strict-aliasing etc. and it doesn't make any
difference.

The seg fault happens in the implementation of operator +=() where we're
appending to an empty string, so this->data is NULL.  That method starts
like this (after the standard pushes etc.):

   0x00400a51 <+17>:mov(%rdi),%r14

which puts this->data (null) into %r14.  Later on, with no intervening
reset of r14, we see this:

   0x00400ac5 <+133>:   cmp%r12,%r14
   0x00400ac8 <+136>:   je 0x400b18 
   0x00400aca <+138>:   subl   $0x1,-0x8(%r14)

We don't take the jump, and this (+138) is where we get the segfault
because r14 is still 0.  This is in the if-statement in the release()
method where it subtracts 1 from count... but it should never get here
because this->data (r14) is NULL!

  (gdb) i r rip
  rip0x400aca 0x400aca 
  (gdb) i r r14
  r140x0  0

Anyone have any thoughts about this?  I know the inline new/delete is
weird but it's required to repro the bug, and we need our own new/delete
because we're using our own allocator.
#include 

inline __attribute__((always_inline)) void* operator new[](size_t size) throw()
{
void* p = malloc(size);
if (p == 0) {
exit(1);
}
return p;
}

inline __attribute__((always_inline)) void operator delete[](void* ptr) throw()
{
free(ptr);
}

class String
{
public:
String() : data(NULL) {}
String(const char* string);

~String() { release(); }

String& operator =(const String& string);
String& operator +=(const char* str);

size_t getLength() const { return data ? getData()->length : 0; }
const char* getString() const { return data ? data : ""; }

operator const char*() const { return getString(); }

private:
char* data;

struct Data
{
size_t length;
unsigned int count;
};

char* allocate(size_t length);

Data* getData() const { return (Data*)(data - sizeof(Data)); }

void release()
{
if (data) {
Data* sd = getData();
if (--sd->count == 0) {
delete[] (char*)sd;
}

data = NULL;
}
}
};

#include 
#include 

String::String(const char* string)
{
data = NULL;

if (string) {
size_t length = strlen(string);
allocate(length);
memcpy(data, string, length);
}
}

char* String::allocate(size_t length)
{
release();

data = new char[sizeof(Data) + length + 1] + sizeof(Data);
getData()->count = 1;
getData()->length = length;
data[length] = '\0';

return data;
}

String& String::operator =(const String& string)
{
if (data != string.data) {
release();

if ((data = string.data)) {
++getData()->count;
}
}
return *this;
}

String& String::operator +=(const char* str)
{
size_t len1 = getLength();
size_t len2 = strlen(str);
String string;
char* p = string.allocate(len1 + len2);

memcpy(p, data, len1);
memcpy(p + len1, str, len2);

*this = string;

return *this;
}

int main()
{
String init;
String value("foo");
init += value;

printf("init = %s\n", init.getString());
return 0;
}