Re: [PATCH][RFC] Poison bitmap_head->obstack
On 12/7/18 1:10 AM, Richard Biener wrote:
On Thu, 6 Dec 2018, Martin Sebor wrote:
On 12/5/18 7:58 AM, Richard Biener wrote:
On Wed, 5 Dec 2018, Jeff Law wrote:
On 12/4/18 6:16 AM, Richard Biener wrote:
This tries to make bugs like that in PR88317 harder to create by
introducing a bitmap_release function that can be used as
pendant to bitmap_initialize for non-allocated bitmap heads.
The function makes sure to poison the bitmaps obstack member
so the obstack the bitmap was initialized with can be safely
released.
The patch also adds a default constructor to bitmap_head
doing the same, but for C++ reason initializes to a
all-zero bitmap_obstack rather than 0xdeadbeef because
the latter isn't possible in constexpr context (it is
by using unions but then things start to look even more ugly).
The stage1 compiler might end up with a few extra runtime
initializers but constexpr makes sure they'll vanish for
later stages.
I had to paper over that you-may-not-use-memset-to-zero classes
with non-trivial constructors warning in two places and I
had to teach gengtype about CONSTEXPR (probably did so in
an awkward way - suggestions and pointers into gengtype
appreciated).
Bootstrapped (with host GCC 4.8) on x86_64-unknown-linux-gnu,
testing in progress.
The LRA issue seems to be rare enough (on x86_64...) that
I didn't trip over it sofar.
Comments? Do we want this? Not sure how we can easily
discover all bitmap_clear () users that should really
use bitmap_release (suggestion for a better name appreciated
as well - I thought about bitmap_uninitialize)
Richard.
2018-12-04 Richard Biener
* bitmap.c (bitmap_head::crashme): Define.
* bitmap.h (bitmap_head): Add constexpr default constructor
poisoning the obstack member.
(bitmap_head::crashme): Declare.
(bitmap_release): New function clearing a bitmap and poisoning
the obstack member.
* gengtype.c (main): Make it recognize CONSTEXPR.
* lra-constraints.c (lra_inheritance): Use bitmap_release
instead of bitmap_clear.
* ira.c (ira): Work around warning.
* regrename.c (create_new_chain): Likewise.
I don't see enough complexity in here to be concerning -- so if it makes
it harder to make mistakes, then I'm for it.
Any comment about the -Wclass-memaccess workaround sprinkling around two
(void *) conversions? I didn't dig deep enough to look for a more
appropriate solution, also because there were some issues with older
host compilers and workarounds we installed elsewhere...
Using '*head = du_head ();' is the solution I like to encourage
to zero out typed objects. When the memory isn't initialized
and the type has a user-defined ctor, bypassing it is undefined
even if the ctor is a no-op (the ctor starts the lifetime of
the object). In that case, placement new is the appropriate
way to bring the object to life and value-initialize it:
new (head) du_head ();
If that's not good enough or portable enough to ancient compilers
then I would suggest adding a comment to explain the intent of
the cast.
Yes, I know. But we have workarounds like the following and I
didn't want to open up another hole just for this debugging feature
#ifdef BROKEN_VALUE_INITIALIZATION
/* Versions of GCC before 4.4 sometimes leave certain objects
uninitialized when value initialized, though if the type has
user defined default ctor, that ctor is invoked. As a workaround
perform clearing first and then the value initialization, which
fixes the case when value initialization doesn't initialize due to
the bugs and should initialize to all zeros, but still allows
vectors for types with user defined default ctor that initializes
some or all elements to non-zero. If T has no user defined
default ctor and some non-static data members have user defined
default ctors that initialize to non-zero the workaround will
still not work properly; in that case we just need to provide
user defined default ctor. */
memset (dst, '\0', sizeof (T) * n);
#endif
for ( ; n; ++dst, --n)
::new (static_cast(dst)) T ();
Why not then introduce a couple of helpers to do it without
duplicating the workaround each time we find ourselves faced
with this problem? Something like this perhaps:
template
inline void value_init (void *p, size_t n)
{
#ifndef BROKEN_VALUE_INITIALIZATION
for (T *q = static_cast (p); n; --n, ++q)
::new (q) T();
#else
memset (p, 0, n * sizeof (T));
#endif
}
template
inline void value_assign (T *p, size_t n)
{
#ifndef BROKEN_VALUE_INITIALIZATION
for ( ; n; --n, ++p)
*p = T ();
#else
memset (p, 0, n * sizeof (T));
#endif
}
That way the code will be clean and portable at the same time,
and the workarounds easy to find and remove if we choose once
the broken compilers are no longer supported.
Martin
Richard.
Martin
Otherwise yes, it makes it
Re: [PATCH][RFC] Poison bitmap_head->obstack
On Thu, 6 Dec 2018, Martin Sebor wrote: > On 12/5/18 7:58 AM, Richard Biener wrote: > > On Wed, 5 Dec 2018, Jeff Law wrote: > > > > > On 12/4/18 6:16 AM, Richard Biener wrote: > > > > > > > > This tries to make bugs like that in PR88317 harder to create by > > > > introducing a bitmap_release function that can be used as > > > > pendant to bitmap_initialize for non-allocated bitmap heads. > > > > The function makes sure to poison the bitmaps obstack member > > > > so the obstack the bitmap was initialized with can be safely > > > > released. > > > > > > > > The patch also adds a default constructor to bitmap_head > > > > doing the same, but for C++ reason initializes to a > > > > all-zero bitmap_obstack rather than 0xdeadbeef because > > > > the latter isn't possible in constexpr context (it is > > > > by using unions but then things start to look even more ugly). > > > > > > > > The stage1 compiler might end up with a few extra runtime > > > > initializers but constexpr makes sure they'll vanish for > > > > later stages. > > > > > > > > I had to paper over that you-may-not-use-memset-to-zero classes > > > > with non-trivial constructors warning in two places and I > > > > had to teach gengtype about CONSTEXPR (probably did so in > > > > an awkward way - suggestions and pointers into gengtype > > > > appreciated). > > > > > > > > Bootstrapped (with host GCC 4.8) on x86_64-unknown-linux-gnu, > > > > testing in progress. > > > > > > > > The LRA issue seems to be rare enough (on x86_64...) that > > > > I didn't trip over it sofar. > > > > > > > > Comments? Do we want this? Not sure how we can easily > > > > discover all bitmap_clear () users that should really > > > > use bitmap_release (suggestion for a better name appreciated > > > > as well - I thought about bitmap_uninitialize) > > > > > > > > Richard. > > > > > > > > 2018-12-04 Richard Biener > > > > > > > > * bitmap.c (bitmap_head::crashme): Define. > > > > * bitmap.h (bitmap_head): Add constexpr default constructor > > > > poisoning the obstack member. > > > > (bitmap_head::crashme): Declare. > > > > (bitmap_release): New function clearing a bitmap and poisoning > > > > the obstack member. > > > > * gengtype.c (main): Make it recognize CONSTEXPR. > > > > > > > > * lra-constraints.c (lra_inheritance): Use bitmap_release > > > > instead of bitmap_clear. > > > > > > > > * ira.c (ira): Work around warning. > > > > * regrename.c (create_new_chain): Likewise. > > > I don't see enough complexity in here to be concerning -- so if it makes > > > it harder to make mistakes, then I'm for it. > > > > Any comment about the -Wclass-memaccess workaround sprinkling around two > > (void *) conversions? I didn't dig deep enough to look for a more > > appropriate solution, also because there were some issues with older > > host compilers and workarounds we installed elsewhere... > > Using '*head = du_head ();' is the solution I like to encourage > to zero out typed objects. When the memory isn't initialized > and the type has a user-defined ctor, bypassing it is undefined > even if the ctor is a no-op (the ctor starts the lifetime of > the object). In that case, placement new is the appropriate > way to bring the object to life and value-initialize it: > > new (head) du_head (); > > If that's not good enough or portable enough to ancient compilers > then I would suggest adding a comment to explain the intent of > the cast. Yes, I know. But we have workarounds like the following and I didn't want to open up another hole just for this debugging feature #ifdef BROKEN_VALUE_INITIALIZATION /* Versions of GCC before 4.4 sometimes leave certain objects uninitialized when value initialized, though if the type has user defined default ctor, that ctor is invoked. As a workaround perform clearing first and then the value initialization, which fixes the case when value initialization doesn't initialize due to the bugs and should initialize to all zeros, but still allows vectors for types with user defined default ctor that initializes some or all elements to non-zero. If T has no user defined default ctor and some non-static data members have user defined default ctors that initialize to non-zero the workaround will still not work properly; in that case we just need to provide user defined default ctor. */ memset (dst, '\0', sizeof (T) * n); #endif for ( ; n; ++dst, --n) ::new (static_cast(dst)) T (); Richard. > Martin > > > > > Otherwise yes, it makes it harder to do mistakes. I'll probably > > use bitmap_head::crashme instead of 0xdeadbeef in bitmap_release. > > And of course we'd need to hunt down users of bitmap_clear that > > should be bitmap_release instead... > > > > Thanks, > > Richard. > > > > -- Richard Biener SUSE LINUX GmbH, GF: Felix Imendoerffer, Jane Smithard, Graham
Re: [PATCH][RFC] Poison bitmap_head->obstack
On 12/5/18 7:58 AM, Richard Biener wrote: On Wed, 5 Dec 2018, Jeff Law wrote: On 12/4/18 6:16 AM, Richard Biener wrote: This tries to make bugs like that in PR88317 harder to create by introducing a bitmap_release function that can be used as pendant to bitmap_initialize for non-allocated bitmap heads. The function makes sure to poison the bitmaps obstack member so the obstack the bitmap was initialized with can be safely released. The patch also adds a default constructor to bitmap_head doing the same, but for C++ reason initializes to a all-zero bitmap_obstack rather than 0xdeadbeef because the latter isn't possible in constexpr context (it is by using unions but then things start to look even more ugly). The stage1 compiler might end up with a few extra runtime initializers but constexpr makes sure they'll vanish for later stages. I had to paper over that you-may-not-use-memset-to-zero classes with non-trivial constructors warning in two places and I had to teach gengtype about CONSTEXPR (probably did so in an awkward way - suggestions and pointers into gengtype appreciated). Bootstrapped (with host GCC 4.8) on x86_64-unknown-linux-gnu, testing in progress. The LRA issue seems to be rare enough (on x86_64...) that I didn't trip over it sofar. Comments? Do we want this? Not sure how we can easily discover all bitmap_clear () users that should really use bitmap_release (suggestion for a better name appreciated as well - I thought about bitmap_uninitialize) Richard. 2018-12-04 Richard Biener * bitmap.c (bitmap_head::crashme): Define. * bitmap.h (bitmap_head): Add constexpr default constructor poisoning the obstack member. (bitmap_head::crashme): Declare. (bitmap_release): New function clearing a bitmap and poisoning the obstack member. * gengtype.c (main): Make it recognize CONSTEXPR. * lra-constraints.c (lra_inheritance): Use bitmap_release instead of bitmap_clear. * ira.c (ira): Work around warning. * regrename.c (create_new_chain): Likewise. I don't see enough complexity in here to be concerning -- so if it makes it harder to make mistakes, then I'm for it. Any comment about the -Wclass-memaccess workaround sprinkling around two (void *) conversions? I didn't dig deep enough to look for a more appropriate solution, also because there were some issues with older host compilers and workarounds we installed elsewhere... Using '*head = du_head ();' is the solution I like to encourage to zero out typed objects. When the memory isn't initialized and the type has a user-defined ctor, bypassing it is undefined even if the ctor is a no-op (the ctor starts the lifetime of the object). In that case, placement new is the appropriate way to bring the object to life and value-initialize it: new (head) du_head (); If that's not good enough or portable enough to ancient compilers then I would suggest adding a comment to explain the intent of the cast. Martin Otherwise yes, it makes it harder to do mistakes. I'll probably use bitmap_head::crashme instead of 0xdeadbeef in bitmap_release. And of course we'd need to hunt down users of bitmap_clear that should be bitmap_release instead... Thanks, Richard.
Re: [PATCH][RFC] Poison bitmap_head->obstack
On Wed, 5 Dec 2018, Jeff Law wrote: > On 12/5/18 7:58 AM, Richard Biener wrote: > > On Wed, 5 Dec 2018, Jeff Law wrote: > > > >> On 12/4/18 6:16 AM, Richard Biener wrote: > >>> > >>> This tries to make bugs like that in PR88317 harder to create by > >>> introducing a bitmap_release function that can be used as > >>> pendant to bitmap_initialize for non-allocated bitmap heads. > >>> The function makes sure to poison the bitmaps obstack member > >>> so the obstack the bitmap was initialized with can be safely > >>> released. > >>> > >>> The patch also adds a default constructor to bitmap_head > >>> doing the same, but for C++ reason initializes to a > >>> all-zero bitmap_obstack rather than 0xdeadbeef because > >>> the latter isn't possible in constexpr context (it is > >>> by using unions but then things start to look even more ugly). > >>> > >>> The stage1 compiler might end up with a few extra runtime > >>> initializers but constexpr makes sure they'll vanish for > >>> later stages. > >>> > >>> I had to paper over that you-may-not-use-memset-to-zero classes > >>> with non-trivial constructors warning in two places and I > >>> had to teach gengtype about CONSTEXPR (probably did so in > >>> an awkward way - suggestions and pointers into gengtype > >>> appreciated). > >>> > >>> Bootstrapped (with host GCC 4.8) on x86_64-unknown-linux-gnu, > >>> testing in progress. > >>> > >>> The LRA issue seems to be rare enough (on x86_64...) that > >>> I didn't trip over it sofar. > >>> > >>> Comments? Do we want this? Not sure how we can easily > >>> discover all bitmap_clear () users that should really > >>> use bitmap_release (suggestion for a better name appreciated > >>> as well - I thought about bitmap_uninitialize) > >>> > >>> Richard. > >>> > >>> 2018-12-04 Richard Biener > >>> > >>> * bitmap.c (bitmap_head::crashme): Define. > >>> * bitmap.h (bitmap_head): Add constexpr default constructor > >>> poisoning the obstack member. > >>> (bitmap_head::crashme): Declare. > >>> (bitmap_release): New function clearing a bitmap and poisoning > >>> the obstack member. > >>> * gengtype.c (main): Make it recognize CONSTEXPR. > >>> > >>> * lra-constraints.c (lra_inheritance): Use bitmap_release > >>> instead of bitmap_clear. > >>> > >>> * ira.c (ira): Work around warning. > >>> * regrename.c (create_new_chain): Likewise. > >> I don't see enough complexity in here to be concerning -- so if it makes > >> it harder to make mistakes, then I'm for it. > > > > Any comment about the -Wclass-memaccess workaround sprinkling around two > > (void *) conversions? I didn't dig deep enough to look for a more > > appropriate solution, also because there were some issues with older > > host compilers and workarounds we installed elsewhere... > Not really. It was just a couple casts and a normal looking ctor, so it > didn't seem terrible. Someone with more C++-fu may have a better > suggestion, but it seemed reasonable to me. > > > > > Otherwise yes, it makes it harder to do mistakes. I'll probably > > use bitmap_head::crashme instead of 0xdeadbeef in bitmap_release. > > And of course we'd need to hunt down users of bitmap_clear that > > should be bitmap_release instead... > Right, but when we trip this kind of thing we'll know to starting > digging around the bitmap_clear calls :-) That's a huge head start. OK. I'll commit the patch later today then. Currently testing with the following followup after I adjusted all 'static bitmap_head ' vars in gcc/*.c. I noticed some oddities there, like using GC allocation for such bitmaps but the bitmaps being not marked GTY (and being short-lived), and sel-sched.c exporting a bitmap_head via a pointer, not using the bitmap_head directly anywhere. Bootstrap & regtest running on x86_64-unknown-linux-gnu. >From 6c90c1c10f0f91a7a37feadd4f583ed8aaf5bcc7 Mon Sep 17 00:00:00 2001 From: Richard Guenther Date: Thu, 6 Dec 2018 10:28:30 +0100 Subject: [PATCH] bitmap-poison-followup 2018-12-06 Richard Biener * df-problems.c (df_rd_local_compute): Use bitmap_release. (df_live_free): Likewise. (df_md_local_compute): Likewise. (df_md_free): Release df_md_scratch bitmap. * loop-invariant.c (calculate_loop_reg_pressure): Use bitmap_release. * sched-deps.c (true_dependency_cache, output_dependency_cache, anti_dependency_cache, control_dependency_cache, spec_dependency_cache): Use bitmap instead of bitmap_head *. * sched-ebb.c (schedule_ebbs_init): Initialize non-GTY dont_calc_deps as bitmap allocated from obstack not GC. (schedule_ebbs_finish): Use bitmap_release. * sched-rgn.c (schedule_insns): Initialize non-GTY not_in_df as bitmap allocated from obstack not GC. Use bitmap_release. * sel-sched.c (_forced_ebb_heads): Remove premature optimization. (sel_region_init): Allocate forced_ebb_heads. (sel_region_finish): Free
Re: [PATCH][RFC] Poison bitmap_head->obstack
On 12/5/18 7:58 AM, Richard Biener wrote: > On Wed, 5 Dec 2018, Jeff Law wrote: > >> On 12/4/18 6:16 AM, Richard Biener wrote: >>> >>> This tries to make bugs like that in PR88317 harder to create by >>> introducing a bitmap_release function that can be used as >>> pendant to bitmap_initialize for non-allocated bitmap heads. >>> The function makes sure to poison the bitmaps obstack member >>> so the obstack the bitmap was initialized with can be safely >>> released. >>> >>> The patch also adds a default constructor to bitmap_head >>> doing the same, but for C++ reason initializes to a >>> all-zero bitmap_obstack rather than 0xdeadbeef because >>> the latter isn't possible in constexpr context (it is >>> by using unions but then things start to look even more ugly). >>> >>> The stage1 compiler might end up with a few extra runtime >>> initializers but constexpr makes sure they'll vanish for >>> later stages. >>> >>> I had to paper over that you-may-not-use-memset-to-zero classes >>> with non-trivial constructors warning in two places and I >>> had to teach gengtype about CONSTEXPR (probably did so in >>> an awkward way - suggestions and pointers into gengtype >>> appreciated). >>> >>> Bootstrapped (with host GCC 4.8) on x86_64-unknown-linux-gnu, >>> testing in progress. >>> >>> The LRA issue seems to be rare enough (on x86_64...) that >>> I didn't trip over it sofar. >>> >>> Comments? Do we want this? Not sure how we can easily >>> discover all bitmap_clear () users that should really >>> use bitmap_release (suggestion for a better name appreciated >>> as well - I thought about bitmap_uninitialize) >>> >>> Richard. >>> >>> 2018-12-04 Richard Biener >>> >>> * bitmap.c (bitmap_head::crashme): Define. >>> * bitmap.h (bitmap_head): Add constexpr default constructor >>> poisoning the obstack member. >>> (bitmap_head::crashme): Declare. >>> (bitmap_release): New function clearing a bitmap and poisoning >>> the obstack member. >>> * gengtype.c (main): Make it recognize CONSTEXPR. >>> >>> * lra-constraints.c (lra_inheritance): Use bitmap_release >>> instead of bitmap_clear. >>> >>> * ira.c (ira): Work around warning. >>> * regrename.c (create_new_chain): Likewise. >> I don't see enough complexity in here to be concerning -- so if it makes >> it harder to make mistakes, then I'm for it. > > Any comment about the -Wclass-memaccess workaround sprinkling around two > (void *) conversions? I didn't dig deep enough to look for a more > appropriate solution, also because there were some issues with older > host compilers and workarounds we installed elsewhere... Not really. It was just a couple casts and a normal looking ctor, so it didn't seem terrible. Someone with more C++-fu may have a better suggestion, but it seemed reasonable to me. > > Otherwise yes, it makes it harder to do mistakes. I'll probably > use bitmap_head::crashme instead of 0xdeadbeef in bitmap_release. > And of course we'd need to hunt down users of bitmap_clear that > should be bitmap_release instead... Right, but when we trip this kind of thing we'll know to starting digging around the bitmap_clear calls :-) That's a huge head start. jeff
Re: [PATCH][RFC] Poison bitmap_head->obstack
On Wed, 5 Dec 2018, Jeff Law wrote: > On 12/4/18 6:16 AM, Richard Biener wrote: > > > > This tries to make bugs like that in PR88317 harder to create by > > introducing a bitmap_release function that can be used as > > pendant to bitmap_initialize for non-allocated bitmap heads. > > The function makes sure to poison the bitmaps obstack member > > so the obstack the bitmap was initialized with can be safely > > released. > > > > The patch also adds a default constructor to bitmap_head > > doing the same, but for C++ reason initializes to a > > all-zero bitmap_obstack rather than 0xdeadbeef because > > the latter isn't possible in constexpr context (it is > > by using unions but then things start to look even more ugly). > > > > The stage1 compiler might end up with a few extra runtime > > initializers but constexpr makes sure they'll vanish for > > later stages. > > > > I had to paper over that you-may-not-use-memset-to-zero classes > > with non-trivial constructors warning in two places and I > > had to teach gengtype about CONSTEXPR (probably did so in > > an awkward way - suggestions and pointers into gengtype > > appreciated). > > > > Bootstrapped (with host GCC 4.8) on x86_64-unknown-linux-gnu, > > testing in progress. > > > > The LRA issue seems to be rare enough (on x86_64...) that > > I didn't trip over it sofar. > > > > Comments? Do we want this? Not sure how we can easily > > discover all bitmap_clear () users that should really > > use bitmap_release (suggestion for a better name appreciated > > as well - I thought about bitmap_uninitialize) > > > > Richard. > > > > 2018-12-04 Richard Biener > > > > * bitmap.c (bitmap_head::crashme): Define. > > * bitmap.h (bitmap_head): Add constexpr default constructor > > poisoning the obstack member. > > (bitmap_head::crashme): Declare. > > (bitmap_release): New function clearing a bitmap and poisoning > > the obstack member. > > * gengtype.c (main): Make it recognize CONSTEXPR. > > > > * lra-constraints.c (lra_inheritance): Use bitmap_release > > instead of bitmap_clear. > > > > * ira.c (ira): Work around warning. > > * regrename.c (create_new_chain): Likewise. > I don't see enough complexity in here to be concerning -- so if it makes > it harder to make mistakes, then I'm for it. Any comment about the -Wclass-memaccess workaround sprinkling around two (void *) conversions? I didn't dig deep enough to look for a more appropriate solution, also because there were some issues with older host compilers and workarounds we installed elsewhere... Otherwise yes, it makes it harder to do mistakes. I'll probably use bitmap_head::crashme instead of 0xdeadbeef in bitmap_release. And of course we'd need to hunt down users of bitmap_clear that should be bitmap_release instead... Thanks, Richard.
Re: [PATCH][RFC] Poison bitmap_head->obstack
On 12/4/18 6:16 AM, Richard Biener wrote: > > This tries to make bugs like that in PR88317 harder to create by > introducing a bitmap_release function that can be used as > pendant to bitmap_initialize for non-allocated bitmap heads. > The function makes sure to poison the bitmaps obstack member > so the obstack the bitmap was initialized with can be safely > released. > > The patch also adds a default constructor to bitmap_head > doing the same, but for C++ reason initializes to a > all-zero bitmap_obstack rather than 0xdeadbeef because > the latter isn't possible in constexpr context (it is > by using unions but then things start to look even more ugly). > > The stage1 compiler might end up with a few extra runtime > initializers but constexpr makes sure they'll vanish for > later stages. > > I had to paper over that you-may-not-use-memset-to-zero classes > with non-trivial constructors warning in two places and I > had to teach gengtype about CONSTEXPR (probably did so in > an awkward way - suggestions and pointers into gengtype > appreciated). > > Bootstrapped (with host GCC 4.8) on x86_64-unknown-linux-gnu, > testing in progress. > > The LRA issue seems to be rare enough (on x86_64...) that > I didn't trip over it sofar. > > Comments? Do we want this? Not sure how we can easily > discover all bitmap_clear () users that should really > use bitmap_release (suggestion for a better name appreciated > as well - I thought about bitmap_uninitialize) > > Richard. > > 2018-12-04 Richard Biener > > * bitmap.c (bitmap_head::crashme): Define. > * bitmap.h (bitmap_head): Add constexpr default constructor > poisoning the obstack member. > (bitmap_head::crashme): Declare. > (bitmap_release): New function clearing a bitmap and poisoning > the obstack member. > * gengtype.c (main): Make it recognize CONSTEXPR. > > * lra-constraints.c (lra_inheritance): Use bitmap_release > instead of bitmap_clear. > > * ira.c (ira): Work around warning. > * regrename.c (create_new_chain): Likewise. I don't see enough complexity in here to be concerning -- so if it makes it harder to make mistakes, then I'm for it. jeff
