Re: [PATCH] pch: Add support for PCH for relocatable executables
On Sat, Nov 13, 2021 at 08:32:41PM +, Iain Sandoe wrote: > IMO both this series > - which restores the ability to work with PIE exes but requires a known > address for the PCH > and the series I posted > - which allows a configuration to opt out of PCH anyway > > could be useful - for Darwin I prefer this series. Yeah, I think we want both and let the users choose. Finding a hole can be indeed hard on 32-bit VA, but no OS I've seen randomizes across the whole 44 or 48 or how many bits VA, otherwise e.g. address sanitizer or thread sanitizer would have no chance to work either. Having the PCH blob be relocatable would be achievable too, we have all the information in the GTY for it after all when we are able to relocate it at PCH saving time, but don't do that currently because it would be more expensive at PCH restore time. But perhaps better to do that as a fallback if we don't manage to get the right slot. Jakub
Re: [PATCH] pch: Add support for PCH for relocatable executables
Hi Folks, IMO both this series - which restores the ability to work with PIE exes but requires a known address for the PCH and the series I posted - which allows a configuration to opt out of PCH anyway could be useful - for Darwin I prefer this series. of course, it would be very nice to have a relocatable impl (or the tree streamer) .. I fear that relying on finding a fixed hole in the VM addresses is probably fragile w.r.t OS updates. > On 10 Nov 2021, at 20:24, Iain Sandoe wrote: >> On 10 Nov 2021, at 08:14, Iain Sandoe wrote: > >>> On 9 Nov 2021, at 12:18, Jakub Jelinek via Gcc-patches >>> wrote: >>> >>> On Tue, Nov 09, 2021 at 11:40:08AM +, Iain Sandoe wrote: There were two issues, of which one remains and probably affects all targets. > 2. This problem remains. > > This problem is also present on master without making any changes to the PCH > implementation - if one fixes up the read-in to simulate a corrupted file, > cc1 hangs > > (which means it’s no barrier to the revised PCH implementation) >> That seems reasonable for the case that we call fatal_error from ggc-common, >> but >> I don’t think it will work if fancy_abort is called (for e.g. a segv) - we >> might need to >> make a local fancy_abort() as well for that specific file, perhaps. >> >> Or in some way defer overwriting the data until we’ve succeeded in >> reading/relocating >> the whole file (not sure what the largest PCH is we might encounter). > > (answering my own question) around 150Mb for largest libstdc++ and similar > for an > Objective-C include of Foundation + AppKit etc. > > The underlying reason here is that diagnostics have become much more > sophisticated, > and they do all sorts of context checking and include the libcpp stuff > directly which is a lot > of GTY(()) stuff. > > I cannot immediately see any small set of state that we can save / restore > around the > PCH read in, I was wrong about that… patch posted that fixes most of this issue. === To add to Jakub's two patches that do the heavy lifting - two configure changes (I have also darwin-local changes which are under test at the moment with the intention to apply them anyway). 0001-configure-gcc-Add-enable-pie-tools.patch Description: Binary data 0002-configure-Add-top-level-configure-support-for-enable.patch Description: Binary data
Re: [PATCH] pch: Add support for PCH for relocatable executables
Hi Folks,
> On 10 Nov 2021, at 08:14, Iain Sandoe wrote:
>> On 9 Nov 2021, at 12:18, Jakub Jelinek via Gcc-patches
>> wrote:
>>
>> On Tue, Nov 09, 2021 at 11:40:08AM +, Iain Sandoe wrote:
>>> There were two issues, of which one remains and probably affects all
>>> targets.
>>> 2. This problem remains.
This problem is also present on master without making any changes to the PCH
implementation - if one fixes up the read-in to simulate a corrupted file, cc1
hangs
(which means it’s no barrier to the revised PCH implementation)
>>> - if we try to emit a diagnostic when the PCH read-in has failed, it seems
>>> that
>>> cc1 hangs somewhere in trying to lookup line table info.
>>>
>>> - this was happening with the Darwin fixed PCH memory address because it
>>> was trying to report a fatal error in being unable to read the file (or
>>> trying to
>>> execute fancy_abort, in response to a segv).
>>
>> I guess once we:
>> /* Read in all the scalar variables. */
>> for (rt = gt_pch_scalar_rtab; *rt; rt++)
>> for (rti = *rt; rti->base != NULL; rti++)
>> if (fread (rti->base, rti->stride, 1, f) != 1)
>> fatal_error (input_location, "cannot read PCH file: %m");
>>
>> /* Read in all the global pointers, in 6 easy loops. */
>> for (rt = gt_ggc_rtab; *rt; rt++)
>> for (rti = *rt; rti->base != NULL; rti++)
>> for (i = 0; i < rti->nelt; i++)
>> if (fread ((char *)rti->base + rti->stride * i,
>> sizeof (void *), 1, f) != 1)
>> fatal_error (input_location, "cannot read PCH file: %m");
>> we overwrite the GTY(()) marked global vars including
>> extern GTY(()) class line_maps *line_table;
>> with pointers into the area we haven't mapped yet (or if the error happens
>> after that mmap but before everything is fixed up (e.g. the new relocation
>> processing), it is no wonder it doesn't work well.
indeed.
>>
>> Could we save line_table (and perhaps a few other vars) into non-GTY! copies
>> of them in ggc-common.c and instead of those fatal_error (input_location,
>> ...)
>> calls in gt_pch_restore and ggc_pch_read call fatal_pch_error (...) where
>> void
>> fatal_pch_error (const char *gmsg)
>> {
>> line_table = saved_line_table;
>> // Restore anything else that is needed for fatal_error
>> fatal_error (input_location, gmsg);
>> }
>
> That seems reasonable for the case that we call fatal_error from ggc-common,
> but
> I don’t think it will work if fancy_abort is called (for e.g. a segv) - we
> might need to
> make a local fancy_abort() as well for that specific file, perhaps.
>
> Or in some way defer overwriting the data until we’ve succeeded in
> reading/relocating
> the whole file (not sure what the largest PCH is we might encounter).
(answering my own question) around 150Mb for largest libstdc++ and similar for
an
Objective-C include of Foundation + AppKit etc.
The underlying reason here is that diagnostics have become much more
sophisticated,
and they do all sorts of context checking and include the libcpp stuff directly
which is a lot
of GTY(()) stuff.
I cannot immediately see any small set of state that we can save / restore
around the
PCH read in,
Perhaps what would be more realistic would be a call into the diagnostics stuff
to say
“disable fancy stuff and just report the minimum”,
as noted, I don’t think this (second) issue is actually a barrier to making the
PCH change
since it’s preexisting - I just ran into it while debugging suitable VM
addresses to use with
ASLR on.
thoughts?
Iain
Re: [PATCH] pch: Add support for PCH for relocatable executables
> On 9 Nov 2021, at 12:18, Jakub Jelinek via Gcc-patches
> wrote:
>
> On Tue, Nov 09, 2021 at 11:40:08AM +, Iain Sandoe wrote:
>> There were two issues, of which one remains and probably affects all targets.
>>
>> 1. The Darwin PCH memory allocation scheme used a system that works reliably
>>for no-PIE but not for PIE
>>
>> .. I hacked in a similar scheme to the mmap one used on Linux .. the suspect
>> stuff
>> there is in choosing some place in the map that is likely to succeed…
>>
>> With that I get passes on all c-family pch.exp (I didn’t try to bootstrap).
>
> Yeah, certainly.
Overnight testing for i686, powerpc and x86_64 darwin suggests I’ve found some
suitable compromise map addresses (but that scheme has always seemed a bit
fragile if the ASLR parameters get updated for a new OS edition).
>> 2. This problem remains.
>>
>> - if we try to emit a diagnostic when the PCH read-in has failed, it seems
>> that
>> cc1 hangs somewhere in trying to lookup line table info.
>>
>> - this was happening with the Darwin fixed PCH memory address because it
>> was trying to report a fatal error in being unable to read the file (or
>> trying to
>> execute fancy_abort, in response to a segv).
>
> I guess once we:
> /* Read in all the scalar variables. */
> for (rt = gt_pch_scalar_rtab; *rt; rt++)
>for (rti = *rt; rti->base != NULL; rti++)
> if (fread (rti->base, rti->stride, 1, f) != 1)
>fatal_error (input_location, "cannot read PCH file: %m");
>
> /* Read in all the global pointers, in 6 easy loops. */
> for (rt = gt_ggc_rtab; *rt; rt++)
>for (rti = *rt; rti->base != NULL; rti++)
> for (i = 0; i < rti->nelt; i++)
>if (fread ((char *)rti->base + rti->stride * i,
> sizeof (void *), 1, f) != 1)
> fatal_error (input_location, "cannot read PCH file: %m");
> we overwrite the GTY(()) marked global vars including
> extern GTY(()) class line_maps *line_table;
> with pointers into the area we haven't mapped yet (or if the error happens
> after that mmap but before everything is fixed up (e.g. the new relocation
> processing), it is no wonder it doesn't work well.
>
> Could we save line_table (and perhaps a few other vars) into non-GTY! copies
> of them in ggc-common.c and instead of those fatal_error (input_location, ...)
> calls in gt_pch_restore and ggc_pch_read call fatal_pch_error (...) where
> void
> fatal_pch_error (const char *gmsg)
> {
> line_table = saved_line_table;
> // Restore anything else that is needed for fatal_error
> fatal_error (input_location, gmsg);
> }
That seems reasonable for the case that we call fatal_error from ggc-common, but
I don’t think it will work if fancy_abort is called (for e.g. a segv) - we
might need to
make a local fancy_abort() as well for that specific file, perhaps.
Or in some way defer overwriting the data until we’ve succeeded in
reading/relocating
the whole file (not sure what the largest PCH is we might encounter).
ISTR that we force clear everything before starting the read, since I had
problems with
phasing diagnostic output when making a previous change to this area, so the
snapshot
might be needed quite early.
Iain
>
> Jakub
>
Re: [PATCH] pch: Add support for PCH for relocatable executables
On Tue, Nov 09, 2021 at 10:23:19AM -0500, Andrew MacLeod wrote:
> yeah, that doesnt work because range_query is a pure virtual. However, there
> also does not seem to be any reason why we need to jump thru hoops since
> get_range_query() doesn't need to be in function.h.. If I relocate it to
> value-query.h like so it seems to work quite well... How about this?
Ah, ok. Indeed while one can extern global_range_query global_ranges;
with incomplete global_range_query type, inheritance will not be known.
> --- a/gcc/value-query.h
> +++ b/gcc/value-query.h
> @@ -127,6 +127,17 @@ public:
> };
>
> extern global_range_query global_ranges;
> +inline range_query *get_global_range_query () { return _ranges; }
Formatting, there should be empty line after global_ranges, and
inline range_query *
get_global_range_entry ()
{
return _ranges;
}
> +
> +/* Returns the currently active range access class. When there is no active
> + range class, global ranges are used. Never returns null. */
> +
> +ATTRIBUTE_RETURNS_NONNULL inline range_query *
> +get_range_query (const struct function *fun)
> +{
> + return fun->x_range_query ? fun->x_range_query : _ranges;
> +}
> +
> extern value_range gimple_range_global (tree name);
> extern bool update_global_range (irange , tree name);
>
Ok for trunk with the above nits fixed if it passes bootstrap/regtest,
thanks.
Jakub
Re: [PATCH] pch: Add support for PCH for relocatable executables
On 11/9/21 9:58 AM, Jakub Jelinek wrote:
On Tue, Nov 09, 2021 at 09:41:08AM -0500, Andrew MacLeod wrote:
Yeah, Im not particular about how we do this... I think thats perfectly
reasonable. Would something like the following solve this issue?
Yes, but see below.
commit 17a5b03c95549b5488bc8dd2af4f6e2cc9ddf098
Author: Andrew MacLeod
Date: Tue Nov 9 09:29:23 2021 -0500
Keep x_range_query NULL for global ranges.
Instead of x_range_query alwasy pointing to an object, have it default to
NULL and return a pointer to the global query in that case.
* function.c (allocate_struct_function): Set x_range_query to NULL.
* function.h (get_range_query): Return context query or global.
* gimple-range.cc (enable_ranger): Check current query is NULL.
(disable_ranger): Clear function current query field.
* value_query.cc (global_range_query_ptr): New.
* value-query.h (global_ranges): Remove.
diff --git a/gcc/function.c b/gcc/function.c
index af3d57b32a3..8768c5fcf22 100644
--- a/gcc/function.c
+++ b/gcc/function.c
@@ -4874,7 +4874,7 @@ allocate_struct_function (tree fndecl, bool abstract_p)
cfun->debug_nonbind_markers = lang_hooks.emits_begin_stmt
&& MAY_HAVE_DEBUG_MARKER_STMTS;
- cfun->x_range_query = _ranges;
+ cfun->x_range_query = NULL;
This isn't needed, at the start of function we do
cfun = ggc_cleared_alloc ();
which already zero initializes the whole structure, including x_range_query.
So instead this can be removed.
--- a/gcc/function.h
+++ b/gcc/function.h
@@ -725,7 +725,9 @@ extern void used_types_insert (tree);
ATTRIBUTE_RETURNS_NONNULL inline range_query *
get_range_query (const struct function *fun)
{
- return fun->x_range_query;
+ // From value-query.h
+ extern range_query *global_range_query_ptr;
+ return fun->x_range_query ? fun->x_range_query : global_range_query_ptr;
Wouldn't it be better to do:
extern range_query global_ranges;
return fun->x_range_query ? fun->x_range_query : _ranges;
I think declaring a variable extern can be done with incomplete type
and is cheaper than ptr, because for the latter you need to
read the pointer value from memory, while for you can just
compute the address of the var which you need to compute for reading
ptr from memory too.
yeah, that doesnt work because range_query is a pure virtual. However,
there also does not seem to be any reason why we need to jump thru hoops
since get_range_query() doesn't need to be in function.h.. If I
relocate it to value-query.h like so it seems to work quite well...
How about this?
Andrew
commit 0d5b27e95b7aef4415163e4277de06b48437d6f8
Author: Andrew MacLeod
Date: Tue Nov 9 09:29:23 2021 -0500
Keep x_range_query NULL for global ranges.
Instead of x_range_query always pointing to an object, have it default to
NULL and return a pointer to the global query in that case.
* function.c (allocate_struct_function): Don't set x_range_query.
* function.h (get_range_query): Move to value-query.h.
* gimple-range.cc (enable_ranger): Check that query is currently NULL.
(disable_ranger): Clear function current query field.
* value_query.cc (get_global_range_query): Relocate to:
* value-query.h (get_global_range_query): Here and inline.
(get_range_query): Relocate here from function.h.
diff --git a/gcc/function.c b/gcc/function.c
index af3d57b32a3..61b3bd036b8 100644
--- a/gcc/function.c
+++ b/gcc/function.c
@@ -4873,8 +4873,6 @@ allocate_struct_function (tree fndecl, bool abstract_p)
binding annotations among them. */
cfun->debug_nonbind_markers = lang_hooks.emits_begin_stmt
&& MAY_HAVE_DEBUG_MARKER_STMTS;
-
- cfun->x_range_query = _ranges;
}
/* This is like allocate_struct_function, but pushes a new cfun for FNDECL
diff --git a/gcc/function.h b/gcc/function.h
index 36003e7576a..899430833ce 100644
--- a/gcc/function.h
+++ b/gcc/function.h
@@ -719,15 +719,4 @@ extern const char *current_function_name (void);
extern void used_types_insert (tree);
-/* Returns the currently active range access class. When there is no active
- range class, global ranges are used. Never returns null. */
-
-ATTRIBUTE_RETURNS_NONNULL inline range_query *
-get_range_query (const struct function *fun)
-{
- return fun->x_range_query;
-}
-
-extern range_query *get_global_range_query ();
-
#endif /* GCC_FUNCTION_H */
diff --git a/gcc/gimple-range.cc b/gcc/gimple-range.cc
index 87dba6e81d8..a2b68b2bc80 100644
--- a/gcc/gimple-range.cc
+++ b/gcc/gimple-range.cc
@@ -467,6 +467,7 @@ enable_ranger (struct function *fun)
{
gimple_ranger *r;
+ gcc_checking_assert (!fun->x_range_query);
r = new gimple_ranger;
fun->x_range_query = r;
@@ -479,7 +480,7 @@ enable_ranger (struct function *fun)
void
disable_ranger (struct function *fun)
{
+
Re: [PATCH] pch: Add support for PCH for relocatable executables
On Tue, Nov 09, 2021 at 09:41:08AM -0500, Andrew MacLeod wrote:
> Yeah, Im not particular about how we do this... I think thats perfectly
> reasonable. Would something like the following solve this issue?
Yes, but see below.
> commit 17a5b03c95549b5488bc8dd2af4f6e2cc9ddf098
> Author: Andrew MacLeod
> Date: Tue Nov 9 09:29:23 2021 -0500
>
> Keep x_range_query NULL for global ranges.
>
> Instead of x_range_query alwasy pointing to an object, have it default to
> NULL and return a pointer to the global query in that case.
>
> * function.c (allocate_struct_function): Set x_range_query to
> NULL.
> * function.h (get_range_query): Return context query or global.
> * gimple-range.cc (enable_ranger): Check current query is NULL.
> (disable_ranger): Clear function current query field.
> * value_query.cc (global_range_query_ptr): New.
> * value-query.h (global_ranges): Remove.
>
> diff --git a/gcc/function.c b/gcc/function.c
> index af3d57b32a3..8768c5fcf22 100644
> --- a/gcc/function.c
> +++ b/gcc/function.c
> @@ -4874,7 +4874,7 @@ allocate_struct_function (tree fndecl, bool abstract_p)
>cfun->debug_nonbind_markers = lang_hooks.emits_begin_stmt
> && MAY_HAVE_DEBUG_MARKER_STMTS;
>
> - cfun->x_range_query = _ranges;
> + cfun->x_range_query = NULL;
This isn't needed, at the start of function we do
cfun = ggc_cleared_alloc ();
which already zero initializes the whole structure, including x_range_query.
So instead this can be removed.
> --- a/gcc/function.h
> +++ b/gcc/function.h
> @@ -725,7 +725,9 @@ extern void used_types_insert (tree);
> ATTRIBUTE_RETURNS_NONNULL inline range_query *
> get_range_query (const struct function *fun)
> {
> - return fun->x_range_query;
> + // From value-query.h
> + extern range_query *global_range_query_ptr;
> + return fun->x_range_query ? fun->x_range_query : global_range_query_ptr;
Wouldn't it be better to do:
extern range_query global_ranges;
return fun->x_range_query ? fun->x_range_query : _ranges;
I think declaring a variable extern can be done with incomplete type
and is cheaper than ptr, because for the latter you need to
read the pointer value from memory, while for you can just
compute the address of the var which you need to compute for reading
ptr from memory too.
Jakub
Re: [PATCH] pch: Add support for PCH for relocatable executables
On 11/9/21 7:29 AM, Jakub Jelinek wrote:
On Tue, Nov 09, 2021 at 01:03:38PM +0100, Richard Biener wrote:
Apparently the range_of_expr can handle some tree cases through
range_query::get_tree_range, like INTEGER_CSTs, ADDR_EXPRs,
and some binary and unary ops.
But that shouldn't need a range query object ... this was all
available pre-ranger and just got stuffed there for no good reason?
resolving the binary ops requires calls back into range_of_expr to
resolve operands. It could be split out if needed/desired.
That is for Andrew/Aldy to answer.
All I can say is that get_tree_range is a non-static member function
of range_query and therefore it needs non-NULL query object.
But I must say I wonder if all this pain is worth it, if it wouldn't
be easier to keep cfun->x_range_query NULL most of the time and use
ATTRIBUTE_RETURNS_NONNULL inline range_query *
get_range_query (const struct function *fun)
{
return fun->x_range_query ? fun->x_range_query : _ranges;
}
(of course, the function then would need to be in some header
where global_ranges is declared).
Jakub
Yeah, Im not particular about how we do this... I think thats perfectly
reasonable. Would something like the following solve this issue?
It creates a global-range class pointer, initializes it to point to the
global query, and we can simply hide its existence and refer to it
directly from function.h if you thinks thats reasonable and will work OK
for this. Then we dont have any inclusion issues.
Let me know and I'll run it thru the gauntlet.
Andrew
commit 17a5b03c95549b5488bc8dd2af4f6e2cc9ddf098
Author: Andrew MacLeod
Date: Tue Nov 9 09:29:23 2021 -0500
Keep x_range_query NULL for global ranges.
Instead of x_range_query alwasy pointing to an object, have it default to
NULL and return a pointer to the global query in that case.
* function.c (allocate_struct_function): Set x_range_query to NULL.
* function.h (get_range_query): Return context query or global.
* gimple-range.cc (enable_ranger): Check current query is NULL.
(disable_ranger): Clear function current query field.
* value_query.cc (global_range_query_ptr): New.
* value-query.h (global_ranges): Remove.
diff --git a/gcc/function.c b/gcc/function.c
index af3d57b32a3..8768c5fcf22 100644
--- a/gcc/function.c
+++ b/gcc/function.c
@@ -4874,7 +4874,7 @@ allocate_struct_function (tree fndecl, bool abstract_p)
cfun->debug_nonbind_markers = lang_hooks.emits_begin_stmt
&& MAY_HAVE_DEBUG_MARKER_STMTS;
- cfun->x_range_query = _ranges;
+ cfun->x_range_query = NULL;
}
/* This is like allocate_struct_function, but pushes a new cfun for FNDECL
diff --git a/gcc/function.h b/gcc/function.h
index 36003e7576a..3c1b2aa2b90 100644
--- a/gcc/function.h
+++ b/gcc/function.h
@@ -725,7 +725,9 @@ extern void used_types_insert (tree);
ATTRIBUTE_RETURNS_NONNULL inline range_query *
get_range_query (const struct function *fun)
{
- return fun->x_range_query;
+ // From value-query.h
+ extern range_query *global_range_query_ptr;
+ return fun->x_range_query ? fun->x_range_query : global_range_query_ptr;
}
extern range_query *get_global_range_query ();
diff --git a/gcc/gimple-range.cc b/gcc/gimple-range.cc
index 87dba6e81d8..a2b68b2bc80 100644
--- a/gcc/gimple-range.cc
+++ b/gcc/gimple-range.cc
@@ -467,6 +467,7 @@ enable_ranger (struct function *fun)
{
gimple_ranger *r;
+ gcc_checking_assert (!fun->x_range_query);
r = new gimple_ranger;
fun->x_range_query = r;
@@ -479,7 +480,7 @@ enable_ranger (struct function *fun)
void
disable_ranger (struct function *fun)
{
+ gcc_checking_assert (fun->x_range_query);
delete fun->x_range_query;
-
- fun->x_range_query = _ranges;
+ fun->x_range_query = NULL;
}
diff --git a/gcc/value-query.cc b/gcc/value-query.cc
index 17ebd86ce5f..8d1b27d9bfb 100644
--- a/gcc/value-query.cc
+++ b/gcc/value-query.cc
@@ -433,7 +433,9 @@ gimple_range_global (tree name)
// --
// global_range_query implementation.
+// This is utlized by function.h get_range_query() only.
global_range_query global_ranges;
+range_query *global_range_query_ptr = _ranges;
// Like get_range_query, but for accessing global ranges.
diff --git a/gcc/value-query.h b/gcc/value-query.h
index 5161d23714b..f56abc4777c 100644
--- a/gcc/value-query.h
+++ b/gcc/value-query.h
@@ -126,7 +126,6 @@ public:
bool range_of_expr (irange , tree expr, gimple * = NULL) OVERRIDE;
};
-extern global_range_query global_ranges;
extern value_range gimple_range_global (tree name);
extern bool update_global_range (irange , tree name);
Re: [PATCH] pch: Add support for PCH for relocatable executables
On Tue, Nov 09, 2021 at 01:03:38PM +0100, Richard Biener wrote:
> > Apparently the range_of_expr can handle some tree cases through
> > range_query::get_tree_range, like INTEGER_CSTs, ADDR_EXPRs,
> > and some binary and unary ops.
>
> But that shouldn't need a range query object ... this was all
> available pre-ranger and just got stuffed there for no good reason?
That is for Andrew/Aldy to answer.
All I can say is that get_tree_range is a non-static member function
of range_query and therefore it needs non-NULL query object.
But I must say I wonder if all this pain is worth it, if it wouldn't
be easier to keep cfun->x_range_query NULL most of the time and use
ATTRIBUTE_RETURNS_NONNULL inline range_query *
get_range_query (const struct function *fun)
{
return fun->x_range_query ? fun->x_range_query : _ranges;
}
(of course, the function then would need to be in some header
where global_ranges is declared).
Jakub
Re: [PATCH] pch: Add support for PCH for relocatable executables
On Tue, Nov 09, 2021 at 11:40:08AM +, Iain Sandoe wrote:
> There were two issues, of which one remains and probably affects all targets.
>
> 1. The Darwin PCH memory allocation scheme used a system that works reliably
> for no-PIE but not for PIE
>
> .. I hacked in a similar scheme to the mmap one used on Linux .. the suspect
> stuff
>there is in choosing some place in the map that is likely to succeed…
>
> With that I get passes on all c-family pch.exp (I didn’t try to bootstrap).
Yeah, certainly.
> 2. This problem remains.
>
> - if we try to emit a diagnostic when the PCH read-in has failed, it seems
> that
>cc1 hangs somewhere in trying to lookup line table info.
>
> - this was happening with the Darwin fixed PCH memory address because it
>was trying to report a fatal error in being unable to read the file (or
> trying to
> execute fancy_abort, in response to a segv).
I guess once we:
/* Read in all the scalar variables. */
for (rt = gt_pch_scalar_rtab; *rt; rt++)
for (rti = *rt; rti->base != NULL; rti++)
if (fread (rti->base, rti->stride, 1, f) != 1)
fatal_error (input_location, "cannot read PCH file: %m");
/* Read in all the global pointers, in 6 easy loops. */
for (rt = gt_ggc_rtab; *rt; rt++)
for (rti = *rt; rti->base != NULL; rti++)
for (i = 0; i < rti->nelt; i++)
if (fread ((char *)rti->base + rti->stride * i,
sizeof (void *), 1, f) != 1)
fatal_error (input_location, "cannot read PCH file: %m");
we overwrite the GTY(()) marked global vars including
extern GTY(()) class line_maps *line_table;
with pointers into the area we haven't mapped yet (or if the error happens
after that mmap but before everything is fixed up (e.g. the new relocation
processing), it is no wonder it doesn't work well.
Could we save line_table (and perhaps a few other vars) into non-GTY! copies
of them in ggc-common.c and instead of those fatal_error (input_location, ...)
calls in gt_pch_restore and ggc_pch_read call fatal_pch_error (...) where
void
fatal_pch_error (const char *gmsg)
{
line_table = saved_line_table;
// Restore anything else that is needed for fatal_error
fatal_error (input_location, gmsg);
}
Jakub
Re: [PATCH] pch: Add support for PCH for relocatable executables
On Tue, 9 Nov 2021, Jakub Jelinek wrote:
> On Tue, Nov 09, 2021 at 10:44:45AM +0100, Jakub Jelinek via Gcc-patches wrote:
> > On Tue, Nov 09, 2021 at 08:12:05AM +0100, Richard Biener wrote:
> > > > So, here is 1), 2), 3) implemented. With this patch alone,
> > > > g++.dg/pch/system-2.C test ICEs. This is because GCC 12 has added
> > > > function::x_range_query member, which is set to _ranges on
> > > > cfun creation and is:
> > > > range_query * GTY ((skip)) x_range_query;
> > > > which means when a PIE binary writes PCH and a PIE binary loaded
> > > > at a different address loads it, cfun->x_range_query might be a garbage
> > > > pointer. We can either apply a patch like the attached one after
> > > > this inline patch, but then probably callback is misnamed and we should
> > > > rename it to relocate_and_skip or something similar. Or we could
> > > > e.g. during gimplification overwrite cfun->x_range_query =
> > > > _ranges.
> > >
> > > I think struct function allocation should initialize it to NULL and
> > > the init to _ranges be done only when we do init_tree_ssa?
> > > In fact x_range_query could be moved to the gimple_df substructure
> > > to make that clear.
> >
> > Agreed, Andrew/Aldy, what do you think?
> >
> > > Hopefully PCH happens before init_tree_ssa.
> >
> > I think it does.
>
> Unfortunately, seems cfun->x_range_query is used already in the FEs :(.
>
> I was trying:
>
> --- gcc/function.h.jj 2021-08-31 22:55:23.072795814 +0200
> +++ gcc/function.h2021-11-09 11:33:22.656779018 +0100
> @@ -312,8 +312,9 @@ struct GTY(()) function {
>
>/* Range query mechanism for functions. The default is to pick up
> global ranges. If a pass wants on-demand ranges OTOH, it must
> - call enable/disable_ranger(). The pointer is never null. It
> - should be queried by calling get_range_query(). */
> + call enable/disable_ranger(). The pointer is never null in between
> + init_tree_ssa and delete_tree_ssa. It should be queried by calling
> + get_range_query(). */
>range_query * GTY ((skip)) x_range_query;
>
>/* Last statement uid. */
> --- gcc/function.c.jj 2021-07-20 22:31:11.088835781 +0200
> +++ gcc/function.c2021-11-09 11:33:47.695424319 +0100
> @@ -4873,8 +4873,6 @@ allocate_struct_function (tree fndecl, b
> binding annotations among them. */
>cfun->debug_nonbind_markers = lang_hooks.emits_begin_stmt
> && MAY_HAVE_DEBUG_MARKER_STMTS;
> -
> - cfun->x_range_query = _ranges;
> }
>
> /* This is like allocate_struct_function, but pushes a new cfun for FNDECL
> --- gcc/tree-ssa.c.jj 2021-11-03 23:02:44.367985554 +0100
> +++ gcc/tree-ssa.c2021-11-09 12:02:07.095351378 +0100
> @@ -44,6 +44,7 @@ along with GCC; see the file COPYING3.
> #include "stringpool.h"
> #include "attribs.h"
> #include "asan.h"
> +#include "value-query.h"
>
> /* Pointer map of variable mappings, keyed by edge. */
> static hash_map > *edge_var_maps;
> @@ -1224,6 +1225,7 @@ init_tree_ssa (struct function *fn, int
> {
>fn->gimple_df = ggc_cleared_alloc ();
>fn->gimple_df->default_defs = hash_table::create_ggc (20);
> + fn->x_range_query = _ranges;
>pt_solution_reset (>gimple_df->escaped);
>init_ssanames (fn, size);
> }
> @@ -1246,6 +1248,7 @@ delete_tree_ssa (struct function *fn)
> delete fn->gimple_df->decls_to_pointers;
>fn->gimple_df->decls_to_pointers = NULL;
>fn->gimple_df = NULL;
> + fn->x_range_query = NULL;
>
>/* We no longer need the edge variable maps. */
>redirect_edge_var_map_empty ();
>
> but that ICEs with:
> #0 0x56d27348 in get_range (val=val@entry=0x7fffe9f8c2d0,
> stmt=0x7fffbb80, stmt@entry=0x0, minmax=minmax@entry=0x7fffbc10,
> rvals=0x0)
> at ../../gcc/tree-ssa-strlen.c:217
> #1 0x56a2fe73 in get_offset_range (x=0x7fffe9f8c2d0, stmt=0x0,
> r=0x7fffbd70, rvals=) at ../../gcc/pointer-query.cc:92
> #2 0x56a33d3e in handle_array_ref (aref=0x7fffe7e17620,
> addr=, ostype=1, pref=0x7fffc000, snlim=...,
> qry=, stmt=)
> at ../../gcc/pointer-query.cc:1621
> #3 0x56a3669d in compute_objsize (ptr=0x7fffe81b3100,
> stmt=, ostype=1, pref=0x7fffc000, ptr_qry=0x7fffbf00)
> at ../../gcc/pointer-query.cc:2154
> #4 0x56a368e4 in compute_objsize (ptr=ptr@entry=0x7fffe81b3100,
> stmt=stmt@entry=0x0, ostype=ostype@entry=1, pref=pref@entry=0x7fffc000,
> rvals=rvals@entry=0x0)
> at ../../gcc/pointer-query.cc:2172
> #5 0x56383f09 in compute_objsize (pref=0x7fffc000, ostype=1,
> ptr=0x7fffe81b3100) at ../../gcc/pointer-query.h:262
> #6 warn_placement_new_too_small (type=0x7fffe9f8a3f0, nelts=0x7fffe81b3160,
> size=0x7fffe9f8c108, oper=0x7fffe81b3100) at ../../gcc/cp/init.c:2621
> #7 0x5638cf9e in build_new_1 (placement=,
> type=0x7fffe9f8a3f0, nelts=, init=0x7fffc3d0,
> globally_qualified_p=, complain=3)
> at ../../gcc/cp/init.c:3287
> #8 0x5638dd92 in build_new
Re: [PATCH] pch: Add support for PCH for relocatable executables
> On 9 Nov 2021, at 08:07, Iain Sandoe wrote: > > > >> On 9 Nov 2021, at 07:12, Richard Biener wrote: >> >> On Mon, 8 Nov 2021, Jakub Jelinek wrote: >> >>> On Mon, Nov 08, 2021 at 12:46:04PM +0100, Jakub Jelinek via Gcc-patches >>> wrote: So, if we want to make PCH work for PIEs, I'd say we can: 1) add a new GTY option, say callback, which would act like skip for non-PCH and for PCH would make us skip it but remember for address bias translation 2) drop the skip for tree_translation_unit_decl::language 3) change get_unnamed_section to have const char * as last argument instead of const void *, change unnamed_section::data also to const char * and update everything related to that 4) maybe add a host hook whether it is ok to support binaries changing addresses (the only thing I'm worried is if some host that uses function descriptors allocates them dynamically instead of having them somewhere in the executable) 5) maybe add a gengtype warning if it sees in GTY tracked structure a function pointer without that new callback option >>> >>> So, here is 1), 2), 3) implemented. With this patch alone, >>> g++.dg/pch/system-2.C test ICEs. This is because GCC 12 has added >>> function::x_range_query member, which is set to _ranges on >>> cfun creation and is: >>> range_query * GTY ((skip)) x_range_query; >>> which means when a PIE binary writes PCH and a PIE binary loaded >>> at a different address loads it, cfun->x_range_query might be a garbage >>> pointer. We can either apply a patch like the attached one after >>> this inline patch, but then probably callback is misnamed and we should >>> rename it to relocate_and_skip or something similar. Or we could >>> e.g. during gimplification overwrite cfun->x_range_query = _ranges. >> >> I think struct function allocation should initialize it to NULL and >> the init to _ranges be done only when we do init_tree_ssa? >> In fact x_range_query could be moved to the gimple_df substructure >> to make that clear. >> >> Hopefully PCH happens before init_tree_ssa. >> >>> Other than that make check-gcc check-g++ passes RUNTESTFLAGS=pch.exp. >>> >>> Not really sure about PA or IA-64 function descriptors, are any of those >>> allocated by the dynamic linker rather than created by the static linker? >>> I guess instead of removing the c-pch.c changes we could remember there >>> not just a function pointer, but also a data pointer and compare if both >>> are either the same or have the same load bias and punt only if they >>> have different bias. Though, on architecture where all function pointers >>> would be dynamically allocated and could change any time even that wouldn't >>> be really a reliable check. >>> >>> Note, on stdc++.h.gch/O2g.gch there are just those 10 relocations without >>> the second patch, with it a few more, but nothing huge. And for non-PIEs >>> there isn't really any extra work on the load side except freading two >>> scalar >>> values and fseek. >>> >>> Thoughts on this? > > - thanks for doing it, I still think a working solution is better than a > disable. > > - This would be a fix for PR 71934 > > - I applied the patches on Darwin and there’s still some case(s) to find and > fix > cc1/cc1plus hang on pch input, when any diagnostic is emitted (doing some > linemap lookup) .. hopefully, I can find some time to figure out what is not > happy. There were two issues, of which one remains and probably affects all targets. 1. The Darwin PCH memory allocation scheme used a system that works reliably for no-PIE but not for PIE .. I hacked in a similar scheme to the mmap one used on Linux .. the suspect stuff there is in choosing some place in the map that is likely to succeed… With that I get passes on all c-family pch.exp (I didn’t try to bootstrap). 2. This problem remains. - if we try to emit a diagnostic when the PCH read-in has failed, it seems that cc1 hangs somewhere in trying to lookup line table info. - this was happening with the Darwin fixed PCH memory address because it was trying to report a fatal error in being unable to read the file (or trying to execute fancy_abort, in response to a segv). .. I didn’t try this on Linux - but I would imagine it would be quick to see if the problem manifests, one would only need to simulate an error. Iain > >> I'm not keen on further life support for PCH, but then if the work is >> done it's hard to reject it... >> >> Note there might be still platforms not supporting PCH and so Iains >> patches still look useful to me. > > Yeah, what would be most useful at this stage is some way to get those > tested against some wider set of package builds to see if anything fires > that breaks stuff. > >> I also wonder whether we want to require active marking of types >> with PCH support and assert when PCH write runs into objects without? > > anything that
Re: [PATCH] pch: Add support for PCH for relocatable executables
On Tue, Nov 09, 2021 at 10:44:45AM +0100, Jakub Jelinek via Gcc-patches wrote:
> On Tue, Nov 09, 2021 at 08:12:05AM +0100, Richard Biener wrote:
> > > So, here is 1), 2), 3) implemented. With this patch alone,
> > > g++.dg/pch/system-2.C test ICEs. This is because GCC 12 has added
> > > function::x_range_query member, which is set to _ranges on
> > > cfun creation and is:
> > > range_query * GTY ((skip)) x_range_query;
> > > which means when a PIE binary writes PCH and a PIE binary loaded
> > > at a different address loads it, cfun->x_range_query might be a garbage
> > > pointer. We can either apply a patch like the attached one after
> > > this inline patch, but then probably callback is misnamed and we should
> > > rename it to relocate_and_skip or something similar. Or we could
> > > e.g. during gimplification overwrite cfun->x_range_query = _ranges.
> >
> > I think struct function allocation should initialize it to NULL and
> > the init to _ranges be done only when we do init_tree_ssa?
> > In fact x_range_query could be moved to the gimple_df substructure
> > to make that clear.
>
> Agreed, Andrew/Aldy, what do you think?
>
> > Hopefully PCH happens before init_tree_ssa.
>
> I think it does.
Unfortunately, seems cfun->x_range_query is used already in the FEs :(.
I was trying:
--- gcc/function.h.jj 2021-08-31 22:55:23.072795814 +0200
+++ gcc/function.h 2021-11-09 11:33:22.656779018 +0100
@@ -312,8 +312,9 @@ struct GTY(()) function {
/* Range query mechanism for functions. The default is to pick up
global ranges. If a pass wants on-demand ranges OTOH, it must
- call enable/disable_ranger(). The pointer is never null. It
- should be queried by calling get_range_query(). */
+ call enable/disable_ranger(). The pointer is never null in between
+ init_tree_ssa and delete_tree_ssa. It should be queried by calling
+ get_range_query(). */
range_query * GTY ((skip)) x_range_query;
/* Last statement uid. */
--- gcc/function.c.jj 2021-07-20 22:31:11.088835781 +0200
+++ gcc/function.c 2021-11-09 11:33:47.695424319 +0100
@@ -4873,8 +4873,6 @@ allocate_struct_function (tree fndecl, b
binding annotations among them. */
cfun->debug_nonbind_markers = lang_hooks.emits_begin_stmt
&& MAY_HAVE_DEBUG_MARKER_STMTS;
-
- cfun->x_range_query = _ranges;
}
/* This is like allocate_struct_function, but pushes a new cfun for FNDECL
--- gcc/tree-ssa.c.jj 2021-11-03 23:02:44.367985554 +0100
+++ gcc/tree-ssa.c 2021-11-09 12:02:07.095351378 +0100
@@ -44,6 +44,7 @@ along with GCC; see the file COPYING3.
#include "stringpool.h"
#include "attribs.h"
#include "asan.h"
+#include "value-query.h"
/* Pointer map of variable mappings, keyed by edge. */
static hash_map > *edge_var_maps;
@@ -1224,6 +1225,7 @@ init_tree_ssa (struct function *fn, int
{
fn->gimple_df = ggc_cleared_alloc ();
fn->gimple_df->default_defs = hash_table::create_ggc (20);
+ fn->x_range_query = _ranges;
pt_solution_reset (>gimple_df->escaped);
init_ssanames (fn, size);
}
@@ -1246,6 +1248,7 @@ delete_tree_ssa (struct function *fn)
delete fn->gimple_df->decls_to_pointers;
fn->gimple_df->decls_to_pointers = NULL;
fn->gimple_df = NULL;
+ fn->x_range_query = NULL;
/* We no longer need the edge variable maps. */
redirect_edge_var_map_empty ();
but that ICEs with:
#0 0x56d27348 in get_range (val=val@entry=0x7fffe9f8c2d0,
stmt=0x7fffbb80, stmt@entry=0x0, minmax=minmax@entry=0x7fffbc10,
rvals=0x0)
at ../../gcc/tree-ssa-strlen.c:217
#1 0x56a2fe73 in get_offset_range (x=0x7fffe9f8c2d0, stmt=0x0,
r=0x7fffbd70, rvals=) at ../../gcc/pointer-query.cc:92
#2 0x56a33d3e in handle_array_ref (aref=0x7fffe7e17620,
addr=, ostype=1, pref=0x7fffc000, snlim=..., qry=, stmt=)
at ../../gcc/pointer-query.cc:1621
#3 0x56a3669d in compute_objsize (ptr=0x7fffe81b3100, stmt=, ostype=1, pref=0x7fffc000, ptr_qry=0x7fffbf00) at
../../gcc/pointer-query.cc:2154
#4 0x56a368e4 in compute_objsize (ptr=ptr@entry=0x7fffe81b3100,
stmt=stmt@entry=0x0, ostype=ostype@entry=1, pref=pref@entry=0x7fffc000,
rvals=rvals@entry=0x0)
at ../../gcc/pointer-query.cc:2172
#5 0x56383f09 in compute_objsize (pref=0x7fffc000, ostype=1,
ptr=0x7fffe81b3100) at ../../gcc/pointer-query.h:262
#6 warn_placement_new_too_small (type=0x7fffe9f8a3f0, nelts=0x7fffe81b3160,
size=0x7fffe9f8c108, oper=0x7fffe81b3100) at ../../gcc/cp/init.c:2621
#7 0x5638cf9e in build_new_1 (placement=,
type=0x7fffe9f8a3f0, nelts=, init=0x7fffc3d0,
globally_qualified_p=, complain=3)
at ../../gcc/cp/init.c:3287
#8 0x5638dd92 in build_new (loc=,
placement=placement@entry=0x7fffc3c8, type=,
type@entry=0x7fffe9f8a3f0, nelts=0x7fffe81b3160,
nelts@entry=0x7fffe81b3120, init=init@entry=0x7fffc3d0,
use_global_new=use_global_new@entry=0, complain=3) at ../../gcc/cp/init.c:3838
Re: [PATCH] pch: Add support for PCH for relocatable executables
On Mon, Nov 08, 2021 at 04:03:09PM -0500, John David Anglin wrote:
> On 2021-11-08 2:48 p.m., Jakub Jelinek wrote:
> > Not really sure about PA or IA-64 function descriptors, are any of those
> > allocated by the dynamic linker rather than created by the static linker?
> On PA, the static linker creates all function descriptors. The dynamic
> linker is responsible for
> resolving descriptors when lazy binding is used.
>
> The primary difference between 32 and 64-bit descriptors is that there can be
> multiple descriptors
> that resolve to the same function in the 32-bit run time. In the 64-bit
> case, there is one official
> procedure descriptor for each function.
What I (or the PCH patch I've posted) cares about is if one does:
int foo () { return 1; }
int bar () { return 2; }
int v1, v2;
int
main ()
{
__builtin_printf ("%p %p %p %p\n", (void *) , (void *) , , );
return 0;
}
whether either the addresses are always the same (non-relocatable binary),
or they differ, but the differences between say the first address and the
other addresses are the same all the time (i.e. if one does what the patch
is doing,
+ if (pch_save != _pch_save)
+{
+ uintptr_t bias = (uintptr_t) _pch_save - (uintptr_t) pch_save;
^^^
+ void **ptrs = XNEWVEC (void *, num_callbacks);
+ unsigned i;
+
+ if (fread (ptrs, sizeof (void *), num_callbacks, f) != num_callbacks)
+ fatal_error (input_location, "cannot read PCH file: %m");
+ for (i = 0; i < num_callbacks; ++i)
+ {
+ memcpy (_save, ptrs[i], sizeof (pch_save));
+ pch_save = (void (*) (FILE *)) ((uintptr_t) pch_save + bias);
^
+ memcpy (ptrs[i], _save, sizeof (pch_save));
+ }
+ XDELETE (ptrs);
+}
whether it will work correctly.
If there are any hosts where it wouldn't, we'd need to disable PCH or at
least PCH with different load bias on such host.
Jakub
Re: [PATCH] pch: Add support for PCH for relocatable executables
On Tue, Nov 09, 2021 at 08:12:05AM +0100, Richard Biener wrote: > > So, here is 1), 2), 3) implemented. With this patch alone, > > g++.dg/pch/system-2.C test ICEs. This is because GCC 12 has added > > function::x_range_query member, which is set to _ranges on > > cfun creation and is: > > range_query * GTY ((skip)) x_range_query; > > which means when a PIE binary writes PCH and a PIE binary loaded > > at a different address loads it, cfun->x_range_query might be a garbage > > pointer. We can either apply a patch like the attached one after > > this inline patch, but then probably callback is misnamed and we should > > rename it to relocate_and_skip or something similar. Or we could > > e.g. during gimplification overwrite cfun->x_range_query = _ranges. > > I think struct function allocation should initialize it to NULL and > the init to _ranges be done only when we do init_tree_ssa? > In fact x_range_query could be moved to the gimple_df substructure > to make that clear. Agreed, Andrew/Aldy, what do you think? > Hopefully PCH happens before init_tree_ssa. I think it does. > I'm not keen on further life support for PCH, but then if the work is > done it's hard to reject it... > > Note there might be still platforms not supporting PCH and so Iains > patches still look useful to me. Sure, I think it is fine to opt out of PCH support if needed, just that PIE shouldn't automatically imply that. > I also wonder whether we want to require active marking of types > with PCH support and assert when PCH write runs into objects without? At least for scalar types I'd prefer not to mark them. For aggregates, why not, but my gengtype knowledge is very limited, it has been a pain to add the TYPE_CALLBACK stuff in there already... Jakub
Re: [PATCH] pch: Add support for PCH for relocatable executables
> On 9 Nov 2021, at 07:12, Richard Biener wrote: > > On Mon, 8 Nov 2021, Jakub Jelinek wrote: > >> On Mon, Nov 08, 2021 at 12:46:04PM +0100, Jakub Jelinek via Gcc-patches >> wrote: >>> So, if we want to make PCH work for PIEs, I'd say we can: >>> 1) add a new GTY option, say callback, which would act like >>> skip for non-PCH and for PCH would make us skip it but >>> remember for address bias translation >>> 2) drop the skip for tree_translation_unit_decl::language >>> 3) change get_unnamed_section to have const char * as >>> last argument instead of const void *, change >>> unnamed_section::data also to const char * and update >>> everything related to that >>> 4) maybe add a host hook whether it is ok to support binaries >>> changing addresses (the only thing I'm worried is if >>> some host that uses function descriptors allocates them >>> dynamically instead of having them somewhere in the >>> executable) >>> 5) maybe add a gengtype warning if it sees in GTY tracked >>> structure a function pointer without that new callback >>> option >> >> So, here is 1), 2), 3) implemented. With this patch alone, >> g++.dg/pch/system-2.C test ICEs. This is because GCC 12 has added >> function::x_range_query member, which is set to _ranges on >> cfun creation and is: >> range_query * GTY ((skip)) x_range_query; >> which means when a PIE binary writes PCH and a PIE binary loaded >> at a different address loads it, cfun->x_range_query might be a garbage >> pointer. We can either apply a patch like the attached one after >> this inline patch, but then probably callback is misnamed and we should >> rename it to relocate_and_skip or something similar. Or we could >> e.g. during gimplification overwrite cfun->x_range_query = _ranges. > > I think struct function allocation should initialize it to NULL and > the init to _ranges be done only when we do init_tree_ssa? > In fact x_range_query could be moved to the gimple_df substructure > to make that clear. > > Hopefully PCH happens before init_tree_ssa. > >> Other than that make check-gcc check-g++ passes RUNTESTFLAGS=pch.exp. >> >> Not really sure about PA or IA-64 function descriptors, are any of those >> allocated by the dynamic linker rather than created by the static linker? >> I guess instead of removing the c-pch.c changes we could remember there >> not just a function pointer, but also a data pointer and compare if both >> are either the same or have the same load bias and punt only if they >> have different bias. Though, on architecture where all function pointers >> would be dynamically allocated and could change any time even that wouldn't >> be really a reliable check. >> >> Note, on stdc++.h.gch/O2g.gch there are just those 10 relocations without >> the second patch, with it a few more, but nothing huge. And for non-PIEs >> there isn't really any extra work on the load side except freading two scalar >> values and fseek. >> >> Thoughts on this? - thanks for doing it, I still think a working solution is better than a disable. - This would be a fix for PR 71934 - I applied the patches on Darwin and there’s still some case(s) to find and fix cc1/cc1plus hang on pch input, when any diagnostic is emitted (doing some linemap lookup) .. hopefully, I can find some time to figure out what is not happy. > I'm not keen on further life support for PCH, but then if the work is > done it's hard to reject it... > > Note there might be still platforms not supporting PCH and so Iains > patches still look useful to me. Yeah, what would be most useful at this stage is some way to get those tested against some wider set of package builds to see if anything fires that breaks stuff. > I also wonder whether we want to require active marking of types > with PCH support and assert when PCH write runs into objects without? anything that makes GTY markup more robust is a Good Idea. Iain > > Richard. > >> 2021-11-08 Jakub Jelinek >> >> gcc/ >> * ggc.h (gt_pch_note_callback): Declare. >> * gengtype.h (enum typekind): Add TYPE_CALLBACK. >> (callback_type): Declare. >> * gengtype.c (dbgprint_count_type_at): Handle TYPE_CALLBACK. >> (callback_type): New variable. >> (process_gc_options): Add CALLBACK argument, handle callback >> option. >> (set_gc_used_type): Adjust process_gc_options caller, if callback, >> set type to _type. >> (output_mangled_typename): Handle TYPE_CALLBACK. >> (walk_type): Likewise. Handle callback option. >> (write_types_process_field): Handle TYPE_CALLBACK. >> (write_types_local_user_process_field): Likewise. >> (write_types_local_process_field): Likewise. >> (write_root): Likewise. >> (dump_typekind): Likewise. >> (dump_type): Likewise. >> * gengtype-state.c (type_lineloc): Handle TYPE_CALLBACK. >> (state_writer::write_state_callback_type): New method. >> (state_writer::write_state_type): Handle
Re: [PATCH] pch: Add support for PCH for relocatable executables
On Mon, 8 Nov 2021, Jakub Jelinek wrote: > On Mon, Nov 08, 2021 at 12:46:04PM +0100, Jakub Jelinek via Gcc-patches wrote: > > So, if we want to make PCH work for PIEs, I'd say we can: > > 1) add a new GTY option, say callback, which would act like > >skip for non-PCH and for PCH would make us skip it but > >remember for address bias translation > > 2) drop the skip for tree_translation_unit_decl::language > > 3) change get_unnamed_section to have const char * as > >last argument instead of const void *, change > >unnamed_section::data also to const char * and update > >everything related to that > > 4) maybe add a host hook whether it is ok to support binaries > >changing addresses (the only thing I'm worried is if > >some host that uses function descriptors allocates them > >dynamically instead of having them somewhere in the > >executable) > > 5) maybe add a gengtype warning if it sees in GTY tracked > >structure a function pointer without that new callback > >option > > So, here is 1), 2), 3) implemented. With this patch alone, > g++.dg/pch/system-2.C test ICEs. This is because GCC 12 has added > function::x_range_query member, which is set to _ranges on > cfun creation and is: > range_query * GTY ((skip)) x_range_query; > which means when a PIE binary writes PCH and a PIE binary loaded > at a different address loads it, cfun->x_range_query might be a garbage > pointer. We can either apply a patch like the attached one after > this inline patch, but then probably callback is misnamed and we should > rename it to relocate_and_skip or something similar. Or we could > e.g. during gimplification overwrite cfun->x_range_query = _ranges. I think struct function allocation should initialize it to NULL and the init to _ranges be done only when we do init_tree_ssa? In fact x_range_query could be moved to the gimple_df substructure to make that clear. Hopefully PCH happens before init_tree_ssa. > Other than that make check-gcc check-g++ passes RUNTESTFLAGS=pch.exp. > > Not really sure about PA or IA-64 function descriptors, are any of those > allocated by the dynamic linker rather than created by the static linker? > I guess instead of removing the c-pch.c changes we could remember there > not just a function pointer, but also a data pointer and compare if both > are either the same or have the same load bias and punt only if they > have different bias. Though, on architecture where all function pointers > would be dynamically allocated and could change any time even that wouldn't > be really a reliable check. > > Note, on stdc++.h.gch/O2g.gch there are just those 10 relocations without > the second patch, with it a few more, but nothing huge. And for non-PIEs > there isn't really any extra work on the load side except freading two scalar > values and fseek. > > Thoughts on this? I'm not keen on further life support for PCH, but then if the work is done it's hard to reject it... Note there might be still platforms not supporting PCH and so Iains patches still look useful to me. I also wonder whether we want to require active marking of types with PCH support and assert when PCH write runs into objects without? Richard. > 2021-11-08 Jakub Jelinek > > gcc/ > * ggc.h (gt_pch_note_callback): Declare. > * gengtype.h (enum typekind): Add TYPE_CALLBACK. > (callback_type): Declare. > * gengtype.c (dbgprint_count_type_at): Handle TYPE_CALLBACK. > (callback_type): New variable. > (process_gc_options): Add CALLBACK argument, handle callback > option. > (set_gc_used_type): Adjust process_gc_options caller, if callback, > set type to _type. > (output_mangled_typename): Handle TYPE_CALLBACK. > (walk_type): Likewise. Handle callback option. > (write_types_process_field): Handle TYPE_CALLBACK. > (write_types_local_user_process_field): Likewise. > (write_types_local_process_field): Likewise. > (write_root): Likewise. > (dump_typekind): Likewise. > (dump_type): Likewise. > * gengtype-state.c (type_lineloc): Handle TYPE_CALLBACK. > (state_writer::write_state_callback_type): New method. > (state_writer::write_state_type): Handle TYPE_CALLBACK. > (read_state_callback_type): New function. > (read_state_type): Handle TYPE_CALLBACK. > * ggc-common.c (callback_vec): New variable. > (gt_pch_note_callback): New function. > (gt_pch_save): Stream out gt_pch_save function address and relocation > table. > (gt_pch_restore): Stream in saved gt_pch_save function address and > relocation table and apply relocations if needed. > * doc/gty.texi (callback): Document new GTY option. > * varasm.c (get_unnamed_section): Change callback argument's type and > last argument's type from const void * to const char *. > (output_section_asm_op): Change argument's type from const void * > to
Re: [PATCH] pch: Add support for PCH for relocatable executables
On 2021-11-08 2:48 p.m., Jakub Jelinek wrote: Not really sure about PA or IA-64 function descriptors, are any of those allocated by the dynamic linker rather than created by the static linker? On PA, the static linker creates all function descriptors. The dynamic linker is responsible for resolving descriptors when lazy binding is used. The primary difference between 32 and 64-bit descriptors is that there can be multiple descriptors that resolve to the same function in the 32-bit run time. In the 64-bit case, there is one official procedure descriptor for each function. I guess instead of removing the c-pch.c changes we could remember there not just a function pointer, but also a data pointer and compare if both are either the same or have the same load bias and punt only if they have different bias. Though, on architecture where all function pointers would be dynamically allocated and could change any time even that wouldn't be really a reliable check. There is no call to dynamically allocate a descriptor but it is possible for code to dynamically build a descriptor. Dave -- John David Anglin dave.ang...@bell.net
[PATCH] pch: Add support for PCH for relocatable executables
On Mon, Nov 08, 2021 at 12:46:04PM +0100, Jakub Jelinek via Gcc-patches wrote: > So, if we want to make PCH work for PIEs, I'd say we can: > 1) add a new GTY option, say callback, which would act like >skip for non-PCH and for PCH would make us skip it but >remember for address bias translation > 2) drop the skip for tree_translation_unit_decl::language > 3) change get_unnamed_section to have const char * as >last argument instead of const void *, change >unnamed_section::data also to const char * and update >everything related to that > 4) maybe add a host hook whether it is ok to support binaries >changing addresses (the only thing I'm worried is if >some host that uses function descriptors allocates them >dynamically instead of having them somewhere in the >executable) > 5) maybe add a gengtype warning if it sees in GTY tracked >structure a function pointer without that new callback >option So, here is 1), 2), 3) implemented. With this patch alone, g++.dg/pch/system-2.C test ICEs. This is because GCC 12 has added function::x_range_query member, which is set to _ranges on cfun creation and is: range_query * GTY ((skip)) x_range_query; which means when a PIE binary writes PCH and a PIE binary loaded at a different address loads it, cfun->x_range_query might be a garbage pointer. We can either apply a patch like the attached one after this inline patch, but then probably callback is misnamed and we should rename it to relocate_and_skip or something similar. Or we could e.g. during gimplification overwrite cfun->x_range_query = _ranges. Other than that make check-gcc check-g++ passes RUNTESTFLAGS=pch.exp. Not really sure about PA or IA-64 function descriptors, are any of those allocated by the dynamic linker rather than created by the static linker? I guess instead of removing the c-pch.c changes we could remember there not just a function pointer, but also a data pointer and compare if both are either the same or have the same load bias and punt only if they have different bias. Though, on architecture where all function pointers would be dynamically allocated and could change any time even that wouldn't be really a reliable check. Note, on stdc++.h.gch/O2g.gch there are just those 10 relocations without the second patch, with it a few more, but nothing huge. And for non-PIEs there isn't really any extra work on the load side except freading two scalar values and fseek. Thoughts on this? 2021-11-08 Jakub Jelinek gcc/ * ggc.h (gt_pch_note_callback): Declare. * gengtype.h (enum typekind): Add TYPE_CALLBACK. (callback_type): Declare. * gengtype.c (dbgprint_count_type_at): Handle TYPE_CALLBACK. (callback_type): New variable. (process_gc_options): Add CALLBACK argument, handle callback option. (set_gc_used_type): Adjust process_gc_options caller, if callback, set type to _type. (output_mangled_typename): Handle TYPE_CALLBACK. (walk_type): Likewise. Handle callback option. (write_types_process_field): Handle TYPE_CALLBACK. (write_types_local_user_process_field): Likewise. (write_types_local_process_field): Likewise. (write_root): Likewise. (dump_typekind): Likewise. (dump_type): Likewise. * gengtype-state.c (type_lineloc): Handle TYPE_CALLBACK. (state_writer::write_state_callback_type): New method. (state_writer::write_state_type): Handle TYPE_CALLBACK. (read_state_callback_type): New function. (read_state_type): Handle TYPE_CALLBACK. * ggc-common.c (callback_vec): New variable. (gt_pch_note_callback): New function. (gt_pch_save): Stream out gt_pch_save function address and relocation table. (gt_pch_restore): Stream in saved gt_pch_save function address and relocation table and apply relocations if needed. * doc/gty.texi (callback): Document new GTY option. * varasm.c (get_unnamed_section): Change callback argument's type and last argument's type from const void * to const char *. (output_section_asm_op): Change argument's type from const void * to const char *, remove unnecessary cast. * tree-core.h (struct tree_translation_unit_decl): Drop GTY((skip)) from language member. * output.h (unnamed_section_callback): Change argument type from const void * to const char *. (struct unnamed_section): Use GTY((callback)) instead of GTY((skip)) for callback member. Change data member type from const void * to const char *. (struct noswitch_section): Use GTY((callback)) instead of GTY((skip)) for callback member. (get_unnamed_section): Change callback argument's type and last argument's type from const void * to const char *. (output_section_asm_op): Change argument's type from const void * to
