Re: Question on -fwrapv and -fwrapv-pointer

[Qing Zhao via Gcc-patches]

> On Sep 15, 2023, at 12:53 PM, Xi Ruoyao  wrote:
> 
> On Fri, 2023-09-15 at 15:37 +, Qing Zhao wrote:
>> 
>> 
>>> On Sep 15, 2023, at 11:29 AM, Richard Biener
>>>  wrote:
>>> 
>>> 
>>> 
 Am 15.09.2023 um 17:25 schrieb Qing Zhao :
 
 
 
> On Sep 15, 2023, at 8:41 AM, Arsen Arsenović 
> wrote:
> 
> 
> Qing Zhao  writes:
> 
>> Even though unsigned integer overflow is well defined, it
>> might be
>> unintentional, shall we warn user about this?
> 
> This would be better addressed by providing operators or
> functions that
> do overflow checking in the language, so that they can be
> explicitly
> used where overflow is unexpected.
 
 Yes, that will be very helpful to prevent unexpected overflow in
 the program in general.
 However, this will mainly benefit new codes.
 
 For the existing C codes, especially large applications, we still
 need to identify all the places 
 Where the overflow is unexpected, and fix them. 
 
 One good example is linux kernel. 
 
> One could easily imagine a scenario
> where overflow is not expected in some region of code but is in
> the
> larger application.
 
 Yes, that’s exactly the same situation Linux kernel faces now, the
 unexpected Overflow and 
 expected wrap-around are mixed together inside one module. 
 It’s hard to detect the unexpected overflow under such situation
 based on the current GCC. 
>>> 
>>> But that’s hardly GCCs fault nor can GCC fix that in any way.  Only
>>> the programmer can distinguish both cases.
>> 
>> Right, compiler cannot fix this. 
>> But can provide some tools to help the user to detect this more
>> conveniently. 
>> 
>> Right now, GCC provides two set of options for different types:
>> 
>>  A. Turn the overflow to expected wrap-around (remove UB);
>>  B. Detect overflow;
>> 
>> A   B
>>  remove UB  -fsanitize=…
>> signed -fwrapv  signed-integer-overflow
>> pointer-fwrapv-pointer  pointer-overflow (broken in Clang)
>> 
>> However, Options in A and B excluded with each other. They cannot mix
>> together for a single file.
>> 
>> What’s requested from Kernel is:
>> 
>> compiler needs to provide a functionality that can mix these two
>> together for a file. 
>> 
>> i.e, apply A (convert UB to defined behavior WRAP-AROUND) only to part
>> of the program.  And then add -fsnaitize=*overflow to detect all other
>> Unexpected overflows in the program.
>> 
>> This is currently missing from GCC, I guess?
> 
> If overflow is really so rare, we should just enable -fsanitize=signed-
> integer-overflow globally and special case the code paths where we want
> wrapping.  It's easy in 2023:
> 
> /* b + c may wrap here because ... ... */
> ckd_add(, b, c);
> 
> Or
> 
> /* if b + c overflows, we have a severe issue, let's panic even if
>   sanitizer disabled */
> if (chk_add(, b, c))
>  panic("b + c overflows but it shouldn't (b = %d, c = %d)", b, c);

Yes, this might be exactly the situation that Linux kernel faces and asks for 
help:

In some part of the program, it tried to detect the overflow itself and handled 
specifically,  therefore, 
It want this part of the program to be excluded from automagical overflow 
detection applied by 
the -fsanitize=signed-integer-overflow. 

Please see my reply to Richard’s email for more info on this.

Thanks.

Qing

> 
> -- 
> Xi Ruoyao 
> School of Aerospace Science and Technology, Xidian University

Re: Question on -fwrapv and -fwrapv-pointer

[Qing Zhao via Gcc-patches]

> On Sep 15, 2023, at 1:26 PM, Richard Biener  
> wrote:
> 
> 
> 
>> Am 15.09.2023 um 17:37 schrieb Qing Zhao :
>> 
>> 
>> 
 On Sep 15, 2023, at 11:29 AM, Richard Biener  
 wrote:
 
 
 
> Am 15.09.2023 um 17:25 schrieb Qing Zhao :
 
 
 
> On Sep 15, 2023, at 8:41 AM, Arsen Arsenović  wrote:
> 
> 
> Qing Zhao  writes:
> 
>> Even though unsigned integer overflow is well defined, it might be
>> unintentional, shall we warn user about this?
> 
> This would be better addressed by providing operators or functions that
> do overflow checking in the language, so that they can be explicitly
> used where overflow is unexpected.
 
 Yes, that will be very helpful to prevent unexpected overflow in the 
 program in general.
 However, this will mainly benefit new codes.
 
 For the existing C codes, especially large applications, we still need to 
 identify all the places 
 Where the overflow is unexpected, and fix them. 
 
 One good example is linux kernel. 
 
> One could easily imagine a scenario
> where overflow is not expected in some region of code but is in the
> larger application.
 
 Yes, that’s exactly the same situation Linux kernel faces now, the 
 unexpected Overflow and 
 expected wrap-around are mixed together inside one module. 
 It’s hard to detect the unexpected overflow under such situation based on 
 the current GCC. 
>>> 
>>> But that’s hardly GCCs fault nor can GCC fix that in any way.  Only the 
>>> programmer can distinguish both cases.
>> 
>> Right, compiler cannot fix this. 
>> But can provide some tools to help the user to detect this more 
>> conveniently. 
>> 
>> Right now, GCC provides two set of options for different types:
>> 
>> A. Turn the overflow to expected wrap-around (remove UB);
>> B. Detect overflow;
>> 
>>   AB
>>  remove UB-fsanitize=…
>> signed   -fwrapvsigned-integer-overflow
>> pointer   -fwrapv-pointerpointer-overflow (broken in Clang)
>> 
>> However, Options in A and B excluded with each other. They cannot mix 
>> together for a single file.
>> 
>> What’s requested from Kernel is:
>> 
>> compiler needs to provide a functionality that can mix these two together 
>> for a file. 
>> 
>> i.e, apply A (convert UB to defined behavior WRAP-AROUND) only to part of 
>> the program.  And then add -fsnaitize=*overflow to detect all other
>> Unexpected overflows in the program.
>> 
>> This is currently missing from GCC, I guess?
> 
> How can GCC know which part of the program wants wrapping and which 
> sanitizing?

GCC doesn’t know, but the user knows. 

Then just provide the user a way to mark part of the program to be wrapping 
around and excluded from sanitizing? 

Currently, GCC provides 

__attribute__(optimize ("wrapv"))  

To mark the specific function to be wrapped around.

However, this attribute does not work for linux kernel due to the following 
reason:

Attribute optimize should be only used for debugging purpose;
The kernel has banned its usage;

So, a new attribute was requested from Linux kernel security: 

 request wrap-around behavior for specific function (PR102317)
__attribute__((wrapv)) 

Is this request reasonable?

Qing

> Richard 
> 
>> Qing
>> 
>> 
>> 
>> 
>> 
>>> 
>>> Richard 
>>> 
 Thanks.
 
 Qing
> -- 
> Arsen Arsenović

Re: Question on -fwrapv and -fwrapv-pointer

[Qing Zhao via Gcc-patches]

> On Sep 15, 2023, at 11:29 AM, Richard Biener  
> wrote:
> 
> 
> 
>> Am 15.09.2023 um 17:25 schrieb Qing Zhao :
>> 
>> 
>> 
>>> On Sep 15, 2023, at 8:41 AM, Arsen Arsenović  wrote:
>>> 
>>> 
>>> Qing Zhao  writes:
>>> 
 Even though unsigned integer overflow is well defined, it might be
 unintentional, shall we warn user about this?
>>> 
>>> This would be better addressed by providing operators or functions that
>>> do overflow checking in the language, so that they can be explicitly
>>> used where overflow is unexpected.
>> 
>> Yes, that will be very helpful to prevent unexpected overflow in the program 
>> in general.
>> However, this will mainly benefit new codes.
>> 
>> For the existing C codes, especially large applications, we still need to 
>> identify all the places 
>> Where the overflow is unexpected, and fix them. 
>> 
>> One good example is linux kernel. 
>> 
>>> One could easily imagine a scenario
>>> where overflow is not expected in some region of code but is in the
>>> larger application.
>> 
>> Yes, that’s exactly the same situation Linux kernel faces now, the 
>> unexpected Overflow and 
>> expected wrap-around are mixed together inside one module. 
>> It’s hard to detect the unexpected overflow under such situation based on 
>> the current GCC. 
> 
> But that’s hardly GCCs fault nor can GCC fix that in any way.  Only the 
> programmer can distinguish both cases.

Right, compiler cannot fix this. 
But can provide some tools to help the user to detect this more conveniently. 

Right now, GCC provides two set of options for different types:

 A. Turn the overflow to expected wrap-around (remove UB);
 B. Detect overflow;

A   B
 remove UB  -fsanitize=…
signed -fwrapv  signed-integer-overflow
pointer-fwrapv-pointer  pointer-overflow (broken in Clang)

However, Options in A and B excluded with each other. They cannot mix together 
for a single file.

What’s requested from Kernel is:

compiler needs to provide a functionality that can mix these two together for a 
file. 

i.e, apply A (convert UB to defined behavior WRAP-AROUND) only to part of the 
program.  And then add -fsnaitize=*overflow to detect all other
Unexpected overflows in the program.

This is currently missing from GCC, I guess?

Qing





> 
> Richard 
> 
>> Thanks.
>> 
>> Qing
>>> -- 
>>> Arsen Arsenović

Re: Question on -fwrapv and -fwrapv-pointer

[Qing Zhao via Gcc-patches]

> On Sep 15, 2023, at 8:41 AM, Arsen Arsenović  wrote:
> 
> 
> Qing Zhao  writes:
> 
>> Even though unsigned integer overflow is well defined, it might be
>> unintentional, shall we warn user about this?
> 
> This would be better addressed by providing operators or functions that
> do overflow checking in the language, so that they can be explicitly
> used where overflow is unexpected.

Yes, that will be very helpful to prevent unexpected overflow in the program in 
general.
However, this will mainly benefit new codes. 

For the existing C codes, especially large applications, we still need to 
identify all the places 
Where the overflow is unexpected, and fix them. 

One good example is linux kernel. 

>  One could easily imagine a scenario
> where overflow is not expected in some region of code but is in the
> larger application.

Yes, that’s exactly the same situation Linux kernel faces now, the unexpected 
Overflow and 
expected wrap-around are mixed together inside one module. 
It’s hard to detect the unexpected overflow under such situation based on the 
current GCC. 

Thanks.

Qing
> -- 
> Arsen Arsenović

Re: Question on -fwrapv and -fwrapv-pointer

[Qing Zhao via Gcc-patches]

> On Sep 15, 2023, at 3:43 AM, Xi Ruoyao  wrote:
> 
> On Thu, 2023-09-14 at 21:41 +, Qing Zhao wrote:
 CLANG already provided -fsanitize=unsigned-integer-overflow. GCC
 might need to do the same.
>>> 
>>> NO. There is no such thing as unsigned integer overflow. That option
>>> is badly designed and the GCC community has rejected a few times now
>>> having that sanitizer before. It is bad form to have a sanitizer for
>>> well defined code.
>> 
>> Even though unsigned integer overflow is well defined, it might be
>> unintentional, shall we warn user about this?
> 
> *Everything* could be unintentional and should be warned then.  GCC is a
> compiler, not an advanced AI educating the programmers.

Well, you are right in some sense. -:)

However, overflow is one important source for security flaws, it’s important  
for compilers to detect
overflows in the programs in general. 

Qing
> 
> -- 
> Xi Ruoyao 
> School of Aerospace Science and Technology, Xidian University

Re: [PATCH] tree optimization/111407--SSA corruption due to widening_mul opt

[Qing Zhao via Gcc-patches]

thanks.

Committed as https://gcc.gnu.org/pipermail/gcc-cvs/2023-September/389614.html

Qing
> On Sep 15, 2023, at 2:12 AM, Richard Biener  
> wrote:
> 
> On Thu, Sep 14, 2023 at 3:25 PM Qing Zhao via Gcc-patches
>  wrote:
>> 
>> on conflict across an abnormal edge
>> 
>> This is a bug in tree-ssa-math-opts.cc, when applying the widening mul
>> optimization, the compiler needs to check whether the operand is in a
>> ABNORMAL PHI, if YES, we should avoid the transformation.
>> 
>> bootstrapped and regression tested on both aarch64 and x86, no issue.
>> 
>> Okay for committing?
> 
> OK.
> 
>> thanks.
>> 
>> Qing
>> 
>> =
>> 
>>PR tree-optimization/111407
>> 
>> gcc/ChangeLog:
>> 
>>* tree-ssa-math-opts.cc (convert_mult_to_widen): Avoid the transform
>>when one of the operands is subject to abnormal coalescing.
>> 
>> gcc/testsuite/ChangeLog:
>> 
>>* gcc.dg/pr111407.c: New test.
>> ---
>> gcc/testsuite/gcc.dg/pr111407.c | 21 +
>> gcc/tree-ssa-math-opts.cc   |  8 
>> 2 files changed, 29 insertions(+)
>> create mode 100644 gcc/testsuite/gcc.dg/pr111407.c
>> 
>> diff --git a/gcc/testsuite/gcc.dg/pr111407.c 
>> b/gcc/testsuite/gcc.dg/pr111407.c
>> new file mode 100644
>> index 000..a171074753f
>> --- /dev/null
>> +++ b/gcc/testsuite/gcc.dg/pr111407.c
>> @@ -0,0 +1,21 @@
>> +/* PR tree-optimization/111407*/
>> +/* { dg-do compile } */
>> +/* { dg-options "-O2" } */
>> +enum { SEND_TOFILE } __sigsetjmp();
>> +void fclose();
>> +void foldergets();
>> +void sendpart_stats(int *p1, int a1, int b1) {
>> + int *a = p1;
>> + fclose();
>> + p1 = 0;
>> + long t = b1;
>> + if (__sigsetjmp()) {
>> +   {
>> + long t1 = a1;
>> + a1+=1;
>> + fclose(a1*(long)t1);
>> +   }
>> + }
>> + if (p1)
>> +   fclose();
>> +}
>> diff --git a/gcc/tree-ssa-math-opts.cc b/gcc/tree-ssa-math-opts.cc
>> index 3db69ad5733..51c14d6bad9 100644
>> --- a/gcc/tree-ssa-math-opts.cc
>> +++ b/gcc/tree-ssa-math-opts.cc
>> @@ -2755,6 +2755,14 @@ convert_mult_to_widen (gimple *stmt, 
>> gimple_stmt_iterator *gsi)
>>   if (!is_widening_mult_p (stmt, , , , ))
>> return false;
>> 
>> +  /* if any one of rhs1 and rhs2 is subject to abnormal coalescing,
>> + avoid the tranform. */
>> +  if ((TREE_CODE (rhs1) == SSA_NAME
>> +   && SSA_NAME_OCCURS_IN_ABNORMAL_PHI (rhs1))
>> +  || (TREE_CODE (rhs2) == SSA_NAME
>> + && SSA_NAME_OCCURS_IN_ABNORMAL_PHI (rhs2)))
>> +return false;
>> +
>>   to_mode = SCALAR_INT_TYPE_MODE (type);
>>   from_mode = SCALAR_INT_TYPE_MODE (type1);
>>   if (to_mode == from_mode)
>> --
>> 2.31.1
>> 

Re: Question on -fwrapv and -fwrapv-pointer

[Qing Zhao via Gcc-patches]

> On Sep 14, 2023, at 4:57 PM, Andrew Pinski  wrote:
> 
> On Thu, Sep 14, 2023 at 1:50 PM Qing Zhao via Gcc-patches
>  wrote:
>> 
>> 
>> 
>>> On Sep 14, 2023, at 12:18 PM, Xi Ruoyao  wrote:
>>> 
>>> On Thu, 2023-09-14 at 15:57 +, Qing Zhao via Gcc-patches wrote:
>>>> Currently, GCC behaves as following:
>>>> 
>>>> /* True if overflow wraps around for the given integral or pointer type.  
>>>> That
>>>>   is, TYPE_MAX + 1 == TYPE_MIN.  */
>>>> #define TYPE_OVERFLOW_WRAPS(TYPE) \
>>>>  (POINTER_TYPE_P (TYPE)\
>>>>   ? flag_wrapv_pointer \
>>>>   : (ANY_INTEGRAL_TYPE_CHECK(TYPE)->base.u.bits.unsigned_flag  \
>>>>  || flag_wrapv))
>>>> 
>>>> /* True if overflow is undefined for the given integral or pointer type.
>>>>   We may optimize on the assumption that values in the type never overflow.
>>>> 
>>>>   IMPORTANT NOTE: Any optimization based on TYPE_OVERFLOW_UNDEFINED
>>>>   must issue a warning based on warn_strict_overflow.  In some cases
>>>>   it will be appropriate to issue the warning immediately, and in
>>>>   other cases it will be appropriate to simply set a flag and let the
>>>>   caller decide whether a warning is appropriate or not.  */
>>>> #define TYPE_OVERFLOW_UNDEFINED(TYPE)   \
>>>>  (POINTER_TYPE_P (TYPE)\
>>>>   ? !flag_wrapv_pointer\
>>>>   : (!ANY_INTEGRAL_TYPE_CHECK(TYPE)->base.u.bits.unsigned_flag \
>>>>  && !flag_wrapv && !flag_trapv))
>>>> 
>>>> The logic above seems treating the pointer default as signed integer, 
>>>> right?
>>> 
>>> It only says the pointers cannot overflow, not the pointers are signed.
>>> 
>>> printf("%d\n", (char *)(intptr_t)-1 > (char *)(intptr_t)1);
>>> 
>>> produces 1 instead of 0.  Technically this is invoking undefined
>>> behavior and a conforming implementation can output anything.  But
>>> consider a 32-bit bare metal target where the linker can locate a "char
>>> x[512]" at [0x7f00, 0x8100).  The standard then requires [512]
>>>> [0], but if we do a signed comparison here we'll end up "[512] <
>>> [0]", this is non-conforming.
>> 
>> So, are both the above examples showing that pointer based comparisons are 
>> similar as unsigned integer comparison?  -:)
>> Do we have examples on treating the pointer arithmetic as signed integer 
>> arithmetic? (Really curious on this….)
>> 
>> But anyway, if we cannot treat pointer type consistently as signed or 
>> unsigned, shall we still need to catch pointer overflow?
>> 
>> Currently, In GCC, we have -fsanitize=signed-integer-overflow to catch 
>> signed integer overflow.
>> But we don’t have options to catch unsigned integer overflow and pointer 
>> overflow.
>> 
>> Shall we add two more options to catch unsigned integer overflow and pointer 
>> overflow, like:
>> 
>> -fsanitize=unsigned-integer-overflow
>> -fsanitize=pointer-overflow
>> 
>> CLANG already provided -fsanitize=unsigned-integer-overflow. GCC might need 
>> to do the same.
> 
> NO. There is no such thing as unsigned integer overflow. That option
> is badly designed and the GCC community has rejected a few times now
> having that sanitizer before. It is bad form to have a sanitizer for
> well defined code.

Even though unsigned integer overflow is well defined, it might be 
unintentional, shall we warn user about this?

> 
> Now -fsanitize=pointer-overflow is already there for GCC which was
> added in r8-2238-gc9b39a4955f56fe609ef5478 . LLVM/clang also provides
> it in the same timeframe too .
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80998

Got it. Thanks a lot for the info.

Qing
> 
> Thanks,
> Andrew
> 
>> 
>> And both Clang and GCC might also need to add -fsanitize=pointer-overflow?
>> 
>>> IIUC, pointers are not integers, at all.  If we treat them as integers
>>> in the brain we'll end up invoking undefined behavior sooner or later.
>>> Thus the wrapping/overflowing behavior of pointer is controlled by a
>>> different option than integers.
>> 
>> However, the wrapping/overflowing behavior of pointers is still based on the 
>> corresponding integer(or unsigned integer) wrapping/overflowing, right?
>> Do we have special pointer wrapping/overflowing?
>> 
>> Qing
>> 
>>> 
>>> --
>>> Xi Ruoyao 
>>> School of Aerospace Science and Technology, Xidian University

Re: Question on -fwrapv and -fwrapv-pointer

[Qing Zhao via Gcc-patches]

> On Sep 14, 2023, at 12:18 PM, Xi Ruoyao  wrote:
> 
> On Thu, 2023-09-14 at 15:57 +0000, Qing Zhao via Gcc-patches wrote:
>> Currently, GCC behaves as following:
>> 
>> /* True if overflow wraps around for the given integral or pointer type.  
>> That
>>is, TYPE_MAX + 1 == TYPE_MIN.  */
>> #define TYPE_OVERFLOW_WRAPS(TYPE) \
>>   (POINTER_TYPE_P (TYPE)\
>>? flag_wrapv_pointer \
>>: (ANY_INTEGRAL_TYPE_CHECK(TYPE)->base.u.bits.unsigned_flag  \
>>   || flag_wrapv))
>> 
>> /* True if overflow is undefined for the given integral or pointer type.
>>We may optimize on the assumption that values in the type never overflow.
>> 
>>IMPORTANT NOTE: Any optimization based on TYPE_OVERFLOW_UNDEFINED
>>must issue a warning based on warn_strict_overflow.  In some cases
>>it will be appropriate to issue the warning immediately, and in
>>other cases it will be appropriate to simply set a flag and let the
>>caller decide whether a warning is appropriate or not.  */
>> #define TYPE_OVERFLOW_UNDEFINED(TYPE)   \
>>   (POINTER_TYPE_P (TYPE)\
>>? !flag_wrapv_pointer\
>>: (!ANY_INTEGRAL_TYPE_CHECK(TYPE)->base.u.bits.unsigned_flag \
>>   && !flag_wrapv && !flag_trapv))
>> 
>> The logic above seems treating the pointer default as signed integer, right?
> 
> It only says the pointers cannot overflow, not the pointers are signed.
> 
> printf("%d\n", (char *)(intptr_t)-1 > (char *)(intptr_t)1);
> 
> produces 1 instead of 0.  Technically this is invoking undefined
> behavior and a conforming implementation can output anything.  But
> consider a 32-bit bare metal target where the linker can locate a "char
> x[512]" at [0x7f00, 0x8100).  The standard then requires [512]
>> [0], but if we do a signed comparison here we'll end up "[512] <
> [0]", this is non-conforming.

So, are both the above examples showing that pointer based comparisons are 
similar as unsigned integer comparison?  -:)
Do we have examples on treating the pointer arithmetic as signed integer 
arithmetic? (Really curious on this….)

But anyway, if we cannot treat pointer type consistently as signed or unsigned, 
shall we still need to catch pointer overflow? 

Currently, In GCC, we have -fsanitize=signed-integer-overflow to catch signed 
integer overflow.
But we don’t have options to catch unsigned integer overflow and pointer 
overflow. 

Shall we add two more options to catch unsigned integer overflow and pointer 
overflow, like:

-fsanitize=unsigned-integer-overflow
-fsanitize=pointer-overflow

CLANG already provided -fsanitize=unsigned-integer-overflow. GCC might need to 
do the same.

And both Clang and GCC might also need to add -fsanitize=pointer-overflow?

> IIUC, pointers are not integers, at all.  If we treat them as integers
> in the brain we'll end up invoking undefined behavior sooner or later. 
> Thus the wrapping/overflowing behavior of pointer is controlled by a
> different option than integers.

However, the wrapping/overflowing behavior of pointers is still based on the 
corresponding integer(or unsigned integer) wrapping/overflowing, right? 
Do we have special pointer wrapping/overflowing?

Qing

> 
> -- 
> Xi Ruoyao 
> School of Aerospace Science and Technology, Xidian University

Re: Question on -fwrapv and -fwrapv-pointer

[Qing Zhao via Gcc-patches]

> On Sep 14, 2023, at 11:12 AM, Richard Biener  
> wrote:
> 
> 
> 
>> Am 14.09.2023 um 17:01 schrieb Qing Zhao :
>> 
>> Thanks for the info.
>> 
>>> On Sep 14, 2023, at 10:06 AM, Richard Biener  
>>> wrote:
>>> 
>>>> On Thu, Sep 14, 2023 at 3:42 PM Qing Zhao via Gcc-patches
>>>>  wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> I have several questions on these options:
>>>> 
>>>> 1.are pointers treated as signed integers in general? (I thought that 
>>>> pointers are addresses to the memory, should be treated as unsigned 
>>>> integer…)
>>>> 2. If Yes, why?
>>>> 3. why a separate option for pointesr -fwrapv-pointer in addition to 
>>>> -fwrapv if they are treated as signed integers?
>>> 
>>> Pointers are unsigned, they might sign-extend to Pmode though.
>> If they are unsigned, why they are sign-extend to Pmode? Is there any 
>> special reason for this? 
> 
> Some targets require this.  See POINTERS_EXTEND_UNSIGNED

Okay, I see.
> 
>> In another word, can we consistently treat pointers as unsigned? 
> 
> We do, but on GIMPLE it doesn’t matter.

Currently, GCC behaves as following:

/* True if overflow wraps around for the given integral or pointer type.  That
   is, TYPE_MAX + 1 == TYPE_MIN.  */
#define TYPE_OVERFLOW_WRAPS(TYPE) \
  (POINTER_TYPE_P (TYPE)\
   ? flag_wrapv_pointer \
   : (ANY_INTEGRAL_TYPE_CHECK(TYPE)->base.u.bits.unsigned_flag  \
  || flag_wrapv))

/* True if overflow is undefined for the given integral or pointer type.
   We may optimize on the assumption that values in the type never overflow.

   IMPORTANT NOTE: Any optimization based on TYPE_OVERFLOW_UNDEFINED
   must issue a warning based on warn_strict_overflow.  In some cases
   it will be appropriate to issue the warning immediately, and in
   other cases it will be appropriate to simply set a flag and let the
   caller decide whether a warning is appropriate or not.  */
#define TYPE_OVERFLOW_UNDEFINED(TYPE)   \
  (POINTER_TYPE_P (TYPE)\
   ? !flag_wrapv_pointer\
   : (!ANY_INTEGRAL_TYPE_CHECK(TYPE)->base.u.bits.unsigned_flag \
  && !flag_wrapv && !flag_trapv))

The logic above seems treating the pointer default as signed integer, right?

Qing

> 
>>> -fwrapv-pointer is to enable wrapping over zero,
>> 
>> If we always treat pointers are unsigned, then we don’t need the 
>> -fwrapv-pointer anymore, right? 
> 
> No, the naming is just ‚bad‘
> 
>> 
>>> we don't have many places using this, ISTR kernel folks requested to
>>> disable specific folding - digging in history
>>> might reveal the case/PR.
>> 
>> Do you mean that this -fwrapv-pointer was introduced for kernel?
> 
> I think it was introduced when removing the separate fstrict-overflow flag 
> and since that covered also some pointer transforms the wraps-pointer flag 
> was introduced.
> 
>> 
>> I will try to dig a little bit here.
>> 
>> thanks.
>> 
>> Qing
>>> 
>>> Richard.
>>> 
>>>> Thanks for your help.
>>>> 
>>>> Qing
>>>> 
>> 

Re: Question on -fwrapv and -fwrapv-pointer

[Qing Zhao via Gcc-patches]

Thanks for the info.

> On Sep 14, 2023, at 10:06 AM, Richard Biener  
> wrote:
> 
> On Thu, Sep 14, 2023 at 3:42 PM Qing Zhao via Gcc-patches
>  wrote:
>> 
>> Hi,
>> 
>> I have several questions on these options:
>> 
>> 1.are pointers treated as signed integers in general? (I thought that 
>> pointers are addresses to the memory, should be treated as unsigned integer…)
>> 2. If Yes, why?
>> 3. why a separate option for pointesr -fwrapv-pointer in addition to -fwrapv 
>> if they are treated as signed integers?
> 
> Pointers are unsigned, they might sign-extend to Pmode though.
If they are unsigned, why they are sign-extend to Pmode? Is there any special 
reason for this? 
In another word, can we consistently treat pointers as unsigned? 

> -fwrapv-pointer is to enable wrapping over zero,

If we always treat pointers are unsigned, then we don’t need the 
-fwrapv-pointer anymore, right? 

> we don't have many places using this, ISTR kernel folks requested to
> disable specific folding - digging in history
> might reveal the case/PR.

Do you mean that this -fwrapv-pointer was introduced for kernel?

I will try to dig a little bit here.

thanks.

Qing
> 
> Richard.
> 
>> Thanks for your help.
>> 
>> Qing
>> 

Question on -fwrapv and -fwrapv-pointer

[Qing Zhao via Gcc-patches]

Hi,

I have several questions on these options:

1.are pointers treated as signed integers in general? (I thought that pointers 
are addresses to the memory, should be treated as unsigned integer…)
2. If Yes, why? 
3. why a separate option for pointesr -fwrapv-pointer in addition to -fwrapv if 
they are treated as signed integers?

Thanks for your help.

Qing

[PATCH] tree optimization/111407--SSA corruption due to widening_mul opt

[Qing Zhao via Gcc-patches]

on conflict across an abnormal edge

This is a bug in tree-ssa-math-opts.cc, when applying the widening mul
optimization, the compiler needs to check whether the operand is in a
ABNORMAL PHI, if YES, we should avoid the transformation.

bootstrapped and regression tested on both aarch64 and x86, no issue.

Okay for committing?

thanks.

Qing

=

PR tree-optimization/111407

gcc/ChangeLog:

* tree-ssa-math-opts.cc (convert_mult_to_widen): Avoid the transform
when one of the operands is subject to abnormal coalescing.

gcc/testsuite/ChangeLog:

* gcc.dg/pr111407.c: New test.
---
 gcc/testsuite/gcc.dg/pr111407.c | 21 +
 gcc/tree-ssa-math-opts.cc   |  8 
 2 files changed, 29 insertions(+)
 create mode 100644 gcc/testsuite/gcc.dg/pr111407.c

diff --git a/gcc/testsuite/gcc.dg/pr111407.c b/gcc/testsuite/gcc.dg/pr111407.c
new file mode 100644
index 000..a171074753f
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/pr111407.c
@@ -0,0 +1,21 @@
+/* PR tree-optimization/111407*/
+/* { dg-do compile } */
+/* { dg-options "-O2" } */
+enum { SEND_TOFILE } __sigsetjmp();
+void fclose();
+void foldergets();
+void sendpart_stats(int *p1, int a1, int b1) {
+ int *a = p1;
+ fclose();
+ p1 = 0;
+ long t = b1;
+ if (__sigsetjmp()) {
+   {
+ long t1 = a1;
+ a1+=1;
+ fclose(a1*(long)t1);
+   }
+ }
+ if (p1)
+   fclose();
+}
diff --git a/gcc/tree-ssa-math-opts.cc b/gcc/tree-ssa-math-opts.cc
index 3db69ad5733..51c14d6bad9 100644
--- a/gcc/tree-ssa-math-opts.cc
+++ b/gcc/tree-ssa-math-opts.cc
@@ -2755,6 +2755,14 @@ convert_mult_to_widen (gimple *stmt, 
gimple_stmt_iterator *gsi)
   if (!is_widening_mult_p (stmt, , , , ))
 return false;
 
+  /* if any one of rhs1 and rhs2 is subject to abnormal coalescing,
+ avoid the tranform. */
+  if ((TREE_CODE (rhs1) == SSA_NAME
+   && SSA_NAME_OCCURS_IN_ABNORMAL_PHI (rhs1))
+  || (TREE_CODE (rhs2) == SSA_NAME
+ && SSA_NAME_OCCURS_IN_ABNORMAL_PHI (rhs2)))
+return false;
+
   to_mode = SCALAR_INT_TYPE_MODE (type);
   from_mode = SCALAR_INT_TYPE_MODE (type1);
   if (to_mode == from_mode)
-- 
2.31.1

Re: [V3][PATCH 2/3] Use the counted_by atribute info in builtin object size [PR108896]

[Qing Zhao via Gcc-patches]

Ping.

thanks.

Qing

> On Aug 25, 2023, at 11:24 AM, Qing Zhao  wrote:
> 
> Use the counted_by atribute info in builtin object size to compute the
> subobject size for flexible array members.
> 
> gcc/ChangeLog:
> 
>   PR C/108896
>   * tree-object-size.cc (addr_object_size): Use the counted_by
>   attribute info.
>   * tree.cc (component_ref_has_counted_by_p): New function.
>   (component_ref_get_counted_by): New function.
>   * tree.h (component_ref_has_counted_by_p): New prototype.
>   (component_ref_get_counted_by): New prototype.
> 
> gcc/testsuite/ChangeLog:
> 
>   PR C/108896
>   * gcc.dg/flex-array-counted-by-2.c: New test.
>   * gcc.dg/flex-array-counted-by-3.c: New test.
> ---
> .../gcc.dg/flex-array-counted-by-2.c  |  74 ++
> .../gcc.dg/flex-array-counted-by-3.c  | 210 ++
> gcc/tree-object-size.cc   |  37 ++-
> gcc/tree.cc   |  95 +++-
> gcc/tree.h|  10 +
> 5 files changed, 418 insertions(+), 8 deletions(-)
> create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
> create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-3.c
> 
> diff --git a/gcc/testsuite/gcc.dg/flex-array-counted-by-2.c 
> b/gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
> new file mode 100644
> index ..ec580c1f1f01
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
> @@ -0,0 +1,74 @@
> +/* test the attribute counted_by and its usage in
> + * __builtin_dynamic_object_size.  */ 
> +/* { dg-do run } */
> +/* { dg-options "-O2" } */
> +
> +#include "builtin-object-size-common.h"
> +
> +#define expect(p, _v) do { \
> +size_t v = _v; \
> +if (p == v) \
> + __builtin_printf ("ok:  %s == %zd\n", #p, p); \
> +else \
> + {  \
> +   __builtin_printf ("WAT: %s == %zd (expected %zd)\n", #p, p, v); \
> +   FAIL (); \
> + } \
> +} while (0);
> +
> +struct flex {
> +  int b;
> +  int c[];
> +} *array_flex;
> +
> +struct annotated {
> +  int b;
> +  int c[] __attribute__ ((counted_by (b)));
> +} *array_annotated;
> +
> +struct nested_annotated {
> +  struct {
> +union {
> +  int b;
> +  float f;   
> +};
> +int n;
> +  };
> +  int c[] __attribute__ ((counted_by (b)));
> +} *array_nested_annotated;
> +
> +void __attribute__((__noinline__)) setup (int normal_count, int attr_count)
> +{
> +  array_flex
> += (struct flex *)malloc (sizeof (struct flex)
> +  + normal_count *  sizeof (int));
> +  array_flex->b = normal_count;
> +
> +  array_annotated
> += (struct annotated *)malloc (sizeof (struct annotated)
> +   + attr_count *  sizeof (int));
> +  array_annotated->b = attr_count;
> +
> +  array_nested_annotated
> += (struct nested_annotated *)malloc (sizeof (struct nested_annotated)
> +  + attr_count *  sizeof (int));
> +  array_nested_annotated->b = attr_count;
> +
> +  return;
> +}
> +
> +void __attribute__((__noinline__)) test ()
> +{
> +expect(__builtin_dynamic_object_size(array_flex->c, 1), -1);
> +expect(__builtin_dynamic_object_size(array_annotated->c, 1),
> +array_annotated->b * sizeof (int));
> +expect(__builtin_dynamic_object_size(array_nested_annotated->c, 1),
> +array_nested_annotated->b * sizeof (int));
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +  setup (10,10);   
> +  test ();
> +  DONE ();
> +}
> diff --git a/gcc/testsuite/gcc.dg/flex-array-counted-by-3.c 
> b/gcc/testsuite/gcc.dg/flex-array-counted-by-3.c
> new file mode 100644
> index ..a0c3cb88ec71
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/flex-array-counted-by-3.c
> @@ -0,0 +1,210 @@
> +/* test the attribute counted_by and its usage in
> +__builtin_dynamic_object_size: what's the correct behavior when the
> +allocation size mismatched with the value of counted_by attribute?  */
> +/* { dg-do run } */
> +/* { dg-options "-O -fstrict-flex-arrays=3" } */
> +
> +#include "builtin-object-size-common.h"
> +
> +struct annotated {
> +  size_t foo;
> +  char others;
> +  char array[] __attribute__((counted_by (foo)));
> +};
> +
> +#define expect(p, _v) do { \
> +size_t v = _v; \
> +if (p == v) \
> +__builtin_printf ("ok:  %s == %zd\n", #p, p); \
> +else \
> +{  \
> +  __builtin_printf ("WAT: %s == %zd (expected %zd)\n", #p, p, v); \
> +   FAIL (); \
> +} \
> +} while (0);
> +
> +#define noinline __attribute__((__noinline__))
> +#define SIZE_BUMP 10 
> +#define MAX(a, b) ((a) > (b) ? (a) : (b))
> +#define MIN(a, b) ((a) < (b) ? (a) : (b))
> +
> +/* In general, Due to type casting, the type for the pointee of a pointer
> +   does not say anything about the object it points to,
> +   So, __builtin_object_size can not directly use the type of the pointee
> +   to decide the size of the object the pointer points to.
> +
> +   

Re: [V3][PATCH 3/3] Use the counted_by attribute information in bound sanitizer[PR108896]

[Qing Zhao via Gcc-patches]

Ping.

thanks.

Qing

> On Aug 25, 2023, at 11:24 AM, Qing Zhao  wrote:
> 
> Use the counted_by attribute information in bound sanitizer.
> 
> gcc/c-family/ChangeLog:
> 
>   PR C/108896
>   * c-ubsan.cc (ubsan_instrument_bounds): Use counted_by attribute
>   information.
> 
> gcc/testsuite/ChangeLog:
> 
>   PR C/108896
>   * gcc.dg/ubsan/flex-array-counted-by-bounds.c: New test.
>   * gcc.dg/ubsan/flex-array-counted-by-bounds-2.c: New test.
> ---
> gcc/c-family/c-ubsan.cc   | 16 +++
> .../ubsan/flex-array-counted-by-bounds-2.c| 27 +++
> .../ubsan/flex-array-counted-by-bounds.c  | 46 +++
> 3 files changed, 89 insertions(+)
> create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
> create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c
> 
> diff --git a/gcc/c-family/c-ubsan.cc b/gcc/c-family/c-ubsan.cc
> index 51aa83a378d2..a99e8433069f 100644
> --- a/gcc/c-family/c-ubsan.cc
> +++ b/gcc/c-family/c-ubsan.cc
> @@ -362,6 +362,10 @@ ubsan_instrument_bounds (location_t loc, tree array, 
> tree *index,
> {
>   tree type = TREE_TYPE (array);
>   tree domain = TYPE_DOMAIN (type);
> +  /* whether the array ref is a flexible array member with valid counted_by
> + attribute.  */
> +  bool fam_has_count_attr = false;
> +  tree counted_by = NULL_TREE;
> 
>   if (domain == NULL_TREE)
> return NULL_TREE;
> @@ -375,6 +379,17 @@ ubsan_instrument_bounds (location_t loc, tree array, 
> tree *index,
> && COMPLETE_TYPE_P (type)
> && integer_zerop (TYPE_SIZE (type)))
>   bound = build_int_cst (TREE_TYPE (TYPE_MIN_VALUE (domain)), -1);
> +  /* If the array ref is to flexible array member field which has
> +  counted_by attribute.  We can use the information from the
> +  attribute as the bound to instrument the reference.  */
> +  else if ((counted_by = component_ref_get_counted_by (array))
> + != NULL_TREE)
> + {
> +   fam_has_count_attr = true;
> +   bound = fold_build2 (MINUS_EXPR, TREE_TYPE (counted_by),
> +counted_by,
> +build_int_cst (TREE_TYPE (counted_by), 1));
> + }
>   else
>   return NULL_TREE;
> }
> @@ -387,6 +402,7 @@ ubsan_instrument_bounds (location_t loc, tree array, tree 
> *index,
>  -fsanitize=bounds-strict.  */
>   tree base = get_base_address (array);
>   if (!sanitize_flags_p (SANITIZE_BOUNDS_STRICT)
> +  && !fam_has_count_attr
>   && TREE_CODE (array) == COMPONENT_REF
>   && base && (INDIRECT_REF_P (base) || TREE_CODE (base) == MEM_REF))
> {
> diff --git a/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c 
> b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
> new file mode 100644
> index ..77ec333509d0
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
> @@ -0,0 +1,27 @@
> +/* test the attribute counted_by and its usage in
> +   bounds sanitizer combined with VLA.  */
> +/* { dg-do run } */
> +/* { dg-options "-fsanitize=bounds" } */
> +
> +#include 
> +
> +void __attribute__((__noinline__)) setup_and_test_vla (int n, int m)
> +{
> +   struct foo {
> +   int n;
> +   int p[][n] __attribute__((counted_by(n)));
> +   } *f;
> +
> +   f = (struct foo *) malloc (sizeof(struct foo) + m*sizeof(int[n]));
> +   f->n = m;
> +   f->p[m][n-1]=1;
> +   return;
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +  setup_and_test_vla (10, 11);
> +  return 0;
> +}
> +
> +/* { dg-output "17:8: runtime error: index 11 out of bounds for type" } */
> diff --git a/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c 
> b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c
> new file mode 100644
> index ..81eaeb3f2681
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c
> @@ -0,0 +1,46 @@
> +/* test the attribute counted_by and its usage in
> +   bounds sanitizer.  */
> +/* { dg-do run } */
> +/* { dg-options "-fsanitize=bounds" } */
> +
> +#include 
> +
> +struct flex {
> +  int b;
> +  int c[];
> +} *array_flex;
> +
> +struct annotated {
> +  int b;
> +  int c[] __attribute__ ((counted_by (b)));
> +} *array_annotated;
> +
> +void __attribute__((__noinline__)) setup (int normal_count, int 
> annotated_count)
> +{
> +  array_flex
> += (struct flex *)malloc (sizeof (struct flex)
> +  + normal_count *  sizeof (int));
> +  array_flex->b = normal_count;
> +
> +  array_annotated
> += (struct annotated *)malloc (sizeof (struct annotated)
> +   + annotated_count *  sizeof (int));
> +  array_annotated->b = annotated_count;
> +
> +  return;
> +}
> +
> +void __attribute__((__noinline__)) test (int normal_index, int 
> annotated_index)
> +{
> +  array_flex->c[normal_index] = 1;
> +  array_annotated->c[annotated_index] = 2;
> +}
> +
> +int main(int argc, char *argv[])
> +{
> 

Re: [V3][PATCH 1/3] Provide counted_by attribute to flexible array member field (PR108896)

[Qing Zhao via Gcc-patches]

PIng.

thanks.

Qing

> On Aug 25, 2023, at 11:24 AM, Qing Zhao  wrote:
> 
> Provide a new counted_by attribute to flexible array member field.
> 
> 'counted_by (COUNT)'
> The 'counted_by' attribute may be attached to the flexible array
> member of a structure.  It indicates that the number of the
> elements of the array is given by the field named "COUNT" in the
> same structure as the flexible array member.  GCC uses this
> information to improve the results of the array bound sanitizer and
> the '__builtin_dynamic_object_size'.
> 
> For instance, the following code:
> 
>  struct P {
>size_t count;
>char other;
>char array[] __attribute__ ((counted_by (count)));
>  } *p;
> 
> specifies that the 'array' is a flexible array member whose number
> of elements is given by the field 'count' in the same structure.
> 
> The field that represents the number of the elements should have an
> integer type.  An explicit 'counted_by' annotation defines a
> relationship between two objects, 'p->array' and 'p->count', that
> 'p->array' has _at least_ 'p->count' number of elements available.
> This relationship must hold even after any of these related objects
> are updated.  It's the user's responsibility to make sure this
> relationship to be kept all the time.  Otherwise the results of the
> array bound sanitizer and the '__builtin_dynamic_object_size' might
> be incorrect.
> 
> For instance, in the following example, the allocated array has
> less elements than what's specified by the 'sbuf->count', this is
> an user error.  As a result, out-of-bounds access to the array
> might not be detected.
> 
>  #define SIZE_BUMP 10
>  struct P *sbuf;
>  void alloc_buf (size_t nelems)
>  {
>sbuf = (struct P *) malloc (MAX (sizeof (struct P),
>   (offsetof (struct P, array[0])
>+ nelems * sizeof (char;
>sbuf->count = nelems + SIZE_BUMP;
>/* This is invalid when the sbuf->array has less than sbuf->count
>   elements.  */
>  }
> 
> In the following example, the 2nd update to the field 'sbuf->count'
> of the above structure will permit out-of-bounds access to the
> array 'sbuf>array' as well.
> 
>  #define SIZE_BUMP 10
>  struct P *sbuf;
>  void alloc_buf (size_t nelems)
>  {
>sbuf = (struct P *) malloc (MAX (sizeof (struct P),
>   (offsetof (struct P, array[0])
>+ (nelems + SIZE_BUMP) * sizeof 
> (char;
>sbuf->count = nelems;
>/* This is valid when the sbuf->array has at least sbuf->count
>   elements.  */
>  }
>  void use_buf (int index)
>  {
>sbuf->count = sbuf->count + SIZE_BUMP + 1;
>/* Now the value of sbuf->count is larger than the number
>   of elements of sbuf->array.  */
>sbuf->array[index] = 0;
>/* then the out-of-bound access to this array
>   might not be detected.  */
>  }
> 
> gcc/c-family/ChangeLog:
> 
>   PR C/108896
>   * c-attribs.cc (handle_counted_by_attribute): New function.
>   (attribute_takes_identifier_p): Add counted_by attribute to the list.
>   * c-common.cc (c_flexible_array_member_type_p): ...To this.
>   * c-common.h (c_flexible_array_member_type_p): New prototype.
> 
> gcc/c/ChangeLog:
> 
>   PR C/108896
>   * c-decl.cc (flexible_array_member_type_p): Renamed and moved to...
>   (add_flexible_array_elts_to_size): Use renamed function.
>   (is_flexible_array_member_p): Use renamed function.
>   (verify_counted_by_attribute): New function.
>   (finish_struct): Use renamed function and verify counted_by
>   attribute.
> 
> gcc/ChangeLog:
> 
>   PR C/108896
>   * doc/extend.texi: Document attribute counted_by.
>   * tree.cc (get_named_field): New function.
>   * tree.h (get_named_field): New prototype.
> 
> gcc/testsuite/ChangeLog:
> 
>   PR C/108896
>   * gcc.dg/flex-array-counted-by.c: New test.
> ---
> gcc/c-family/c-attribs.cc| 54 -
> gcc/c-family/c-common.cc | 13 
> gcc/c-family/c-common.h  |  1 +
> gcc/c/c-decl.cc  | 79 +++-
> gcc/doc/extend.texi  | 77 +++
> gcc/testsuite/gcc.dg/flex-array-counted-by.c | 40 ++
> gcc/tree.cc  | 40 ++
> gcc/tree.h   |  5 ++
> 8 files changed, 291 insertions(+), 18 deletions(-)
> create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by.c
> 
> diff --git 

Re: [V3][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

Ping.  

Thanks.

Qing

> On Aug 25, 2023, at 11:24 AM, Qing Zhao  wrote:
> 
> This is the 3rd version of the patch, per our discussion based on the
> review comments for the 1st and 2nd version, the major changes in this
> version are:
> 
> ***Against 1st version:
> 1. change the name "element_count" to "counted_by";
> 2. change the parameter for the attribute from a STRING to an
> Identifier;
> 3. Add logic and testing cases to handle anonymous structure/unions;
> 4. Clarify documentation to permit the situation when the allocation
> size is larger than what's specified by "counted_by", at the same time,
> it's user's error if allocation size is smaller than what's specified by
> "counted_by";
> 5. Add a complete testing case for using counted_by attribute in
> __builtin_dynamic_object_size when there is mismatch between the
> allocation size and the value of "counted_by", the expecting behavior
> for each case and the explanation on why in the comments. 
> 
> ***Against 2rd version:
> 1. Identify a tree node sharing issue and fixed it in the routine
>   "component_ref_get_counted_ty" of tree.cc;
> 2. Update the documentation and testing cases with the clear usage
>   of the fomula to compute the allocation size:
> MAX (sizeof (struct A), offsetof (struct A, array[0]) + counted_by * 
> sizeof(element))
>   (the algorithm used in tree-object-size.cc is correct).
> 
> In this set of patches, the major functionality provided is:
> 
> 1. a new attribute "counted_by";
> 2. use this new attribute in bound sanitizer;
> 3. use this new attribute in dynamic object size for subobject size;
> 
> As discussed, I plan to add two more separate patches sets after this initial
> patch set is approved and committed.
> 
> set 1. A new warning option and a new sanitizer option for the user error
>  when the allocation size is smaller than the value of "counted_by".
> set 2. An improvement to __builtin_dynamic_object_size  for whole-object
>  size of the structure with FAM annaoted with counted_by. 
> 
> there are also some existing bugs in tree-object-size.cc identified
> during the study, and PRs were filed to record them. these bugs will 
> be fixed seperately with individual patches:
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111030
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111040
> 
> Bootstrapped and regression tested on both aarch64 and X86, no issue.
> 
> Please see more details on the description of this work on:
> 
> https://gcc.gnu.org/pipermail/gcc-patches/2023-May/619708.html
> 
> and more discussions on
> https://gcc.gnu.org/pipermail/gcc-patches/2023-August/626376.html
> 
> Okay for committing?
> 
> thanks.
> 
> Qing
> 
> Qing Zhao (3):
>  Provide counted_by attribute to flexible array member field (PR108896)
>  Use the counted_by atribute info in builtin object size [PR108896]
>  Use the counted_by attribute information in bound sanitizer[PR108896]
> 
> gcc/c-family/c-attribs.cc |  54 -
> gcc/c-family/c-common.cc  |  13 ++
> gcc/c-family/c-common.h   |   1 +
> gcc/c-family/c-ubsan.cc   |  16 ++
> gcc/c/c-decl.cc   |  79 +--
> gcc/doc/extend.texi   |  77 +++
> .../gcc.dg/flex-array-counted-by-2.c  |  74 ++
> .../gcc.dg/flex-array-counted-by-3.c  | 210 ++
> gcc/testsuite/gcc.dg/flex-array-counted-by.c  |  40 
> .../ubsan/flex-array-counted-by-bounds-2.c|  27 +++
> .../ubsan/flex-array-counted-by-bounds.c  |  46 
> gcc/tree-object-size.cc   |  37 ++-
> gcc/tree.cc   | 133 +++
> gcc/tree.h|  15 ++
> 14 files changed, 797 insertions(+), 25 deletions(-)
> create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
> create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-3.c
> create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by.c
> create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
> create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c
> 
> -- 
> 2.31.1
> 

Re: RFC: Introduce -fhardened to enable security-related flags

[Qing Zhao via Gcc-patches]

> On Aug 29, 2023, at 3:42 PM, Marek Polacek via Gcc-patches 
>  wrote:
> 
> Improving the security of software has been a major trend in the recent
> years.  Fortunately, GCC offers a wide variety of flags that enable extra
> hardening.  These flags aren't enabled by default, though.  And since
> there are a lot of hardening flags, with more to come, it's been difficult
> to keep on top of them; more so for the users of GCC who ought not to be
> expected to keep track of all the new options.
> 
> To alleviate some of the problems I mentioned, we thought it would
> be useful to provide a new umbrella option that enables a reasonable set
> of hardening flags.  What's "reasonable" in this context is not easy to
> pin down.  Surely, there must be no ABI impact, the option cannot cause
> severe performance issues, and, I suspect, it should not cause build
> errors by enabling stricter compile-time errors (such as, -Wimplicit-int,
> -Wint-conversion).  Including a controversial option in -fhardened
> would likely cause that users would not use -fhardened at all.  It's
> roughly akin to -Wall or -O2 -- those also enable a reasonable set of
> options, and evolve over time, and are not kept in sync with other
> compilers.
> 
> Currently, -fhardened enables:
> 
>  -D_FORTIFY_SOURCE=3 (or =2 for older glibcs)
>  -D_GLIBCXX_ASSERTIONS
>  -ftrivial-auto-var-init=zero
>  -fPIE  -pie  -Wl,-z,relro,-z,now
>  -fstack-protector-strong
>  -fstack-clash-protection
>  -fcf-protection=full (x86 GNU/Linux only)
> 
> -fsanitize=undefined is specifically not enabled.  -fstrict-flex-arrays is
> also liable to break a lot of code so I didn't include it.
> 
> Appended is a proof-of-concept patch.  It doesn't implement --help=hardened
> yet.  A fairly crucial point is that -fhardened will not override options
> that were specified on the command line (before or after -fhardened).  For
> example,
> 
> -D_FORTIFY_SOURCE=1 -fhardened
> 
> means that _FORTIFY_SOURCE=1 will be used.  Similarly,
> 
>  -fhardened -fstack-protector
> 
> will not enable -fstack-protector-strong.
> 
> Thoughts?

In general, I think that it is a very good idea to provide umbrella options
 for software security purpose.  Thanks a lot for this work!

1. I do agree with Martin, multiple-level control for this purpose might be 
needed,
similar as multiple levels for warnings, and multiple levels for optimizations.

Similar as optimization options, can we organize all the security options 
together 
In our manual, then the user will have a good central place to get more and 
complete
Information of the security features our compiler provides? 

2. What’s the major criteria to decide which security feature should go into 
this list?
Later, when we have new security features, how to decide whether to add them to
This list or not?
I am wondering why -fzero-call-used-regs is not included in the list and also 
Why chose -ftrivial-auto-var-init=zero instead of 
-ftrivial-auto-var-init=pattern? 

3. Looks like currently, -fhardened excludes all compilation-time Warning 
options for security purpose,
(For example, -Warray-bounds, --Wstringop-overflow…)
And also excludes all sanitizer options for security purpose 
(-fsanitizer=undifined)

So, shall we also provide an umbrella option for compilation-time warning 
options for security purpose
And a umbrella option for sanitizer options (is the -fsanitizer=undefined this 
one)?

Just some thoughts. -:).

Qing

[V3][PATCH 3/3] Use the counted_by attribute information in bound sanitizer[PR108896]

[Qing Zhao via Gcc-patches]

Use the counted_by attribute information in bound sanitizer.

gcc/c-family/ChangeLog:

PR C/108896
* c-ubsan.cc (ubsan_instrument_bounds): Use counted_by attribute
information.

gcc/testsuite/ChangeLog:

PR C/108896
* gcc.dg/ubsan/flex-array-counted-by-bounds.c: New test.
* gcc.dg/ubsan/flex-array-counted-by-bounds-2.c: New test.
---
 gcc/c-family/c-ubsan.cc   | 16 +++
 .../ubsan/flex-array-counted-by-bounds-2.c| 27 +++
 .../ubsan/flex-array-counted-by-bounds.c  | 46 +++
 3 files changed, 89 insertions(+)
 create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
 create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c

diff --git a/gcc/c-family/c-ubsan.cc b/gcc/c-family/c-ubsan.cc
index 51aa83a378d2..a99e8433069f 100644
--- a/gcc/c-family/c-ubsan.cc
+++ b/gcc/c-family/c-ubsan.cc
@@ -362,6 +362,10 @@ ubsan_instrument_bounds (location_t loc, tree array, tree 
*index,
 {
   tree type = TREE_TYPE (array);
   tree domain = TYPE_DOMAIN (type);
+  /* whether the array ref is a flexible array member with valid counted_by
+ attribute.  */
+  bool fam_has_count_attr = false;
+  tree counted_by = NULL_TREE;
 
   if (domain == NULL_TREE)
 return NULL_TREE;
@@ -375,6 +379,17 @@ ubsan_instrument_bounds (location_t loc, tree array, tree 
*index,
  && COMPLETE_TYPE_P (type)
  && integer_zerop (TYPE_SIZE (type)))
bound = build_int_cst (TREE_TYPE (TYPE_MIN_VALUE (domain)), -1);
+  /* If the array ref is to flexible array member field which has
+counted_by attribute.  We can use the information from the
+attribute as the bound to instrument the reference.  */
+  else if ((counted_by = component_ref_get_counted_by (array))
+   != NULL_TREE)
+   {
+ fam_has_count_attr = true;
+ bound = fold_build2 (MINUS_EXPR, TREE_TYPE (counted_by),
+  counted_by,
+  build_int_cst (TREE_TYPE (counted_by), 1));
+   }
   else
return NULL_TREE;
 }
@@ -387,6 +402,7 @@ ubsan_instrument_bounds (location_t loc, tree array, tree 
*index,
  -fsanitize=bounds-strict.  */
   tree base = get_base_address (array);
   if (!sanitize_flags_p (SANITIZE_BOUNDS_STRICT)
+  && !fam_has_count_attr
   && TREE_CODE (array) == COMPONENT_REF
   && base && (INDIRECT_REF_P (base) || TREE_CODE (base) == MEM_REF))
 {
diff --git a/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c 
b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
new file mode 100644
index ..77ec333509d0
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
@@ -0,0 +1,27 @@
+/* test the attribute counted_by and its usage in
+   bounds sanitizer combined with VLA.  */
+/* { dg-do run } */
+/* { dg-options "-fsanitize=bounds" } */
+
+#include 
+
+void __attribute__((__noinline__)) setup_and_test_vla (int n, int m)
+{
+   struct foo {
+   int n;
+   int p[][n] __attribute__((counted_by(n)));
+   } *f;
+
+   f = (struct foo *) malloc (sizeof(struct foo) + m*sizeof(int[n]));
+   f->n = m;
+   f->p[m][n-1]=1;
+   return;
+}
+
+int main(int argc, char *argv[])
+{
+  setup_and_test_vla (10, 11);
+  return 0;
+}
+
+/* { dg-output "17:8: runtime error: index 11 out of bounds for type" } */
diff --git a/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c 
b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c
new file mode 100644
index ..81eaeb3f2681
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c
@@ -0,0 +1,46 @@
+/* test the attribute counted_by and its usage in
+   bounds sanitizer.  */
+/* { dg-do run } */
+/* { dg-options "-fsanitize=bounds" } */
+
+#include 
+
+struct flex {
+  int b;
+  int c[];
+} *array_flex;
+
+struct annotated {
+  int b;
+  int c[] __attribute__ ((counted_by (b)));
+} *array_annotated;
+
+void __attribute__((__noinline__)) setup (int normal_count, int 
annotated_count)
+{
+  array_flex
+= (struct flex *)malloc (sizeof (struct flex)
++ normal_count *  sizeof (int));
+  array_flex->b = normal_count;
+
+  array_annotated
+= (struct annotated *)malloc (sizeof (struct annotated)
+ + annotated_count *  sizeof (int));
+  array_annotated->b = annotated_count;
+
+  return;
+}
+
+void __attribute__((__noinline__)) test (int normal_index, int annotated_index)
+{
+  array_flex->c[normal_index] = 1;
+  array_annotated->c[annotated_index] = 2;
+}
+
+int main(int argc, char *argv[])
+{
+  setup (10, 10);   
+  test (10, 10);
+  return 0;
+}
+
+/* { dg-output "36:21: runtime error: index 10 out of bounds for type" } */
-- 
2.31.1

[V3][PATCH 2/3] Use the counted_by atribute info in builtin object size [PR108896]

[Qing Zhao via Gcc-patches]

Use the counted_by atribute info in builtin object size to compute the
subobject size for flexible array members.

gcc/ChangeLog:

PR C/108896
* tree-object-size.cc (addr_object_size): Use the counted_by
attribute info.
* tree.cc (component_ref_has_counted_by_p): New function.
(component_ref_get_counted_by): New function.
* tree.h (component_ref_has_counted_by_p): New prototype.
(component_ref_get_counted_by): New prototype.

gcc/testsuite/ChangeLog:

PR C/108896
* gcc.dg/flex-array-counted-by-2.c: New test.
* gcc.dg/flex-array-counted-by-3.c: New test.
---
 .../gcc.dg/flex-array-counted-by-2.c  |  74 ++
 .../gcc.dg/flex-array-counted-by-3.c  | 210 ++
 gcc/tree-object-size.cc   |  37 ++-
 gcc/tree.cc   |  95 +++-
 gcc/tree.h|  10 +
 5 files changed, 418 insertions(+), 8 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-3.c

diff --git a/gcc/testsuite/gcc.dg/flex-array-counted-by-2.c 
b/gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
new file mode 100644
index ..ec580c1f1f01
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
@@ -0,0 +1,74 @@
+/* test the attribute counted_by and its usage in
+ * __builtin_dynamic_object_size.  */ 
+/* { dg-do run } */
+/* { dg-options "-O2" } */
+
+#include "builtin-object-size-common.h"
+
+#define expect(p, _v) do { \
+size_t v = _v; \
+if (p == v) \
+   __builtin_printf ("ok:  %s == %zd\n", #p, p); \
+else \
+   {  \
+ __builtin_printf ("WAT: %s == %zd (expected %zd)\n", #p, p, v); \
+ FAIL (); \
+   } \
+} while (0);
+
+struct flex {
+  int b;
+  int c[];
+} *array_flex;
+
+struct annotated {
+  int b;
+  int c[] __attribute__ ((counted_by (b)));
+} *array_annotated;
+
+struct nested_annotated {
+  struct {
+union {
+  int b;
+  float f; 
+};
+int n;
+  };
+  int c[] __attribute__ ((counted_by (b)));
+} *array_nested_annotated;
+
+void __attribute__((__noinline__)) setup (int normal_count, int attr_count)
+{
+  array_flex
+= (struct flex *)malloc (sizeof (struct flex)
++ normal_count *  sizeof (int));
+  array_flex->b = normal_count;
+
+  array_annotated
+= (struct annotated *)malloc (sizeof (struct annotated)
+ + attr_count *  sizeof (int));
+  array_annotated->b = attr_count;
+
+  array_nested_annotated
+= (struct nested_annotated *)malloc (sizeof (struct nested_annotated)
++ attr_count *  sizeof (int));
+  array_nested_annotated->b = attr_count;
+
+  return;
+}
+
+void __attribute__((__noinline__)) test ()
+{
+expect(__builtin_dynamic_object_size(array_flex->c, 1), -1);
+expect(__builtin_dynamic_object_size(array_annotated->c, 1),
+  array_annotated->b * sizeof (int));
+expect(__builtin_dynamic_object_size(array_nested_annotated->c, 1),
+  array_nested_annotated->b * sizeof (int));
+}
+
+int main(int argc, char *argv[])
+{
+  setup (10,10);   
+  test ();
+  DONE ();
+}
diff --git a/gcc/testsuite/gcc.dg/flex-array-counted-by-3.c 
b/gcc/testsuite/gcc.dg/flex-array-counted-by-3.c
new file mode 100644
index ..a0c3cb88ec71
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/flex-array-counted-by-3.c
@@ -0,0 +1,210 @@
+/* test the attribute counted_by and its usage in
+__builtin_dynamic_object_size: what's the correct behavior when the
+allocation size mismatched with the value of counted_by attribute?  */
+/* { dg-do run } */
+/* { dg-options "-O -fstrict-flex-arrays=3" } */
+
+#include "builtin-object-size-common.h"
+
+struct annotated {
+  size_t foo;
+  char others;
+  char array[] __attribute__((counted_by (foo)));
+};
+
+#define expect(p, _v) do { \
+size_t v = _v; \
+if (p == v) \
+__builtin_printf ("ok:  %s == %zd\n", #p, p); \
+else \
+{  \
+  __builtin_printf ("WAT: %s == %zd (expected %zd)\n", #p, p, v); \
+ FAIL (); \
+} \
+} while (0);
+
+#define noinline __attribute__((__noinline__))
+#define SIZE_BUMP 10 
+#define MAX(a, b) ((a) > (b) ? (a) : (b))
+#define MIN(a, b) ((a) < (b) ? (a) : (b))
+
+/* In general, Due to type casting, the type for the pointee of a pointer
+   does not say anything about the object it points to,
+   So, __builtin_object_size can not directly use the type of the pointee
+   to decide the size of the object the pointer points to.
+
+   there are only two reliable ways:
+   A. observed allocations  (call to the allocation functions in the routine)
+   B. observed accesses (read or write access to the location of the
+ pointer points to)
+
+   that provide information about the type/existence of an object at
+   the corresponding 

[V3][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

This is the 3rd version of the patch, per our discussion based on the
review comments for the 1st and 2nd version, the major changes in this
version are:

***Against 1st version:
1. change the name "element_count" to "counted_by";
2. change the parameter for the attribute from a STRING to an
Identifier;
3. Add logic and testing cases to handle anonymous structure/unions;
4. Clarify documentation to permit the situation when the allocation
size is larger than what's specified by "counted_by", at the same time,
it's user's error if allocation size is smaller than what's specified by
"counted_by";
5. Add a complete testing case for using counted_by attribute in
__builtin_dynamic_object_size when there is mismatch between the
allocation size and the value of "counted_by", the expecting behavior
for each case and the explanation on why in the comments. 

***Against 2rd version:
1. Identify a tree node sharing issue and fixed it in the routine
   "component_ref_get_counted_ty" of tree.cc;
2. Update the documentation and testing cases with the clear usage
   of the fomula to compute the allocation size:
MAX (sizeof (struct A), offsetof (struct A, array[0]) + counted_by * 
sizeof(element))
   (the algorithm used in tree-object-size.cc is correct).

In this set of patches, the major functionality provided is:

1. a new attribute "counted_by";
2. use this new attribute in bound sanitizer;
3. use this new attribute in dynamic object size for subobject size;

As discussed, I plan to add two more separate patches sets after this initial
patch set is approved and committed.

set 1. A new warning option and a new sanitizer option for the user error
  when the allocation size is smaller than the value of "counted_by".
set 2. An improvement to __builtin_dynamic_object_size  for whole-object
  size of the structure with FAM annaoted with counted_by. 

there are also some existing bugs in tree-object-size.cc identified
during the study, and PRs were filed to record them. these bugs will 
be fixed seperately with individual patches:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111030
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111040

Bootstrapped and regression tested on both aarch64 and X86, no issue.

Please see more details on the description of this work on:

https://gcc.gnu.org/pipermail/gcc-patches/2023-May/619708.html

and more discussions on
https://gcc.gnu.org/pipermail/gcc-patches/2023-August/626376.html

Okay for committing?

thanks.

Qing

Qing Zhao (3):
  Provide counted_by attribute to flexible array member field (PR108896)
  Use the counted_by atribute info in builtin object size [PR108896]
  Use the counted_by attribute information in bound sanitizer[PR108896]

 gcc/c-family/c-attribs.cc |  54 -
 gcc/c-family/c-common.cc  |  13 ++
 gcc/c-family/c-common.h   |   1 +
 gcc/c-family/c-ubsan.cc   |  16 ++
 gcc/c/c-decl.cc   |  79 +--
 gcc/doc/extend.texi   |  77 +++
 .../gcc.dg/flex-array-counted-by-2.c  |  74 ++
 .../gcc.dg/flex-array-counted-by-3.c  | 210 ++
 gcc/testsuite/gcc.dg/flex-array-counted-by.c  |  40 
 .../ubsan/flex-array-counted-by-bounds-2.c|  27 +++
 .../ubsan/flex-array-counted-by-bounds.c  |  46 
 gcc/tree-object-size.cc   |  37 ++-
 gcc/tree.cc   | 133 +++
 gcc/tree.h|  15 ++
 14 files changed, 797 insertions(+), 25 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-3.c
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by.c
 create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
 create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c

-- 
2.31.1

[V3][PATCH 1/3] Provide counted_by attribute to flexible array member field (PR108896)

[Qing Zhao via Gcc-patches]

Provide a new counted_by attribute to flexible array member field.

'counted_by (COUNT)'
 The 'counted_by' attribute may be attached to the flexible array
 member of a structure.  It indicates that the number of the
 elements of the array is given by the field named "COUNT" in the
 same structure as the flexible array member.  GCC uses this
 information to improve the results of the array bound sanitizer and
 the '__builtin_dynamic_object_size'.

 For instance, the following code:

  struct P {
size_t count;
char other;
char array[] __attribute__ ((counted_by (count)));
  } *p;

 specifies that the 'array' is a flexible array member whose number
 of elements is given by the field 'count' in the same structure.

 The field that represents the number of the elements should have an
 integer type.  An explicit 'counted_by' annotation defines a
 relationship between two objects, 'p->array' and 'p->count', that
 'p->array' has _at least_ 'p->count' number of elements available.
 This relationship must hold even after any of these related objects
 are updated.  It's the user's responsibility to make sure this
 relationship to be kept all the time.  Otherwise the results of the
 array bound sanitizer and the '__builtin_dynamic_object_size' might
 be incorrect.

 For instance, in the following example, the allocated array has
 less elements than what's specified by the 'sbuf->count', this is
 an user error.  As a result, out-of-bounds access to the array
 might not be detected.

  #define SIZE_BUMP 10
  struct P *sbuf;
  void alloc_buf (size_t nelems)
  {
sbuf = (struct P *) malloc (MAX (sizeof (struct P),
   (offsetof (struct P, array[0])
+ nelems * sizeof (char;
sbuf->count = nelems + SIZE_BUMP;
/* This is invalid when the sbuf->array has less than sbuf->count
   elements.  */
  }

 In the following example, the 2nd update to the field 'sbuf->count'
 of the above structure will permit out-of-bounds access to the
 array 'sbuf>array' as well.

  #define SIZE_BUMP 10
  struct P *sbuf;
  void alloc_buf (size_t nelems)
  {
sbuf = (struct P *) malloc (MAX (sizeof (struct P),
   (offsetof (struct P, array[0])
+ (nelems + SIZE_BUMP) * sizeof 
(char;
sbuf->count = nelems;
/* This is valid when the sbuf->array has at least sbuf->count
   elements.  */
  }
  void use_buf (int index)
  {
sbuf->count = sbuf->count + SIZE_BUMP + 1;
/* Now the value of sbuf->count is larger than the number
   of elements of sbuf->array.  */
sbuf->array[index] = 0;
/* then the out-of-bound access to this array
   might not be detected.  */
  }

gcc/c-family/ChangeLog:

PR C/108896
* c-attribs.cc (handle_counted_by_attribute): New function.
(attribute_takes_identifier_p): Add counted_by attribute to the list.
* c-common.cc (c_flexible_array_member_type_p): ...To this.
* c-common.h (c_flexible_array_member_type_p): New prototype.

gcc/c/ChangeLog:

PR C/108896
* c-decl.cc (flexible_array_member_type_p): Renamed and moved to...
(add_flexible_array_elts_to_size): Use renamed function.
(is_flexible_array_member_p): Use renamed function.
(verify_counted_by_attribute): New function.
(finish_struct): Use renamed function and verify counted_by
attribute.

gcc/ChangeLog:

PR C/108896
* doc/extend.texi: Document attribute counted_by.
* tree.cc (get_named_field): New function.
* tree.h (get_named_field): New prototype.

gcc/testsuite/ChangeLog:

PR C/108896
* gcc.dg/flex-array-counted-by.c: New test.
---
 gcc/c-family/c-attribs.cc| 54 -
 gcc/c-family/c-common.cc | 13 
 gcc/c-family/c-common.h  |  1 +
 gcc/c/c-decl.cc  | 79 +++-
 gcc/doc/extend.texi  | 77 +++
 gcc/testsuite/gcc.dg/flex-array-counted-by.c | 40 ++
 gcc/tree.cc  | 40 ++
 gcc/tree.h   |  5 ++
 8 files changed, 291 insertions(+), 18 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by.c

diff --git a/gcc/c-family/c-attribs.cc b/gcc/c-family/c-attribs.cc
index e2792ca6898b..65e4f6639109 100644
--- a/gcc/c-family/c-attribs.cc
+++ b/gcc/c-family/c-attribs.cc
@@ -103,6 +103,8 @@ static tree 

Re: Another bug for __builtin_object_size? (Or expected behavior)

[Qing Zhao via Gcc-patches]

> On Aug 18, 2023, at 12:00 PM, Qing Zhao via Gcc-patches 
>  wrote:
> 
> 
> 
>> On Aug 17, 2023, at 5:32 PM, Siddhesh Poyarekar  wrote:
>> 
>> On 2023-08-17 17:25, Qing Zhao wrote:
>>>> It's not exactly the same issue, the earlier discussion was about choosing 
>>>> sizes in the same pass while the current one is about choosing between 
>>>> passes, but I agree it "rhymes".  This is what I was alluding to 
>>>> originally (for OST_MINIMUM use MIN_EXPR if both passes returned a pass) 
>>>> but I haven't thought about it hard enough to be 100% confident that it's 
>>>> the better solution, especially for OST_MAXIMUM.
>>> We have two different sources to get SIZE information for the subobject:
>>> 1. From the TYPESIZE information embedded in the IR;
>>> 2. From the initialization information propagated from data flow, this 
>>> includes both malloc call and the DECL_INIT.
>>> We need to choose between these two when both available, (these two 
>>> information could be
>>> in the same pass as we discussed before, or in different passes which is 
>>> shown in this discussion).
>>> I think that the MIN_EXPR might be the right choice (especially for 
>>> OST_MAXIMUM) -:)
>> 
>> It's worth a shot I guess.  We could emit something like the following in 
>> early_object_sizes_execute_one:
>> 
>> sz = (__bos(o->sub, ost) == unknown
>>   ? early_size
>>   : MIN_EXPR (__bos(o->sub, ost), early_size));
>> 
>> and see if it sticks.
> 
> I came up with the following change for tree-object-size.cc:
> 
> diff --git a/gcc/tree-object-size.cc b/gcc/tree-object-size.cc
> index a62af0500563..e1b2008c6dcc 100644
> --- a/gcc/tree-object-size.cc
> +++ b/gcc/tree-object-size.cc
> @@ -2016,10 +2016,22 @@ do_valueize (tree t)
>   return t;
> }
> 
> -/* Process a __builtin_object_size or __builtin_dynamic_object_size call in
> -   CALL early for subobjects before any object information is lost due to
> -   optimization.  Insert a MIN or MAX expression of the result and
> -   __builtin_object_size at I so that it may be processed in the second pass.
> +/* Process a __builtin_object_size or __builtin_dynamic_object_size call
> +   early for subobjects before any object information is lost due to
> +   optimization.
> +
> +   We have two different sources to get the size information for subobjects:
> +   A. The TYPE information of the subobject in the IR;
> +   B. The initialization information propagated through data flow.
> +   In the early pass, only A is available.
> +   B might be available in the second pass.
> +
> +   If both A and B are available, we should choose the minimum one between
> +   these two.
> +
> +   Insert a MIN expression of the result from the early pass and the original
> +   __builtin_object_size call at I so that it may be processed in the second 
> pass.
> +
>__builtin_dynamic_object_size is treated like __builtin_object_size here
>since we're only looking for constant bounds.  */
> 
> @@ -2036,7 +2048,7 @@ early_object_sizes_execute_one (gimple_stmt_iterator 
> *i, gimple *call)
>   unsigned HOST_WIDE_INT object_size_type = tree_to_uhwi (ost);
>   tree ptr = gimple_call_arg (call, 0);
> 
> -  if (object_size_type != 1 && object_size_type != 3)
> +  if (object_size_type & OST_SUBOBJECT == 0)
> return;
> 
>   if (TREE_CODE (ptr) != ADDR_EXPR && TREE_CODE (ptr) != SSA_NAME)
> @@ -2050,9 +2062,8 @@ early_object_sizes_execute_one (gimple_stmt_iterator 
> *i, gimple *call)
> 
>   tree tem = make_ssa_name (type);
>   gimple_call_set_lhs (call, tem);
> -  enum tree_code code = object_size_type & OST_MINIMUM ? MAX_EXPR : MIN_EXPR;
>   tree cst = fold_convert (type, bytes);
> -  gimple *g = gimple_build_assign (lhs, code, tem, cst);
> +  gimple *g = gimple_build_assign (lhs, MIN_EXPR, tem, cst);
>   gsi_insert_after (i, g, GSI_NEW_STMT);
>   update_stmt (call);
> }
> 
> Let me know if you see any issue with the change.

I tested the above change, everything is fine except one testing case in 
gcc.dg/builtin-object-size-4.c

I reduced the failed case to the following small one:

#include 
#include 
#include 

/* Tests for strdup/strndup.  */
size_t
__attribute__ ((noinline))
test9 (void)
{
  const char *ptr = "abcdefghijklmnopqrstuvwxyz";
  char *res = strndup (ptr, 21);
  int n = 0;
  if ((n = __builtin_object_size (res, 3)) != 22)
printf("FAIL is %d\n", n);

  free (res);
}

int
main (void)
{
  test9 ();
}
[opc@qinzhao-ol8u3-x86 gcc]$ sh t
FAIL is 1

I debugged into tree-object-size.cc, the routine “strdup_object

Re: Another bug for __builtin_object_size? (Or expected behavior)

[Qing Zhao via Gcc-patches]

> On Aug 17, 2023, at 5:32 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-17 17:25, Qing Zhao wrote:
>>> It's not exactly the same issue, the earlier discussion was about choosing 
>>> sizes in the same pass while the current one is about choosing between 
>>> passes, but I agree it "rhymes".  This is what I was alluding to originally 
>>> (for OST_MINIMUM use MIN_EXPR if both passes returned a pass) but I haven't 
>>> thought about it hard enough to be 100% confident that it's the better 
>>> solution, especially for OST_MAXIMUM.
>> We have two different sources to get SIZE information for the subobject:
>> 1. From the TYPESIZE information embedded in the IR;
>> 2. From the initialization information propagated from data flow, this 
>> includes both malloc call and the DECL_INIT.
>> We need to choose between these two when both available, (these two 
>> information could be
>>  in the same pass as we discussed before, or in different passes which is 
>> shown in this discussion).
>> I think that the MIN_EXPR might be the right choice (especially for 
>> OST_MAXIMUM) -:)
> 
> It's worth a shot I guess.  We could emit something like the following in 
> early_object_sizes_execute_one:
> 
>  sz = (__bos(o->sub, ost) == unknown
>? early_size
>: MIN_EXPR (__bos(o->sub, ost), early_size));
> 
> and see if it sticks.

I came up with the following change for tree-object-size.cc:

diff --git a/gcc/tree-object-size.cc b/gcc/tree-object-size.cc
index a62af0500563..e1b2008c6dcc 100644
--- a/gcc/tree-object-size.cc
+++ b/gcc/tree-object-size.cc
@@ -2016,10 +2016,22 @@ do_valueize (tree t)
   return t;
 }
 
-/* Process a __builtin_object_size or __builtin_dynamic_object_size call in
-   CALL early for subobjects before any object information is lost due to
-   optimization.  Insert a MIN or MAX expression of the result and
-   __builtin_object_size at I so that it may be processed in the second pass.
+/* Process a __builtin_object_size or __builtin_dynamic_object_size call
+   early for subobjects before any object information is lost due to
+   optimization.
+
+   We have two different sources to get the size information for subobjects:
+   A. The TYPE information of the subobject in the IR;
+   B. The initialization information propagated through data flow.
+   In the early pass, only A is available.
+   B might be available in the second pass.
+
+   If both A and B are available, we should choose the minimum one between
+   these two.
+
+   Insert a MIN expression of the result from the early pass and the original
+   __builtin_object_size call at I so that it may be processed in the second 
pass.
+
__builtin_dynamic_object_size is treated like __builtin_object_size here
since we're only looking for constant bounds.  */
 
@@ -2036,7 +2048,7 @@ early_object_sizes_execute_one (gimple_stmt_iterator *i, 
gimple *call)
   unsigned HOST_WIDE_INT object_size_type = tree_to_uhwi (ost);
   tree ptr = gimple_call_arg (call, 0);
 
-  if (object_size_type != 1 && object_size_type != 3)
+  if (object_size_type & OST_SUBOBJECT == 0)
 return;
 
   if (TREE_CODE (ptr) != ADDR_EXPR && TREE_CODE (ptr) != SSA_NAME)
@@ -2050,9 +2062,8 @@ early_object_sizes_execute_one (gimple_stmt_iterator *i, 
gimple *call)
 
   tree tem = make_ssa_name (type);
   gimple_call_set_lhs (call, tem);
-  enum tree_code code = object_size_type & OST_MINIMUM ? MAX_EXPR : MIN_EXPR;
   tree cst = fold_convert (type, bytes);
-  gimple *g = gimple_build_assign (lhs, code, tem, cst);
+  gimple *g = gimple_build_assign (lhs, MIN_EXPR, tem, cst);
   gsi_insert_after (i, g, GSI_NEW_STMT);
   update_stmt (call);
 }

Let me know if you see any issue with the change.

thanks.

Qing

> 
> Thanks,
> Sid

Re: Another bug for __builtin_object_size? (Or expected behavior)

[Qing Zhao via Gcc-patches]

> On Aug 17, 2023, at 4:57 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-17 16:23, Qing Zhao wrote:
 Then, I think whatever MIN or MAX, the early phase has more precise 
 information than the later phase, we should use its result if it’s NOT 
 UNKNOWN?
>>> 
>>> We can't be sure about that though, can we?  For example for something like 
>>> this:
>>> 
>>> struct S
>>> {
>>>  int a;
>>>  char b[10];
>>>  int c;
>>> };
>>> 
>>> size_t foo (struct S *s)
>>> {
>>>  return __builtin_object_size (s->b, 1);
>>> }
>>> 
>>> size_t bar ()
>>> {
>>>  struct S *in = malloc (8);
>>> 
>>>  return foo (in);
>>> }
>>> 
>>> returns 10 for __builtin_object_size in early_objsz but when it sees the 
>>> malloc in the later objsz pass, it returns 4:
>>> 
>>> $ gcc/cc1 -fdump-tree-objsz-details -quiet -o - -O bug.c
>>> ...
>>> foo:
>>> .LFB0:
>>> .cfi_startproc
>>> movl$10, %eax
>>> ret
>>> .cfi_endproc
>>> ...
>>> bar:
>>> .LFB1:
>>> .cfi_startproc
>>> movl$4, %eax
>>> ret
>>> .cfi_endproc
>>> ...
>>> 
>>> In fact, this ends up returning the wrong result for OST_MINIMUM:
>>> 
>>> $ gcc/cc1 -fdump-tree-objsz-details -quiet -o - -O bug.c
>>> ...
>>> foo:
>>> .LFB0:
>>> .cfi_startproc
>>> movl$10, %eax
>>> ret
>>> .cfi_endproc
>>> ...
>>> bar:
>>> .LFB1:
>>> .cfi_startproc
>>> movl$10, %eax
>>> ret
>>> .cfi_endproc
>>> ...
>>> 
>>> bar ought to have returned 4 too (and I'm betting the later objsz must have 
>>> seen that) but it got overridden by the earlier estimate of 10.
>> Okay, I see.
>> Then is this the similar issue we discussed previously?  (As following:)
>> "
>>> Hi, Sid and Jakub,
>>> I have a question in the following source portion of the routine 
>>> “addr_object_size” of gcc/tree-object-size.cc:
>>>  743   bytes = compute_object_offset (TREE_OPERAND (ptr, 0), var);
>>>  744   if (bytes != error_mark_node)
>>>  745 {
>>>  746   bytes = size_for_offset (var_size, bytes);
>>>  747   if (var != pt_var && pt_var_size && TREE_CODE (pt_var) == 
>>> MEM_REF)
>>>  748 {
>>>  749   tree bytes2 = compute_object_offset (TREE_OPERAND (ptr, 
>>> 0),
>>>  750pt_var);
>>>  751   if (bytes2 != error_mark_node)
>>>  752 {
>>>  753   bytes2 = size_for_offset (pt_var_size, bytes2);
>>>  754   bytes = size_binop (MIN_EXPR, bytes, bytes2);
>>>  755 }
>>>  756 }
>>>  757 }
>>> At line 754, why we always use “MIN_EXPR” whenever it’s for OST_MINIMUM or 
>>> not?
>>> Shall we use
>>> (object_size_type & OST_MINIMUM
>>> ? MIN_EXPR : MAX_EXPR)
>> That MIN_EXPR is not for OST_MINIMUM.  It is to cater for allocations like 
>> this:
>> typedef struct
>> {
>>  int a;
>> } A;
>> size_t f()
>> {
>>  A *p = malloc (1);
>>  return __builtin_object_size (p, 0);
>> }
>> where the returned size should be 1 and not sizeof (int).  The mode doesn't 
>> really matter in this case.
>> “
>> If this is the same issue, I think we can use the same solution: always use 
>> MIN_EXPR,
>> What do you think?
> 
> It's not exactly the same issue, the earlier discussion was about choosing 
> sizes in the same pass while the current one is about choosing between 
> passes, but I agree it "rhymes".  This is what I was alluding to originally 
> (for OST_MINIMUM use MIN_EXPR if both passes returned a pass) but I haven't 
> thought about it hard enough to be 100% confident that it's the better 
> solution, especially for OST_MAXIMUM.

We have two different sources to get SIZE information for the subobject:

1. From the TYPESIZE information embedded in the IR;
2. From the initialization information propagated from data flow, this includes 
both malloc call and the DECL_INIT. 

We need to choose between these two when both available, (these two information 
could be
 in the same pass as we discussed before, or in different passes which is shown 
in this discussion). 

I think that the MIN_EXPR might be the right choice (especially for 
OST_MAXIMUM) -:)

Qing
> 
> Thanks,
> Sid

Re: Another bug for __builtin_object_size? (Or expected behavior)

[Qing Zhao via Gcc-patches]

> On Aug 17, 2023, at 3:59 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-17 15:27, Qing Zhao wrote:
>>> Yes, that's it.  Maybe it's more correct if instead of MAX_EXPR if for 
>>> OST_MINIMUM we stick with the early_objsz answer if it's non-zero.  I'm not 
>>> sure if that's the case for maximum size though, my gut says it isn't.
>> So, the major purpose for adding the early object size phase is for 
>> computing SUBobjects size more precisely before the subobject information 
>> lost?
> 
> I suppose it's more about being able to do it at all, rather than precision.

Without the subobject information in IR, our later object size phase uses the 
whole object size as an estimation as it currently does. -:)
> 
>> Then, I think whatever MIN or MAX, the early phase has more precise 
>> information than the later phase, we should use its result if it’s NOT 
>> UNKNOWN?
> 
> We can't be sure about that though, can we?  For example for something like 
> this:
> 
> struct S
> {
>  int a;
>  char b[10];
>  int c;
> };
> 
> size_t foo (struct S *s)
> {
>  return __builtin_object_size (s->b, 1);
> }
> 
> size_t bar ()
> {
>  struct S *in = malloc (8);
> 
>  return foo (in);
> }
> 
> returns 10 for __builtin_object_size in early_objsz but when it sees the 
> malloc in the later objsz pass, it returns 4:
> 
> $ gcc/cc1 -fdump-tree-objsz-details -quiet -o - -O bug.c
> ...
> foo:
> .LFB0:
>   .cfi_startproc
>   movl$10, %eax
>   ret
>   .cfi_endproc
> ...
> bar:
> .LFB1:
>   .cfi_startproc
>   movl$4, %eax
>   ret
>   .cfi_endproc
> ...
> 
> In fact, this ends up returning the wrong result for OST_MINIMUM:
> 
> $ gcc/cc1 -fdump-tree-objsz-details -quiet -o - -O bug.c
> ...
> foo:
> .LFB0:
>   .cfi_startproc
>   movl$10, %eax
>   ret
>   .cfi_endproc
> ...
> bar:
> .LFB1:
>   .cfi_startproc
>   movl$10, %eax
>   ret
>   .cfi_endproc
> ...
> 
> bar ought to have returned 4 too (and I'm betting the later objsz must have 
> seen that) but it got overridden by the earlier estimate of 10.

Okay, I see. 

Then is this the similar issue we discussed previously?  (As following:)

"
> Hi, Sid and Jakub,
> I have a question in the following source portion of the routine 
> “addr_object_size” of gcc/tree-object-size.cc:
>  743   bytes = compute_object_offset (TREE_OPERAND (ptr, 0), var);
>  744   if (bytes != error_mark_node)
>  745 {
>  746   bytes = size_for_offset (var_size, bytes);
>  747   if (var != pt_var && pt_var_size && TREE_CODE (pt_var) == 
> MEM_REF)
>  748 {
>  749   tree bytes2 = compute_object_offset (TREE_OPERAND (ptr, 0),
>  750pt_var);
>  751   if (bytes2 != error_mark_node)
>  752 {
>  753   bytes2 = size_for_offset (pt_var_size, bytes2);
>  754   bytes = size_binop (MIN_EXPR, bytes, bytes2);
>  755 }
>  756 }
>  757 }
> At line 754, why we always use “MIN_EXPR” whenever it’s for OST_MINIMUM or 
> not?
> Shall we use
> (object_size_type & OST_MINIMUM
> ? MIN_EXPR : MAX_EXPR)

That MIN_EXPR is not for OST_MINIMUM.  It is to cater for allocations like this:

typedef struct
{
 int a;
} A;

size_t f()
{
 A *p = malloc (1);

 return __builtin_object_size (p, 0);
}

where the returned size should be 1 and not sizeof (int).  The mode doesn't 
really matter in this case.
“

If this is the same issue, I think we can use the same solution: always use 
MIN_EXPR, 
What do you think?

Qing

> 
> We probably need smarter heuristics on choosing between the estimate of the 
> early_objsz and late objsz passes each by itself isn't good enough for 
> subobjects.
> 
> Thanks,
> Sid

Re: Another bug for __builtin_object_size? (Or expected behavior)

[Qing Zhao via Gcc-patches]

> On Aug 17, 2023, at 1:49 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-17 09:58, Qing Zhao wrote:
>>> So this is a (sort of) known issue, which necessitated the early_objsz pass 
>>> to get an estimate before a subobject reference was optimized to a MEM_REF.
>> Do you mean that after a subobject reference was optimized to a MEM_REF, 
>> there is no way to compute the size of the subobject anymore?
> 
> Yes, in cases where the TYPE_SIZE is lost and there's no other allocation 
> information to fall back on.

Okay, I see.

> 
>>>  However it looks like the MIN/MAX hack doesn't work in this case for 
>>> OST_MINIMUM; it should probably get the minimum of the two passes if both 
>>> passes were successful, or only the result of the pass that was successful.
>> You mean that the following line:
>> 2053   enum tree_code code = object_size_type & OST_MINIMUM ? MAX_EXPR : 
>> MIN_EXPR;
>> Might need to be changed to:
>> 2053   enum tree_code code =  MIN_EXPR;
> 
> Yes, that's it.  Maybe it's more correct if instead of MAX_EXPR if for 
> OST_MINIMUM we stick with the early_objsz answer if it's non-zero.  I'm not 
> sure if that's the case for maximum size though, my gut says it isn't.

So, the major purpose for adding the early object size phase is for computing 
SUBobjects size more precisely before the subobject information lost?

Then, I think whatever MIN or MAX, the early phase has more precise information 
than the later phase, we should use its result if it’s NOT UNKNOWN?

Qing
> 
> Thanks,
> Sid

Re: Another bug for __builtin_object_size? (Or expected behavior)

[Qing Zhao via Gcc-patches]

> On Aug 17, 2023, at 7:00 AM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-16 11:59, Qing Zhao wrote:
>> Jakub and Sid,
>> During my study, I found an interesting behavior for the following small 
>> testing case:
>> #include 
>> #include 
>> struct fixed {
>>   size_t foo;
>>   char b;
>>   char array[10];
>> } q = {};
>> #define noinline __attribute__((__noinline__))
>> static void noinline bar ()
>> {
>>   struct fixed *p = 
>>   printf("the__bos of MAX p->array sub is %d \n", 
>> __builtin_object_size(p->array, 1));
>>   printf("the__bos of MIN p->array sub is %d \n", 
>> __builtin_object_size(p->array, 3));
>>   return;
>> }
>> int main ()
>> {
>>   bar ();
>>   return 0;
>> }
>> [opc@qinzhao-aarch64-ol8 108896]$ sh t
>> /home/opc/Install/latest-d/bin/gcc -O -fstrict-flex-arrays=3 t2.c
>> the__bos of MAX p->array sub is 10
>> the__bos of MIN p->array sub is 15
>> I assume that the Minimum size in the sub-object should be 10 too (i.e 
>> __builtin_object_size(p->array, 3) should be 10 too).
>> So, first question: Is this correct or wrong behavior for 
>> __builtin_object_size(p->array, 3)?
>> The second question is, when I debugged into why 
>> __builtin_object_size(p->array, 3) returns 15 instead of 10, I observed the 
>> following:
>> 1. In “early_objz” phase, The IR for p->array is:
>> (gdb) call debug_generic_expr(ptr)
>> _5->array
>> And the pt_var is:
>> (gdb) call debug_generic_expr(pt_var)
>> *p_5
>> As a result, the following condition in tree-object-size.cc:
>>  585   if (pt_var != TREE_OPERAND (ptr, 0))
>> Was satisfied, and then the algorithm for computing the SUBOBJECT was 
>> invoked and the size of the subobject 10 was used.
>> and then an MAX_EXPR was inserted after the __builtin_object_size call as:
>>   _3 = _5->array;
>>   _10 = __builtin_object_size (_3, 3);
>>   _4 = MAX_EXPR <_10, 10>;
>> Till now, everything looks fine.
>> 2. within “ccp1” phase, when folding the call  to __builtin_object_size, the 
>> IR for the p-:>array is:
>> (gdb) call debug_generic_expr(ptr)
>>   [(void *) + 9B]
>> And the pt_var is:
>> (gdb) call debug_generic_expr(pt_var)
>> MEM  [(void *) + 9B]
>> As a result, the following condition in tree-object-size.cc:
>>  585   if (pt_var != TREE_OPERAND (ptr, 0))
>> Was NOT satisfied, therefore the algorithm for computing the SUBOBJECT was 
>> NOT invoked at all, as a result, the size in the whole object, 15, was used.
>> And then finally, MAX_EXPR (_10, 10) becomes MAX_EXPR (15, 10), 15 is the 
>> final result.
>> Based on the above, is there any issue with the current algorithm?
> 
> So this is a (sort of) known issue, which necessitated the early_objsz pass 
> to get an estimate before a subobject reference was optimized to a MEM_REF.

Do you mean that after a subobject reference was optimized to a MEM_REF, there 
is no way to compute the size of the subobject anymore?

>  However it looks like the MIN/MAX hack doesn't work in this case for 
> OST_MINIMUM; it should probably get the minimum of the two passes if both 
> passes were successful, or only the result of the pass that was successful.

You mean that the following line:
2053   enum tree_code code = object_size_type & OST_MINIMUM ? MAX_EXPR : 
MIN_EXPR;
Might need to be changed to:
2053   enum tree_code code =  MIN_EXPR;

?

thanks.

Qing
> 
> Thanks,
> Sid

Re: [V2][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

Hi, Kees,

Thanks for the testing case. 
Yes, I noticed this issue too, and already fixed it in my private branch. 

With the latest patch, the compilation has no issue:
[opc@qinzhao-ol8u3-x86 108896]$ sh t
/home/opc/Install/latest-d/bin/gcc -O2 -c -o /dev/null bug.c
[opc@qinzhao-ol8u3-x86 108896]$ 

Qing

> On Aug 17, 2023, at 2:38 AM, Kees Cook  wrote:
> 
> On Wed, Aug 16, 2023 at 10:31:30PM -0700, Kees Cook wrote:
>> On Fri, Aug 04, 2023 at 07:44:28PM +, Qing Zhao wrote:
>>> This is the 2nd version of the patch, per our discussion based on the
>>> review comments for the 1st version, the major changes in this version
>> 
>> I've been using Coccinelle to find and annotate[1] structures (193 so
>> far...), and I've encountered 2 cases of GCC internal errors. I'm working
>> on a minimized test case, but just in case these details are immediately
>> helpful, here's what I'm seeing:
> 
> Okay, I got it minimized:
> 
> $ cat poc.c
> struct a {
>  unsigned long c;
>  char d[] __attribute__((__counted_by__(c)));
> } *b;
> 
> void f(long);
> 
> void e(void) {
>  long g = __builtin_dynamic_object_size(b->d, 1);
>  f(g);
> }
> $ gcc -O2 -c -o /dev/null poc.c
> poc.c: In function 'e':
> poc.c:8:6: error: incorrect sharing of tree nodes
>8 | void e(void) {
>  |  ^
> *b.0_1
> _2 = _1->d;
> during GIMPLE pass: objsz
> poc.c:8:6: internal compiler error: verify_gimple failed
> 0xfe97fd verify_gimple_in_cfg(function*, bool, bool)
>../../../../gcc/gcc/tree-cfg.cc:5646
> 0xe84894 execute_function_todo
>../../../../gcc/gcc/passes.cc:2088
> 0xe84dee execute_todo
>../../../../gcc/gcc/passes.cc:2142
> 
> -- 
> Kees Cook

Re: [V2][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

Hi, 

After some more studying and consideration, the following is my thoughts:

For a structure with FMA annotated with counted_by attribute: (the following 
small example)


struct annotated {
size_t foo;
char b;
char array[] __attribute__((counted_by (foo)));
};

#define noinline __attribute__((__noinline__))
#define MAX(a, b)  ((a) > (b) ? (a) :  (b))

static struct annotated * noinline alloc_buf (size_t length)
{
  struct annotated *p;
  p = (struct annotated *) malloc (MAX (sizeof (struct annotated),
(offsetof 
(struct annotated, array[0])
+ (length) * 
sizeof (char; 
  p->foo = length;
  return p;
}

int main ()
{
  struct annotated *p = alloc_buf (10);
  printf("the__bdos of max p->array whole is %d \n", 
__builtin_dynamic_object_size(p->array, 0)); 
  printf("the__bdos of max p->array sub is %d \n", 
__builtin_dynamic_object_size(p->array, 1));
  printf("the__bdos of min p->array whole is %d \n", 
__builtin_dynamic_object_size(p->array, 2));
  printf("the__bdos of min p->array sub is %d \n", 
__builtin_dynamic_object_size(p->array, 3));   
}

=

 The actual allocation of the structure and the layout of the structure p is 
fixed at compilation time,
A. We know the offsetof (p->array) during compilation time, (it’s 9)
B. We also know the size of the p->array though the counted_by attribute, 
it’s p->foo * sizeof (char).

  1.  for subobject size (1/3 modes), Both A and B are know at compilation 
time, whatever it’s MAX or MIN, we 
can determine  the size of the subobject p->array is:  p->foo * 
sizeof(char) without estimation. 

  2.  for whole object size (0/2 modes), since we don’t have any info on the 
actual allocation or structure 
Initialization, we don’t know the size for the whole object whatever it’s 
MAX or MIN. 

So, the problem to decide which formula to use ((sizeof (x) + N * sizeof(elt), 
or offsetof + N * sizeof(elt)) is actually
the programmer’s job when allocating memory for the structure with FMA. (It’s 
not compiler’s job).  

Since this size computation is really confusing for the structure with FMA, I 
think that adding some clarification in
the documentation might be necessary to provide more details and guidance to 
the end-users.

Let me know if I miss anything here.

Thanks a lot.

Qing




> On Aug 10, 2023, at 11:18 AM, Martin Uecker  wrote:
> The access attribute gives the size directly. The counted_by gives
> a length for the array which needs to be translated into a size
> via a formula. There are different formulas in use. The question 
> is which formula should bdos trust?
> 
> Whatever you pick, if this is not consistent with the actual
> allocation or use, then it will cause problems either by
> breaking code or not detecting buffer overruns.
> 
> So it needs to be consistent with what GCC allocates for a
> var with FAM and initialization and also the user needs to 
> be told what the right choice is so that he can use the right
> size for allocation and argument to memcpy / memset etc.

> On Aug 10, 2023, at 1:06 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-10 12:39, Jakub Jelinek wrote:
>> On Thu, Aug 10, 2023 at 12:30:06PM -0400, Siddhesh Poyarekar wrote:
>>> The definition of __bos/__bdos allows us the freedom to *estimate* rather
>>> than be precise, so I'd go for sizeof(x) + N * sizeof(*x.a) since it's bound
>>> to give the more conservative answer of the two.
>> To be precise, we have the 0/1 modes vs. 2/3.  So, when not determining
>> __bos/__bdos from actual allocation size or size of an stack object or
>> size of data section object but something else (say counted_by), perhaps
>> 0/1 modes should give the upper estimate of sizeof (x) + N * sizeof(elt)
>> and 2/3 modes should give a lower estimate, so offsetof + N * sizeof(elt),
>> then user code can continue testing if both modes are equal to have
>> exact number.
> 
> Ack, that's fair.
> 
> Thanks,
> Sid

Re: Another bug for __builtin_object_size? (Or expected behavior)

[Qing Zhao via Gcc-patches]

FYI, I filed a new PR https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111040
to record this issue. 

Qing
> On Aug 16, 2023, at 11:59 AM, Qing Zhao via Gcc-patches 
>  wrote:
> 
> Jakub and Sid,
> 
> During my study, I found an interesting behavior for the following small 
> testing case:
> 
> #include 
> #include 
> 
> struct fixed {
>  size_t foo;
>  char b;
>  char array[10]; 
> } q = {};
> 
> #define noinline __attribute__((__noinline__))
> 
> static void noinline bar ()
> {
>  struct fixed *p = 
> 
>  printf("the__bos of MAX p->array sub is %d \n", 
> __builtin_object_size(p->array, 1)); 
>  printf("the__bos of MIN p->array sub is %d \n", 
> __builtin_object_size(p->array, 3)); 
> 
>  return;
> }
> 
> int main ()
> {
>  bar ();
>  return 0;
> }
> [opc@qinzhao-aarch64-ol8 108896]$ sh t
> /home/opc/Install/latest-d/bin/gcc -O -fstrict-flex-arrays=3 t2.c
> the__bos of MAX p->array sub is 10 
> the__bos of MIN p->array sub is 15 
> 
> I assume that the Minimum size in the sub-object should be 10 too (i.e 
> __builtin_object_size(p->array, 3) should be 10 too). 
> 
> So, first question: Is this correct or wrong behavior for 
> __builtin_object_size(p->array, 3)?
> 
> The second question is, when I debugged into why 
> __builtin_object_size(p->array, 3) returns 15 instead of 10, I observed the 
> following:
> 
> 1. In “early_objz” phase, The IR for p->array is:
> (gdb) call debug_generic_expr(ptr)
> _5->array
> 
> And the pt_var is:
> (gdb) call debug_generic_expr(pt_var)
> *p_5
> 
> As a result, the following condition in tree-object-size.cc:
> 
> 585   if (pt_var != TREE_OPERAND (ptr, 0))
> 
> Was satisfied, and then the algorithm for computing the SUBOBJECT was invoked 
> and the size of the subobject 10 was used. 
> 
> and then an MAX_EXPR was inserted after the __builtin_object_size call as:
>  _3 = _5->array;
>  _10 = __builtin_object_size (_3, 3);
>  _4 = MAX_EXPR <_10, 10>;
> 
> Till now, everything looks fine.
> 
> 2. within “ccp1” phase, when folding the call  to __builtin_object_size, the 
> IR for the p-:>array is:
> (gdb) call debug_generic_expr(ptr)
>   [(void *) + 9B]
> 
> And the pt_var is:
> (gdb) call debug_generic_expr(pt_var)
> MEM  [(void *) + 9B]
> 
> As a result, the following condition in tree-object-size.cc:
> 
> 585   if (pt_var != TREE_OPERAND (ptr, 0))
> 
> Was NOT satisfied, therefore the algorithm for computing the SUBOBJECT was 
> NOT invoked at all, as a result, the size in the whole object, 15, was used. 
> 
> And then finally, MAX_EXPR (_10, 10) becomes MAX_EXPR (15, 10), 15 is the 
> final result.
> 
> Based on the above, is there any issue with the current algorithm?
> 
> Thanks a lot for the help.
> 
> Qing 
> 
> 

Another bug for __builtin_object_size? (Or expected behavior)

[Qing Zhao via Gcc-patches]

Jakub and Sid,

During my study, I found an interesting behavior for the following small 
testing case:

#include 
#include 

struct fixed {
  size_t foo;
  char b;
  char array[10]; 
} q = {};

#define noinline __attribute__((__noinline__))

static void noinline bar ()
{
  struct fixed *p = 

  printf("the__bos of MAX p->array sub is %d \n", 
__builtin_object_size(p->array, 1)); 
  printf("the__bos of MIN p->array sub is %d \n", 
__builtin_object_size(p->array, 3)); 

  return;
}

int main ()
{
  bar ();
  return 0;
}
[opc@qinzhao-aarch64-ol8 108896]$ sh t
/home/opc/Install/latest-d/bin/gcc -O -fstrict-flex-arrays=3 t2.c
the__bos of MAX p->array sub is 10 
the__bos of MIN p->array sub is 15 

I assume that the Minimum size in the sub-object should be 10 too (i.e 
__builtin_object_size(p->array, 3) should be 10 too). 

So, first question: Is this correct or wrong behavior for 
__builtin_object_size(p->array, 3)?

The second question is, when I debugged into why 
__builtin_object_size(p->array, 3) returns 15 instead of 10, I observed the 
following:

1. In “early_objz” phase, The IR for p->array is:
(gdb) call debug_generic_expr(ptr)
_5->array

And the pt_var is:
(gdb) call debug_generic_expr(pt_var)
*p_5

As a result, the following condition in tree-object-size.cc:

 585   if (pt_var != TREE_OPERAND (ptr, 0))

Was satisfied, and then the algorithm for computing the SUBOBJECT was invoked 
and the size of the subobject 10 was used. 

and then an MAX_EXPR was inserted after the __builtin_object_size call as:
  _3 = _5->array;
  _10 = __builtin_object_size (_3, 3);
  _4 = MAX_EXPR <_10, 10>;

Till now, everything looks fine.

2. within “ccp1” phase, when folding the call  to __builtin_object_size, the IR 
for the p-:>array is:
(gdb) call debug_generic_expr(ptr)
  [(void *) + 9B]

And the pt_var is:
(gdb) call debug_generic_expr(pt_var)
MEM  [(void *) + 9B]

As a result, the following condition in tree-object-size.cc:

 585   if (pt_var != TREE_OPERAND (ptr, 0))

Was NOT satisfied, therefore the algorithm for computing the SUBOBJECT was NOT 
invoked at all, as a result, the size in the whole object, 15, was used. 

And then finally, MAX_EXPR (_10, 10) becomes MAX_EXPR (15, 10), 15 is the final 
result.

Based on the above, is there any issue with the current algorithm?

Thanks a lot for the help.

Qing 

Re: Is this a bug for __builtin_dynamic_object_size?

[Qing Zhao via Gcc-patches]

Thanks.

I just filed a PR https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111030 to record 
this issue and added you to the CC list.

Qing
> On Aug 15, 2023, at 6:57 AM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-14 19:12, Qing Zhao wrote:
>> Hi, Sid,
>> For the following testing case:
>> #include 
>> #define noinline __attribute__((__noinline__))
>> static void noinline alloc_buf_more (int index)
>> {
>>   struct annotated {
>> long foo;
>> char b;
>> char array[index];
>> long c;
>>   } q, *p;
>>   p = 
>>   printf("the__bdos of p->array whole max is %d \n", 
>> __builtin_dynamic_object_size(p->array, 0));
>>   printf("the__bdos of p->array sub max is %d \n", 
>> __builtin_dynamic_object_size(p->array, 1));
>>   printf("the__bdos of p->array whole min is %d \n", 
>> __builtin_dynamic_object_size(p->array, 2));
>>   printf("the__bdos of p->array sub min is %d \n", 
>> __builtin_dynamic_object_size(p->array, 3));
>>   return;
>> }
>> int main ()
>> {
>>   alloc_buf_more (10);
>>   return 0;
>> }
>> If I compile it with the latest upstream gcc and run it:
>> /home/opc/Install/latest-d/bin/gcc -O t.c
>> the__bdos of p->array whole max is 23
>> the__bdos of p->array sub max is 23
>> the__bdos of p->array whole min is 23
>> the__bdos of p->array sub min is 23
>> In which__builtin_dynamic_object_size(p->array, 0) and 
>> __builtin_dynamic_object_size(p->array, 1) return the same size, this seems 
>> wrong to me.
>> There is one line in tree-object-size.cc might relate to this bug: (in the 
>> routine “addr_object_size”)
>>  603   if (! TYPE_SIZE_UNIT (TREE_TYPE (var))
>>  604   || ! tree_fits_uhwi_p (TYPE_SIZE_UNIT (TREE_TYPE (var)))
>>  605   || (pt_var_size && TREE_CODE (pt_var_size) == INTEGER_CST
>>  606   && tree_int_cst_lt (pt_var_size,
>>  607   TYPE_SIZE_UNIT (TREE_TYPE 
>> (var)
>>  608 var = pt_var;
>> I suspect that the above line 604 “ ! tree_fits_uhwi_p (TYPE_SIZE_UNIT 
>> (TREE_TYPE (var)))” relates to this bug, since the TYPESIZE of the VLA 
>> “array” is not a unsigned HOST_WIDE_INT, but we still can use its TYPESIZE 
>> for dynamic_object_size?
>> What do you think?
> 
> Thanks, yes that doesn't work.  I'm trying to revive the patch I had 
> submitted earlier[1] in the year and fix this issue too in that process.  In 
> general the subobject size computation doesn't handle variable sizes at all; 
> it depends on whole object+offset to get size information, which ends up 
> working only for flex arrays at the end of objects.
> 
> Sid
> 
> [1] https://gcc.gnu.org/pipermail/gcc-patches/2022-December/608914.html

Is this a bug for __builtin_dynamic_object_size?

[Qing Zhao via Gcc-patches]

Hi, Sid,

For the following testing case:

#include 

#define noinline __attribute__((__noinline__))

static void noinline alloc_buf_more (int index)
{
  struct annotated {
long foo;
char b;
char array[index];
long c;
  } q, *p;

  p = 

  printf("the__bdos of p->array whole max is %d \n", 
__builtin_dynamic_object_size(p->array, 0)); 
  printf("the__bdos of p->array sub max is %d \n", 
__builtin_dynamic_object_size(p->array, 1));  
  printf("the__bdos of p->array whole min is %d \n", 
__builtin_dynamic_object_size(p->array, 2)); 
  printf("the__bdos of p->array sub min is %d \n", 
__builtin_dynamic_object_size(p->array, 3)); 

  return;
}

int main ()
{
  alloc_buf_more (10);
  return 0;
}

If I compile it with the latest upstream gcc and run it:

/home/opc/Install/latest-d/bin/gcc -O t.c
the__bdos of p->array whole max is 23 
the__bdos of p->array sub max is 23 
the__bdos of p->array whole min is 23 
the__bdos of p->array sub min is 23 

In which__builtin_dynamic_object_size(p->array, 0) and 
__builtin_dynamic_object_size(p->array, 1) return the same size, this seems 
wrong to me. 

There is one line in tree-object-size.cc might relate to this bug: (in the 
routine “addr_object_size”)

 603   if (! TYPE_SIZE_UNIT (TREE_TYPE (var))
 604   || ! tree_fits_uhwi_p (TYPE_SIZE_UNIT (TREE_TYPE (var)))
 605   || (pt_var_size && TREE_CODE (pt_var_size) == INTEGER_CST
 606   && tree_int_cst_lt (pt_var_size,
 607   TYPE_SIZE_UNIT (TREE_TYPE (var)
 608 var = pt_var;

I suspect that the above line 604 “ ! tree_fits_uhwi_p (TYPE_SIZE_UNIT 
(TREE_TYPE (var)))” relates to this bug, since the TYPESIZE of the VLA “array” 
is not a unsigned HOST_WIDE_INT, but we still can use its TYPESIZE for 
dynamic_object_size?

What do you think?

Thanks.

Qing

Re: [V2][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Aug 10, 2023, at 12:39 PM, Jakub Jelinek  wrote:
> 
> On Thu, Aug 10, 2023 at 12:30:06PM -0400, Siddhesh Poyarekar wrote:
>> The definition of __bos/__bdos allows us the freedom to *estimate* rather
>> than be precise, so I'd go for sizeof(x) + N * sizeof(*x.a) since it's bound
>> to give the more conservative answer of the two.
> 
> To be precise, we have the 0/1 modes vs. 2/3.  So, when not determining
> __bos/__bdos from actual allocation size or size of an stack object or
> size of data section object but something else (say counted_by), perhaps
> 0/1 modes should give the upper estimate of sizeof (x) + N * sizeof(elt)
> and 2/3 modes should give a lower estimate, so offsetof + N * sizeof(elt),
> then user code can continue testing if both modes are equal to have
> exact number.

Yes, this sounds reasonable to me.

Qing
> 
>   Jakub
> 

Re: [V2][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

Hi, Martin,

> On Aug 10, 2023, at 11:18 AM, Martin Uecker  wrote:
> 
> Am Donnerstag, dem 10.08.2023 um 10:58 -0400 schrieb Siddhesh Poyarekar:
>> On 2023-08-10 10:47, Martin Uecker wrote:
>>> Am Donnerstag, dem 10.08.2023 um 16:42 +0200 schrieb Jakub Jelinek:
 On Thu, Aug 10, 2023 at 04:38:21PM +0200, Martin Uecker wrote:
> Am Donnerstag, dem 10.08.2023 um 13:59 + schrieb Qing Zhao:
>> 
>>> On Aug 10, 2023, at 2:58 AM, Martin Uecker  wrote:
>>> 
>>> Am Mittwoch, dem 09.08.2023 um 20:10 + schrieb Qing Zhao:
 
> On Aug 9, 2023, at 12:21 PM, Michael Matz  wrote:
>>> 
> 
>>> I am not sure for the reason given above. The following
>>> code would not work:
>>> 
>>> struct foo_flex { int a; short b; char t[]; } x;
>>> x.a = 1;
>>> struct foo_flex *p = malloc(sizeof(x) + x.a);
>>> if (!p) abort();
>>> memcpy(p, , sizeof(x)); // initialize struct
>>> 
>> Okay.
>> Then, the user still should use the sizeof(struct foo_flex) + N * 
>> sizeof(foo->t) for the allocation, even though this might allocate more 
>> bytes than necessary. (But this is safe)
>> 
>> Let me know if I still miss anything.
> 
> The question is not only what the user should use to
> allocate, but also what BDOS should return.  In my
> example the user uses the sizeof() + N * sizeof
> formula and the memcpy is safe, but it would be flagged
> as a buffer overrun if BDOS uses the offsetof formula.
 
 BDOS/BOS (at least the 0 level) should return what is actually
 allocated for the var, what size was passed to malloc and if it
 is a var with flex array member with initialization what is actually the
 size on the stack or in .data/.rodata etc.
>>> 
>>> Agreed.
>>> 
>>> But what about a struct with FAM with the new "counted_by" attribute
>>> if the original allocation is not visible?
>> 
>> There's precedent for this through the __access__ attribute; __bos 
>> trusts what the attribute says about the allocation.
> 
> The access attribute gives the size directly. The counted_by gives
> a length for the array which needs to be translated into a size
> via a formula. There are different formulas in use. The question 
> is which formula should bdos trust?
> 
> Whatever you pick, if this is not consistent with the actual
> allocation or use, then it will cause problems either by
> breaking code or not detecting buffer overruns.
> 
> So it needs to be consistent with what GCC allocates for a
> var with FAM and initialization and also the user needs to 
> be told what the right choice is so that he can use the right
> size for allocation and argument to memcpy / memset etc.

All agreed. Thanks a lot for raising these issues and providing helpful 
suggestions. 
I will double check on these and make sure __bos behaves correctly for 
structure with FAM.

Might come back with more questions for discussion…-:).

Thanks.

Qing
> 
> Martin

Re: [V2][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Aug 10, 2023, at 2:58 AM, Martin Uecker  wrote:
> 
> Am Mittwoch, dem 09.08.2023 um 20:10 + schrieb Qing Zhao:
>> 
>>> On Aug 9, 2023, at 12:21 PM, Michael Matz  wrote:
> 
> ...
>> 
>> By definition, the sizeof() of a struct with FAM might not be the same as 
>> the non-FAM one. 
>> i.e, for the following two structures, one with FAM, the other with fixed 
>> array:
>> 
>> struct foo_flex { int a; short b; char t[]; } x = { .t = { 1, 2, 3 } };
>> struct foo_fix {int a; short b; char t[3]; } 
>> 
>> With current GCC:
>> sizeof(foo_flex) == 8
>> sizeof(foo_fix) == 12
>> 
>> I think that the current behavior of sizeof for structure with FAM in GCC is 
>> correct. 
> 
> Yes, sadly the sizeof has to be like this as required by ISO C.
Agreed. Yes, if the size information of the FAM can be integrated into the 
TYPE system in C standard, we will not have such issue anymore. 

> 
>> 
>> The major issue is what was pointed out by Martin in the previous email:
>> 
>> Whether using the following formula is correct to compute the allocation?
>> 
>> sizeof(struct foo_flex) + N * sizeof(foo->t);
> 
> That formula is safe for allocation, but might allocate more padding
> than the minimum amount and
Yes. 
> it might allocate less storage than a
> similar array with fixed array.
Yes. 
> 
>> As pointed out  in the wikipedia, the value computed by this formula might
>>  be bigger than the actual size since “sizeof(struct foo_flex)” might 
>> include 
>> paddings that are used as part of the array.
>> 
>> So the more accurate formula should be
>> 
>> offset(struct foo_flex, t[0]) + N * sizeof(foo->t);
>> 
>> With GCC, offset(struct foo_flex,t[0]) == 6, which is also correct. 
> 
> This formula might be considered incorrect / dangerous because
> it might allocate less storage than sizeof(struct foo_flex). 
> 
> 
> https://godbolt.org/z/8accq75f3

I see, thanks.
>> 
> ...
>>> As in: I think sizeof for both structs should return 12, and 12 bytes 
>>> should be reserved for objects of such types.
>>> 
>>> And then the next question is what __builtin_object_size should do with 
>>> these: should it return the size with or without padding at end (i.e. 
>>> could/should it return 9 even if sizeof is 12).  I can see arguments for 
>>> both.
>> 
>> Currently, GCC’s __builtin_object_size use the following formula to compute 
>> the object size for
>> The structure with FAM:
>> 
>> offset(struct foo_flex, t[0]) + N * sizeof(foo->t);
>> 
>> I think it’s correct. 
>> 
>> I think that the users might need to use this formula to compute the 
>> allocation size for a structure with FAM too.
> 
> I am not sure for the reason given above. The following
> code would not work:
> 
> struct foo_flex { int a; short b; char t[]; } x;
> x.a = 1;
> struct foo_flex *p = malloc(sizeof(x) + x.a);
> if (!p) abort();
> memcpy(p, , sizeof(x)); // initialize struct
> 
Okay. 
Then, the user still should use the sizeof(struct foo_flex) + N * 
sizeof(foo->t) for the allocation, even though this might allocate more bytes 
than necessary. (But this is safe)

Let me know if I still miss anything.

Thanks a lot for the explanation.

Qing
> Martin
> 
> 
> 

Re: [V2][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Aug 8, 2023, at 10:54 AM, Martin Uecker  wrote:
> 
> 
> 
> I am sure this has been discussed before, but seeing that you
> test for a specific formula, let me point out the following:
> 
> There at least three different size expression which could
> make sense. Consider
> 
> short foo { int a; short b; char t[]; }; 
> 
> Most people seem to use
> 
> sizeof(struct foo) + N * sizeof(foo->t);
> 
> which for N == 3 yields 11 bytes on x86-64 because the formula
> adds the padding of the original struct. There is an example
> in the  C standard that uses this formula.
> 
> 
> But he minimum size of an object which stores N elements is
> 
> max(sizeof (struct s), offsetof(struct s, t[n]))
> 
> which is 9 bytes. 
> 
> This is what clang uses for statically allocated objects with
> initialization, while GCC uses the rule above (11 bytes). But 
> bdos / bos  then returns the smaller size of 9 which is a bit
> confusing.

As I checked the algorithm for bos in GCC,  it uses a similar formula as the 
following to compute the object size:

offset(struct foo, t[0]) + N * sizeof(*foo->t);

Which seems correct to me.  (Therefore bos returns 9 for this example).
> 
> 
> https://godbolt.org/z/K1hvaK1ns
> 
> https://github.com/llvm/llvm-project/issues/62929
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956
> 
> 
> Then there is also the size of a similar array where the FAM
> is replaced with an array of static size:
> 
> struct foo { int a; short b; char t[3]; }; 
> 
> This would make the most sense to me, but it has 12 bytes
> because the padding is according to the usual alignment
> rules.

My understanding is, since a structure with FAM cannot be an element of an 
array, 
 the tailing padding might not be necessary?

Qing

> 
> 
> Martin
> 
> 
> 
> Am Montag, dem 07.08.2023 um 09:16 -0700 schrieb Kees Cook:
>> On Fri, Aug 04, 2023 at 07:44:28PM +, Qing Zhao wrote:
>>> This is the 2nd version of the patch, per our discussion based on the
>>> review comments for the 1st version, the major changes in this version
>>> are:
>> 
>> Thanks for the update!
>> 
>>> 
>>> 1. change the name "element_count" to "counted_by";
>>> 2. change the parameter for the attribute from a STRING to an
>>> Identifier;
>>> 3. Add logic and testing cases to handle anonymous structure/unions;
>>> 4. Clarify documentation to permit the situation when the allocation
>>> size is larger than what's specified by "counted_by", at the same time,
>>> it's user's error if allocation size is smaller than what's specified by
>>> "counted_by";
>>> 5. Add a complete testing case for using counted_by attribute in
>>> __builtin_dynamic_object_size when there is mismatch between the
>>> allocation size and the value of "counted_by", the expecting behavior
>>> for each case and the explanation on why in the comments. 
>> 
>> All the "normal" test cases I have are passing; this is wonderful! :)
>> 
>> I'm still seeing unexpected situations when I've intentionally set
>> counted_by to be smaller than alloc_size, but I assume it's due to not
>> yet having the patch you mention below.
>> 
>>> As discussed, I plan to add two more separate patch sets after this initial
>>> patch set is approved and committed.
>>> 
>>> set 1. A new warning option and a new sanitizer option for the user error
>>>when the allocation size is smaller than the value of "counted_by".
>>> set 2. An improvement to __builtin_dynamic_object_size  for the following
>>>case:
>>> 
>>> struct A
>>> {
>>> size_t foo;
>>> int array[] __attribute__((counted_by (foo)));
>>> };
>>> 
>>> extern struct fix * alloc_buf ();
>>> 
>>> int main ()
>>> {
>>> struct fix *p = alloc_buf ();
>>> __builtin_object_size(p->array, 0) == sizeof(struct A) + p->foo * 
>>> sizeof(int);
>>>   /* with the current algorithm, it’s UNKNOWN */ 
>>> __builtin_object_size(p->array, 2) == sizeof(struct A) + p->foo * 
>>> sizeof(int);
>>>   /* with the current algorithm, it’s UNKNOWN */
>>> }
>> 
>> Should the above be bdos instead of bos?
>> 
>>> Bootstrapped and regression tested on both aarch64 and X86, no issue.
>> 
>> I've updated the Linux kernel's macros for the name change and done
>> build tests with my first pass at "easy" cases for adding counted_by:
>> https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/commit/?h=devel/counted_by=adc5b3cb48a049563dc673f348eab7b6beba8a9b
>> 
>> Everything is working as expected. :)
>> 
>> -Kees
>> 
> 
> -- 
> Univ.-Prof. Dr. rer. nat. Martin Uecker
> Graz University of Technology
> Institute of Biomedical Imaging

Re: [V2][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Aug 9, 2023, at 12:21 PM, Michael Matz  wrote:
> 
> Hello,
> 
> On Wed, 9 Aug 2023, Qing Zhao wrote:
> 
>> Although this is an old FAM related issue that does not relate to my current 
>> patch 
>> (and might need to be resolved in a separate patch).  I think that it’s 
>> necessary to have
>> more discussion on this old issue and resolve it. 
>> 
>> The first thing that I’d like to confirm is:
>> 
>> What the exact memory layout for the following structure x?
>> 
>> struct foo { int a; short b; char t[]; } x = { .t = { 1, 2, 3 } };
>> 
>> And the key that is confusing me is, where should the field “t” start? 
>> 
>> A.  Starting at offset 8 as the following:
>> 
>> a4-bytes
>> b2-bytes
>> padding   2-bytes
>> t3-bytes
> 
> Why should there be padding before 't'?  It's a char array (FAM or not), 
> so it can be (and should be) placed directly after 'b'.  So ...

Yes, you are right. My mistake -:)
> 
>> B. Starting at offset 6 as the following:
>> 
>> a4-bytes
>> b2-bytes
>> t3-bytes
> 
> ... this is the correct layout, when seen in isolation.  The discussion 
> revolves around what should come after 't': if it's a non-FAM struct (with 
> t[3]), then it's clear that there needs to be padding after it, so to pad 
> out the whole struct to be 12 bytes long (for sizeof() purpose), as 
> required by its alignment (due to the int field 'a').

right.
> 
> So, should the equivalent FAM struct also have this sizeof()?  If no: 
> there should be a good argument why it shouldn't be similar to the non-FAM 
> one.

The sizeof() of a structure with FAM is defined as: (after I searched online,
 I think that the one in Wikipedia is the most reasonable one):
https://en.wikipedia.org/wiki/Flexible_array_member

==
Effect on struct size and padding
The sizeof operator on such a struct gives the size of the structure as if the 
flexible 
array member were empty. This may include padding added to accommodate the
 flexible member; the compiler is also free to re-use such padding as part of 
the array itself.[2]

It is common to allocate sizeof(struct) + array_len*sizeof(array element) 
bytes. 

This is not wrong, but it may allocate a few more bytes than necessary: the 
compiler
 may be re-purposing some of the padding that is included in sizeof(struct). 
Should 
this be a concern, macros are available[3] to compute the minimum size while 
ensuring that the compiler's padding is not disrupted. 

As the array may start in the padding before the end of the structure, its 
content
 should always be accessed via indexing (arr[i]) or  offsetof, not sizeof.
==

By definition, the sizeof() of a struct with FAM might not be the same as the 
non-FAM one. 
i.e, for the following two structures, one with FAM, the other with fixed array:

struct foo_flex { int a; short b; char t[]; } x = { .t = { 1, 2, 3 } };
struct foo_fix {int a; short b; char t[3]; } 

With current GCC:
sizeof(foo_flex) == 8
sizeof(foo_fix) == 12

I think that the current behavior of sizeof for structure with FAM in GCC is 
correct. 

The major issue is what was pointed out by Martin in the previous email:

Whether using the following formula is correct to compute the allocation?

sizeof(struct foo_flex) + N * sizeof(foo->t);

As pointed out  in the wikipedia, the value computed by this formula might
 be bigger than the actual size since “sizeof(struct foo_flex)” might include 
paddings that are used as part of the array.

So the more accurate formula should be

offset(struct foo_flex, t[0]) + N * sizeof(foo->t);

With GCC, offset(struct foo_flex,t[0]) == 6, which is also correct. 


> Then there is an argument that the compiler would be fine, when allocating 
> a single object of such type (not as part of an array!), to only reserve 9 
> bytes of space for the FAM-struct.  Then the question is: should it also 
> do that for a non-FAM struct (based on the argument that the padding 
> behind 't' isn't accessible, and hence doesn't need to be alloced).  I 
> think it would be quite unexpected for the compiler to actually reserve 
> less space than sizeof() implies, so I personally don't buy that argument.  
> For FAM or non-FAM structs.

For the question, whether the compiler needs to allocate paddings after the FAM 
field,
 I don’t know the answer, and it’s not specified in the standard either. 
Does it matter?

> 
> Note that if one choses to allocate less space than sizeof implies that 
> this will have quite some consequences for code generation, in that 
> sometimes the instruction sequences (e.g. for copying) need to be careful 
> to never access tail padding that should be there in array context, but 
> isn't there in single-object context.  I think this alone should make it 
> clear that it's advisable that sizeof() and allocated size agree.

Sizeof by definition is return the size of the TYPE, not the size of the 
allocated object.

Another thing I need to point out is,  a 

Re: [V2][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

Hi, Martin,

Thanks for raising this issue.

Although this is an old FAM related issue that does not relate to my current 
patch 
(and might need to be resolved in a separate patch).  I think that it’s 
necessary to have
more discussion on this old issue and resolve it. 

The first thing that I’d like to confirm is:

What the exact memory layout for the following structure x?

struct foo { int a; short b; char t[]; } x = { .t = { 1, 2, 3 } };

And the key that is confusing me is, where should the field “t” start? 

A.  Starting at offset 8 as the following:

a   4-bytes
b   2-bytes
padding   2-bytes
t   3-bytes

B. Starting at offset 6 as the following:

a   4-bytes
b   2-bytes
t   3-bytes

From my understanding, A should be correct. However, when I debugged into gcc, 
I found that the following

tree
byte_position (const_tree field)
{
  return byte_from_pos (DECL_FIELD_OFFSET (field),
DECL_FIELD_BIT_OFFSET (field));
}

Returned 6 for the field “t”:

498   tree pos = byte_position (last);
(gdb) n
499   size = fold_build2 (PLUS_EXPR, TREE_TYPE (size), pos, compsize);
(gdb) call debug_generic_expr(pos)
6

So, I suspect that there is a bug in GCC which incorrectly represent the offset 
of the FAM field in the IR.

Thanks.

Qing
> On Aug 8, 2023, at 10:54 AM, Martin Uecker  wrote:
> 
> 
> 
> I am sure this has been discussed before, but seeing that you
> test for a specific formula, let me point out the following:
> 
> There at least three different size expression which could
> make sense. Consider
> 
> short foo { int a; short b; char t[]; }; 
> 
> Most people seem to use
> 
> sizeof(struct foo) + N * sizeof(foo->t);
> 
> which for N == 3 yields 11 bytes on x86-64 because the formula
> adds the padding of the original struct. There is an example
> in the  C standard that uses this formula.
> 
> 
> But he minimum size of an object which stores N elements is
> 
> max(sizeof (struct s), offsetof(struct s, t[n]))
> 
> which is 9 bytes. 
> 
> This is what clang uses for statically allocated objects with
> initialization, while GCC uses the rule above (11 bytes). But 
> bdos / bos  then returns the smaller size of 9 which is a bit
> confusing.
> 
> 
> https://godbolt.org/z/K1hvaK1ns
> 
> https://github.com/llvm/llvm-project/issues/62929
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109956
> 
> 
> Then there is also the size of a similar array where the FAM
> is replaced with an array of static size:
> 
> struct foo { int a; short b; char t[3]; }; 
> 
> This would make the most sense to me, but it has 12 bytes
> because the padding is according to the usual alignment
> rules.
> 
> 
> Martin
> 
> 
> 
> Am Montag, dem 07.08.2023 um 09:16 -0700 schrieb Kees Cook:
>> On Fri, Aug 04, 2023 at 07:44:28PM +, Qing Zhao wrote:
>>> This is the 2nd version of the patch, per our discussion based on the
>>> review comments for the 1st version, the major changes in this version
>>> are:
>> 
>> Thanks for the update!
>> 
>>> 
>>> 1. change the name "element_count" to "counted_by";
>>> 2. change the parameter for the attribute from a STRING to an
>>> Identifier;
>>> 3. Add logic and testing cases to handle anonymous structure/unions;
>>> 4. Clarify documentation to permit the situation when the allocation
>>> size is larger than what's specified by "counted_by", at the same time,
>>> it's user's error if allocation size is smaller than what's specified by
>>> "counted_by";
>>> 5. Add a complete testing case for using counted_by attribute in
>>> __builtin_dynamic_object_size when there is mismatch between the
>>> allocation size and the value of "counted_by", the expecting behavior
>>> for each case and the explanation on why in the comments. 
>> 
>> All the "normal" test cases I have are passing; this is wonderful! :)
>> 
>> I'm still seeing unexpected situations when I've intentionally set
>> counted_by to be smaller than alloc_size, but I assume it's due to not
>> yet having the patch you mention below.
>> 
>>> As discussed, I plan to add two more separate patch sets after this initial
>>> patch set is approved and committed.
>>> 
>>> set 1. A new warning option and a new sanitizer option for the user error
>>>when the allocation size is smaller than the value of "counted_by".
>>> set 2. An improvement to __builtin_dynamic_object_size  for the following
>>>case:
>>> 
>>> struct A
>>> {
>>> size_t foo;
>>> int array[] __attribute__((counted_by (foo)));
>>> };
>>> 
>>> extern struct fix * alloc_buf ();
>>> 
>>> int main ()
>>> {
>>> struct fix *p = alloc_buf ();
>>> __builtin_object_size(p->array, 0) == sizeof(struct A) + p->foo * 
>>> sizeof(int);
>>>   /* with the current algorithm, it’s UNKNOWN */ 
>>> __builtin_object_size(p->array, 2) == sizeof(struct A) + p->foo * 
>>> sizeof(int);
>>>   /* with the current algorithm, it’s UNKNOWN */
>>> }
>> 
>> Should the above be bdos instead of bos?
>> 
>>> 

Re: [V2][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Aug 7, 2023, at 12:16 PM, Kees Cook  wrote:
> 
> On Fri, Aug 04, 2023 at 07:44:28PM +, Qing Zhao wrote:
>> This is the 2nd version of the patch, per our discussion based on the
>> review comments for the 1st version, the major changes in this version
>> are:
> 
> Thanks for the update!
> 
>> 
>> 1. change the name "element_count" to "counted_by";
>> 2. change the parameter for the attribute from a STRING to an
>> Identifier;
>> 3. Add logic and testing cases to handle anonymous structure/unions;
>> 4. Clarify documentation to permit the situation when the allocation
>> size is larger than what's specified by "counted_by", at the same time,
>> it's user's error if allocation size is smaller than what's specified by
>> "counted_by";
>> 5. Add a complete testing case for using counted_by attribute in
>> __builtin_dynamic_object_size when there is mismatch between the
>> allocation size and the value of "counted_by", the expecting behavior
>> for each case and the explanation on why in the comments. 
> 
> All the "normal" test cases I have are passing; this is wonderful! :)
> 
> I'm still seeing unexpected situations when I've intentionally set
> counted_by to be smaller than alloc_size, but I assume it's due to not
> yet having the patch you mention below.

What’s the testing case for the one that failed? 
If it’s 

__builtin_dynamic_object_size(p->array, 0/2) without the allocation information 
in the routine, 
then with the current algorithm, gcc cannot deduce the size for the whole 
object.

If not such case, let me know.

> 
>> As discussed, I plan to add two more separate patch sets after this initial
>> patch set is approved and committed.
>> 
>> set 1. A new warning option and a new sanitizer option for the user error
>>   when the allocation size is smaller than the value of "counted_by".
>> set 2. An improvement to __builtin_dynamic_object_size  for the following
>>   case:
>> 
>> struct A
>> {
>> size_t foo;
>> int array[] __attribute__((counted_by (foo)));
>> };
>> 
>> extern struct fix * alloc_buf ();
>> 
>> int main ()
>> {
>> struct fix *p = alloc_buf ();
>> __builtin_object_size(p->array, 0) == sizeof(struct A) + p->foo * 
>> sizeof(int);
>>  /* with the current algorithm, it’s UNKNOWN */ 
>> __builtin_object_size(p->array, 2) == sizeof(struct A) + p->foo * 
>> sizeof(int);
>>  /* with the current algorithm, it’s UNKNOWN */
>> }
> 
> Should the above be bdos instead of bos?

Yes, sorry for the typo.  -:)
> 
>> Bootstrapped and regression tested on both aarch64 and X86, no issue.
> 
> I've updated the Linux kernel's macros for the name change and done
> build tests with my first pass at "easy" cases for adding counted_by:
> https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/commit/?h=devel/counted_by=adc5b3cb48a049563dc673f348eab7b6beba8a9b

> 
> Everything is working as expected. :)

Thanks a lot for the testing!

Qing
> 
> -Kees
> 
> -- 
> Kees Cook
> 
> 

[V2][PATCH] gcc-14/changes.html: Deprecate a GCC C extension on flexible array members.

[Qing Zhao via Gcc-patches]

Hi,

This is the 2nd version of the patch.
Comparing to the 1st version, the only change is to address Richard's
comment on refering a warning option for diagnosing deprecated behavior.


Okay for committing?

thanks.

Qing

==

*htdocs/gcc-14/changes.html (Caveats): Add notice about deprecating a C
extension about flexible array members.
---
 htdocs/gcc-14/changes.html | 13 -
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/htdocs/gcc-14/changes.html b/htdocs/gcc-14/changes.html
index dad1ba53..eae25f1a 100644
--- a/htdocs/gcc-14/changes.html
+++ b/htdocs/gcc-14/changes.html
@@ -30,7 +30,18 @@ a work-in-progress.
 
 Caveats
 
-  ...
+  C:
+  Support for the GCC extension, a structure containing a C99 flexible 
array
+  member, or a union containing such a structure, is not the last field of
+  another structure, is deprecated. Refer to
+  https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html;>
+  Zero Length Arrays.
+  Any code relying on this extension should be modifed to ensure that
+  C99 flexible array members only end up at the ends of structures.
+  Please use the warning option
+  https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wflex-array-member-not-at-end;>-Wflex-array-member-not-at-end
 to
+  identify all such cases in the source code and modify them.
+  
 
 
 
-- 
2.31.1

[V2][PATCH 3/3] Use the counted_by attribute information in bound sanitizer[PR108896]

[Qing Zhao via Gcc-patches]

gcc/c-family/ChangeLog:

PR C/108896
* c-ubsan.cc (ubsan_instrument_bounds): Use counted_by attribute
information.

gcc/testsuite/ChangeLog:

PR C/108896
* gcc.dg/ubsan/flex-array-counted-by-bounds.c: New test.
* gcc.dg/ubsan/flex-array-counted-by-bounds-2.c: New test.
---
 gcc/c-family/c-ubsan.cc   | 16 +++
 .../ubsan/flex-array-counted-by-bounds-2.c| 27 +++
 .../ubsan/flex-array-counted-by-bounds.c  | 46 +++
 3 files changed, 89 insertions(+)
 create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
 create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c

diff --git a/gcc/c-family/c-ubsan.cc b/gcc/c-family/c-ubsan.cc
index 51aa83a378d2..a99e8433069f 100644
--- a/gcc/c-family/c-ubsan.cc
+++ b/gcc/c-family/c-ubsan.cc
@@ -362,6 +362,10 @@ ubsan_instrument_bounds (location_t loc, tree array, tree 
*index,
 {
   tree type = TREE_TYPE (array);
   tree domain = TYPE_DOMAIN (type);
+  /* whether the array ref is a flexible array member with valid counted_by
+ attribute.  */
+  bool fam_has_count_attr = false;
+  tree counted_by = NULL_TREE;
 
   if (domain == NULL_TREE)
 return NULL_TREE;
@@ -375,6 +379,17 @@ ubsan_instrument_bounds (location_t loc, tree array, tree 
*index,
  && COMPLETE_TYPE_P (type)
  && integer_zerop (TYPE_SIZE (type)))
bound = build_int_cst (TREE_TYPE (TYPE_MIN_VALUE (domain)), -1);
+  /* If the array ref is to flexible array member field which has
+counted_by attribute.  We can use the information from the
+attribute as the bound to instrument the reference.  */
+  else if ((counted_by = component_ref_get_counted_by (array))
+   != NULL_TREE)
+   {
+ fam_has_count_attr = true;
+ bound = fold_build2 (MINUS_EXPR, TREE_TYPE (counted_by),
+  counted_by,
+  build_int_cst (TREE_TYPE (counted_by), 1));
+   }
   else
return NULL_TREE;
 }
@@ -387,6 +402,7 @@ ubsan_instrument_bounds (location_t loc, tree array, tree 
*index,
  -fsanitize=bounds-strict.  */
   tree base = get_base_address (array);
   if (!sanitize_flags_p (SANITIZE_BOUNDS_STRICT)
+  && !fam_has_count_attr
   && TREE_CODE (array) == COMPONENT_REF
   && base && (INDIRECT_REF_P (base) || TREE_CODE (base) == MEM_REF))
 {
diff --git a/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c 
b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
new file mode 100644
index ..77ec333509d0
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
@@ -0,0 +1,27 @@
+/* test the attribute counted_by and its usage in
+   bounds sanitizer combined with VLA.  */
+/* { dg-do run } */
+/* { dg-options "-fsanitize=bounds" } */
+
+#include 
+
+void __attribute__((__noinline__)) setup_and_test_vla (int n, int m)
+{
+   struct foo {
+   int n;
+   int p[][n] __attribute__((counted_by(n)));
+   } *f;
+
+   f = (struct foo *) malloc (sizeof(struct foo) + m*sizeof(int[n]));
+   f->n = m;
+   f->p[m][n-1]=1;
+   return;
+}
+
+int main(int argc, char *argv[])
+{
+  setup_and_test_vla (10, 11);
+  return 0;
+}
+
+/* { dg-output "17:8: runtime error: index 11 out of bounds for type" } */
diff --git a/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c 
b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c
new file mode 100644
index ..81eaeb3f2681
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c
@@ -0,0 +1,46 @@
+/* test the attribute counted_by and its usage in
+   bounds sanitizer.  */
+/* { dg-do run } */
+/* { dg-options "-fsanitize=bounds" } */
+
+#include 
+
+struct flex {
+  int b;
+  int c[];
+} *array_flex;
+
+struct annotated {
+  int b;
+  int c[] __attribute__ ((counted_by (b)));
+} *array_annotated;
+
+void __attribute__((__noinline__)) setup (int normal_count, int 
annotated_count)
+{
+  array_flex
+= (struct flex *)malloc (sizeof (struct flex)
++ normal_count *  sizeof (int));
+  array_flex->b = normal_count;
+
+  array_annotated
+= (struct annotated *)malloc (sizeof (struct annotated)
+ + annotated_count *  sizeof (int));
+  array_annotated->b = annotated_count;
+
+  return;
+}
+
+void __attribute__((__noinline__)) test (int normal_index, int annotated_index)
+{
+  array_flex->c[normal_index] = 1;
+  array_annotated->c[annotated_index] = 2;
+}
+
+int main(int argc, char *argv[])
+{
+  setup (10, 10);   
+  test (10, 10);
+  return 0;
+}
+
+/* { dg-output "36:21: runtime error: index 10 out of bounds for type" } */
-- 
2.31.1

[V2][PATCH 2/3] Use the counted_by atribute info in builtin object size [PR108896]

[Qing Zhao via Gcc-patches]

gcc/ChangeLog:

PR C/108896
* tree-object-size.cc (addr_object_size): Use the counted_by
attribute info.
* tree.cc (component_ref_has_counted_by_p): New function.
(component_ref_get_counted_by): New function.
* tree.h (component_ref_has_counted_by_p): New prototype.
(component_ref_get_counted_by): New prototype.

gcc/testsuite/ChangeLog:

PR C/108896
* gcc.dg/flex-array-counted-by-2.c: New test.
* gcc.dg/flex-array-counted-by-3.c: New test.
---
 .../gcc.dg/flex-array-counted-by-2.c  |  74 +++
 .../gcc.dg/flex-array-counted-by-3.c  | 197 ++
 gcc/tree-object-size.cc   |  37 +++-
 gcc/tree.cc   |  95 -
 gcc/tree.h|  10 +
 5 files changed, 405 insertions(+), 8 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-3.c

diff --git a/gcc/testsuite/gcc.dg/flex-array-counted-by-2.c 
b/gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
new file mode 100644
index ..ec580c1f1f01
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
@@ -0,0 +1,74 @@
+/* test the attribute counted_by and its usage in
+ * __builtin_dynamic_object_size.  */ 
+/* { dg-do run } */
+/* { dg-options "-O2" } */
+
+#include "builtin-object-size-common.h"
+
+#define expect(p, _v) do { \
+size_t v = _v; \
+if (p == v) \
+   __builtin_printf ("ok:  %s == %zd\n", #p, p); \
+else \
+   {  \
+ __builtin_printf ("WAT: %s == %zd (expected %zd)\n", #p, p, v); \
+ FAIL (); \
+   } \
+} while (0);
+
+struct flex {
+  int b;
+  int c[];
+} *array_flex;
+
+struct annotated {
+  int b;
+  int c[] __attribute__ ((counted_by (b)));
+} *array_annotated;
+
+struct nested_annotated {
+  struct {
+union {
+  int b;
+  float f; 
+};
+int n;
+  };
+  int c[] __attribute__ ((counted_by (b)));
+} *array_nested_annotated;
+
+void __attribute__((__noinline__)) setup (int normal_count, int attr_count)
+{
+  array_flex
+= (struct flex *)malloc (sizeof (struct flex)
++ normal_count *  sizeof (int));
+  array_flex->b = normal_count;
+
+  array_annotated
+= (struct annotated *)malloc (sizeof (struct annotated)
+ + attr_count *  sizeof (int));
+  array_annotated->b = attr_count;
+
+  array_nested_annotated
+= (struct nested_annotated *)malloc (sizeof (struct nested_annotated)
++ attr_count *  sizeof (int));
+  array_nested_annotated->b = attr_count;
+
+  return;
+}
+
+void __attribute__((__noinline__)) test ()
+{
+expect(__builtin_dynamic_object_size(array_flex->c, 1), -1);
+expect(__builtin_dynamic_object_size(array_annotated->c, 1),
+  array_annotated->b * sizeof (int));
+expect(__builtin_dynamic_object_size(array_nested_annotated->c, 1),
+  array_nested_annotated->b * sizeof (int));
+}
+
+int main(int argc, char *argv[])
+{
+  setup (10,10);   
+  test ();
+  DONE ();
+}
diff --git a/gcc/testsuite/gcc.dg/flex-array-counted-by-3.c 
b/gcc/testsuite/gcc.dg/flex-array-counted-by-3.c
new file mode 100644
index ..22ef2af31c20
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/flex-array-counted-by-3.c
@@ -0,0 +1,197 @@
+/* test the attribute counted_by and its usage in
+__builtin_dynamic_object_size: what's the correct behavior when the allocaiton
+size mismatched with the value of counted_by attribute?  */
+/* { dg-do run } */
+/* { dg-options "-O -fstrict-flex-arrays=3" } */
+
+#include "builtin-object-size-common.h"
+
+struct annotated {
+  size_t foo;
+  int array[] __attribute__((counted_by (foo)));
+};
+
+#define expect(p, _v) do { \
+size_t v = _v; \
+if (p == v) \
+__builtin_printf ("ok:  %s == %zd\n", #p, p); \
+else \
+{  \
+  __builtin_printf ("WAT: %s == %zd (expected %zd)\n", #p, p, v); \
+ FAIL (); \
+} \
+} while (0);
+
+#define noinline __attribute__((__noinline__))
+#define SIZE_BUMP 5
+
+/* In general, Due to type casting, the type for the pointee of a pointer
+   does not say anything about the object it points to,
+   So, __builtin_object_size can not directly use the type of the pointee
+   to decide the size of the object the pointer points to.
+
+   there are only two reliable ways:
+   A. observed allocations  (call to the allocation functions in the routine)
+   B. observed accesses (read or write access to the location of the
+ pointer points to)
+
+   that provide information about the type/existence of an object at
+   the corresponding address.
+
+   for A, we use the "alloc_size" attribute for the corresponding allocation
+   functions to determine the object size;
+
+   For B, we use the SIZE info of the TYPE attached to the corresponding 
access.
+   

[V2][PATCH 1/3] Provide counted_by attribute to flexible array member field (PR108896)

[Qing Zhao via Gcc-patches]

'counted_by (COUNT)'
 The 'counted_by' attribute may be attached to the flexible array
 member of a structure.  It indicates that the number of the
 elements of the array is given by the field named "COUNT" in the
 same structure as the flexible array member.  GCC uses this
 information to improve the results of the array bound sanitizer and
 the '__builtin_dynamic_object_size'.

 For instance, the following code:

  struct P {
size_t count;
int array[] __attribute__ ((counted_by (count)));
  } *p;

 specifies that the 'array' is a flexible array member whose number
 of elements is given by the field 'count' in the same structure.

 The field that represents the number of the elements should have an
 integer type.  An explicit 'counted_by' annotation defines a
 relationship between two objects, 'p->array' and 'p->count', that
 'p->array' has _at least_ 'p->count' number of elements available.
 This relationship must hold even after any of these related objects
 are updated.  It's the user's responsibility to make sure this
 relationship to be kept all the time.  Otherwise the results of the
 array bound sanitizer and the '__builtin_dynamic_object_size' might
 be incorrect.

 For instance, in the following example, the allocated array has
 less elements than what's specified by the 'sbuf->count', this is
 an user error.  As a result, out-of-bounds access to the array
 might not be detected.

  #define SIZE_BUMP 10
  struct P *sbuf;
  void alloc_buf (size_t nelems)
  {
sbuf = (int *) malloc (sizeof (struct P) + sizeof (int) * nelems);
sbuf->count = nelems + SIZE_BUMP;
/* This is invalid when the sbuf->array has less than sbuf->count
   elements.  */
  }

 In the following example, the 2nd update to the field 'sbuf->count'
 of the above structure will permit out-of-bounds access to the
 array 'sbuf>array' as well.

  #define SIZE_BUMP 10
  struct P *sbuf;
  void alloc_buf (size_t nelems)
  {
sbuf = (int *) malloc (sizeof (struct P)
 + sizeof (int) * (nelems + SIZE_BUMP));
sbuf->count = nelems;
/* This is valid when the sbuf->array has at least sbuf->count
   elements.  */
  }
  void use_buf (int index)
  {
sbuf->count = sbuf->count + SIZE_BUMP + 1;
/* Now the value of sbuf->count is larger than the number
   of elements of sbuf->array.  */
sbuf->array[index] = 0;
/* then the out-of-bound access to this array
   might not be detected.  */
  }

gcc/c-family/ChangeLog:

PR C/108896
* c-attribs.cc (handle_counted_by_attribute): New function.
(attribute_takes_identifier_p): Add counted_by attribute to the list.
* c-common.cc (c_flexible_array_member_type_p): ...To this.
* c-common.h (c_flexible_array_member_type_p): New prototype.

gcc/c/ChangeLog:

PR C/108896
* c-decl.cc (flexible_array_member_type_p): Renamed and moved to...
(add_flexible_array_elts_to_size): Use renamed function.
(is_flexible_array_member_p): Use renamed function.
(verify_counted_by_attribute): New function.
(finish_struct): Use renamed function and verify counted_by
attribute.

gcc/ChangeLog:

PR C/108896
* doc/extend.texi: Document attribute counted_by.
* tree.cc (get_named_field): New function.
* tree.h (get_named_field): New prototype.

gcc/testsuite/ChangeLog:

PR C/108896
* gcc.dg/flex-array-counted-by.c: New test.
---
 gcc/c-family/c-attribs.cc| 54 -
 gcc/c-family/c-common.cc | 13 
 gcc/c-family/c-common.h  |  1 +
 gcc/c/c-decl.cc  | 79 +++-
 gcc/doc/extend.texi  | 73 ++
 gcc/testsuite/gcc.dg/flex-array-counted-by.c | 40 ++
 gcc/tree.cc  | 40 ++
 gcc/tree.h   |  5 ++
 8 files changed, 287 insertions(+), 18 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by.c

diff --git a/gcc/c-family/c-attribs.cc b/gcc/c-family/c-attribs.cc
index e2792ca6898b..65e4f6639109 100644
--- a/gcc/c-family/c-attribs.cc
+++ b/gcc/c-family/c-attribs.cc
@@ -103,6 +103,8 @@ static tree handle_warn_if_not_aligned_attribute (tree *, 
tree, tree,
  int, bool *);
 static tree handle_strict_flex_array_attribute (tree *, tree, tree,
 int, bool *);
+static tree handle_counted_by_attribute (tree *, tree, tree,
+  

[V2][PATCH 0/3] New attribute "counted_by" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

Hi,

This is the 2nd version of the patch, per our discussion based on the
review comments for the 1st version, the major changes in this version
are:

1. change the name "element_count" to "counted_by";
2. change the parameter for the attribute from a STRING to an
Identifier;
3. Add logic and testing cases to handle anonymous structure/unions;
4. Clarify documentation to permit the situation when the allocation
size is larger than what's specified by "counted_by", at the same time,
it's user's error if allocation size is smaller than what's specified by
"counted_by";
5. Add a complete testing case for using counted_by attribute in
__builtin_dynamic_object_size when there is mismatch between the
allocation size and the value of "counted_by", the expecting behavior
for each case and the explanation on why in the comments. 

As discussed, I plan to add two more separate patch sets after this initial
patch set is approved and committed.

set 1. A new warning option and a new sanitizer option for the user error
   when the allocation size is smaller than the value of "counted_by".
set 2. An improvement to __builtin_dynamic_object_size  for the following
   case:

struct A
{
size_t foo;
int array[] __attribute__((counted_by (foo)));
};

extern struct fix * alloc_buf ();

int main ()
{
struct fix *p = alloc_buf ();
__builtin_object_size(p->array, 0) == sizeof(struct A) + p->foo * sizeof(int);
  /* with the current algorithm, it’s UNKNOWN */ 
__builtin_object_size(p->array, 2) == sizeof(struct A) + p->foo * sizeof(int);
  /* with the current algorithm, it’s UNKNOWN */
}

Bootstrapped and regression tested on both aarch64 and X86, no issue.

Please see more details on the description of this work on:

https://gcc.gnu.org/pipermail/gcc-patches/2023-May/619708.html

Okay for committing?

thanks.

Qing

Qing Zhao (3):
  Provide counted_by attribute to flexible array member field (PR108896)
  Use the counted_by atribute info in builtin object size [PR108896]
  Use the counted_by attribute information in bound sanitizer[PR108896]

 gcc/c-family/c-attribs.cc |  54 -
 gcc/c-family/c-common.cc  |  13 ++
 gcc/c-family/c-common.h   |   1 +
 gcc/c-family/c-ubsan.cc   |  16 ++
 gcc/c/c-decl.cc   |  79 +--
 gcc/doc/extend.texi   |  73 +++
 .../gcc.dg/flex-array-counted-by-2.c  |  74 +++
 .../gcc.dg/flex-array-counted-by-3.c  | 197 ++
 gcc/testsuite/gcc.dg/flex-array-counted-by.c  |  40 
 .../ubsan/flex-array-counted-by-bounds-2.c|  27 +++
 .../ubsan/flex-array-counted-by-bounds.c  |  46 
 gcc/tree-object-size.cc   |  37 +++-
 gcc/tree.cc   | 133 
 gcc/tree.h|  15 ++
 14 files changed, 780 insertions(+), 25 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-2.c
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by-3.c
 create mode 100644 gcc/testsuite/gcc.dg/flex-array-counted-by.c
 create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds-2.c
 create mode 100644 gcc/testsuite/gcc.dg/ubsan/flex-array-counted-by-bounds.c

-- 
2.31.1

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

> On Aug 4, 2023, at 3:09 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-04 15:06, Qing Zhao wrote:
>>> Yes, that's what I'm thinking.
>>> 
> so `q` must be pointing to a single element.  So you could deduce:
> 
> 1. the minimum size of the whole object that q points to.
 You mean that the TYPE will determine the minimum size of the whole 
 object?  (Does this include the size of the flexible array member, or only 
 the other part of the structure except the flexible array member?)
>>> 
>>> Only the constant sized part of the structure.
>> Okay. I see.
>> But if the “counted_by” info is available, then from p->array, we can deduce 
>> the minimum size too, as sizeof(struct A) + q->foo * sizeof(int), right?
> 
> Yes.
> 
>>> 
> Actually for minimum size we'd also need a guarantee that 
> `alloc_buf_more` returns a valid allocated object.
 Why? Please explain a little bit here.
>>> 
>>> So `alloc_buf_more` could return NULL, a valid pointer or an invalid 
>>> pointer.  So, we could end up returning a non-zero minimum size for an 
>>> invalid or NULL pointer, which is incorrect, we don't know that.
>> I see what’ s you mean now.
>> However, if we already see p->array, then the p is guaranteed a valid 
>> pointer and not a NULL, right?  (We are discussing on 
>> __builtin_dynamic_object_size (q->array, 2), we see q->array already)
> 
> Yes, you could argue that for p->array, I agree, but not for p.

Agreed. Yes, for p->array, observed access. -:)

Looks like we can improve __builtin_dynamic_object_size  for the following case:
struct A
{
 size_t foo;
 int array[] __attribute__((counted_by (foo)));
};

extern struct fix * alloc_buf ();

int main ()
{
 struct fix *p = alloc_buf ();
 __builtin_object_size(p->array, 0) == sizeof(struct A) + p->foo * sizeof(int); 
  /* with the current algorithm, it’s UNKNOWN */ 
 __builtin_object_size(p->array, 2) == sizeof(struct A) + p->foo * sizeof(int); 
  /* with the current algorithm, it’s UNKNOWN */
}

I will add this improvement to __builtin_dynamic_object_size for FAM with 
“counted_by” attribute in a later patch after the initial patch is committed.

Thanks a lot for the help.

Qing
> 
>>> 
>>> We won't need the object validity guarantee for (2) beyond, e.g. guarding 
>>> against a new NULL pointer dereference because it's a *maximum* estimate; 
>>> an invalid or NULL pointer would have 0 size.  So for such cases, __bos(q, 
>>> 0) could return
>>> 
>>> sizeof(*q) + (q ? q->foo:0)
>>> 
>>> and __bos(q->array, 0) could be
>>> 
>>> sizeof(*q) + q->foo - offsetof(q, array)
>>> 
>>> There's no need to guard against a dereference in the second case because 
>>> the q->array dereference already assumes that q is valid.
>> q->array should also guarantee that q is a valid pointer for minimum size, 
>> right? Or do I miss anything here?
> 
> Yes.
> 
> Thanks,
> Sid

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

> On Aug 4, 2023, at 12:36 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-04 11:27, Qing Zhao wrote:
>>> On Aug 4, 2023, at 10:40 AM, Siddhesh Poyarekar  wrote:
>>> 
>>> On 2023-08-03 13:34, Qing Zhao wrote:
 One thing I need to point out first is, currently, even for regular fixed 
 size array in the structure,
 We have this same issue, for example:
 #define LENGTH 10
 struct fix {
   size_t foo;
   int array[LENGTH];
 };
 …
 int main ()
 {
   struct fix *p;
   p = alloc_buf_more ();
   expect(__builtin_object_size(p->array, 1), LENGTH * sizeof(int));
   expect(__builtin_object_size(p->array, 0), -1);
 }
 Currently, for __builtin_object_size(p->array, 0),  GCC return UNKNOWN for 
 it.
 This is not a special issue for flexible array member.
>>> 
>>> That's fine for fixed arrays at the end of a struct because the "whole 
>>> object" size could be anything; `p` could be pointing to the beginning of 
>>> an array for all we know.  If however `array` is strictly a flex array, 
>>> i.e.:
>>> 
>>> ```
>>> struct A
>>> {
>>>  size_t foo;
>>>  int array[];
>>> };
>>> ```
>>> 
>>> then there's no way in valid C to have an array of `struct fix`,
>> Yes!!   this is exactly the place that makes difference between structures 
>> with fixed arrays and the ones with flexible arrays.
>> With such difference, I guess that using the type of the structure with 
>> flexible array member for p->array to get the size of the whole object p 
>> point to might be reasonable?
> 
> Yes, that's what I'm thinking.
> 
>>> so `q` must be pointing to a single element.  So you could deduce:
>>> 
>>> 1. the minimum size of the whole object that q points to.
>> You mean that the TYPE will determine the minimum size of the whole object?  
>> (Does this include the size of the flexible array member, or only the other 
>> part of the structure except the flexible array member?)
> 
> Only the constant sized part of the structure.
Okay. I see.
But if the “counted_by” info is available, then from p->array, we can deduce 
the minimum size too, as sizeof(struct A) + q->foo * sizeof(int), right?
> 
>>> Actually for minimum size we'd also need a guarantee that `alloc_buf_more` 
>>> returns a valid allocated object.
>> Why? Please explain a little bit here.
> 
> So `alloc_buf_more` could return NULL, a valid pointer or an invalid pointer. 
>  So, we could end up returning a non-zero minimum size for an invalid or NULL 
> pointer, which is incorrect, we don't know that.

I see what’ s you mean now.

However, if we already see p->array, then the p is guaranteed a valid pointer 
and not a NULL, right?  (We are discussing on __builtin_dynamic_object_size 
(q->array, 2), we see q->array already)

> 
> We won't need the object validity guarantee for (2) beyond, e.g. guarding 
> against a new NULL pointer dereference because it's a *maximum* estimate; an 
> invalid or NULL pointer would have 0 size.  So for such cases, __bos(q, 0) 
> could return
> 
> sizeof(*q) + (q ? q->foo:0)
> 
> and __bos(q->array, 0) could be
> 
> sizeof(*q) + q->foo - offsetof(q, array)
> 
> There's no need to guard against a dereference in the second case because the 
> q->array dereference already assumes that q is valid.

q->array should also guarantee that q is a valid pointer for minimum size, 
right? Or do I miss anything here?

thanks.

Qing
> 
>>> 
>>> and
>>> 
>>> 2. if you're able to determine the size of the flex array (through 
>>> __element_count__(foo) for example), you could even determine the maximum 
>>> size of the whole object.
>>> 
>>> For (2) though, you'd break applications that overallocate and then expect 
>>> to be able to use that overallocation despite the space not being reflected 
>>> in the __element_count__.  I think it's a bug in the application and I 
>>> can't see a way for an application to be able to do this in a valid way so 
>>> I'm inclined towards breaking it.
>> Currently, we allow the situation when the allocation size for the whole 
>> object is larger than the value reflected in the “counted_by” attribute (the 
>> old name is __element_count__). But don’t allow the other way around (i.e, 
>> when the allocation size for the whole object is smaller than the value 
>> reflected in the “counted_by” attribute.
> 
> Right, that's going to be the "break".  For underallocation __bos will only 
> end up overestimating the space available, which is not ideal, but won't end 
> up breaking compatibility.
> 
>>> 
>>> Of course, the fact that gcc allows flex arrays to be in the middle of 
>>> structs breaks the base assumption but that's something we need to get rid 
>>> of anyway since there's no way for valid C programs to use that safely.
>> Since GCC14, we started to deprecate this extension (allow flex array to be 
>> in the middle of structs).
>> https://gcc.gnu.org/pipermail/gcc-cvs/2023-June/385730.html
> 
> Yes, that's what I'm banking on.
> 
> Thanks,
> Sid

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

> On Aug 4, 2023, at 10:42 AM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-04 10:40, Siddhesh Poyarekar wrote:
>> On 2023-08-03 13:34, Qing Zhao wrote:
>>> One thing I need to point out first is, currently, even for regular fixed 
>>> size array in the structure,
>>> We have this same issue, for example:
>>> 
>>> #define LENGTH 10
>>> 
>>> struct fix {
>>>size_t foo;
>>>int array[LENGTH];
>>> };
>>> 
>>> …
>>> int main ()
>>> {
>>>struct fix *p;
>>>p = alloc_buf_more ();
>>> 
>>>expect(__builtin_object_size(p->array, 1), LENGTH * sizeof(int));
>>>expect(__builtin_object_size(p->array, 0), -1);
>>> }
>>> 
>>> Currently, for __builtin_object_size(p->array, 0),  GCC return UNKNOWN for 
>>> it.
>>> This is not a special issue for flexible array member.
>> That's fine for fixed arrays at the end of a struct because the "whole 
>> object" size could be anything; `p` could be pointing to the beginning of an 
>> array for all we know.  If however `array` is strictly a flex array, i.e.:
>> ```
>> struct A
>> {
>>   size_t foo;
>>   int array[];
>> };
>> ```
>> then there's no way in valid C to have an array of `struct fix`, so `q` must 
>> be pointing to a single element.  So you could deduce:
>> 1. the minimum size of the whole object that q points to.
> 
> Actually for minimum size we'd also need a guarantee that `alloc_buf_more` 
> returns a valid allocated object.

Why? Please explain a little bit here.

thanks.

Qing
> 
> Sid

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

> On Aug 4, 2023, at 10:40 AM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-03 13:34, Qing Zhao wrote:
>> One thing I need to point out first is, currently, even for regular fixed 
>> size array in the structure,
>> We have this same issue, for example:
>> #define LENGTH 10
>> struct fix {
>>   size_t foo;
>>   int array[LENGTH];
>> };
>> …
>> int main ()
>> {
>>   struct fix *p;
>>   p = alloc_buf_more ();
>>   expect(__builtin_object_size(p->array, 1), LENGTH * sizeof(int));
>>   expect(__builtin_object_size(p->array, 0), -1);
>> }
>> Currently, for __builtin_object_size(p->array, 0),  GCC return UNKNOWN for 
>> it.
>> This is not a special issue for flexible array member.
> 
> That's fine for fixed arrays at the end of a struct because the "whole 
> object" size could be anything; `p` could be pointing to the beginning of an 
> array for all we know.  If however `array` is strictly a flex array, i.e.:
> 
> ```
> struct A
> {
>  size_t foo;
>  int array[];
> };
> ```
> 
> then there's no way in valid C to have an array of `struct fix`,

Yes!!   this is exactly the place that makes difference between structures with 
fixed arrays and the ones with flexible arrays. 

With such difference, I guess that using the type of the structure with 
flexible array member for p->array to get the size of the whole object p point 
to might be reasonable? 

> so `q` must be pointing to a single element.  So you could deduce:
> 
> 1. the minimum size of the whole object that q points to.

You mean that the TYPE will determine the minimum size of the whole object?  
(Does this include the size of the flexible array member, or only the other 
part of the structure except the flexible array member?)

> 
> and
> 
> 2. if you're able to determine the size of the flex array (through 
> __element_count__(foo) for example), you could even determine the maximum 
> size of the whole object.
> 
> For (2) though, you'd break applications that overallocate and then expect to 
> be able to use that overallocation despite the space not being reflected in 
> the __element_count__.  I think it's a bug in the application and I can't see 
> a way for an application to be able to do this in a valid way so I'm inclined 
> towards breaking it.

Currently, we allow the situation when the allocation size for the whole object 
is larger than the value reflected in the “counted_by” attribute (the old name 
is __element_count__). But don’t allow the other way around (i.e, when the 
allocation size for the whole object is smaller than the value reflected in the 
“counted_by” attribute. 
> 
> Of course, the fact that gcc allows flex arrays to be in the middle of 
> structs breaks the base assumption but that's something we need to get rid of 
> anyway since there's no way for valid C programs to use that safely.

Since GCC14, we started to deprecate this extension (allow flex array to be in 
the middle of structs).
https://gcc.gnu.org/pipermail/gcc-cvs/2023-June/385730.html

Thanks.

Qing


> 
> Thanks,
> Sid

Re: [PATCH] Add documentation for -Wflex-array-member-not-at-end.

[Qing Zhao via Gcc-patches]

Thanks.

I just updated the doc per your suggestion and committed as:

https://gcc.gnu.org/pipermail/gcc-cvs/2023-August/387588.html

Qing
> On Aug 3, 2023, at 1:29 PM, Joseph Myers  wrote:
> 
> On Thu, 3 Aug 2023, Qing Zhao via Gcc-patches wrote:
> 
>> +@opindex Wflex-array-member-not-at-end
>> +@opindex Wno-flex-array-member-not-at-end
>> +@item -Wflex-array-member-not-at-end
> 
> I'd expect this to have @r{(C and C++ only)} to indicate what languages 
> the option applies to.  OK with that change.
> 
> -- 
> Joseph S. Myers
> jos...@codesourcery.com

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

> On Aug 4, 2023, at 3:38 AM, Kees Cook  wrote:
> 
> On Thu, Aug 03, 2023 at 09:31:24PM +, Qing Zhao wrote:
>> So, the basic question is:
>> 
>> Given the following:
>> 
>> struct fix {
>>  int others;
>>  int array[10];
>> }
>> 
>> extern struct fix * alloc_buf ();
>> 
>> int main ()
>> {
>>  struct fix *p = alloc_buf ();
>>  __builtin_object_size(p->array,0) == ?
>> }
>> 
>> Given p->array, can the compiler determine that p points to an object that 
>> has TYPE struct fix?
>> 
>> If the answer is YES, then the current__builtin_object_size algorithm can be 
>> improved to determine __builtin_object_size(p->array, 0)  with the TYPE of 
>> the struct fix.
> 
> I think it is fine to leave __bos(..., 0) as-is. From the Linux kernel's
> use of __bos, we are almost exclusively only interesting the mode 1, not
> node 0. :)

Okay, that’s good to know.

Qing
> 
> -- 
> Kees Cook

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

So, the basic question is:

Given the following:

struct fix {
  int others;
  int array[10];
}

extern struct fix * alloc_buf ();

int main ()
{
  struct fix *p = alloc_buf ();
  __builtin_object_size(p->array,0) == ?
}

Given p->array, can the compiler determine that p points to an object that has 
TYPE struct fix?

If the answer is YES, then the current__builtin_object_size algorithm can be 
improved to determine __builtin_object_size(p->array, 0)  with the TYPE of the 
struct fix.

Qing


> On Aug 3, 2023, at 1:34 PM, Qing Zhao via Gcc-patches 
>  wrote:
> 
> One thing I need to point out first is, currently, even for regular fixed 
> size array in the structure,
> We have this same issue, for example:
> 
> #define LENGTH 10
> 
> struct fix {
>  size_t foo;
>  int array[LENGTH];
> };
> 
> …
> int main ()
> {
>  struct fix *p;
>  p = alloc_buf_more ();
> 
>  expect(__builtin_object_size(p->array, 1), LENGTH * sizeof(int));
>  expect(__builtin_object_size(p->array, 0), -1);
> }
> 
> Currently, for __builtin_object_size(p->array, 0),  GCC return UNKNOWN for it.
> This is not a special issue for flexible array member.
> 
> Qing
> 
> 
> On Aug 3, 2023, at 1:19 PM, Siddhesh Poyarekar  wrote:
>> 
>> On 2023-08-03 12:43, Qing Zhao wrote:
>>>> Surely we could emit that for __bdos(q->array, 0) though, couldn't we?
>>> For __bdos(q->array, 0), we only have the access info for the sub-object 
>>> q->array, we can surely decide the size of the sub-object q->array, but we 
>>> still cannot
>>> decide the whole object that is pointed by q (the same reason as above), 
>>> right?
>> 
>> It's tricky, I mean we could assume p to be a valid object due to the 
>> dereference and hence assume that q->foo is also valid and that there's at 
>> least sizeof(*q) + q->foo * sizeof (q->array) bytes available.  The question 
>> then is whether q could be pointing to an element of an array of `struct 
>> annotated`.  Could we ever have a valid array of such structs that have a 
>> flex array at the end?  Wouldn't it always be a single object?
>> 
>> In fact for all pointers to such structs with a flex array at the end, could 
>> we always assume that it is a single object and never part of an array, and 
>> hence return sizeof()?
>> 
>> Thanks,
>> Sid
> 

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

> On Aug 3, 2023, at 1:51 PM, Kees Cook  wrote:
> 
> On August 3, 2023 10:34:24 AM PDT, Qing Zhao  wrote:
>> One thing I need to point out first is, currently, even for regular fixed 
>> size array in the structure,
>> We have this same issue, for example:
>> 
>> #define LENGTH 10
>> 
>> struct fix {
>> size_t foo;
>> int array[LENGTH];
>> };
>> 
>> …
>> int main ()
>> {
>> struct fix *p;
>> p = alloc_buf_more ();
>> 
>> expect(__builtin_object_size(p->array, 1), LENGTH * sizeof(int));
>> expect(__builtin_object_size(p->array, 0), -1);
>> }
>> 
>> Currently, for __builtin_object_size(p->array, 0),  GCC return UNKNOWN for 
>> it.
>> This is not a special issue for flexible array member.
> 
> Is this true with -fstrict-flex-arrays=3 ?

Yes. 


Please see the following testing case:
#include 
#include 

#define LENGTH 10
#define SIZE_BUMP 5 
#define noinline __attribute__((__noinline__))

struct fix {
  size_t foo;
  int array[LENGTH]; 
};

#define expect(p, _v) do { \
size_t v = _v; \
if (p == v) \
__builtin_printf ("ok:  %s == %zd\n", #p, p); \
else \
{  \
  __builtin_printf ("WAT: %s == %zd (expected %zd)\n", #p, p, v); \
} \
} while (0);


/* in the following function, malloc allocated more space than size of the 
   struct fix.  Then what's the correct behavior we expect
   the __builtin_object_size should have for the following?
 */

static struct fix * noinline alloc_buf_more ()
{
  struct fix *p;
  p = malloc(sizeof (struct fix) + SIZE_BUMP * sizeof (int)); 

  /*when checking the observed access p->array, we have info on both
observered allocation and observed access, 
A. from observed allocation (alloc_size): (LENGTH + SIZE_BUMP) * sizeof 
(int)
B. from observed access (TYPE): LENGTH * sizeof (int)
   */
   
  /* for MAXIMUM size in the whole object: currently, GCC always used the A.  */
  expect(__builtin_object_size(p->array, 0), (LENGTH + SIZE_BUMP) * 
sizeof(int));

  /* for MAXIMUM size in the sub-object: currently, GCC chose the smaller
 one among these two: B.  */
  expect(__builtin_object_size(p->array, 1), LENGTH * sizeof(int));

  return p;
}

/* in the following function, malloc allocated less space than size of the 
   struct fix.  Then what's the correct behavior we expect
   the __builtin_object_size should have for the following?
 */

static struct fix * noinline alloc_buf_less ()
{
  struct fix *p;
  p = malloc(sizeof (struct fix) - SIZE_BUMP * sizeof (int)); 

  /*when checking the observed access p->array, we have info on both
observered allocation and observed access, 
A. from observed allocation (alloc_size): (LENGTH - SIZE_BUMP) * sizeof 
(int)
B. from observed access (TYPE): LENGTH * sizeof (int)
   */
   
  /* for MAXIMUM size in the whole object: currently, GCC always used the A.  */
  expect(__builtin_object_size(p->array, 0), (LENGTH - SIZE_BUMP) * 
sizeof(int));

  /* for MAXIMUM size in the sub-object: currently, GCC chose the smaller
 one among these two: B.  */
  expect(__builtin_object_size(p->array, 1), (LENGTH - SIZE_BUMP) * 
sizeof(int));

  return p;
}


int main ()
{
  struct fix *p, *q; 
  p = alloc_buf_more ();
  q = alloc_buf_less ();

  expect(__builtin_object_size(p->array, 1), LENGTH * sizeof(int));
  expect(__builtin_object_size(p->array, 0), -1);
  /*when checking the pointer p, we have no observed allocation nor observed 
access.
therefore, we cannot determine the size info here.  */
  expect(__builtin_object_size(p, 1), -1);
  expect(__builtin_object_size(p, 0), -1);

  /*when checking the observed access p->array, we only have info on the
observed access, i.e, the TYPE_SIZE info from the access. We don't have
info on the whole object.  */
  expect(__builtin_object_size(q->array, 1), LENGTH * sizeof(int));
  expect(__builtin_object_size(q->array, 0), -1);
  /*when checking the pointer p, we have no observed allocation nor observed 
access.
therefore, we cannot determine the size info here.  */
  expect(__builtin_object_size(q, 1), -1);
  expect(__builtin_object_size(q, 0), -1);

  return 0;
}

[opc@qinzhao-ol8u3-x86 108896]$ sh t
/home/opc/Install/latest/bin/gcc -O -fstrict-flex-arrays=3 t28.c
ok:  __builtin_object_size(p->array, 0) == 60
ok:  __builtin_object_size(p->array, 1) == 40
ok:  __builtin_object_size(p->array, 0) == 20
ok:  __builtin_object_size(p->array, 1) == 20
ok:  __builtin_object_size(p->array, 1) == 40
ok:  __builtin_object_size(p->array, 0) == -1
ok:  __builtin_object_size(p, 1) == -1
ok:  __builtin_object_size(p, 0) == -1
ok:  __builtin_object_size(q->array, 1) == 40
ok:  __builtin_object_size(q->array, 0) == -1
ok:  __builtin_object_size(q, 1) == -1
ok:  __builtin_object_size(q, 0) == -1
[opc@qinzhao-ol8u3-x86 108896]$ 

> 
> -Kees
> 
>> 
>> Qing
>> 
>> 
>> On Aug 3, 2023, at 1:19 PM, Siddhesh Poyarekar  wrote:
>>> 
>>> On 2023-08-03 12:43, Qing Zhao wrote:
> Surely we could emit that for __bdos(q->array, 0) though, couldn't we?
 For 

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

One thing I need to point out first is, currently, even for regular fixed size 
array in the structure,
We have this same issue, for example:

#define LENGTH 10

struct fix {
  size_t foo;
  int array[LENGTH];
};

…
int main ()
{
  struct fix *p;
  p = alloc_buf_more ();

  expect(__builtin_object_size(p->array, 1), LENGTH * sizeof(int));
  expect(__builtin_object_size(p->array, 0), -1);
}

Currently, for __builtin_object_size(p->array, 0),  GCC return UNKNOWN for it.
This is not a special issue for flexible array member.

Qing


On Aug 3, 2023, at 1:19 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-03 12:43, Qing Zhao wrote:
>>>  Surely we could emit that for __bdos(q->array, 0) though, couldn't we?
>> For __bdos(q->array, 0), we only have the access info for the sub-object 
>> q->array, we can surely decide the size of the sub-object q->array, but we 
>> still cannot
>> decide the whole object that is pointed by q (the same reason as above), 
>> right?
> 
> It's tricky, I mean we could assume p to be a valid object due to the 
> dereference and hence assume that q->foo is also valid and that there's at 
> least sizeof(*q) + q->foo * sizeof (q->array) bytes available.  The question 
> then is whether q could be pointing to an element of an array of `struct 
> annotated`.  Could we ever have a valid array of such structs that have a 
> flex array at the end?  Wouldn't it always be a single object?
> 
> In fact for all pointers to such structs with a flex array at the end, could 
> we always assume that it is a single object and never part of an array, and 
> hence return sizeof()?
> 
> Thanks,
> Sid

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

> On Aug 3, 2023, at 12:15 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-08-02 10:02, Qing Zhao wrote:
>>   /*when checking the observed access p->array, we only have info on the
>> observed access, i.e, the TYPE_SIZE info from the access. We don't have
>> info on the whole object.  */
>>   expect(__builtin_dynamic_object_size(q->array, 1), q->foo * sizeof(int));
>>   expect(__builtin_dynamic_object_size(q->array, 0), -1);
>>   expect(__builtin_dynamic_object_size(q->array, 3), q->foo * sizeof(int));
>>   expect(__builtin_dynamic_object_size(q->array, 2), 0);
>>   /*when checking the pointer p, we have no observed allocation nor observed 
>> access.
>> therefore, we cannot determine the size info here.  */
>>   expect(__builtin_dynamic_object_size(q, 1), -1);
>>   expect(__builtin_dynamic_object_size(q, 0), -1);
>>   expect(__builtin_dynamic_object_size(q, 3), 0);
>>   expect(__builtin_dynamic_object_size(q, 2), 0);
> 
> I'm wondering if we could sizeof (*q) + q->foo for __bdos(q, 0), but I 
> suppose it could mean generating code that potentially dereferences an 
> invalid pointer.

I think for __bdos(q, 0), if there is no allocation information for q, we 
cannot decide the size for its pointee.

>  Surely we could emit that for __bdos(q->array, 0) though, couldn't we?

For __bdos(q->array, 0), we only have the access info for the sub-object 
q->array, we can surely decide the size of the sub-object q->array, but we 
still cannot
decide the whole object that is pointed by q (the same reason as above), right?

Qing
> 
> Thanks,
> Sid

[PATCH] Add documentation for -Wflex-array-member-not-at-end.

[Qing Zhao via Gcc-patches]

When adding the option -Wflex-array-member-not-at-end in the commit
https://gcc.gnu.org/pipermail/gcc-cvs/2023-June/385730.html

the documentation for this new option was missing.

This patch is to add the documentation for this warning option.

bootstrapped and also checked the documentation, no issue.

Okay for committing?

thanks.

Qing

==


'-Wflex-array-member-not-at-end'
 Warn when a structure containing a C99 flexible array member as the
 last field is not at the end of another structure.  This warning
 warns e.g.  about

  struct flex  { int length; char data[]; };
  struct mid_flex { int m; struct flex flex_data; int n; };

gcc/ChangeLog:

* doc/invoke.texi (-Wflex-array-member-not-at-end): Document
new option.
---
 gcc/doc/invoke.texi | 13 +
 1 file changed, 13 insertions(+)

diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index adb10a3528da..0e7d827d355f 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -357,6 +357,7 @@ Objective-C and Objective-C++ Dialects}.
 -Wempty-body  -Wno-endif-labels  -Wenum-compare  -Wenum-conversion
 -Wenum-int-mismatch
 -Werror  -Werror=*  -Wexpansion-to-defined  -Wfatal-errors
+-Wflex-array-member-not-at-end
 -Wfloat-conversion  -Wfloat-equal  -Wformat  -Wformat=2
 -Wno-format-contains-nul  -Wno-format-extra-args
 -Wformat-nonliteral  -Wformat-overflow=@var{n}
@@ -9312,6 +9313,18 @@ value, like assigning a signed integer expression to an 
unsigned
 integer variable. An explicit cast silences the warning. In C, this
 option is enabled also by @option{-Wconversion}.
 
+@opindex Wflex-array-member-not-at-end
+@opindex Wno-flex-array-member-not-at-end
+@item -Wflex-array-member-not-at-end
+Warn when a structure containing a C99 flexible array member as the last
+field is not at the end of another structure.
+This warning warns e.g. about
+
+@smallexample
+struct flex  @{ int length; char data[]; @};
+struct mid_flex @{ int m; struct flex flex_data; int n; @};
+@end smallexample
+
 @opindex Wfloat-conversion
 @opindex Wno-float-conversion
 @item -Wfloat-conversion
-- 
2.31.1

Re: [PATCH] gcc-14/changes.html: Deprecate a GCC C extension on flexible array members.

[Qing Zhao via Gcc-patches]

> On Aug 3, 2023, at 3:10 AM, Richard Biener  wrote:
> 
> On Mon, Jul 10, 2023 at 9:12 PM Qing Zhao via Gcc-patches
>  wrote:
>> 
>> Hi,
>> 
>> This is the change for the GCC14 releaes Notes on the deprecating of a C
>> extension about flexible array members.
>> 
>> Okay for committing?
>> 
>> thanks.
>> 
>> Qing
>> 
>> 
>> 
>> *htdocs/gcc-14/changes.html (Caveats): Add notice about deprecating a C
>> extension about flexible array members.
>> ---
>> htdocs/gcc-14/changes.html | 10 +-
>> 1 file changed, 9 insertions(+), 1 deletion(-)
>> 
>> diff --git a/htdocs/gcc-14/changes.html b/htdocs/gcc-14/changes.html
>> index 3f797642..c7f2ce4d 100644
>> --- a/htdocs/gcc-14/changes.html
>> +++ b/htdocs/gcc-14/changes.html
>> @@ -30,7 +30,15 @@ a work-in-progress.
>> 
>> Caveats
>> 
>> -  ...
>> +  C:
>> +  Support for the GCC extension, a structure containing a C99 flexible 
>> array
>> +  member, or a union containing such a structure, is not the last field 
>> of
>> +  another structure, is deprecated. Refer to
>> +  https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html;>
>> +  Zero Length Arrays.
>> +  Any code relying on this extension should be modifed to ensure that
>> +  C99 flexible array members only end up at the ends of structures.
> 
> If it's deprecated any use should be diagnosed by default with a
> warning, can you
> mention that and make sure we do so?

I have added such warning option: -Wflex-array-member-not-at-end
In commit https://gcc.gnu.org/pipermail/gcc-cvs/2023-June/385730.html

I should add this warning option to the release notes. I will do that.


Another thing is, I just realized that the doc of this new option is missing 
from the invoke.texi.

I will send a new patch for the documentation of this new option first.

>  What would be the most surprising
> example of code that's going to be rejected?

 struct flex  { int length; char data[]; };

  struct mid_flex { int m; struct flex flex_data; int n; };

The above example, the 2nd structure mid_flex will be warned.

Qing
 
> 
> Richard.
> 
>> +  
>> 
>> 
>> 
>> --
>> 2.31.1

Re: [PATCH] gcc-14/changes.html: Deprecate a GCC C extension on flexible array members.

[Qing Zhao via Gcc-patches]

Ping…

thanks.

Qing

> On Jul 10, 2023, at 3:11 PM, Qing Zhao  wrote:
> 
> Hi,
> 
> This is the change for the GCC14 releaes Notes on the deprecating of a C
> extension about flexible array members.
> 
> Okay for committing?
> 
> thanks.
> 
> Qing
> 
> 
> 
> *htdocs/gcc-14/changes.html (Caveats): Add notice about deprecating a C
> extension about flexible array members.
> ---
> htdocs/gcc-14/changes.html | 10 +-
> 1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/htdocs/gcc-14/changes.html b/htdocs/gcc-14/changes.html
> index 3f797642..c7f2ce4d 100644
> --- a/htdocs/gcc-14/changes.html
> +++ b/htdocs/gcc-14/changes.html
> @@ -30,7 +30,15 @@ a work-in-progress.
> 
> Caveats
> 
> -  ...
> +  C:
> +  Support for the GCC extension, a structure containing a C99 flexible 
> array
> +  member, or a union containing such a structure, is not the last field 
> of
> +  another structure, is deprecated. Refer to
> +  https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html;>
> +  Zero Length Arrays.
> +  Any code relying on this extension should be modifed to ensure that
> +  C99 flexible array members only end up at the ends of structures.
> +  
> 
> 
> 
> -- 
> 2.31.1
> 

Re: [PATCH] gcc-13/changes.html: Add and fix URL to -fstrict-flex-array option.

[Qing Zhao via Gcc-patches]

Ping.

This is a very simple patch to correct a URL address in GCC13’s changes.html.
Currently, it’s pointing to a wrong address.

Okay for committing? 

> On Jul 21, 2023, at 3:02 PM, Qing Zhao  wrote:
> 
> Hi,
> 
> In the current GCC13 release note, the URL to the option -fstrict-flex-array
> is wrong (pointing to -Wstrict-flex-array).
> This is the change to correct the URL and also add the URL in another place
> where -fstrict-flex-array is mentioned.
> 
> I have checked the resulting HTML file, works well.
> 
> Okay for committing?
> 
> thanks.
> 
> Qing
> ---
> htdocs/gcc-13/changes.html | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/htdocs/gcc-13/changes.html b/htdocs/gcc-13/changes.html
> index 68e8c5cc..39b63a84 100644
> --- a/htdocs/gcc-13/changes.html
> +++ b/htdocs/gcc-13/changes.html
> @@ -46,7 +46,7 @@ You may also want to check out our
>   will no longer issue warnings for out of
>   bounds accesses to trailing struct members of one-element array type
>   anymore. Instead it diagnoses accesses to trailing arrays according to
> -  -fstrict-flex-arrays. 
> +   href="https://gcc.gnu.org/onlinedocs/gcc-13.1.0/gcc/C-Dialect-Options.html#index-fstrict-flex-arrays;>-fstrict-flex-arrays.
>  
>  href="https://gcc.gnu.org/onlinedocs/gcc-13.1.0/gcc/Static-Analyzer-Options.html;>-fanalyzer
>   is still only suitable for analyzing C code.
>   In particular, using it on C++ is unlikely to give meaningful 
> output.
> @@ -213,7 +213,7 @@ You may also want to check out our
>  flexible array member for the purpose of accessing the elements of such
>  an array. By default, all trailing arrays in aggregates are treated as
>  flexible array members. Use the new command-line option
> -  href="https://gcc.gnu.org/onlinedocs/gcc-13.1.0/gcc/Warning-Options.html#index-Wstrict-flex-arrays;>-fstrict-flex-arrays
> +  href="https://gcc.gnu.org/onlinedocs/gcc-13.1.0/gcc/C-Dialect-Options.html#index-fstrict-flex-arrays;>-fstrict-flex-arrays
>  to control which array members are treated as flexible arrays.
>  
> 
> -- 
> 2.31.1
> 

Re: [C PATCH]: Add Walloc-type to warn about insufficient size in allocations

[Qing Zhao via Gcc-patches]

> On Aug 1, 2023, at 10:31 AM, Martin Uecker  wrote:
> 
> Am Dienstag, dem 01.08.2023 um 13:27 + schrieb Qing Zhao:
>> 
>>> On Aug 1, 2023, at 3:51 AM, Martin Uecker via Gcc-patches 
>>>  wrote:
>>> 
> 
> 
 Hi Martin,
 Just wondering if it'd be a good idea perhaps to warn if alloc size is
 not a multiple of TYPE_SIZE_UNIT instead of just less-than ?
 So it can catch cases like:
 int *p = malloc (sizeof (int) + 2); // probably intended malloc
 (sizeof (int) * 2)
 
 FWIW, this is caught using -fanalyzer:
 f.c: In function 'f':
 f.c:3:12: warning: allocated buffer size is not a multiple of the
 pointee's size [CWE-131] [-Wanalyzer-allocation-size]
3 |   int *p = __builtin_malloc (sizeof(int) + 2);
  |^~
 
 Thanks,
 Prathamesh
>>> 
>>> Yes, this is probably a good idea.  It might need special
>>> logic for flexible array members then...
>> 
>> Why special logic for FAM on such warning? (Not a multiple of TYPE_SIZE_UNIT 
>> for the element).
>> 
> 
> For
> 
> struct { int n; char buf[]; } *p = malloc(sizeof *p + n);
> p->n = n;
> 
> the size would not be a multiple.

But n is still a multiple of sizeof (char), right? Do I miss anything here?

Qing
> 
> Martin
> 
> 
> 
> 

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Aug 1, 2023, at 6:45 PM, Kees Cook  wrote:
> 
> On Mon, Jul 31, 2023 at 08:14:42PM +, Qing Zhao wrote:
>> /* In general, Due to type casting, the type for the pointee of a pointer
>>   does not say anything about the object it points to,
>>   So, __builtin_object_size can not directly use the type of the pointee
>>   to decide the size of the object the pointer points to.
>> 
>>   there are only two reliable ways:
>>   A. observed allocations  (call to the allocation functions in the routine)
>>   B. observed accesses (read or write access to the location of the 
>> pointer points to)
>> 
>>   that provide information about the type/existence of an object at
>>   the corresponding address.
>> 
>>   for A, we use the "alloc_size" attribute for the corresponding allocation
>>   functions to determine the object size;
>> 
>>   For B, we use the SIZE info of the TYPE attached to the corresponding 
>> access.
>>   (We treat counted_by attribute as a complement to the SIZE info of the TYPE
>>for FMA) 
>> 
>>   The only other way in C which ensures that a pointer actually points
>>   to an object of the correct type is 'static':
>> 
>>   void foo(struct P *p[static 1]);   
>> 
>>   See https://gcc.gnu.org/pipermail/gcc-patches/2023-July/624814.html
>>   for more details.  */
> 
> This is a great explanation; thank you!
> 
> In the future I might want to have a new builtin that will allow
> a program to query a pointer when neither A nor B have happened. But
> for the first version of the __counted_by infrastructure, the above
> limitations seen fine.
> 
> For example, maybe __builtin_counted_size(p) (which returns sizeof(*p) +
> sizeof(*p->flex_array_member) * p->counted_by_member). Though since
> there might be multiple flex array members, maybe this can't work. :)

What do you mean by “there might be multiple flex array members”?

Do you mean the following example:

struct annotated {
size_t foo;
int array[] __attribute__((counted_by (foo)));
};

static struct annotated * noinline alloc_buf (int index)
{
  struct annotated *p;
  p = malloc(sizeof (*p) + (index) * sizeof (int));
  p->foo = index;
  return p;
}

Int main ()
{
  struct annotated *p1, *p2;
  p1 = alloc_buf (10);
  p2 = alloc_buf (20);

  __builtin_counted_size(p1)???
  __builtin_counted_size(p2)???
}

Or something else?

Qing
> 
> -Kees
> 
> -- 
> Kees Cook

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Aug 2, 2023, at 2:25 AM, Martin Uecker  wrote:
> 
> Am Dienstag, dem 01.08.2023 um 15:45 -0700 schrieb Kees Cook:
>> On Mon, Jul 31, 2023 at 08:14:42PM +, Qing Zhao wrote:
>>> /* In general, Due to type casting, the type for the pointee of a pointer
>>>   does not say anything about the object it points to,
>>>   So, __builtin_object_size can not directly use the type of the pointee
>>>   to decide the size of the object the pointer points to.
>>> 
>>>   there are only two reliable ways:
>>>   A. observed allocations  (call to the allocation functions in the routine)
>>>   B. observed accesses (read or write access to the location of the 
>>> pointer points to)
>>> 
>>>   that provide information about the type/existence of an object at
>>>   the corresponding address.
>>> 
>>>   for A, we use the "alloc_size" attribute for the corresponding allocation
>>>   functions to determine the object size;
>>> 
>>>   For B, we use the SIZE info of the TYPE attached to the corresponding 
>>> access.
>>>   (We treat counted_by attribute as a complement to the SIZE info of the 
>>> TYPE
>>>for FMA) 
>>> 
>>>   The only other way in C which ensures that a pointer actually points
>>>   to an object of the correct type is 'static':
>>> 
>>>   void foo(struct P *p[static 1]);   
>>> 
>>>   See https://gcc.gnu.org/pipermail/gcc-patches/2023-July/624814.html
>>>   for more details.  */
>> 
>> This is a great explanation; thank you!
>> 
>> In the future I might want to have a new builtin that will allow
>> a program to query a pointer when neither A nor B have happened. But
>> for the first version of the __counted_by infrastructure, the above
>> limitations seen fine.
>> 
>> For example, maybe __builtin_counted_size(p) (which returns sizeof(*p) +
>> sizeof(*p->flex_array_member) * p->counted_by_member). Though since
>> there might be multiple flex array members, maybe this can't work. :)
> 
> We had a _Lengthof proposal for arrays (instead of sizeof/sizeof)
> and thought about how to extend this to structs with FAM. The
> problem is that it can not rely on an attribute.
> 
> With GCC's VLA in structs you could do 
> 
> struct foo { int n; char buf[n_init]; } *p = malloc(sizeof *p);
> p->n_init = n;
> 
> and get sizeof and bounds checking with UBSan
> https://godbolt.org/z/d4nneqs3P
> 
> (but also compiler bugs and other issues)

This works great!

If later the bounds information for the FAM can be integrated into TYPE system 
just like the VLA, 
That will be ideal, then we don’t need to hack the compiler here and there to 
handle the FMA specially.
> 
> 
> Also see my experimental container library, where you can do:
> 
> vec_decl(int);
> vec(int)* v = vec_alloc(int);
> 
> vec_push(, 1);
> vec_push(, 3);
> 
> auto p = _array(v);
> (*p)[1] = 1; // bounds check
> 
> Here, "vec_array()" would give you a regular C array view
> of the vector contant and with correct dynamic size, so you
> can apply "sizeof" and  have bounds checking with UBSan and
> it just works (with clang / GCC without changes). 
> https://github.com/uecker/noplate

Yes, the idea of providing a type-safe library for C also looks promising. 
thanks

Qing
> 
> 
> 
> Martin

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

Okay.  This previous small example was used to show the correct behavior of 
__bos 
for Fixed arrays when the allocation size and the TYPE_SIZE are mismatched. 

Now we agreed on the correct behavior for each of the cases for the fixed array.

Since the new “counted_by” attribute is mainly a complement to the TYPE_SIZE 
for the flexible array member.
So, GCC should just use it similarly as TYPE_SIZE. 

Based on the fixed array example, I came up a small example for the flexible 
array member with “counted_by” attribute,
And the expected correct behavior for each of the cases. 
I also put detailed comments into the example to explain why for each case. 
(Similar as the fixed array example)

Please take a look at this example and let me know any issue you see.
With my private GCC that support “counted_by” attribute, all the cases passed.

Thanks.

Qing.


#include 
#include 

struct annotated {
size_t foo;
int array[] __attribute__((counted_by (foo)));
};

#define expect(p, _v) do { \
size_t v = _v; \
if (p == v) \
__builtin_printf ("ok:  %s == %zd\n", #p, p); \
else \
{  \
  __builtin_printf ("WAT: %s == %zd (expected %zd)\n", #p, p, v); \
} \
} while (0);

#define noinline __attribute__((__noinline__))
#define SIZE_BUMP 5 

/* In general, Due to type casting, the type for the pointee of a pointer
   does not say anything about the object it points to,
   So, __builtin_object_size can not directly use the type of the pointee
   to decide the size of the object the pointer points to.

   there are only two reliable ways:
   A. observed allocations  (call to the allocation functions in the routine)
   B. observed accesses (read or write access to the location of the 
 pointer points to)

   that provide information about the type/existence of an object at
   the corresponding address.

   for A, we use the "alloc_size" attribute for the corresponding allocation
   functions to determine the object size;

   For B, we use the SIZE info of the TYPE attached to the corresponding access.
   (We treat counted_by attribute as a complement to the SIZE info of the TYPE
for FMA) 

   The only other way in C which ensures that a pointer actually points
   to an object of the correct type is 'static':

   void foo(struct P *p[static 1]);   

   See https://gcc.gnu.org/pipermail/gcc-patches/2023-July/624814.html
   for more details.  */

/* in the following function, malloc allocated more space than the value
   of counted_by attribute.  Then what's the correct behavior we expect
   the __builtin_dynamic_object_size should have for each of the cases?  */ 

static struct annotated * noinline alloc_buf_more (int index)
{
  struct annotated *p;
  p = malloc(sizeof (*p) + (index + SIZE_BUMP) * sizeof (int));
  p->foo = index;

  /*when checking the observed access p->array, we have info on both
observered allocation and observed access, 
A. from observed allocation: (index + SIZE_BUMP) * sizeof (int)
B. from observed access: p->foo * sizeof (int)

in the above, p->foo = index.
   */
   
  /* for size in the whole object: always uses A.  */
  /* for size in the sub-object: chose the smaller of A and B.
   * Please see https://gcc.gnu.org/pipermail/gcc-patches/2023-July/625891.html
   * for details on why.  */

  /* for MAXIMUM size in the whole object: use the allocation size 
 for the whole object.  */
  expect(__builtin_dynamic_object_size(p->array, 0), (index + SIZE_BUMP) * 
sizeof(int));

  /* for MAXIMUM size in the sub-object. use the smaller of A and B.  */ 
  expect(__builtin_dynamic_object_size(p->array, 1), (p->foo) * sizeof(int));

  /* for MINIMUM size in the whole object: use the allocation size 
 for the whole object.  */
  expect(__builtin_dynamic_object_size(p->array, 2), (index + SIZE_BUMP) * 
sizeof(int));

  /* for MINIMUM size in the sub-object: use the smaller of A and B.  */
  expect(__builtin_dynamic_object_size(p->array, 3), p->foo * sizeof(int));

  /*when checking the pointer p, we only have info on the observed allocation.
So, the object size info can only been obtained from the call to malloc.
for both MAXIMUM and MINIMUM: A = (index + SIZE_BUMP) * sizeof (int)  */ 
  expect(__builtin_dynamic_object_size(p, 1), sizeof (*p) + (index + SIZE_BUMP) 
* sizeof(int));
  expect(__builtin_dynamic_object_size(p, 0), sizeof (*p) + (index + SIZE_BUMP) 
* sizeof(int));
  expect(__builtin_dynamic_object_size(p, 3), sizeof (*p) + (index + SIZE_BUMP) 
* sizeof(int));
  expect(__builtin_dynamic_object_size(p, 2), sizeof (*p) + (index + SIZE_BUMP) 
* sizeof(int));
  return p;
}

/* in the following function, malloc allocated less space than the value
   of counted_by attribute.  Then what's the correct behavior we expect
   the __builtin_dynamic_object_size should have for each of the cases?
   NOTE: this is an user error, GCC should 

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

> On Jul 31, 2023, at 1:07 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-07-31 13:03, Siddhesh Poyarekar wrote:
>> On 2023-07-31 12:47, Qing Zhao wrote:
>>> Hi, Sid and Jakub,
>>> 
>>> I have a question in the following source portion of the routine 
>>> “addr_object_size” of gcc/tree-object-size.cc:
>>> 
>>>   743   bytes = compute_object_offset (TREE_OPERAND (ptr, 0), var);
>>>   744   if (bytes != error_mark_node)
>>>   745 {
>>>   746   bytes = size_for_offset (var_size, bytes);
>>>   747   if (var != pt_var && pt_var_size && TREE_CODE (pt_var) == 
>>> MEM_REF)
>>>   748 {
>>>   749   tree bytes2 = compute_object_offset (TREE_OPERAND (ptr, 
>>> 0),
>>>   750pt_var);
>>>   751   if (bytes2 != error_mark_node)
>>>   752 {
>>>   753   bytes2 = size_for_offset (pt_var_size, bytes2);
>>>   754   bytes = size_binop (MIN_EXPR, bytes, bytes2);
>>>   755 }
>>>   756 }
>>>   757 }
>>> 
>>> At line 754, why we always use “MIN_EXPR” whenever it’s for OST_MINIMUM or 
>>> not?
>>> Shall we use
>>> 
>>> (object_size_type & OST_MINIMUM
>>>  ? MIN_EXPR : MAX_EXPR)
>>> 
>> That MIN_EXPR is not for OST_MINIMUM.  It is to cater for allocations like 
>> this:
>> typedef struct
>> {
>>   int a;
>> } A;
>> size_t f()
>> {
>>   A *p = malloc (1);
>>   return __builtin_object_size (p, 0);
> 
> Correction, that should be __builtin_object_size (p->a, 0).

Actually, it should be __builtin_object_size(p->a, 1).
For __builtin_object_size(p->a,0), gcc always uses the allocation size for the 
whole object.

GCC’s current behavior is:

For the size of the whole object, GCC currently always uses the allocation 
size. 
And for the size in the sub-object, GCC chose the smaller one among the 
allocation size and the TYPE_SIZE. 

Is this correct behavior?

thanks.

Qing

Please see the following small example to show the above behavior:

=

#include 
#include 

#define LENGTH 10
#define SIZE_BUMP 5 
#define noinline __attribute__((__noinline__))

struct fix {
  size_t foo;
  int array[LENGTH]; 
};

#define expect(p, _v) do { \
size_t v = _v; \
if (p == v) \
__builtin_printf ("ok:  %s == %zd\n", #p, p); \
else \
{  \
  __builtin_printf ("WAT: %s == %zd (expected %zd)\n", #p, p, v); \
} \
} while (0);


/* in the following function, malloc allocated more space than size of the 
   struct fix.  Then what's the correct behavior we expect
   the __builtin_object_size should have for the following?
 */

static struct fix * noinline alloc_buf_more ()
{
  struct fix *p;
  p = malloc(sizeof (struct fix) + SIZE_BUMP * sizeof (int)); 

  /*when checking the observed access p->array, we have info on both
observered allocation and observed access, 
A. from observed allocation (alloc_size): (LENGTH + SIZE_BUMP) * sizeof 
(int)
B. from observed access (TYPE): LENGTH * sizeof (int)
   */
   
  /* for MAXIMUM size in the whole object: currently, GCC always used the A.  */
  expect(__builtin_object_size(p->array, 0), (LENGTH + SIZE_BUMP) * 
sizeof(int));

  /* for MAXIMUM size in the sub-object: currently, GCC chose the smaller
 one among these two: B.  */
  expect(__builtin_object_size(p->array, 1), LENGTH * sizeof(int));

  return p;
}

/* in the following function, malloc allocated less space than size of the 
   struct fix.  Then what's the correct behavior we expect
   the __builtin_object_size should have for the following?
 */

static struct fix * noinline alloc_buf_less ()
{
  struct fix *p;
  p = malloc(sizeof (struct fix) - SIZE_BUMP * sizeof (int)); 

  /*when checking the observed access p->array, we have info on both
observered allocation and observed access, 
A. from observed allocation (alloc_size): (LENGTH - SIZE_BUMP) * sizeof 
(int)
B. from observed access (TYPE): LENGTH * sizeof (int)
   */
   
  /* for MAXIMUM size in the whole object: currently, GCC always used the A.  */
  expect(__builtin_object_size(p->array, 0), (LENGTH - SIZE_BUMP) * 
sizeof(int));

  /* for MAXIMUM size in the sub-object: currently, GCC chose the smaller
 one among these two: B.  */
  expect(__builtin_object_size(p->array, 1), (LENGTH - SIZE_BUMP) * 
sizeof(int));

  return p;
}


int main ()
{
  struct fix *p, *q; 
  p = alloc_buf_more ();
  q = alloc_buf_less ();

  return 0;
}


When compile the above small testing case with upstream gcc with 
-fstrict-flex-array=1:

/home/opc/Install/latest/bin/gcc -O -fstrict-flex-arrays=1 t28.c
ok:  __builtin_object_size(p->array, 0) == 60
ok:  __builtin_object_size(p->array, 1) == 40
ok:  __builtin_object_size(p->array, 0) == 20
ok:  __builtin_object_size(p->array, 1) == 20


> 
>> }
>> where the returned size should be 1 and not sizeof (int).  The mode doesn't 
>> really matter in this case.
>> HTH.
>> Sid

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

  therefore, we cannot determine the size info here.  */
  expect(__builtin_dynamic_object_size(p, 1), -1);
  expect(__builtin_dynamic_object_size(p, 0), -1);
  expect(__builtin_dynamic_object_size(p, 3), 0);
  expect(__builtin_dynamic_object_size(p, 2), 0);

  return 0;
}





> On Jul 19, 2023, at 2:52 PM, Qing Zhao via Gcc-patches 
>  wrote:
> 
> 
>>> 
>>> The point is: allocation size should synced with the value of “counted_by”. 
>>> LLVM’s RFC also have the similar requirement:
>>> https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854#maintaining-correctness-of-bounds-annotations-18
>> 
>> Right, I'm saying it would be nice if __alloc_size was checked as well,
>> in the sense that if it is available, it knows without question what the
>> size of the allocation is. If __alloc_size and __counted_by conflict,
>> the smaller of the two should be the truth.
> 
> I don’t think that  “if __alloc_size and __counted_by conflict, the smaller 
> of the two should be the truth” will work correctly.
> 
> When __alloc_size is larger than the value of __counted_by, it’s okay. 
> But when the value of __counted_by is larger than the __alloc_size, the array 
> bound check or object size sanitizer might not work correctly.
> 
> 
> Please see the following example:
> 
> struct grows {
>   int alloc_count;
>   int valid_count;
>   int  item[] __counted_by(valid_count);
> } *p;
> 
> void __attribute__((__noinline__)) something (int n)
> {
>   p = malloc(sizeof(*p) + sizeof(*p->item) * 100);
>   p->alloc_count = 100;
>   p->valid_count = 102;
>   p->item[n] = 10;// both _alloc_size and the value of 
> __counted_by are available in this routine, the smaller one is , 100;
> 
> }
> 
> void __attribute__((__noinline__))  something_2 (int n)
> {
>   p->item[n] = 10;   // only the value of  __counted_by is available in this 
> routine, which is 102;  
> }
> 
> Int main
> {
>   Something (101);
>   Something_2 (101);
> }
> 
> 
> For the above example, the out-of-bound array access in routine “something” 
> should be able to be caught by the compiler.
> However, the out-of-bound array access in the routine “something_2” will NOT 
> be able to be caught by the compiler.
> 
> Since in the routine “something_2” , the compiler don’t know the alloc_size, 
> the only available info is the counted_by value
> through the attribute.  But this value is bigger than the REAL size of the 
> array. Therefore the compiler cannot detect the 
> out-of-bound array access in the routine something_2
> 
> 
> Based on the above observation, I think we should add the following 
> requirement: 
> 
> The value of “counted_by” should be equal or SMALLER than the real alloc_size 
> for the flexible array member. 
> 
> This is the same requirement as the LLVM RFC. 
> https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854#maintaining-correctness-of-bounds-annotations-18
> 
> "the compiler inserts additional checks to ensure the new buf has at least as 
> many elements as the new count indicates.”
> LLVM has additional requirement in addition to this, we might need to 
> consider those requirement too. 
> 
> Qing
> 
>> But, as I said, if there is some need to explicitly ignore __alloc_size
>> when __counted_by is present, I can live with it; we just need to
>> document it.
>> 
>> If the RFC and you agree that the __counted_by variable can only ever be
>> (re)assigned after the flex array has been (re)allocated, then I guess
>> we'll see how it goes. :) I think most places in the kernel using
>> __counted_by will be fine, but I suspect we may have cases where we need
>> to update it like in the loop I described above. If that's true, we can
>> revisit the requirement then. :)
>> 
>> -Kees
>> 
>> -- 
>> Kees Cook
> 

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

> On Jul 31, 2023, at 2:23 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-07-31 14:13, Qing Zhao wrote:
>> Okay. I see.
>> Then if the size info from the TYPE is smaller than the size info from the 
>> malloc,
>>  then based on the current code, we use the smaller one between these two,
>>  i.e, the size info from the TYPE.  (Even for the OST_MAXIMUM).
>> Is such behavior correct?
> 
> Yes, it's correct even for OST_MAXIMUM.  The smaller one between the two is 
> the more precise estimate, which is why the mode doesn't matter.
> 
>> This is for the new “counted_by” attribute and how to use it in 
>> __builtin_dynamic_object_size.
>> for example:
>> ===
>> struct annotated {
>> size_t foo;
>> int array[] __attribute__((counted_by (foo)));
>> };
>> #define noinline __attribute__((__noinline__))
>> #define SIZE_BUMP 2
>> /* in the following function, malloc allocated more space than the value
>>of counted_by attribute.  Then what's the correct behavior we expect
>>the __builtin_dynamic_object_size should have?  */
>> static struct annotated * noinline alloc_buf (int index)
>> {
>>   struct annotated *p;
>>   p = malloc(sizeof (*p) + (index + SIZE_BUMP) * sizeof (int));
>>   p->foo = index;
>>   /*when checking the observed access p->array, we have info on both
>> observered allocation and observed access,
>> A. from observed allocation: (index + SIZE_BUMP) * sizeof (int)
>> B. from observed access: p->foo * sizeof (int)
>> in the above, p->foo = index.
>>*/
>>   /* for MAXIMUM size, based on the current code, we will use the size info 
>> from the TYPE,
>>  i.e, the “counted_by” attribute, which is the smaller one.   */
>>   expect(__builtin_dynamic_object_size(p->array, 1), (p->foo) * sizeof(int));
> 
> If the counted_by is less than what is allocated, it is the more correct 
> value to return because that's what the application asked for through the 
> attribute.  If the allocated size is less, we return the allocated size 
> because in that case, despite what the application said, the actual allocated 
> size is less and hence that's the safer value.

Thanks a lot for the clear explanation. This makes good sense.
> 
> In fact in the latter case it may even make sense to emit a warning because 
> it is more likely than not to be a bug.

Agreed. 

Here is a patch from Martin on a new similar warning (-Walloc-type):  
https://gcc.gnu.org/pipermail/gcc-patches/2023-July/625172.html. 

I guess that I will also need to issue warning for such cases for the new 
attribute “counted_by”.

Qing

> Thanks,
> Sid

Re: One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

Hi, Sid,

Thanks a lot.

> On Jul 31, 2023, at 1:07 PM, Siddhesh Poyarekar  wrote:
> 
> On 2023-07-31 13:03, Siddhesh Poyarekar wrote:
>> On 2023-07-31 12:47, Qing Zhao wrote:
>>> Hi, Sid and Jakub,
>>> 
>>> I have a question in the following source portion of the routine 
>>> “addr_object_size” of gcc/tree-object-size.cc:
>>> 
>>>   743   bytes = compute_object_offset (TREE_OPERAND (ptr, 0), var);
>>>   744   if (bytes != error_mark_node)
>>>   745 {
>>>   746   bytes = size_for_offset (var_size, bytes);
>>>   747   if (var != pt_var && pt_var_size && TREE_CODE (pt_var) == 
>>> MEM_REF)
>>>   748 {
>>>   749   tree bytes2 = compute_object_offset (TREE_OPERAND (ptr, 
>>> 0),
>>>   750pt_var);
>>>   751   if (bytes2 != error_mark_node)
>>>   752 {
>>>   753   bytes2 = size_for_offset (pt_var_size, bytes2);
>>>   754   bytes = size_binop (MIN_EXPR, bytes, bytes2);
>>>   755 }
>>>   756 }
>>>   757 }
>>> 
>>> At line 754, why we always use “MIN_EXPR” whenever it’s for OST_MINIMUM or 
>>> not?
>>> Shall we use
>>> 
>>> (object_size_type & OST_MINIMUM
>>>  ? MIN_EXPR : MAX_EXPR)
>>> 
>> That MIN_EXPR is not for OST_MINIMUM.  It is to cater for allocations like 
>> this:
>> typedef struct
>> {
>>   int a;
>> } A;
>> size_t f()
>> {
>>   A *p = malloc (1);
>>   return __builtin_object_size (p, 0);
> 
> Correction, that should be __builtin_object_size (>a, 0)

Okay. I see.

Then if the size info from the TYPE is smaller than the size info from the 
malloc,
 then based on the current code, we use the smaller one between these two,
 i.e, the size info from the TYPE.  (Even for the OST_MAXIMUM). 

Is such behavior correct?

This is for the new “counted_by” attribute and how to use it in 
__builtin_dynamic_object_size. 
for example:

===

struct annotated {
size_t foo;
int array[] __attribute__((counted_by (foo)));
};

#define noinline __attribute__((__noinline__))
#define SIZE_BUMP 2 

/* in the following function, malloc allocated more space than the value
   of counted_by attribute.  Then what's the correct behavior we expect
   the __builtin_dynamic_object_size should have?  */

static struct annotated * noinline alloc_buf (int index)
{
  struct annotated *p;
  p = malloc(sizeof (*p) + (index + SIZE_BUMP) * sizeof (int));
  p->foo = index;

  /*when checking the observed access p->array, we have info on both
observered allocation and observed access, 
A. from observed allocation: (index + SIZE_BUMP) * sizeof (int)
B. from observed access: p->foo * sizeof (int)

in the above, p->foo = index.
   */

  /* for MAXIMUM size, based on the current code, we will use the size info 
from the TYPE, 
 i.e, the “counted_by” attribute, which is the smaller one.   */
  expect(__builtin_dynamic_object_size(p->array, 1), (p->foo) * sizeof(int));

  return p;
}


Is the above the correct behavior?

thanks.

Qing
> 
>> }
>> where the returned size should be 1 and not sizeof (int).  The mode doesn't 
>> really matter in this case.
>> HTH.
>> Sid

One question on the source code of tree-object-size.cc

[Qing Zhao via Gcc-patches]

Hi, Sid and Jakub,

I have a question in the following source portion of the routine 
“addr_object_size” of gcc/tree-object-size.cc:

 743   bytes = compute_object_offset (TREE_OPERAND (ptr, 0), var);
 744   if (bytes != error_mark_node)
 745 {
 746   bytes = size_for_offset (var_size, bytes);
 747   if (var != pt_var && pt_var_size && TREE_CODE (pt_var) == 
MEM_REF)
 748 {
 749   tree bytes2 = compute_object_offset (TREE_OPERAND (ptr, 0),
 750pt_var);
 751   if (bytes2 != error_mark_node)
 752 {
 753   bytes2 = size_for_offset (pt_var_size, bytes2);
 754   bytes = size_binop (MIN_EXPR, bytes, bytes2);
 755 }
 756 }
 757 }

At line 754, why we always use “MIN_EXPR” whenever it’s for OST_MINIMUM or not? 
Shall we use 

(object_size_type & OST_MINIMUM
? MIN_EXPR : MAX_EXPR)

Instead?

Thanks a lot for the help.

Qing

Re: [C PATCH]: Add Walloc-type to warn about insufficient size in allocations

[Qing Zhao via Gcc-patches]

> On Jul 21, 2023, at 7:21 AM, Martin Uecker via Gcc-patches 
>  wrote:
> 
> 
> 
> This patch adds a warning for allocations with insufficient size
> based on the "alloc_size" attribute and the type of the pointer 
> the result is assigned to. While it is theoretically legal to
> assign to the wrong pointer type and cast it to the right type
> later, this almost always indicates an error. Since this catches
> common mistakes and is simple to diagnose, it is suggested to
> add this warning.
> 
> 
> Bootstrapped and regression tested on x86. 
> 
> 
> Martin
> 
> 
> 
> Add option Walloc-type that warns about allocations that have
> insufficient storage for the target type of the pointer the
> storage is assigned to.
> 
> gcc:
>   * doc/invoke.texi: Document -Wstrict-flex-arrays option.

The above should be “Document -Walloc-type option”. -:).

Qing
> 
> gcc/c-family:
> 
>   * c.opt (Walloc-type): New option.
> 
> gcc/c:
>   * c-typeck.cc (convert_for_assignment): Add Walloc-type warning.
> 
> gcc/testsuite:
> 
>   * gcc.dg/Walloc-type-1.c: New test.
> 
> 
> diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt
> index 4abdc8d0e77..8b9d148582b 100644
> --- a/gcc/c-family/c.opt
> +++ b/gcc/c-family/c.opt
> @@ -319,6 +319,10 @@ Walloca
> C ObjC C++ ObjC++ Var(warn_alloca) Warning
> Warn on any use of alloca.
> 
> +Walloc-type
> +C ObjC Var(warn_alloc_type) Warning
> +Warn when allocating insufficient storage for the target type of the
> assigned pointer.
> +
> Walloc-size-larger-than=
> C ObjC C++ LTO ObjC++ Var(warn_alloc_size_limit) Joined Host_Wide_Int
> ByteSize Warning Init(HOST_WIDE_INT_MAX)
> -Walloc-size-larger-than=  Warn for calls to allocation
> functions that
> diff --git a/gcc/c/c-typeck.cc b/gcc/c/c-typeck.cc
> index 7cf411155c6..2e392f9c952 100644
> --- a/gcc/c/c-typeck.cc
> +++ b/gcc/c/c-typeck.cc
> @@ -7343,6 +7343,32 @@ convert_for_assignment (location_t location,
> location_t expr_loc, tree type,
>   "request for implicit conversion "
>   "from %qT to %qT not permitted in C++", rhstype,
> type);
> 
> +  /* Warn of new allocations are not big enough for the target
> type.  */
> +  tree fndecl;
> +  if (warn_alloc_type
> +   && TREE_CODE (rhs) == CALL_EXPR
> +   && (fndecl = get_callee_fndecl (rhs)) != NULL_TREE
> +   && DECL_IS_MALLOC (fndecl))
> + {
> +   tree fntype = TREE_TYPE (fndecl);
> +   tree fntypeattrs = TYPE_ATTRIBUTES (fntype);
> +   tree alloc_size = lookup_attribute ("alloc_size",
> fntypeattrs);
> +   if (alloc_size)
> + {
> +   tree args = TREE_VALUE (alloc_size);
> +   int idx = TREE_INT_CST_LOW (TREE_VALUE (args)) - 1;
> +   /* For calloc only use the second argument.  */
> +   if (TREE_CHAIN (args))
> + idx = TREE_INT_CST_LOW (TREE_VALUE (TREE_CHAIN
> (args))) - 1;
> +   tree arg = CALL_EXPR_ARG (rhs, idx);
> +   if (TREE_CODE (arg) == INTEGER_CST
> +   && tree_int_cst_lt (arg, TYPE_SIZE_UNIT (ttl)))
> +  warning_at (location, OPT_Walloc_type, "allocation of
> "
> +  "insufficient size %qE for type %qT with
> "
> +  "size %qE", arg, ttl, TYPE_SIZE_UNIT
> (ttl));
> + }
> + }
> +
>   /* See if the pointers point to incompatible address spaces.  */
>   asl = TYPE_ADDR_SPACE (ttl);
>   asr = TYPE_ADDR_SPACE (ttr);
> diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
> index 88e3c625030..6869bed64c3 100644
> --- a/gcc/doc/invoke.texi
> +++ b/gcc/doc/invoke.texi
> @@ -8076,6 +8076,15 @@ always leads to a call to another @code{cold}
> function such as wrappers of
> C++ @code{throw} or fatal error reporting functions leading to
> @code{abort}.
> @end table
> 
> +@opindex Wno-alloc-type
> +@opindex Walloc-type
> +@item -Walloc-type
> +Warn about calls to allocation functions decorated with attribute
> +@code{alloc_size} that specify insufficient size for the target type
> of
> +the pointer the result is assigned to, including those to the built-in
> +forms of the functions @code{aligned_alloc}, @code{alloca},
> @code{calloc},
> +@code{malloc}, and @code{realloc}.
> +
> @opindex Wno-alloc-zero
> @opindex Walloc-zero
> @item -Walloc-zero
> diff --git a/gcc/testsuite/gcc.dg/Walloc-type-1.c
> b/gcc/testsuite/gcc.dg/Walloc-type-1.c
> new file mode 100644
> index 000..bc62e5e9aa3
> --- /dev/null
> +++ b/gcc/testsuite/gcc.dg/Walloc-type-1.c
> @@ -0,0 +1,37 @@
> +/* Tests the warnings for insufficient allocation size. 
> +   { dg-do compile }
> + * { dg-options "-Walloc-type" } 
> + * */
> +#include 
> +#include 
> +
> +struct b { int x[10]; };
> +
> +void fo0(void)
> +{
> +struct b *p = malloc(sizeof *p);
> +}
> +
> +void fo1(void)
> +{
> +struct b *p = malloc(sizeof p);  /* { dg-
> warning "allocation of insufficient size" } */
> +}
> +
> +void fo2(void)
> +{
> +struct b *p = 

[PATCH] gcc-13/changes.html: Add and fix URL to -fstrict-flex-array option.

[Qing Zhao via Gcc-patches]

Hi,

In the current GCC13 release note, the URL to the option -fstrict-flex-array
is wrong (pointing to -Wstrict-flex-array).
This is the change to correct the URL and also add the URL in another place
where -fstrict-flex-array is mentioned.

I have checked the resulting HTML file, works well.

Okay for committing?

thanks.

Qing
---
 htdocs/gcc-13/changes.html | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/htdocs/gcc-13/changes.html b/htdocs/gcc-13/changes.html
index 68e8c5cc..39b63a84 100644
--- a/htdocs/gcc-13/changes.html
+++ b/htdocs/gcc-13/changes.html
@@ -46,7 +46,7 @@ You may also want to check out our
   will no longer issue warnings for out of
   bounds accesses to trailing struct members of one-element array type
   anymore. Instead it diagnoses accesses to trailing arrays according to
-  -fstrict-flex-arrays. 
+  https://gcc.gnu.org/onlinedocs/gcc-13.1.0/gcc/C-Dialect-Options.html#index-fstrict-flex-arrays;>-fstrict-flex-arrays.
 
 https://gcc.gnu.org/onlinedocs/gcc-13.1.0/gcc/Static-Analyzer-Options.html;>-fanalyzer
   is still only suitable for analyzing C code.
   In particular, using it on C++ is unlikely to give meaningful 
output.
@@ -213,7 +213,7 @@ You may also want to check out our
  flexible array member for the purpose of accessing the elements of such
  an array. By default, all trailing arrays in aggregates are treated as
  flexible array members. Use the new command-line option
- https://gcc.gnu.org/onlinedocs/gcc-13.1.0/gcc/Warning-Options.html#index-Wstrict-flex-arrays;>-fstrict-flex-arrays
+ https://gcc.gnu.org/onlinedocs/gcc-13.1.0/gcc/C-Dialect-Options.html#index-fstrict-flex-arrays;>-fstrict-flex-arrays
  to control which array members are treated as flexible arrays.
  
 
-- 
2.31.1

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

>> 
>> The point is: allocation size should synced with the value of “counted_by”. 
>> LLVM’s RFC also have the similar requirement:
>> https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854#maintaining-correctness-of-bounds-annotations-18
> 
> Right, I'm saying it would be nice if __alloc_size was checked as well,
> in the sense that if it is available, it knows without question what the
> size of the allocation is. If __alloc_size and __counted_by conflict,
> the smaller of the two should be the truth.

I don’t think that  “if __alloc_size and __counted_by conflict, the smaller of 
the two should be the truth” will work correctly.

When __alloc_size is larger than the value of __counted_by, it’s okay. 
But when the value of __counted_by is larger than the __alloc_size, the array 
bound check or object size sanitizer might not work correctly.


Please see the following example:

struct grows {
int alloc_count;
int valid_count;
int  item[] __counted_by(valid_count);
} *p;

void __attribute__((__noinline__)) something (int n)
{
p = malloc(sizeof(*p) + sizeof(*p->item) * 100);
p->alloc_count = 100;
p->valid_count = 102;
p->item[n] = 10;// both _alloc_size and the value of 
__counted_by are available in this routine, the smaller one is , 100;

}

void __attribute__((__noinline__))  something_2 (int n)
{
   p->item[n] = 10;   // only the value of  __counted_by is available in this 
routine, which is 102;  
}

Int main
{
   Something (101);
   Something_2 (101);
}


For the above example, the out-of-bound array access in routine “something” 
should be able to be caught by the compiler.
However, the out-of-bound array access in the routine “something_2” will NOT be 
able to be caught by the compiler.

Since in the routine “something_2” , the compiler don’t know the alloc_size, 
the only available info is the counted_by value
 through the attribute.  But this value is bigger than the REAL size of the 
array. Therefore the compiler cannot detect the 
out-of-bound array access in the routine something_2


Based on the above observation, I think we should add the following 
requirement: 

The value of “counted_by” should be equal or SMALLER than the real alloc_size 
for the flexible array member. 

This is the same requirement as the LLVM RFC. 
https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854#maintaining-correctness-of-bounds-annotations-18

"the compiler inserts additional checks to ensure the new buf has at least as 
many elements as the new count indicates.”
LLVM has additional requirement in addition to this, we might need to consider 
those requirement too. 

Qing

> But, as I said, if there is some need to explicitly ignore __alloc_size
> when __counted_by is present, I can live with it; we just need to
> document it.
> 
> If the RFC and you agree that the __counted_by variable can only ever be
> (re)assigned after the flex array has been (re)allocated, then I guess
> we'll see how it goes. :) I think most places in the kernel using
> __counted_by will be fine, but I suspect we may have cases where we need
> to update it like in the loop I described above. If that's true, we can
> revisit the requirement then. :)
> 
> -Kees
> 
> -- 
> Kees Cook

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

More thoughts on the following example Kees provided: 

> On Jul 17, 2023, at 7:40 PM, Kees Cook  wrote:
>> 
>> The counted_by attribute is used to annotate a Flexible array member on how 
>> many elements it will have.
>> However, if this information can not accurately reflect the real number of 
>> elements for the array allocated, 
>> What’s the purpose of such information? 
> 
> For example, imagine code that allocates space for 100 elements since
> the common case is that the number of elements will grow over time.
> Elements are added as it goes. For example:
> 
> struct grows {
>   int alloc_count;
>   int valid_count;
>   struct element item[] __counted_by(valid_count);
> } *p;
> 
> void something(void)
> {
>   p = malloc(sizeof(*p) + sizeof(*p->item) * 100);
>   p->alloc_count = 100;
>   p->valid_count = 0;
> 
>   /* this loop doesn't check that we don't go over 100. */
>   while (items_to_copy) {
>   struct element *item_ptr = get_next_item();
>   /* __counted_by stays in sync: */
>   p->valid_count++;
>   p->item[p->valid_count - 1] = *item_ptr;
>   }
> }
> 
> We would want to catch cases there p->item[] is accessed with an index
> that is >= p->valid_count, even though the allocation is (currently)
> larger.
> 
> However, if we ever reached valid_count >= alloc_count, we need to trap
> too, since we can still "see" the true allocation size.
> 
> Now, the __alloc_size hint is visible in very few places, so if there is
> a strong reason to do so, I can live with saying that __counted_by takes
> full precedence over __alloc_size. It seems it should be possible to
> compare when both are present, but I can live with __counted_by being
> the universal truth. :)

In the above use case (not sure how popular such user case is?), the major 
questions are:

for one object with flexible array member, 

1. Shall we allow the situation when  the allocated size for the object 
and the number of element for the contained FAM are mismatched?

If the answer to 1 is YES (to support such user cases), then

2.  If there is a mismatch between these two, should the number of element 
impact the allocated
size for the object? (__builtin_object_size())

From the doc of object size checking: 
(https://gcc.gnu.org/onlinedocs/gcc/Object-Size-Checking.html)

=
Built-in Function: size_t __builtin_object_size (const void * ptr, int type)
is a built-in construct that returns a constant number of bytes from ptr to the 
end of the object ptr pointer points to (if known at compile time). To 
determine the sizes of dynamically allocated objects the function relies on the 
allocation functions called to obtain the storage to be declared with the 
alloc_size attribute (see Common Function Attributes). __builtin_object_size 
never evaluates its arguments for side effects. If there are any side effects 
in them, it returns (size_t) -1 for type 0 or 1 and (size_t) 0 for type 2 or 3. 
If there are multiple objects ptr can point to and all of them are known at 
compile time, the returned number is the maximum of remaining byte counts in 
those objects if type & 2 is 0 and minimum if nonzero. If it is not possible to 
determine which objects ptr points to at compile time, __builtin_object_size 
should return (size_t) -1 for type 0 or 1 and (size_t) 0 for type 2 or 3.

=

Based on the current documentation for __bos, I think that the answer should be 
NO, i.e, we should not use the counted_by info to change the REAL allocated 
size for the object. 


3. Then, As pointed out also by Martin, only the bounds check (including  
-Warray-bounds or -fsanitizer=bounds) should be impacted by the counted_by 
information, since these checks are based on the TYPE system, and “counted_by” 
info should be treated as a complement to the TYPE system. 

Let me know your opinions.

Qing

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Jul 18, 2023, at 11:37 AM, Qing Zhao via Gcc-patches 
>  wrote:
> 
> 
> 
>> On Jul 17, 2023, at 7:40 PM, Kees Cook  wrote:
>> 
>> On Mon, Jul 17, 2023 at 09:17:48PM +, Qing Zhao wrote:
>>> 
>>>> On Jul 13, 2023, at 4:31 PM, Kees Cook  wrote:
>>>> 
>>>> In the bug, the problem is that "p" isn't known to be allocated, if I'm
>>>> reading that correctly?
>>> 
>>> I think that the major point in PR109557 
>>> (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109557):
>>> 
>>> for the following pointer p.3_1, 
>>> 
>>> p.3_1 = p;
>>> _2 = __builtin_object_size (p.3_1, 0);
>>> 
>>> Question: why the size of p.3_1 cannot use the TYPE_SIZE of the pointee of 
>>> p when the TYPE_SIZE can be determined at compile time?
>>> 
>>> Answer:  From just knowing the type of the pointee of p, the compiler 
>>> cannot determine the size of the object.  
>> 
>> Why is that? "p" points to "struct P", which has a fixed size. There
>> must be an assumption somewhere that a pointer is allocated, otherwise
>> __bos would almost never work?
> 
> My understanding from the comments in PR109557 was: 
> 
> In general the pointer could point to the first object of an array that has 
> more elements, or to an object of a different type. 
> Without seeing the real allocation to the pointer, the compiler cannot assume 
> that the pointer point to an object that has
> the exact same type as its declaration. 
> 
> Sid and Martin, is the above understand correctly?
> 
> Honestly, I am still not 100% clear on this yet.
> 
> Jakub, Sid and Martin, could  you please explain a little bit more here, 
> especially for the following small example?
> 
> [opc@qinzhao-ol8u3-x86 109557]$ cat t.c
> #include 
> #include 
> struct P {
>  int k;
>  int x[10]; 
> } *p;
> 
> void store(int a, int b) 
> {
>  p = (struct P *)malloc (sizeof (struct P));
>  p->k = a;
>  p->x[b] = 0;
>  assert (__builtin_dynamic_object_size (p->x, 1) == sizeof (int[10]));
>  assert (__builtin_dynamic_object_size (p, 1) == sizeof (struct P));
>  return;
> }
> 
> int main()
> {
>  store(7, 7);
>  assert (__builtin_dynamic_object_size (p->x, 1) == sizeof (int[10]));
>  assert (__builtin_dynamic_object_size (p, 1) == sizeof (struct P));
>  free (p);
> }
> [opc@qinzhao-ol8u3-x86 109557]$ sh t
> /home/opc/Install/latest/bin/gcc -O -fdump-tree-all t.c
> a.out: t.c:21: main: Assertion `__builtin_dynamic_object_size (p->x, 1) == 
> sizeof (int[10])' failed.
> t: line 19: 859944 Aborted (core dumped) ./a.out
> 
> 

A correction to the above compilation option:
/home/opc/Install/latest-d/bin/gcc -O -fstrict-flex-arrays=3 t.c
a.out: t.c:22: main: Assertion `__builtin_dynamic_object_size (p, 1) == sizeof 
(struct P)' failed.
t: line 19: 918833 Aborted (core dumped) ./a.out

(All others keep the same).

Sorry for the mistake.

Qing

> In the above, among the 4 assertions, only the last one failed.
> 
> Why GCC can use the TYPE_SIZE of the pointee of the pointer “p->x” as the 
> size of the object, 
> but cannot use the TYPE_SIZE of the pointee of the pointer “p” as the size of 
> the object? 
> 
> 
>> 
>>> Therefore the bug has been closed. 
>>> 
>>> In your following testing 5:
>>> 
>>>> I'm not sure this is a reasonable behavior, but
>>>> let me get into the specific test, which looks like this:
>>>> 
>>>> TEST(counted_by_seen_by_bdos)
>>>> {
>>>>  struct annotated *p;
>>>>  int index = MAX_INDEX + unconst;
>>>> 
>>>>  p = alloc_annotated(index);
>>>> 
>>>>  REPORT_SIZE(p->array);
>>>> /* 1 */ EXPECT_EQ(sizeof(*p), offsetof(typeof(*p), array));
>>>>  /* Check array size alone. */
>>>> /* 2 */ EXPECT_EQ(__builtin_object_size(p->array, 1), SIZE_MAX);
>>>> /* 3 */ EXPECT_EQ(__builtin_dynamic_object_size(p->array, 1), p->foo * 
>>>> sizeof(*p->array));
>>>>  /* Check check entire object size. */
>>>> /* 4 */ EXPECT_EQ(__builtin_object_size(p, 1), SIZE_MAX);
>>>> /* 5 */ EXPECT_EQ(__builtin_dynamic_object_size(p, 1), sizeof(*p) + p->foo 
>>>> * sizeof(*p->array));
>>>> }
>>>> 
>>>> Test 5 should pass as well, since, again, p can be examined. Passing p
>>>> to __bdos implies it is allocated and the __counted_by annotation can be
>>>>

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Jul 18, 2023, at 12:03 PM, Martin Uecker  wrote:
> 
> Am Dienstag, dem 18.07.2023 um 15:37 + schrieb Qing Zhao:
>> 
>> 
>>> On Jul 17, 2023, at 7:40 PM, Kees Cook 
>>> wrote:
>>> 
>>> On Mon, Jul 17, 2023 at 09:17:48PM +, Qing Zhao wrote:
 
> On Jul 13, 2023, at 4:31 PM, Kees Cook 
> wrote:
> 
> In the bug, the problem is that "p" isn't known to be
> allocated, if I'm
> reading that correctly?
 
 I think that the major point in PR109557
 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109557):
 
 for the following pointer p.3_1, 
 
 p.3_1 = p;
 _2 = __builtin_object_size (p.3_1, 0);
 
 Question: why the size of p.3_1 cannot use the TYPE_SIZE of the
 pointee of p when the TYPE_SIZE can be determined at compile
 time?
 
 Answer:  From just knowing the type of the pointee of p, the
 compiler cannot determine the size of the object.  
>>> 
>>> Why is that? "p" points to "struct P", which has a fixed size.
>>> There
>>> must be an assumption somewhere that a pointer is allocated,
>>> otherwise
>>> __bos would almost never work?
>> 
>> My understanding from the comments in PR109557 was: 
>> 
>> In general the pointer could point to the first object of an array
>> that has more elements, or to an object of a different type. 
>> Without seeing the real allocation to the pointer, the compiler
>> cannot assume that the pointer point to an object that has
>> the exact same type as its declaration. 
>> 
>> Sid and Martin, is the above understand correctly?
> 
> Yes. 
> 
> In the example, it *could* work because the compiler
> could inline 'store' or otherwise use its knowledge about
> the function definition to conclude that 'p' points to
> an object of size 'sizeof (struct P)'. But this is fragile
> because it relies on optimization and will not work across
> TUs.
> 
>> Honestly, I am still not 100% clear on this yet.
> 
>> Jakub, Sid and Martin, could  you please explain a little bit more
>> here, especially for the following small example?
>> 
>> [opc@qinzhao-ol8u3-x86 109557]$ cat t.c
>> #include 
>> #include 
>> struct P {
>>   int k;
>>   int x[10]; 
>> } *p;
>> 
>> void store(int a, int b) 
>> {
>>   p = (struct P *)malloc (sizeof (struct P));
>>   p->k = a;
>>   p->x[b] = 0;
>>   assert (__builtin_dynamic_object_size (p->x, 1) == sizeof
>> (int[10]));
>>   assert (__builtin_dynamic_object_size (p, 1) == sizeof (struct P));
>>   return;
>> }
>> 
>> int main()
>> {
>>   store(7, 7);
>>   assert (__builtin_dynamic_object_size (p->x, 1) == sizeof
>> (int[10]));
>>   assert (__builtin_dynamic_object_size (p, 1) == sizeof (struct P));
>>   free (p);
>> }
>> [opc@qinzhao-ol8u3-x86 109557]$ sh t
>> /home/opc/Install/latest/bin/gcc -O -fdump-tree-all t.c
>> a.out: t.c:21: main: Assertion `__builtin_dynamic_object_size (p->x,
>> 1) == sizeof (int[10])' failed.
>> t: line 19: 859944 Aborted (core dumped) ./a.out
>> 
>> 
>> In the above, among the 4 assertions, only the last one failed.
> 
> I don't know why this is the case. 
> 
>> 
>> Why GCC can use the TYPE_SIZE of the pointee of the pointer “p->x” as
>> the size of the object, 
> 
> I do not think it can do this in general. Is this how it 
> is implemented?

No. -:)

 I guess that the implementation of this should base on your following case,  
“observed accesses”:
Although I am not 100% sure on the definition of “observed accesses”.

p->x  is an access to the field of the object, so compiler can assume that the 
object exist and have
the type associate with this access?

On the other hand, “p” is just a plain pointer, no observed access.



> Thus would then seem incorrect to me.  
> 
>> but cannot use the TYPE_SIZE of the pointee of the pointer “p” as the
>> size of the object? 
> 
> In general, the type of a pointer does not say anything about the
> object it points to, because you could cast the pointer to a different
> type, pass it around, and then cast it back before use. 

Okay, I see.
> 
> Only observed allocations or observed accesses provide information
> about the type / existence of an object at the corresponding address.

What will be included in “observed accesses”?

> 
> The only other way in C which ensures that a pointer actually points
> to an object of the correct type is 'static':
> 
> void foo(struct P *p[static 1]);

Thanks for the info.

Qing
> 
> 
> 
> Martin
> 
> 
>> 
>>> 
 Therefore the bug has been closed. 
 
 In your following testing 5:
 
> I'm not sure this is a reasonable behavior, but
> let me get into the specific test, which looks like this:
> 
> TEST(counted_by_seen_by_bdos)
> {
>   struct annotated *p;
>   int index = MAX_INDEX + unconst;
> 
>   p = alloc_annotated(index);
> 
>   REPORT_SIZE(p->array);
> /* 1 */ EXPECT_EQ(sizeof(*p), offsetof(typeof(*p), array));
>   /* Check array size alone. */
> /* 2 */ 

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Jul 17, 2023, at 7:40 PM, Kees Cook  wrote:
> 
> On Mon, Jul 17, 2023 at 09:17:48PM +, Qing Zhao wrote:
>> 
>>> On Jul 13, 2023, at 4:31 PM, Kees Cook  wrote:
>>> 
>>> In the bug, the problem is that "p" isn't known to be allocated, if I'm
>>> reading that correctly?
>> 
>> I think that the major point in PR109557 
>> (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109557):
>> 
>> for the following pointer p.3_1, 
>> 
>> p.3_1 = p;
>> _2 = __builtin_object_size (p.3_1, 0);
>> 
>> Question: why the size of p.3_1 cannot use the TYPE_SIZE of the pointee of p 
>> when the TYPE_SIZE can be determined at compile time?
>> 
>> Answer:  From just knowing the type of the pointee of p, the compiler cannot 
>> determine the size of the object.  
> 
> Why is that? "p" points to "struct P", which has a fixed size. There
> must be an assumption somewhere that a pointer is allocated, otherwise
> __bos would almost never work?

My understanding from the comments in PR109557 was: 

In general the pointer could point to the first object of an array that has 
more elements, or to an object of a different type. 
Without seeing the real allocation to the pointer, the compiler cannot assume 
that the pointer point to an object that has
the exact same type as its declaration. 

Sid and Martin, is the above understand correctly?

Honestly, I am still not 100% clear on this yet.

Jakub, Sid and Martin, could  you please explain a little bit more here, 
especially for the following small example?

[opc@qinzhao-ol8u3-x86 109557]$ cat t.c
#include 
#include 
struct P {
  int k;
  int x[10]; 
} *p;

void store(int a, int b) 
{
  p = (struct P *)malloc (sizeof (struct P));
  p->k = a;
  p->x[b] = 0;
  assert (__builtin_dynamic_object_size (p->x, 1) == sizeof (int[10]));
  assert (__builtin_dynamic_object_size (p, 1) == sizeof (struct P));
  return;
}

int main()
{
  store(7, 7);
  assert (__builtin_dynamic_object_size (p->x, 1) == sizeof (int[10]));
  assert (__builtin_dynamic_object_size (p, 1) == sizeof (struct P));
  free (p);
}
[opc@qinzhao-ol8u3-x86 109557]$ sh t
/home/opc/Install/latest/bin/gcc -O -fdump-tree-all t.c
a.out: t.c:21: main: Assertion `__builtin_dynamic_object_size (p->x, 1) == 
sizeof (int[10])' failed.
t: line 19: 859944 Aborted (core dumped) ./a.out


In the above, among the 4 assertions, only the last one failed.

Why GCC can use the TYPE_SIZE of the pointee of the pointer “p->x” as the size 
of the object, 
but cannot use the TYPE_SIZE of the pointee of the pointer “p” as the size of 
the object? 


> 
>> Therefore the bug has been closed. 
>> 
>> In your following testing 5:
>> 
>>> I'm not sure this is a reasonable behavior, but
>>> let me get into the specific test, which looks like this:
>>> 
>>> TEST(counted_by_seen_by_bdos)
>>> {
>>>   struct annotated *p;
>>>   int index = MAX_INDEX + unconst;
>>> 
>>>   p = alloc_annotated(index);
>>> 
>>>   REPORT_SIZE(p->array);
>>> /* 1 */ EXPECT_EQ(sizeof(*p), offsetof(typeof(*p), array));
>>>   /* Check array size alone. */
>>> /* 2 */ EXPECT_EQ(__builtin_object_size(p->array, 1), SIZE_MAX);
>>> /* 3 */ EXPECT_EQ(__builtin_dynamic_object_size(p->array, 1), p->foo * 
>>> sizeof(*p->array));
>>>   /* Check check entire object size. */
>>> /* 4 */ EXPECT_EQ(__builtin_object_size(p, 1), SIZE_MAX);
>>> /* 5 */ EXPECT_EQ(__builtin_dynamic_object_size(p, 1), sizeof(*p) + p->foo 
>>> * sizeof(*p->array));
>>> }
>>> 
>>> Test 5 should pass as well, since, again, p can be examined. Passing p
>>> to __bdos implies it is allocated and the __counted_by annotation can be
>>> examined.
>> 
>> Since the call to the routine “alloc_annotated" cannot be inlined, GCC does 
>> not see any allocation calls for the pointer p.
> 
> Correct.
> 
>> At the same time, due to the same reason as PR109986, GCC cannot determine 
>> the size of the object by just knowing the TYPE_SIZE
>> of the pointee of p.  
> 
> So the difference between test 3 and test 5 is that "p" is explicitly
> dereferenced to find "array", and therefore an assumption is established
> that "p" is allocated?

Yes, this might be the assumption that GCC made  -:)
> 
>> So, this is exactly the same issue as PR109557.  It’s an existing behavior 
>> per the current __buitlin_object_size algorithm.
>> I am still not very sure whether the situation in PR109557 can be improved 
>> or not, but either way, it’s a separate issue.
> 
> Okay, so the issue is one of object allocation visibility (or
> assumptions there in)?

Might be, let’s see what Sid or Martin will say on this.
> 
>> Please see the new testing case I added for PR109557, you will see that the 
>> above case 5 is a similar case as the new testing case in PR109557.
> 
> I will ponder this a bit more to see if I can come up with a useful
> test case to replace the part from "test 5" above.
> 
>>> 
>>> If "return p->array[index];" would be caught by the sanitizer, then
>>> it follows that __builtin_dynamic_object_size(p, 1) 

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Jul 13, 2023, at 4:31 PM, Kees Cook  wrote:
> 
> In the bug, the problem is that "p" isn't known to be allocated, if I'm
> reading that correctly?

I think that the major point in PR109557 
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109557):

for the following pointer p.3_1, 

p.3_1 = p;
_2 = __builtin_object_size (p.3_1, 0);

Question: why the size of p.3_1 cannot use the TYPE_SIZE of the pointee of p 
when the TYPE_SIZE can be determined at compile time?

Answer:  From just knowing the type of the pointee of p, the compiler cannot 
determine the size of the object.  

Therefore the bug has been closed. 

In your following testing 5:

> I'm not sure this is a reasonable behavior, but
> let me get into the specific test, which looks like this:
> 
> TEST(counted_by_seen_by_bdos)
> {
>struct annotated *p;
>int index = MAX_INDEX + unconst;
> 
>p = alloc_annotated(index);
> 
>REPORT_SIZE(p->array);
> /* 1 */ EXPECT_EQ(sizeof(*p), offsetof(typeof(*p), array));
>/* Check array size alone. */
> /* 2 */ EXPECT_EQ(__builtin_object_size(p->array, 1), SIZE_MAX);
> /* 3 */ EXPECT_EQ(__builtin_dynamic_object_size(p->array, 1), p->foo * 
> sizeof(*p->array));
>/* Check check entire object size. */
> /* 4 */ EXPECT_EQ(__builtin_object_size(p, 1), SIZE_MAX);
> /* 5 */ EXPECT_EQ(__builtin_dynamic_object_size(p, 1), sizeof(*p) + p->foo * 
> sizeof(*p->array));
> }
> 
> Test 5 should pass as well, since, again, p can be examined. Passing p
> to __bdos implies it is allocated and the __counted_by annotation can be
> examined.

Since the call to the routine “alloc_annotated" cannot be inlined, GCC does not 
see any allocation calls for the pointer p.
At the same time, due to the same reason as PR109986, GCC cannot determine the 
size of the object by just knowing the TYPE_SIZE
of the pointee of p.  

So, this is exactly the same issue as PR109557.  It’s an existing behavior per 
the current __buitlin_object_size algorithm.
I am still not very sure whether the situation in PR109557 can be improved or 
not, but either way, it’s a separate issue.

Please see the new testing case I added for PR109557, you will see that the 
above case 5 is a similar case as the new testing case in PR109557.

> 
> If "return p->array[index];" would be caught by the sanitizer, then
> it follows that __builtin_dynamic_object_size(p, 1) must also know the
> size. i.e. both must examine "p" and its trailing flexible array and
> __counted_by annotation.
> 
>> 
>> 2. The common issue for  the failed testing 3, 4, 9, 10 is:
>> 
>> for the following annotated structure: 
>> 
>> 
>> struct annotated {
>>unsigned long flags;
>>size_t foo;
>>int array[] __attribute__((counted_by (foo)));
>> };
>> 
>> 
>> struct annotated *p;
>> int index = 16;
>> 
>> p = malloc(sizeof(*p) + index * sizeof(*p->array));  // allocated real size 
>> 
>> p->foo = index + 2;  // p->foo was set by a different value than the real 
>> size of p->array as in test 9 and 10
> 
> Right, tests 9 and 10 are checking that the _smallest_ possible value of
> the array is used. (There are two sources of information: the allocation
> size and the size calculated by counted_by. The smaller of the two
> should be used when both are available.)

The counted_by attribute is used to annotate a Flexible array member on how 
many elements it will have.
However, if this information can not accurately reflect the real number of 
elements for the array allocated, 
What’s the purpose of such information? 

>> or
>> p->foo was not set to any value as in test 3 and 4
> 
> For tests 3 and 4, yes, this was my mistake. I have fixed up the tests
> now. Bill noticed the same issue. Sorry for the think-o!
> 
>> 
>> 
>> i.e, the value of p->foo is NOT synced with the number of elements allocated 
>> for the array p->array.  
>> 
>> I think that this should be considered as an user error, and the 
>> documentation of the attribute should include
>> this requirement.  (In the LLVM’s RFC, such requirement was included in the 
>> programing model: 
>> https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854#maintaining-correctness-of-bounds-annotations-18)
>> 
>> We can add a new warning option -Wcounted-by to report such user error if 
>> needed.
>> 
>> What’s your opinion on this?
> 
> I think it is correct to allow mismatch between allocation and
> counted_by as long as only the least of the two is used.

What’s your mean by “only the least of the two is used”?

> This may be
> desirable in a few situations. One example would be a large allocation
> that is slowly filled up by the program.

So, for such situation, whenever the allocation is filled up, the field that 
hold the “counted_by” attribute should be increased at the same time,
Then, the “counted_by” value always sync with the real allocation. 
> I.e. the counted_by member is
> slowly increased during runtime (but not beyond the true 

[PATCH] gcc-14/changes.html: Deprecate a GCC C extension on flexible array members.

[Qing Zhao via Gcc-patches]

Hi,

This is the change for the GCC14 releaes Notes on the deprecating of a C
extension about flexible array members.

Okay for committing?

thanks.

Qing



*htdocs/gcc-14/changes.html (Caveats): Add notice about deprecating a C
extension about flexible array members.
---
 htdocs/gcc-14/changes.html | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/htdocs/gcc-14/changes.html b/htdocs/gcc-14/changes.html
index 3f797642..c7f2ce4d 100644
--- a/htdocs/gcc-14/changes.html
+++ b/htdocs/gcc-14/changes.html
@@ -30,7 +30,15 @@ a work-in-progress.
 
 Caveats
 
-  ...
+  C:
+  Support for the GCC extension, a structure containing a C99 flexible 
array
+  member, or a union containing such a structure, is not the last field of
+  another structure, is deprecated. Refer to
+  https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html;>
+  Zero Length Arrays.
+  Any code relying on this extension should be modifed to ensure that
+  C99 flexible array members only end up at the ends of structures.
+  
 
 
 
-- 
2.31.1

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

The following is the updated documentation on this new attribute, please let me 
know any suggestion and comment:

==

'counted_by (COUNT)'
 The 'counted_by' attribute may be attached to the flexible array
 member of a structure.  It indicates that the number of the
 elements of the array is given by the field named "COUNT" in the
 same structure as the flexible array member.  GCC uses this
 information to improve the results of
 '__builtin_dynamic_object_size' and array bound sanitizer.

 For instance, the following declaration:

  struct P {
size_t count;
int array[] __attribute__ ((counted_by (count)));
  } *p;

 specifies that the 'array' is a flexible array member whose number
 of elements is given by the field 'count' in the same structure.

 The field that represents the number of the elements should have an
 integer type.  An explicit 'counted_by' annotation defines a
 relationship between two objects, 'p->array' and 'p->count', that
 'p->array' has 'p->count' number of elements available.  This
 relationship must hold even after any of these related objects are
 updated.  It's the user's responsibility to make sure this
 relationship to be kept all the time.  Otherwise the results of the
 '__builtin_dynamic_object_size' and array bound sanitizer might be
 incorrect.

 For instance, the following 2nd update to the field 'count' of the
 above structure will permit out-of-bounds access to the array
 'sbuf>array':

  struct P *sbuf;
  void alloc_buf (size_t nelems)
  {
sbuf = (int *) malloc (sizeof (struct P) + sizeof (int) * nelems);
sbuf->count = nelems;
  }
  void use_buf (int index)
  {
sbuf->count++;
/* Now the value of sbuf->count and the number
   of elements of sbuf->array is out of sync.  */
sbuf->array[index] = 0;
/* then the out-of-bound access to this array
   might not be detected.  */
  }

 The users can use the warning option '-Wcounted-by-attribute' to
 detect such user errors during compilation time, or the sanitizer
 option '-fsanitize=counted-by-attribute' to detect such user errors
 during runtime.

=

Qing

> On Jul 7, 2023, at 11:47 AM, Qing Zhao via Gcc-patches 
>  wrote:
> 
> 
> 
>> On Jul 6, 2023, at 5:10 PM, Martin Uecker  wrote:
>> 
>> Am Donnerstag, dem 06.07.2023 um 18:56 + schrieb Qing Zhao:
>>> Hi, Kees,
>>> 
>>> I have updated my V1 patch with the following changes:
>>> A. changed the name to "counted_by"
>>> B. changed the argument from a string to an identifier
>>> C. updated the documentation and testing cases accordingly.
>>> 
>>> And then used this new gcc to test 
>>> https://github.com/kees/kernel-tools/blob/trunk/fortify/array-bounds.c 
>>> (with the following change)
>>> [opc@qinzhao-ol8u3-x86 Kees]$ !1091
>>> diff array-bounds.c array-bounds.c.org
>>> 32c32
>>> < # define __counted_by(member) __attribute__((counted_by (member)))
>>> ---
>>>> # define __counted_by(member)  
>>>> __attribute__((__element_count__(#member)))
>>> 34c34
>>> < # define __counted_by(member)   __attribute__((counted_by (member)))
>>> ---
>>>> # define __counted_by(member)  /* 
>>>> __attribute__((__element_count__(#member))) */
>>> 
>>> Then I got the following result:
>>> [opc@qinzhao-ol8u3-x86 Kees]$ ./array-bounds 2>&1 | grep -v ^'#'
>>> TAP version 13
>>> 1..12
>>> ok 1 global.fixed_size_seen_by_bdos
>>> ok 2 global.fixed_size_enforced_by_sanitizer
>>> not ok 3 global.unknown_size_unknown_to_bdos
>>> not ok 4 global.unknown_size_ignored_by_sanitizer
>>> ok 5 global.alloc_size_seen_by_bdos
>>> ok 6 global.alloc_size_enforced_by_sanitizer
>>> not ok 7 global.element_count_seen_by_bdos
>>> ok 8 global.element_count_enforced_by_sanitizer
>>> not ok 9 global.alloc_size_with_smaller_element_count_seen_by_bdos
>>> not ok 10 global.alloc_size_with_smaller_element_count_enforced_by_sanitizer
>>> ok 11 global.alloc_size_with_bigger_element_count_seen_by_bdos
>>> ok 12 global.alloc_size_with_bigger_element_count_enforced_by_sanitizer
>>> 
>>> The same as your previous results. Then I took a look at all the failed 
>>> testing: 3, 4, 7, 9, and 10. And studied the reasons for all of them.
>>> 
>>> in a summary, there are two major issues:
>>> 1.  The reason for the failed 

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

> On Jul 6, 2023, at 5:10 PM, Martin Uecker  wrote:
> 
> Am Donnerstag, dem 06.07.2023 um 18:56 + schrieb Qing Zhao:
>> Hi, Kees,
>> 
>> I have updated my V1 patch with the following changes:
>> A. changed the name to "counted_by"
>> B. changed the argument from a string to an identifier
>> C. updated the documentation and testing cases accordingly.
>> 
>> And then used this new gcc to test 
>> https://github.com/kees/kernel-tools/blob/trunk/fortify/array-bounds.c (with 
>> the following change)
>> [opc@qinzhao-ol8u3-x86 Kees]$ !1091
>> diff array-bounds.c array-bounds.c.org
>> 32c32
>> < # define __counted_by(member)  __attribute__((counted_by (member)))
>> ---
>>> # define __counted_by(member)   
>>> __attribute__((__element_count__(#member)))
>> 34c34
>> < # define __counted_by(member)   __attribute__((counted_by (member)))
>> ---
>>> # define __counted_by(member)   /* 
>>> __attribute__((__element_count__(#member))) */
>> 
>> Then I got the following result:
>> [opc@qinzhao-ol8u3-x86 Kees]$ ./array-bounds 2>&1 | grep -v ^'#'
>> TAP version 13
>> 1..12
>> ok 1 global.fixed_size_seen_by_bdos
>> ok 2 global.fixed_size_enforced_by_sanitizer
>> not ok 3 global.unknown_size_unknown_to_bdos
>> not ok 4 global.unknown_size_ignored_by_sanitizer
>> ok 5 global.alloc_size_seen_by_bdos
>> ok 6 global.alloc_size_enforced_by_sanitizer
>> not ok 7 global.element_count_seen_by_bdos
>> ok 8 global.element_count_enforced_by_sanitizer
>> not ok 9 global.alloc_size_with_smaller_element_count_seen_by_bdos
>> not ok 10 global.alloc_size_with_smaller_element_count_enforced_by_sanitizer
>> ok 11 global.alloc_size_with_bigger_element_count_seen_by_bdos
>> ok 12 global.alloc_size_with_bigger_element_count_enforced_by_sanitizer
>> 
>> The same as your previous results. Then I took a look at all the failed 
>> testing: 3, 4, 7, 9, and 10. And studied the reasons for all of them.
>> 
>>  in a summary, there are two major issues:
>> 1.  The reason for the failed testing 7 is the same issue as I observed in 
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109557
>> Which is not a bug, it’s an expected behavior. 
>> 
>> 2. The common issue for  the failed testing 3, 4, 9, 10 is:
>> 
>> for the following annotated structure: 
>> 
>> 
>> struct annotated {
>> unsigned long flags;
>> size_t foo;
>> int array[] __attribute__((counted_by (foo)));
>> };
>> 
>> 
>> struct annotated *p;
>> int index = 16;
>> 
>> p = malloc(sizeof(*p) + index * sizeof(*p->array));  // allocated real size 
>> 
>> p->foo = index + 2;  // p->foo was set by a different value than the real 
>> size of p->array as in test 9 and 10
>> or
>> p->foo was not set to any value as in test 3 and 4
>> 
>> 
>> 
>> i.e, the value of p->foo is NOT synced with the number of elements allocated 
>> for the array p->array.  
>> 
>> I think that this should be considered as an user error, and the 
>> documentation of the attribute should include
>> this requirement.  (In the LLVM’s RFC, such requirement was included in the 
>> programing model: 
>> https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854#maintaining-correctness-of-bounds-annotations-18)
>> 
>> We can add a new warning option -Wcounted-by to report such user error if 
>> needed.
>> 
>> What’s your opinion on this?
> 
> 
> Additionally, we could also have a sanitizer that
> checks this at run-time.

Yes, that’s also a nice feature to have. 
I think that the main point here is to catch such user errors during 
compilation time or run time. 

I will add one or two separate patches for these compilation warning and 
sanitizer feature.


> 
> Personally, I am still not very happy that in the
> following example the two 'n's refer to different
> entities:
> 
> void f(int n)
> {
>struct foo {
>int n;   
>int (*p[])[n] [[counted_by(n)]];
>};
> }
> 
Me either )-:


> But I guess it will be difficult to convince everybody
> that it would be wise to use a new syntax for
> disambiguation:
> 
> void f(int n)
> {
>struct foo {
>int n;   
>int (*p[])[n] [[counted_by(.n)]];
>};
> }
> 

I guess that it’s quite hard to convince everyone that the new syntax is the 
best solution at this moment. 
And it might not worth the effort at this time.

We can do the new syntax later if necessary.

thanks.

Qing

> Martin
> 
> 
>> 
>> thanks.
>> 
>> Qing
>> 
>> 
>>> On May 26, 2023, at 4:40 PM, Kees Cook  wrote:
>>> 
>>> On Thu, May 25, 2023 at 04:14:47PM +, Qing Zhao wrote:
 GCC will pass the number of elements info from the attached attribute to 
 both 
 __builtin_dynamic_object_size and bounds sanitizer to check the 
 out-of-bounds
 or dynamic object size issues during runtime for flexible array members.
 
 This new feature will provide nice protection to flexible array members 
 (which
 currently are completely ignored by both __builtin_dynamic_object_size and
 bounds 

Re: [V1][PATCH 0/3] New attribute "element_count" to annotate bounds for C99 FAM(PR108896)

[Qing Zhao via Gcc-patches]

Hi, Kees,

I have updated my V1 patch with the following changes:
A. changed the name to "counted_by"
B. changed the argument from a string to an identifier
C. updated the documentation and testing cases accordingly.

And then used this new gcc to test 
https://github.com/kees/kernel-tools/blob/trunk/fortify/array-bounds.c (with 
the following change)
[opc@qinzhao-ol8u3-x86 Kees]$ !1091
diff array-bounds.c array-bounds.c.org
32c32
< # define __counted_by(member) __attribute__((counted_by (member)))
---
> # define __counted_by(member) __attribute__((__element_count__(#member)))
34c34
< # define __counted_by(member)   __attribute__((counted_by (member)))
---
> # define __counted_by(member) /* __attribute__((__element_count__(#member))) 
> */

Then I got the following result:
[opc@qinzhao-ol8u3-x86 Kees]$ ./array-bounds 2>&1 | grep -v ^'#'
TAP version 13
1..12
ok 1 global.fixed_size_seen_by_bdos
ok 2 global.fixed_size_enforced_by_sanitizer
not ok 3 global.unknown_size_unknown_to_bdos
not ok 4 global.unknown_size_ignored_by_sanitizer
ok 5 global.alloc_size_seen_by_bdos
ok 6 global.alloc_size_enforced_by_sanitizer
not ok 7 global.element_count_seen_by_bdos
ok 8 global.element_count_enforced_by_sanitizer
not ok 9 global.alloc_size_with_smaller_element_count_seen_by_bdos
not ok 10 global.alloc_size_with_smaller_element_count_enforced_by_sanitizer
ok 11 global.alloc_size_with_bigger_element_count_seen_by_bdos
ok 12 global.alloc_size_with_bigger_element_count_enforced_by_sanitizer

The same as your previous results. Then I took a look at all the failed 
testing: 3, 4, 7, 9, and 10. And studied the reasons for all of them.

 in a summary, there are two major issues:
1.  The reason for the failed testing 7 is the same issue as I observed in 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109557
Which is not a bug, it’s an expected behavior. 

2. The common issue for  the failed testing 3, 4, 9, 10 is:

for the following annotated structure: 


struct annotated {
unsigned long flags;
size_t foo;
int array[] __attribute__((counted_by (foo)));
};


struct annotated *p;
int index = 16;

p = malloc(sizeof(*p) + index * sizeof(*p->array));  // allocated real size 

p->foo = index + 2;  // p->foo was set by a different value than the real size 
of p->array as in test 9 and 10
or
p->foo was not set to any value as in test 3 and 4



i.e, the value of p->foo is NOT synced with the number of elements allocated 
for the array p->array.  

I think that this should be considered as an user error, and the documentation 
of the attribute should include
this requirement.  (In the LLVM’s RFC, such requirement was included in the 
programing model: 
https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854#maintaining-correctness-of-bounds-annotations-18)

We can add a new warning option -Wcounted-by to report such user error if 
needed.

What’s your opinion on this?

thanks.

Qing


> On May 26, 2023, at 4:40 PM, Kees Cook  wrote:
> 
> On Thu, May 25, 2023 at 04:14:47PM +, Qing Zhao wrote:
>> GCC will pass the number of elements info from the attached attribute to 
>> both 
>> __builtin_dynamic_object_size and bounds sanitizer to check the out-of-bounds
>> or dynamic object size issues during runtime for flexible array members.
>> 
>> This new feature will provide nice protection to flexible array members 
>> (which
>> currently are completely ignored by both __builtin_dynamic_object_size and
>> bounds sanitizers).
> 
> Testing went pretty well, though I think I found some bdos issues:
> 
> - some things that bdos can't know the size of, and correctly returned
>  SIZE_MAX in the past, now thinks are 0-sized.
> - while bdos correctly knows the size of an element_count-annotated
>  flexible array, it doesn't know the size of the containing object
>  (i.e. it returns SIZE_MAX).
> 
> Also, I think I found a precedence issue:
> 
> - if both __alloc_size and 'element_count' are in use, the _smallest_
>  of the two is what I would expect to be enforced by the sanitizer
>  and reported by __bdos. As is, alloc_size appears to be used when
>  it is available, regardless of what 'element_count' shows.
> 
> I've updated my test cases to show it more clearly, but here is the
> before/after:
> 
> 
> GCC 13 (correctly does not implement "element_count"):
> 
> $ ./array-bounds 2>&1 | grep -v ^'#'
> TAP version 13
> 1..12
> ok 1 global.fixed_size_seen_by_bdos
> ok 2 global.fixed_size_enforced_by_sanitizer
> ok 3 global.unknown_size_unknown_to_bdos
> ok 4 global.unknown_size_ignored_by_sanitizer
> ok 5 global.alloc_size_seen_by_bdos
> ok 6 global.alloc_size_enforced_by_sanitizer
> not ok 7 global.element_count_seen_by_bdos
> not ok 8 global.element_count_enforced_by_sanitizer
> not ok 9 global.alloc_size_with_smaller_element_count_seen_by_bdos
> not ok 10 global.alloc_size_with_smaller_element_count_enforced_by_sanitizer
> ok 11 global.alloc_size_with_bigger_element_count_seen_by_bdos
> ok 12 

Re: [PATCH] Introduce hardbool attribute for C

[Qing Zhao via Gcc-patches]

Hi, ALexandre,

Thank you for the explanation.
I am now clear with the interaction between hardbool and 
-ftrivial-auto-var-init, and also agree
that clarifying the documentation on their interaction is good enough. 

Qing

> On Jun 29, 2023, at 6:30 AM, Alexandre Oliva  wrote:
> 
> On Jun 28, 2023, Qing Zhao  wrote:
> 
>> In summary, Ada’s Boolean variables (whether it’s hardened or not) are
>> represented as
>> enumeration types in GNU IR.
> 
> Not quite.  Only boolean types with representation clauses are.  Those
> without (such as Standard.Boolean) are BOOLEAN_TYPEs.  But those without
> a representation clause are not so relevant and could be disregarded,
> for purposes of this conversation.
> 
>> FE takes care of the converting between non-boolean_type_node
>> enumeration types and boolean_type_node as needed, no special handling
>> in Middle end.
> 
>> So, is this exactly the same situation as the new hardbool attribute
>> for C being implemented in this patch?
> 
> That's correct.
> 
>> (Another question, for Ada’s Boolean variables, does the ada FE also
>> insert BUILT_IN_TRAP when
>>  The value is neither true_value nor false_value?)
> 
> Ada raises exceptions when validity checking fails; such as upon using a
> boolean variable with a representation clause holds a value that is
> neither true nor false.
> 
>>> The middle-end doesn't know (and ATM cannot know) that those represented
>>> as enumeration types are conceptually booleans, so they are treated as
>>> enumeration types, not as booleans.
> 
>> They should know it’s a boolean if using the lookup_attribute to get
>> the attribute info -:)
> 
> I meant boolean types that have a representation clause but are not
> hardbools.  Those don't have any attribute whatsoever.
> 
>>> You mean more than what's in the patch posted last week?
>> No, the updated doc is good I think.
> 
> Great, thanks
> 
>> So, from my current understanding, a summary on my major concern and
>> the possible solution to this concern:
> 
> That was a good summary.
> 
>> Is it necessary to fix such inconsistency?
> 
> I don't think it is even desirable.
> 
> Initialization of static variables is well-defined, one is allowed to
> count on a specific value after initialization, and we have that
> covered.
> 
> Automatic variables, OTOH, when not explicitly initialized, may hold
> undefined, random, even malformed values.  Picking an initializer to
> make them predictable needs not follow the semantics of zero
> initialization for static variables.  =pattern makes it clear that using
> something other than zero initialization is useful to catch errors.  The
> Ada language standard even suggests that compilers may set uninitialized
> variables to out-of-range values so as to catch this sort of error.  So,
> though it might seem desirable, for symmetry, to have automatic
> variables implicitly initialized similarly to static variables, it's not
> clear that doing so serves a useful purpose, at least for such types as
> hardened booleans, that are *intended* to catch malformed values.
> 
> -- 
> Alexandre Oliva, happy hackerhttps://FSFLA.org/blogs/lxo/
>   Free Software Activist   GNU Toolchain Engineer
> Disinformation flourishes because many people care deeply about injustice
> but very few check the facts.  Ask me about 

Re: [PATCH] Introduce hardbool attribute for C

[Qing Zhao via Gcc-patches]

> On Jun 28, 2023, at 3:26 AM, Alexandre Oliva  wrote:
> 
> I'd probably have arranged for the front-end to create the initializer
> value, because expansion time is too late to figure it out: we may not
> even have the front-end at hand any more, in case of lto compilation.
>>> 
 Is the hardbool attribute information available during the rtl expansion 
 phase?
>>> 
>>> It is in the sense that the attribute lives on, but c_hardbool_type_attr
>>> is a frontend function, it cannot be called from e.g. lto1.
>> does lookup_attribute work for this attribute during rtl expansion?
>> (Still a little confusing here)
> 
> Yes, the hardbool attribute would be there in C.
> But not in Ada.
> 
> And that should be fine, because Ada hardbool is handled entirely in the
> frontend, as are non-hardened booleans with representation clauses, that
> become enumeration types without any distinguishing feature.
> 
>>> The hardbool attribute is also implemented in Ada, but there it only
>>> affects validity checking in the front end: Boolean types in Ada are
>>> Enumeration types, and there is standard syntax to specify the
>>> representations for true and false.  AFAICT, once we translate GNAT IR
>>> to GNU IR, hardened booleans would not be recognizable as boolean types.
>>> Even non-hardened booleans with representation clauses would.
> 
>> So, right now, the GNU IR represents Ada’s boolean type as enumeration type? 
> 
> All Ada boolean types are defined by the language as enumeration types:
> 
>  There is a predefined enumeration type named Boolean, [declared in the
>  visible part of package Standard].  It has the two enumeration
>  literals False and True ordered with the relation False < True.  Any
>  descendant of the predefined type Boolean is called a boolean type.
> 
> However, boolean types without representation clauses are mapped to the
> language-independent boolean_type_node.  Those that do are mapped to
> enumeration types.

>>> So
>>> handling these differently from other enumeration types, to make them
>>> closer to booleans, would be a bit of a challenge,
> 
>> is there any special handling in GNU IR when representing Ada’s
>> boolean type as enumeration type?
>> Any issue right now?
> 
> Not that I'm aware of.  The front end takes care of converting between
> non-boolean_type_node enumeration types and boolean_type_node as needed,
> so that the GNU IR needs no extra information.

In summary, Ada’s Boolean variables (whether it’s hardened or not) are 
represented as
enumeration types in GNU IR. FE takes care of the converting between 
non-boolean_type_node enumeration types and boolean_type_node as needed, 
no special handling in Middle end. 

So, is this exactly the same situation as the new hardbool attribute for C 
being implemented in 
this patch?

(Another question, for Ada’s Boolean variables, does the ada FE also insert 
BUILT_IN_TRAP when
  The value is neither true_value nor false_value?)

>>> and a
>>> backwards-compatibility issue (because such booleans have already been
>>> handled in the present way since the introduction of -ftrivial-* back in
>>> GCC12)
> 
>> With the new hardbool attribute added for C, an original bool type
>> becomes an enumeration type logically,
> 
> There's no change to the original bool type.
> 
> Only hardbool types are represented as enumeration types in C.
> 
> In Ada, boolean types with representation clauses are still represented
> as enumeration types, whether or not they're hardbool.
> 
>> But such information is not passed to middle end through GNU IR, So,
>> in GCC middle-end, We still treat such type as boolean, not an
>> enumeration type.
> 
> The middle-end doesn't know (and ATM cannot know) that those represented
> as enumeration types are conceptually booleans, so they are treated as
> enumeration types, not as booleans.
They should know it’s a boolean if using the lookup_attribute to get the 
attribute info -:)
> 
>>> static hbool zeroinit; /* False, stored as (char)0x5a.  */
>>> auto hbool uninit; /* Undefined, may trap.  */
> 
>> For the hardbool variable "uninit", -ftrivial-auto-var-init=zero will
>> initialize it to zero, and it will trap during runtime.
>> And at the same time, -ftrivial-auto-var-init=pattern will initialize
>> it to 0xfe, and it will trap during runtime, too.
> 
>> I think these are good behaviors, just need to be documented. 
> 
> You mean more than what's in the patch posted last week?
No, the updated doc is good I think.

> 
>>> 
 And this is a very reasonable initial value for Boolean variables,
>>> 
>>> Agreed.  The all-zeros bit pattern is not so great for booleans that use
>>> alternate representations, though, such as the following standard Ada:
>>> 
>>> type MyBool is new Boolean;
>>> for MyBool use (16#5a#, 16#a5#);
>>> for MyBool'Size use 8;
>>> 
>>> or for biased variables such as:
>>> 
>>> X : Integer range 254 .. 507;
>>> for X'Size use 8; -- bits, so a biased representation is required.

Re: [PATCH v3] Introduce strub: machine-independent stack scrubbing

[Qing Zhao via Gcc-patches]

Hi, Alexandre,

Thanks a lot for the work. I think that this will be a valuable feature to be 
added for GCC’s security functionality. 

I have several questions on this patch:


1.  The implementation of register scrubbing,  -fzero-call-used-regs,  is to 
insert the register zeroing sequence in the routine’s epilogue,
So each routine will be responsible to clean its own call-clobbered registers 
before returning. 
 This is simple and straightforward, no change to the function’s interface.

 I am wondering why stack scrubbing, proposed in this patch series, cannot 
do the stack scrubbing in the routine’s epilogue similar as
register scrubbing?

 There are the following benefits from doing the stack scrubbing in the 
callee’s epilogue:
  A.  The size of the stack need to be cleaned is known by itself, no need 
to pass this information to other routines,
Therefore  functions' interface change can be avoided; no need to 
change the caller’s body, no need for cloning the callee, etc.
  B.   As a result, the runtime overhead of stack scrubbing should be 
reduced.
  C.  If we do the stack scrubbing in a very late stage and in the 
routine’s epilogue, similar as register scrubbing, we don’t need 
to deal with the complicated call-chain staff anymore, right?


   So, what’s the fundamental issues that stack scrubbing cannot be done by the 
routine itself but its caller?

2.  I have concerns on the runtime performance overhead, do you have any data 
on this for your current implementation?

3. You mentioned that there are several “modes” for this feature, could you 
please provide more details on the modes and their description?

thanks.

Qing



> On Jun 16, 2023, at 2:09 AM, Alexandre Oliva via Gcc-patches 
>  wrote:
> 
> 
> This patch adds the strub attribute for function and variable types,
> command-line options, passes and adjustments to implement it,
> documentation, and tests.
> 
> Stack scrubbing is implemented in a machine-independent way: functions
> with strub enabled are modified so that they take an extra stack
> watermark argument, that they update with their stack use, and the
> caller can then zero it out once it regains control, whether by return
> or exception.  There are two ways to go about it: at-calls, that
> modifies the visible interface (signature) of the function, and
> internal, in which the body is moved to a clone, the clone undergoes
> the interface change, and the function becomes a wrapper, preserving
> its original interface, that calls the clone and then clears the stack
> used by it.
> 
> Variables can also be annotated with the strub attribute, so that
> functions that read from them get stack scrubbing enabled implicitly,
> whether at-calls, for functions only usable within a translation unit,
> or internal, for functions whose interfaces must not be modified.
> 
> There is a strict mode, in which functions that have their stack
> scrubbed can only call other functions with stack-scrubbing
> interfaces, or those explicitly marked as callable from strub
> contexts, so that an entire call chain gets scrubbing, at once or
> piecemeal depending on optimization levels.  In the default mode,
> relaxed, this requirement is not enforced by the compiler.
> 
> The implementation adds two IPA passes, one that assigns strub modes
> early on, another that modifies interfaces and adds calls to the
> builtins that jointly implement stack scrubbing.  Another builtin,
> that obtains the stack pointer, is added for use in the implementation
> of the builtins, whether expanded inline or called in libgcc.
> 
> There are new command-line options to change operation modes and to
> force the feature disabled; it is enabled by default, but it has no
> effect and is implicitly disabled if the strub attribute is never
> used.  There are also options meant to use for testing the feature,
> enabling different strubbing modes for all (viable) functions.
> 
> Regstrapped on x86_64-linux-gnu.  Also tested with gcc-13, and with
> various other targets.  Ok to install?
> 
> There have been only minor changes since v2:
> 
> - scrub the stack in the same direction it grows, inline and out-of-line
> 
> - remove need for stack space in __strub_leave
> 
> - add (ultimately not needed) means to avoid using the red zone in
>  __strub_leave
> 
> - introduce and document TARGET_ macros to tune __strub_leave
> 
> - drop a misoptimization in inlined __strub_enter
> 
> - fix handling of cgraph edges without call stmts
> 
> - adjust some testcases (async stack uses; Ada compiler bug fix)
> 
> - drop bits for compatibility with gcc 10
> 
> - preserve the comdat group when resetting a function into a strub
>  wrapper, coping with a symtab_node::reset change in gcc-13
> 
> 
> for  gcc/ChangeLog
> 
>   * Makefile.in (OBJS): Add ipa-strub.o.
>   * builtins.def (BUILT_IN_STACK_ADDRESS): New.
>   (BUILT_IN___STRUB_ENTER): New.
>   (BUILT_IN___STRUB_UPDATE): New.
>   

Re: [V1][PATCH 1/3] Provide element_count attribute to flexible array member field (PR108896)

[Qing Zhao via Gcc-patches]

Hi,

Based on the discussion so far and further consideration, the following is my 
plan for this new attribute:

1.  The syntax of the new attribute will be:

__attribute__((counted_by (count_field_id)));

In the above, count_field_id is the identifier for the field that carries the 
number 
of elements info in the same structure of the FAM. 

For example:

struct object {
..
size_t count:  /* carries the number of elements info for the FAM flex.  */
int flex[] __attribute__((counted_by (count)));
};

2.  Later, if the argument of the this attribute need to be extended to an 
expression, we might need to 
extend the C FE to accept ".count”  in the future. 

Let me know if you have further comments and suggestions.

thanks.

Qing

> On Jun 20, 2023, at 3:40 PM, Qing Zhao via Gcc-patches 
>  wrote:
> 
> 
> 
>> On Jun 16, 2023, at 5:35 PM, Joseph Myers  wrote:
>> 
>> On Fri, 16 Jun 2023, Qing Zhao via Gcc-patches wrote:
>> 
>>>> So for 
>>>> 
>>>> struct foo { int c; int buf[(struct { int d; }){ .d = .c }]; };
>>>> 
>>>> one knows during parsing that the .d is a designator
>>>> and that .c is not.
>>> 
>>> Therefore, the above should be invalid based on this rule since .c is 
>>> not a member in the current structure.
>> 
>> What do you mean by "current structure"?  I think two different concepts 
>> are being conflated: the structure *being initialized* (what the C 
>> standard calls the "current object" for a brace-enclosed initializer 
>> list),
> 
> I think the concept of “current structure” should be stick to this. 
> 
>> and the structure *being defined*.
> Not this.
> 
> (Forgive me about my poor English -:)).
> 
> Then it will be cleaner? 
> 
> What’s your opinion?
> 
> 
>> The former is what's relevant 
>> for designators.  The latter is what's relevant for the suggested new 
>> syntax.  And .c *is* a member of the structure being defined in this 
>> example.
>> 
>> Those two structure types are always different, except for corner cases 
>> with C2x tag compatibility (where an object of structure type might be 
>> initialized in the middle of a redefinition of that type).
> 
> Can you give an example on this?  Thanks.
> 
> Qing
>> 
>> -- 
>> Joseph S. Myers
>> jos...@codesourcery.com

Re: [PATCH] Introduce hardbool attribute for C

[Qing Zhao via Gcc-patches]

Hi, Alexandre,

> On Jun 23, 2023, at 10:38 PM, Alexandre Oliva  wrote:
> 
>> For normal Boolean variables, 0x00 is false, this is a reasonable init
>> value with zero-initialization.
> 
> *nod*.  I was surprised by zero initialization of (non-hardened)
> booleans even when pattern is requested, but not consistently
> (e.g. boolean fields of a larger struct would still get
> pattern-initialized IIUC).  I'd have expected pattern would translate to
> nonzero and thus to true, rather than false.

Such inconsistent behavior was introduced by the following commit:

From c081d0a3b0291297f04a05c833d2ffa8de3a7a1a Mon Sep 17 00:00:00 2001
Subject: [PATCH] middle-end/103033 - drop native_interpret_expr with
 .DEFERRED_INIT expansion

> 
>> For hardbool variables, what 0x00 represents if it’s not false or true
>> value?
> 
> It depends on how hardbool is parameterized.  One may pick 0x00 or 0xFE
> as the representations for true or false, or neither, in which case the
> trivial initializer will end up as a trapping value.

Okay, then, this looks like a  good behavior (trapping with 
-ftrival-auto-var-init  most of the time,
 i.e, when neither 0x00 or 0xFE was chosen as the representations for true or 
false), 
it will urge the user to fix the uninitialized hardbool variables.
Do I miss anything here?
> 
>>> I'd probably have arranged for the front-end to create the initializer
>>> value, because expansion time is too late to figure it out: we may not
>>> even have the front-end at hand any more, in case of lto compilation.
> 
>> Is the hardbool attribute information available during the rtl expansion 
>> phase?
> 
> It is in the sense that the attribute lives on, but c_hardbool_type_attr
> is a frontend function, it cannot be called from e.g. lto1.
does lookup_attribute work for this attribute during rtl expansion? (Still a 
little confusing here)
> 
> The hardbool attribute is also implemented in Ada, but there it only
> affects validity checking in the front end: Boolean types in Ada are
> Enumeration types, and there is standard syntax to specify the
> representations for true and false.  AFAICT, once we translate GNAT IR
> to GNU IR, hardened booleans would not be recognizable as boolean types.
> Even non-hardened booleans with representation clauses would.

So, right now, the GNU IR represents Ada’s boolean type as enumeration type? 

>  So
> handling these differently from other enumeration types, to make them
> closer to booleans, would be a bit of a challenge,

is there any special handling in GNU IR when representing Ada’s boolean type as 
enumeration type?
Any issue right now?

> and a
> backwards-compatibility issue (because such booleans have already been
> handled in the present way since the introduction of -ftrivial-* back in
> GCC12)

With the new hardbool attribute added for C, an original bool type becomes an 
enumeration type logically, 
But such information is not passed to middle end through GNU IR, So, in GCC 
middle-end, 
We still treat such type as boolean, not an enumeration type.
Is this understanding correct?

> 
>>> Now, I acknowledge that the decision to make implicit
>>> zero-initialization of boolean types set them to the value for false,
>>> rather than to all-bits-zero representation, is a departure from common
>>> practice of zero-initialization yielding logical zero.
> 
>> Dont’s quite understand the above, for normal boolean variables,
> 
> Sorry, I meant hardened boolean types.  This was WRT to the design
> decision that led to this bit in the documentation:
> 
> typedef char __attribute__ ((__hardbool__ (0x5a))) hbool;
> [...]
> static hbool zeroinit; /* False, stored as (char)0x5a.  */
> auto hbool uninit; /* Undefined, may trap.  */
For the hardbool variable "uninit",  -ftrivial-auto-var-init=zero will 
initialize it to zero, and it will trap during runtime.
And at the same time, -ftrivial-auto-var-init=pattern will initialize it to 
0xfe, and it will trap during runtime, too.

I think these are good behaviors, just need to be documented. 
> 
>> And this is a very reasonable initial value for Boolean variables,
> 
> Agreed.  The all-zeros bit pattern is not so great for booleans that use
> alternate representations, though, such as the following standard Ada:
> 
>   type MyBool is new Boolean;
>   for MyBool use (16#5a#, 16#a5#);
>   for MyBool'Size use 8;
> 
> or for biased variables such as:
> 
>  X : Integer range 254 .. 507;
>  for X'Size use 8; -- bits, so a biased representation is required.
> 
> Just to make things more interesting, I chose a range for X that causes
> the compiler to represent 0xfe as 0x00 in in the byte that holds X, but
> that places the 0xfe pattern just out of the range :-) So with
> -ftrivial-auto-var-init=zero, X = 254, whereas with
> -ftrivial-auto-var-init=pattern, it fails validity checking, and might
> come out as 508 if that's disabled.

 for the biased variable X,  it was initialized to 254 (the smallest valid 
value in the range) when 

Re: [PATCH v3] Add leafy mode for zero-call-used-regs

[Qing Zhao via Gcc-patches]

> On Jun 23, 2023, at 7:27 PM, Alexandre Oliva  wrote:
> 
> On Jun 23, 2023, Qing Zhao via Gcc-patches  wrote:
> 
>> It’s better to add this definition earlier in the list of the “three
>> basic values”, to make it “four basic values”, like the following:
> 
> Oh, my, sorry for being so dense, I had managed to miss that bit all
> this time somehow :-(
> 
>> The sentence "This value is mainly to provide users a more efficient mode to 
>> zero 
>>call-used registers in leaf functions.” just for your reference,
>> the wording can certainly be improved.  -:)
> 
> :-)  got it, thanks.  How about this?
> 
> 
> Add leafy mode for zero-call-used-regs
> 
> Introduce 'leafy' to auto-select between 'used' and 'all' for leaf and
> nonleaf functions, respectively.
> 
> Regstrapped on x86_64-linux-gnu.  Ok to install?

LGTM now.

However, I am not a reviewer,  you might still need approval from a middle-end 
reviewer.

thanks.

Qing


> 
> 
> for  gcc/ChangeLog
> 
>   * doc/extend.texi (zero-call-used-regs): Document leafy and
>   variants thereof.
>   * flag-types.h (zero_regs_flags): Add LEAFY_MODE, as well as
>   LEAFY and variants.
>   * function.cc (gen_call_ued_regs_seq): Set only_used for leaf
>   functions in leafy mode.
>   * opts.cc (zero_call_used_regs_opts): Add leafy and variants.
> 
> for  gcc/testsuite/ChangeLog
> 
>   * c-c++-common/zero-scratch-regs-leafy-1.c: New.
>   * c-c++-common/zero-scratch-regs-leafy-2.c: New.
>   * gcc.target/i386/zero-scratch-regs-leafy-1.c: New.
>   * gcc.target/i386/zero-scratch-regs-leafy-2.c: New.
> ---
> gcc/doc/extend.texi|   30 ++--
> gcc/flag-types.h   |5 +++
> gcc/function.cc|3 ++
> gcc/opts.cc|4 +++
> .../c-c++-common/zero-scratch-regs-leafy-1.c   |   15 ++
> .../c-c++-common/zero-scratch-regs-leafy-2.c   |   21 ++
> .../gcc.target/i386/zero-scratch-regs-leafy-1.c|   12 
> .../gcc.target/i386/zero-scratch-regs-leafy-2.c|   16 +++
> 8 files changed, 103 insertions(+), 3 deletions(-)
> create mode 100644 gcc/testsuite/c-c++-common/zero-scratch-regs-leafy-1.c
> create mode 100644 gcc/testsuite/c-c++-common/zero-scratch-regs-leafy-2.c
> create mode 100644 gcc/testsuite/gcc.target/i386/zero-scratch-regs-leafy-1.c
> create mode 100644 gcc/testsuite/gcc.target/i386/zero-scratch-regs-leafy-2.c
> 
> diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi
> index 852f6b629bea8..739c40368f556 100644
> --- a/gcc/doc/extend.texi
> +++ b/gcc/doc/extend.texi
> @@ -4349,7 +4349,7 @@ through registers.
> In order to satisfy users with different security needs and control the
> run-time overhead at the same time, the @var{choice} parameter provides a
> flexible way to choose the subset of the call-used registers to be zeroed.
> -The three basic values of @var{choice} are:
> +The four basic values of @var{choice} are:
> 
> @itemize @bullet
> @item
> @@ -4362,10 +4362,16 @@ the function.
> 
> @item
> @samp{all} zeros all call-used registers.
> +
> +@item
> +@samp{leafy} behaves like @samp{used} in a leaf function, and like
> +@samp{all} in a nonleaf function.  This makes for leaner zeroing in leaf
> +functions, where the set of used registers is known, and that may be
> +enough for some purposes of register zeroing.
> @end itemize
> 
> In addition to these three basic choices, it is possible to modify
> -@samp{used} or @samp{all} as follows:
> +@samp{used}, @samp{all}, and @samp{leafy} as follows:
> 
> @itemize @bullet
> @item
> @@ -4412,10 +4418,28 @@ zeros all call-used registers that pass arguments.
> @item all-gpr-arg
> zeros all call-used general purpose registers that pass
> arguments.
> +
> +@item leafy
> +Same as @samp{used} in a leaf function, and same as @samp{all} in a
> +nonleaf function.
> +
> +@item leafy-gpr
> +Same as @samp{used-gpr} in a leaf function, and same as @samp{all-gpr}
> +in a nonleaf function.
> +
> +@item leafy-arg
> +Same as @samp{used-arg} in a leaf function, and same as @samp{all-arg}
> +in a nonleaf function.
> +
> +@item leafy-gpr-arg
> +Same as @samp{used-gpr-arg} in a leaf function, and same as
> +@samp{all-gpr-arg} in a nonleaf function.
> +
> @end table
> 
> Of this list, @samp{used-arg}, @samp{used-gpr-arg}, @samp{all-arg},
> -and @samp{all-gpr-arg} are mainly used for ROP mitigation.
> +@samp{all-gpr-arg}, @samp{leafy-arg}, and @samp{leafy-gpr-arg} are
> +mainly used for ROP mitigation.
> 
> The default for the att

Re: [PATCH] Introduce hardbool attribute for C

[Qing Zhao via Gcc-patches]

> On Jun 21, 2023, at 10:35 PM, Alexandre Oliva  wrote:
> 
> On Jun 21, 2023, Qing Zhao  wrote:
> 
>> I see that you have testing case to check the above built_in_trap call
>> is generated by FE.
>> Do you have a testing case to check the trap is happening at runtime? 
> 
> I have written such tests, using type-punning, but I don't think our
> testing infrastructure could take trapping as success and other results
> as failure.
Okay, I see. 
> 
>> So, when -ftrivial-auto-var-init presents, what’s the behavior for the
>> hardened Boolean auto variables?
> 
> Good question.  This option was not even available when hardbool was
> designed and implemented.  (tests) The deferred_init internal function
> initializes with bit-patterns 0x00 or 0xfe, regardless of type, when the
> data lives in memory, and otherwise forces the 0x00 bit pattern for
> booleans, variable-sized types, types that cannot be accessed with a
> single mode or for modes that don't have a set pattern.

-ftrivial-auto-var-init has been in GCC since GCC12.  
The decision to use 0x00 for zero-initiation, 0xfe for pattern-initiation
has been discussed thoroughly during the design phase. -:)

These initial values (0x00 and 0xfe) are not perfect choices, but they the 
best choices based on functionality and performance tradeoff. 

Since this hardbool attribute will also  be a security feature, users that seek
security features of GCC will likely ask the same question for these two
features.

So, their interaction is better to be at least documented. That’s my main point.

> 
> It's hard for me to tell what "correct" or "expected" would be here.
> Enumerators don't seem to be given special treatment.  Checked
> enumerators, constrained integral subtypes, none of these would get
> well-formed values or even checking at the assignments.

> 
> If I were to design this option myself, I'd probably arrange for it to
> handle booleans (including hardened booleans) by zero-initializing as
> false and pattern-initializing as true, though I realize that this could
> be very confusing if one chose to use 0xfe as the value for false and/or
> 0x00 as the value for true.

For normal Boolean variables, 0x00 is false, this is a reasonable init value 
with zero-initialization.
For hardbool variables, what 0x00 represents if it’s not false or true value?  
A garbage value for this hardbool variable?
> 
> I'd probably have arranged for the front-end to create the initializer
> value, because expansion time is too late to figure it out: we may not
> even have the front-end at hand any more, in case of lto compilation.

Is the hardbool attribute information available during the rtl expansion phase?
> 
> 
> But with the current description and implementation, I guess the
> behavior is correct, if not ideal: the bit patterns refer to the
> representation, rather than to the language interpretation of the value.
> When it comes ot integral types, they may match, but floating-point,
> fixed fractional types, offsets and multipliers, even boolean member of
> larger structs...  not so much: the effect is that of a memset, rather
> than that of an assignment of zero or of the pattern to a variable.
> 
> Now, I acknowledge that the decision to make implicit
> zero-initialization of boolean types set them to the value for false,
> rather than to all-bits-zero representation, is a departure from common
> practice of zero-initialization yielding logical zero.

Dont’s quite understand the above, for normal boolean variables, 
zero-initialization set them to false, 
and also set them to all-bits-zero.(i.e, all-bits-zero is equal to logical 
false for normal boolean variable, right?)
And this is a very reasonable initial value for Boolean variables,


>  That was unusual
> enough that I thought it worth mentioning in the docs.
> 
I don’t see why this is unusual for the normal Boolean variables? Could you 
please explain a little bit?

From my understanding, only with the introduction of “hardbool” attribute, 
all-bits-zero will not be equal to the
logical false anymore. 

> 
>> This might need to be documented and also handled correctly. 
> 
> I suppose the place to document this distinction between logical values
> and representation would be under -ftrivial-auto-var-init.

Yes, the documentation of -ftrivial-auto-var-init could be updated with this 
clarification, mainly for
The new “hardbool” attribute. 

>  That's
> likely where someone using that option would look for guidance on how it
> interacts with unusual types, and where exceptions to general
> expectations WRT initialization would go.  Do you concur?
And at the same time, the doc for the new “hardbool” attribute might be better 
to add such warning too?

> 
> That said, it probalby makes sense to refer to / mention that
> -ftrivial-auto-var-init does not special-case hardened booleans in the
> hardened booleans documentation.

Agreed.

thanks.

Qing

>  I wonder if there are other
> conflicting options I'm not even 

Re: [PATCH zero-call-used-regs] Add leafy mode for zero-call-used-regs

[Qing Zhao via Gcc-patches]

Hi, Alexandre,


> On Jun 21, 2023, at 9:16 PM, Alexandre Oliva  wrote:
> 
> Hello, Qing,
> 
> On Jun 16, 2023, Qing Zhao  wrote:
> 
>> As I mentioned in the previous round of review, I think that the 
>> documentation
>> might need to add more details on what’s the LEAFY mode,
>> The purpose of it, and how to use it, provide more details to the end-users.
> 
> I'm afraid I'm having difficulty picturing what it is that you're
> looking for.  The proposal incorporates, by reference, all the
> documentation for 'used' and for 'all', and the way to use it is no
> different.
> 
>>> +Same as @samp{used} in a leaf function, and same as @samp{all} in a
>>> +nonleaf function.

Oh, yeah.  The definition of “leafy” is here. -:)

It’s better to add this definition earlier in the list of the “three basic 
values”, to make it “four basic values”, like the following:

===
In order to satisfy users with different security needs and control the
run-time overhead at the same time, the @var{choice} parameter provides a
flexible way to choose the subset of the call-used registers to be zeroed.
The four basic values of @var{choice} are:

@itemize @bullet
@item
@samp{skip} doesn't zero any call-used registers.

@item
@samp{used} only zeros call-used registers that are used in the function.
A ``used'' register is one whose content has been set or referenced in
the function.

@item
@samp{all} zeros all call-used registers.
@end itemize

@item
@samp{leafy} Same as @samp{used} in a leaf function, and same as @samp{all} in a
nonleaf function. This value is mainly to provide users a more efficient 
mode to zero 
call-used registers in leaf functions.
@end itemize
==

Then,  in the full list of choice, add the new values of leafy, leafy-gpr, 
leafy-arg, leafy-gpr-arg 

The sentence "This value is mainly to provide users a more efficient mode to 
zero 
call-used registers in leaf functions.” just for your reference,  the 
wording can certainly be improved.  -:)
> 
> If there was documentation on how to choose between e.g. all and used, I
> suppose I could build on that to add this intermediate choice, but...  I
> can't find any such docs, and I'm uncertain on whether adding that would
> be useful to begin with.
> 
> Did you have something else in mind?


Hope this time I am clear (and sorry for the confusion in the previous emails).

thanks.

Qing

> 
> -- 
> Alexandre Oliva, happy hackerhttps://FSFLA.org/blogs/lxo/
>   Free Software Activist   GNU Toolchain Engineer
> Disinformation flourishes because many people care deeply about injustice
> but very few check the facts.  Ask me about 

Re: [PATCH] Introduce hardbool attribute for C

[Qing Zhao via Gcc-patches]

Hi, Alexandre,

> 
> diff --git a/gcc/c/c-typeck.cc b/gcc/c/c-typeck.cc
> index 22e240a3c2a55..f9cc609b54d94 100644
> --- a/gcc/c/c-typeck.cc
> +++ b/gcc/c/c-typeck.cc
> @@ -2226,6 +2226,35 @@ convert_lvalue_to_rvalue (location_t loc, struct 
> c_expr exp,
> exp.value = convert (build_qualified_type (TREE_TYPE (exp.value), 
> TYPE_UNQUALIFIED), exp.value);
>   if (force_non_npc)
> exp.value = build1 (NOP_EXPR, TREE_TYPE (exp.value), exp.value);
> +
> +  {
> +tree false_value, true_value;
> +if (convert_p && !error_operand_p (exp.value)
> + && c_hardbool_type_attr (TREE_TYPE (exp.value),
> +  _value, _value))
> +  {
> + tree t = save_expr (exp.value);
> +
> + mark_exp_read (exp.value);
> +
> + tree trapfn = builtin_decl_explicit (BUILT_IN_TRAP);
> + tree expr = build_call_expr_loc (loc, trapfn, 0);
> + expr = build_compound_expr (loc, expr, boolean_true_node);
> + expr = fold_build3_loc (loc, COND_EXPR, boolean_type_node,
> + fold_build2_loc (loc, NE_EXPR,
> +  boolean_type_node,
> +  t, true_value),
> + expr, boolean_true_node);
> + expr = fold_build3_loc (loc, COND_EXPR, boolean_type_node,
> + fold_build2_loc (loc, NE_EXPR,
> +  boolean_type_node,
> +  t, false_value),
> + expr, boolean_false_node);
> +
> + exp.value = expr;
> +  }
> +  }

I see that you have testing case to check the above built_in_trap call is 
generated by FE. 
Do you have a testing case to check the trap is happening at runtime? 
> +
>   return exp;
> }
> 
> @@ -8488,7 +8517,7 @@ digest_init (location_t init_loc, tree type, tree init, 
> tree origtype,
>   }
>   }
> 
> -  if (code == VECTOR_TYPE)
> +  if (code == VECTOR_TYPE || c_hardbool_type_attr (type))
>   /* Although the types are compatible, we may require a
>  conversion.  */
>   inside_init = convert (type, inside_init);
> diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi
> index 2de212c8c2d84..7b5592502734e 100644
> --- a/gcc/doc/extend.texi
> +++ b/gcc/doc/extend.texi
> @@ -8681,6 +8681,58 @@ initialization will result in future breakage.
> GCC emits warnings based on this attribute by default; use
> @option{-Wno-designated-init} to suppress them.
> 
> +@cindex @code{hardbool} type attribute
> +@item hardbool
> +@itemx hardbool (@var{false_value})
> +@itemx hardbool (@var{false_value}, @var{true_value})
> +This attribute may only be applied to integral types in C, to introduce
> +hardened boolean types.  It turns the integral type into a boolean-like
> +type with the same size and precision, that uses the specified values as
> +representations for @code{false} and @code{true}.  Underneath, it is
> +actually an enumerate type, but its observable behavior is like that of
> +@code{_Bool}, except for the strict internal representations, verified
> +by runtime checks.
> +
> +If @var{true_value} is omitted, the bitwise negation of
> +@var{false_value} is used.  If @var{false_value} is omitted, zero is
> +used.  The named representation values must be different when converted
> +to the original integral type.  Narrower bitfields are rejected if the
> +representations become indistinguishable.
> +
> +Values of such types automatically decay to @code{_Bool}, at which
> +point, the selected representation values are mapped to the
> +corresponding @code{_Bool} values.  When the represented value is not
> +determined, at compile time, to be either @var{false_value} or
> +@var{true_value}, runtime verification calls @code{__builtin_trap} if it
> +is neither.  This is what makes them hardened boolean types.
> +
> +When converting scalar types to such hardened boolean types, implicitly
> +or explicitly, behavior corresponds to a conversion to @code{_Bool},
> +followed by a mapping from @code{false} and @code{true} to
> +@var{false_value} and @var{true_value}, respectively.
> +
> +@smallexample
> +typedef char __attribute__ ((__hardbool__ (0x5a))) hbool;
> +hbool first = 0;   /* False, stored as (char)0x5a.  */
> +hbool second = !first; /* True, stored as ~(char)0x5a.  */
> +
> +static hbool zeroinit; /* False, stored as (char)0x5a.  */
> +auto hbool uninit; /* Undefined, may trap.  */
> +@end smallexample
> +
> +When zero-initializing a variable or field of hardened boolean type
> +(presumably held in static storage) the implied zero initializer gets
> +converted to @code{_Bool}, and then to the hardened boolean type, so
> +that the initial value is the hardened representation for @code{false}.
> +Using that value is well defined.  This is @emph{not} the case when
> +variables and fields of such types are uninitialized (presumably held in
> +automatic or dynamic storage): 

Re: [V1][PATCH 1/3] Provide element_count attribute to flexible array member field (PR108896)

[Qing Zhao via Gcc-patches]

> On Jun 16, 2023, at 5:35 PM, Joseph Myers  wrote:
> 
> On Fri, 16 Jun 2023, Qing Zhao via Gcc-patches wrote:
> 
>>> So for 
>>> 
>>> struct foo { int c; int buf[(struct { int d; }){ .d = .c }]; };
>>> 
>>> one knows during parsing that the .d is a designator
>>> and that .c is not.
>> 
>> Therefore, the above should be invalid based on this rule since .c is 
>> not a member in the current structure.
> 
> What do you mean by "current structure"?  I think two different concepts 
> are being conflated: the structure *being initialized* (what the C 
> standard calls the "current object" for a brace-enclosed initializer 
> list),

I think the concept of “current structure” should be stick to this. 

> and the structure *being defined*.
Not this.

(Forgive me about my poor English -:)).

Then it will be cleaner? 

What’s your opinion?


>  The former is what's relevant 
> for designators.  The latter is what's relevant for the suggested new 
> syntax.  And .c *is* a member of the structure being defined in this 
> example.
> 
> Those two structure types are always different, except for corner cases 
> with C2x tag compatibility (where an object of structure type might be 
> initialized in the middle of a redefinition of that type).

Can you give an example on this?  Thanks.

Qing
> 
> -- 
> Joseph S. Myers
> jos...@codesourcery.com

[V10][PATCH 1/3] Introduce IR bit TYPE_INCLUDES_FLEXARRAY for the GCC extension [PR77650]

[Qing Zhao via Gcc-patches]

on a structure with a C99 flexible array member being nested in
another structure

GCC extension accepts the case when a struct with a flexible array member
is embedded into another struct or union (possibly recursively) as the last
field.
This patch is to introduce the IR bit TYPE_INCLUDES_FLEXARRAY (reuse the
existing IR bit TYPE_NO_NAMED_ARGS_SATDARG_P), set it correctly in C FE,
stream it correctly in Middle-end, and print it during IR dumping.

PR C/77650

gcc/c/ChangeLog:

* c-decl.cc (finish_struct): Set TYPE_INCLUDES_FLEXARRAY for
struct/union type.

gcc/lto/ChangeLog:

* lto-common.cc (compare_tree_sccs_1): Compare bit
TYPE_NO_NAMED_ARGS_STDARG_P or TYPE_INCLUDES_FLEXARRAY properly
for its corresponding type.

gcc/ChangeLog:

* print-tree.cc (print_node): Print new bit type_include_flexarray.
* tree-core.h (struct tree_type_common): Use bit no_named_args_stdarg_p
as type_include_flexarray for RECORD_TYPE or UNION_TYPE.
* tree-streamer-in.cc (unpack_ts_type_common_value_fields): Stream
in bit no_named_args_stdarg_p properly for its corresponding type.
* tree-streamer-out.cc (pack_ts_type_common_value_fields): Stream
out bit no_named_args_stdarg_p properly for its corresponding type.
* tree.h (TYPE_INCLUDES_FLEXARRAY): New macro TYPE_INCLUDES_FLEXARRAY.
---
 gcc/c/c-decl.cc  | 11 +++
 gcc/lto/lto-common.cc|  5 -
 gcc/print-tree.cc|  5 +
 gcc/tree-core.h  |  2 ++
 gcc/tree-streamer-in.cc  |  5 -
 gcc/tree-streamer-out.cc |  5 -
 gcc/tree.h   |  7 ++-
 7 files changed, 36 insertions(+), 4 deletions(-)

diff --git a/gcc/c/c-decl.cc b/gcc/c/c-decl.cc
index 1af51c4acfc..e14f514cb6e 100644
--- a/gcc/c/c-decl.cc
+++ b/gcc/c/c-decl.cc
@@ -9267,6 +9267,17 @@ finish_struct (location_t loc, tree t, tree fieldlist, 
tree attributes,
   /* Set DECL_NOT_FLEXARRAY flag for FIELD_DECL x.  */
   DECL_NOT_FLEXARRAY (x) = !is_flexible_array_member_p (is_last_field, x);
 
+  /* Set TYPE_INCLUDES_FLEXARRAY for the context of x, t.
+when x is an array and is the last field.  */
+  if (TREE_CODE (TREE_TYPE (x)) == ARRAY_TYPE)
+   TYPE_INCLUDES_FLEXARRAY (t)
+ = is_last_field && flexible_array_member_type_p (TREE_TYPE (x));
+  /* Recursively set TYPE_INCLUDES_FLEXARRAY for the context of x, t
+when x is an union or record and is the last field.  */
+  else if (RECORD_OR_UNION_TYPE_P (TREE_TYPE (x)))
+   TYPE_INCLUDES_FLEXARRAY (t)
+ = is_last_field && TYPE_INCLUDES_FLEXARRAY (TREE_TYPE (x));
+
   if (DECL_NAME (x)
  || RECORD_OR_UNION_TYPE_P (TREE_TYPE (x)))
saw_named_field = true;
diff --git a/gcc/lto/lto-common.cc b/gcc/lto/lto-common.cc
index 537570204b3..f6b85bbc6f7 100644
--- a/gcc/lto/lto-common.cc
+++ b/gcc/lto/lto-common.cc
@@ -1275,7 +1275,10 @@ compare_tree_sccs_1 (tree t1, tree t2, tree **map)
   if (AGGREGATE_TYPE_P (t1))
compare_values (TYPE_TYPELESS_STORAGE);
   compare_values (TYPE_EMPTY_P);
-  compare_values (TYPE_NO_NAMED_ARGS_STDARG_P);
+  if (FUNC_OR_METHOD_TYPE_P (t1))
+   compare_values (TYPE_NO_NAMED_ARGS_STDARG_P);
+  if (RECORD_OR_UNION_TYPE_P (t1))
+   compare_values (TYPE_INCLUDES_FLEXARRAY);
   compare_values (TYPE_PACKED);
   compare_values (TYPE_RESTRICT);
   compare_values (TYPE_USER_ALIGN);
diff --git a/gcc/print-tree.cc b/gcc/print-tree.cc
index ccecd3dc6a7..62451b6cf4e 100644
--- a/gcc/print-tree.cc
+++ b/gcc/print-tree.cc
@@ -632,6 +632,11 @@ print_node (FILE *file, const char *prefix, tree node, int 
indent,
  && TYPE_CXX_ODR_P (node))
fputs (" cxx-odr-p", file);
 
+  if ((code == RECORD_TYPE
+  || code == UNION_TYPE)
+ && TYPE_INCLUDES_FLEXARRAY (node))
+   fputs (" includes-flexarray", file);
+
   /* The transparent-union flag is used for different things in
 different nodes.  */
   if ((code == UNION_TYPE || code == RECORD_TYPE)
diff --git a/gcc/tree-core.h b/gcc/tree-core.h
index c48a12b378f..668808a29d0 100644
--- a/gcc/tree-core.h
+++ b/gcc/tree-core.h
@@ -1713,6 +1713,8 @@ struct GTY(()) tree_type_common {
   unsigned typeless_storage : 1;
   unsigned empty_flag : 1;
   unsigned indivisible_p : 1;
+  /* TYPE_NO_NAMED_ARGS_STDARG_P for a stdarg function.
+ Or TYPE_INCLUDES_FLEXARRAY for RECORD_TYPE and UNION_TYPE.  */
   unsigned no_named_args_stdarg_p : 1;
   unsigned spare : 1;
 
diff --git a/gcc/tree-streamer-in.cc b/gcc/tree-streamer-in.cc
index c803800862c..be2bdbb7699 100644
--- a/gcc/tree-streamer-in.cc
+++ b/gcc/tree-streamer-in.cc
@@ -386,7 +386,10 @@ unpack_ts_type_common_value_fields (struct bitpack_d *bp, 
tree expr)
   if (AGGREGATE_TYPE_P (expr))
 TYPE_TYPELESS_STORAGE (expr) = (unsigned) bp_unpack_value (bp, 1);
   TYPE_EMPTY_P (expr) = (unsigned) bp_unpack_value (bp, 1);
-  

[V10][PATCH 3/3] Use TYPE_INCLUDES_FLEXARRAY in __builtin_object_size [PR101832]

[Qing Zhao via Gcc-patches]

__builtin_object_size should treat struct with TYPE_INCLUDES_FLEXARRAY as
flexible size.

gcc/ChangeLog:

PR tree-optimization/101832
* tree-object-size.cc (addr_object_size): Handle structure/union type
when it has flexible size.

gcc/testsuite/ChangeLog:

PR tree-optimization/101832
* gcc.dg/builtin-object-size-pr101832.c: New test.
---
 .../gcc.dg/builtin-object-size-pr101832.c | 134 ++
 gcc/tree-object-size.cc   |  23 ++-
 2 files changed, 156 insertions(+), 1 deletion(-)
 create mode 100644 gcc/testsuite/gcc.dg/builtin-object-size-pr101832.c

diff --git a/gcc/testsuite/gcc.dg/builtin-object-size-pr101832.c 
b/gcc/testsuite/gcc.dg/builtin-object-size-pr101832.c
new file mode 100644
index 000..60078e11634
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/builtin-object-size-pr101832.c
@@ -0,0 +1,134 @@
+/* PR 101832: 
+   GCC extension accepts the case when a struct with a C99 flexible array
+   member is embedded into another struct (possibly recursively).
+   __builtin_object_size will treat such struct as flexible size.
+   However, when a structure with non-C99 flexible array member, i.e, trailing
+   [0], [1], or [4], is embedded into anther struct, the stucture will not
+   be treated as flexible size.  */ 
+/* { dg-do run } */
+/* { dg-options "-O2" } */
+
+#include "builtin-object-size-common.h"
+
+#define expect(p, _v) do { \
+  size_t v = _v; \
+  if (p == v) \
+__builtin_printf ("ok:  %s == %zd\n", #p, p); \
+  else {\
+__builtin_printf ("WAT: %s == %zd (expected %zd)\n", #p, p, v); \
+FAIL (); \
+  } \
+} while (0);
+
+
+struct A {
+  int n;
+  char data[];
+};
+
+struct B {
+  int m;
+  struct A a;
+};
+
+struct C {
+  int q;
+  struct B b;
+};
+
+struct A0 {
+  int n;
+  char data[0];
+};
+
+struct B0 {
+  int m;
+  struct A0 a;
+};
+
+struct C0 {
+  int q;
+  struct B0 b;
+};
+
+struct A1 {
+  int n;
+  char data[1];
+};
+
+struct B1 {
+  int m;
+  struct A1 a;
+};
+
+struct C1 {
+  int q;
+  struct B1 b;
+};
+
+struct An {
+  int n;
+  char data[8];
+};
+
+struct Bn {
+  int m;
+  struct An a;
+};
+
+struct Cn {
+  int q;
+  struct Bn b;
+};
+
+volatile void *magic1, *magic2;
+
+int main (int argc, char *argv[])
+{
+  struct B *outer;
+  struct C *outest;
+
+  /* Make sure optimization can't find some other object size. */
+  outer = (void *)magic1;
+  outest = (void *)magic2;
+
+  expect (__builtin_object_size (>a, 1), -1);
+  expect (__builtin_object_size (>b, 1), -1);
+  expect (__builtin_object_size (>b.a, 1), -1);
+
+  struct B0 *outer0;
+  struct C0 *outest0;
+
+  /* Make sure optimization can't find some other object size. */
+  outer0 = (void *)magic1;
+  outest0 = (void *)magic2;
+
+  expect (__builtin_object_size (>a, 1), sizeof (outer0->a));
+  expect (__builtin_object_size (>b, 1), sizeof (outest0->b));
+  expect (__builtin_object_size (>b.a, 1), sizeof (outest0->b.a));
+
+  struct B1 *outer1;
+  struct C1 *outest1;
+
+  /* Make sure optimization can't find some other object size. */
+  outer1 = (void *)magic1;
+  outest1 = (void *)magic2;
+
+  expect (__builtin_object_size (>a, 1), sizeof (outer1->a));
+  expect (__builtin_object_size (>b, 1), sizeof (outest1->b));
+  expect (__builtin_object_size (>b.a, 1), sizeof (outest1->b.a));
+
+  struct Bn *outern;
+  struct Cn *outestn;
+
+  /* Make sure optimization can't find some other object size. */
+  outern = (void *)magic1;
+  outestn = (void *)magic2;
+
+  expect (__builtin_object_size (>a, 1), sizeof (outern->a));
+  expect (__builtin_object_size (>b, 1), sizeof (outestn->b));
+  expect (__builtin_object_size (>b.a, 1), sizeof (outestn->b.a));
+
+  DONE ();
+  return 0;
+}
diff --git a/gcc/tree-object-size.cc b/gcc/tree-object-size.cc
index 9a936a91983..a62af050056 100644
--- a/gcc/tree-object-size.cc
+++ b/gcc/tree-object-size.cc
@@ -633,11 +633,32 @@ addr_object_size (struct object_size_info *osi, 
const_tree ptr,
v = NULL_TREE;
break;
  case COMPONENT_REF:
-   if (TREE_CODE (TREE_TYPE (v)) != ARRAY_TYPE)
+   /* When the ref is not to an aggregate type, i.e, an array,
+  a record or a union, it will not have flexible size,
+  compute the object size directly.  */
+   if (!AGGREGATE_TYPE_P (TREE_TYPE (v)))
  {
v = NULL_TREE;
break;
  }
+   /* if the ref is to a record or union type, but the type
+  does not include a flexible array recursively, compute
+  the object size directly.  */
+   if (RECORD_OR_UNION_TYPE_P (TREE_TYPE (v)))
+ {
+   if (!TYPE_INCLUDES_FLEXARRAY (TREE_TYPE (v)))
+ {
+   v = NULL_TREE;
+   break;
+  

[V10][PATCH 2/3] Update documentation to clarify a GCC extension [PR77650]

[Qing Zhao via Gcc-patches]

on a structure with a C99 flexible array member being nested in
another structure.

"The GCC extension accepts a structure containing an ISO C99 "flexible array
member", or a union containing such a structure (possibly recursively)
to be a member of a structure.

 There are two situations:

   * A structure containing a C99 flexible array member, or a union
 containing such a structure, is the last field of another structure,
 for example:

  struct flex  { int length; char data[]; };
  union union_flex { int others; struct flex f; };

  struct out_flex_struct { int m; struct flex flex_data; };
  struct out_flex_union { int n; union union_flex flex_data; };

 In the above, both 'out_flex_struct.flex_data.data[]' and
 'out_flex_union.flex_data.f.data[]' are considered as flexible
 arrays too.

   * A structure containing a C99 flexible array member, or a union
 containing such a structure, is not the last field of another structure,
 for example:

  struct flex  { int length; char data[]; };

  struct mid_flex { int m; struct flex flex_data; int n; };

 In the above, accessing a member of the array 'mid_flex.flex_data.data[]'
 might have undefined behavior.  Compilers do not handle such a case
 consistently, Any code relying on this case should be modified to ensure
 that flexible array members only end up at the ends of structures.

 Please use the warning option '-Wflex-array-member-not-at-end' to
 identify all such cases in the source code and modify them.  This extension
 is now deprecated.
"

PR C/77650

gcc/c-family/ChangeLog:

* c.opt: New option -Wflex-array-member-not-at-end.

gcc/c/ChangeLog:

* c-decl.cc (finish_struct): Issue warnings for new option.

gcc/ChangeLog:

* doc/extend.texi: Document GCC extension on a structure containing
a flexible array member to be a member of another structure.

gcc/testsuite/ChangeLog:

* gcc.dg/variable-sized-type-flex-array.c: New test.
---
 gcc/c-family/c.opt|  5 +++
 gcc/c/c-decl.cc   |  9 
 gcc/doc/extend.texi   | 44 ++-
 .../gcc.dg/variable-sized-type-flex-array.c   | 31 +
 4 files changed, 88 insertions(+), 1 deletion(-)
 create mode 100644 gcc/testsuite/gcc.dg/variable-sized-type-flex-array.c

diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt
index cead1995561..fce66b1d225 100644
--- a/gcc/c-family/c.opt
+++ b/gcc/c-family/c.opt
@@ -737,6 +737,11 @@ Wformat-truncation=
 C ObjC C++ LTO ObjC++ Joined RejectNegative UInteger Var(warn_format_trunc) 
Warning LangEnabledBy(C ObjC C++ LTO ObjC++,Wformat=, warn_format >= 1, 0) 
IntegerRange(0, 2)
 Warn about calls to snprintf and similar functions that truncate output.
 
+Wflex-array-member-not-at-end
+C C++ Var(warn_flex_array_member_not_at_end) Warning
+Warn when a structure containing a C99 flexible array member as the last
+field is not at the end of another structure.
+
 Wif-not-aligned
 C ObjC C++ ObjC++ Var(warn_if_not_aligned) Init(1) Warning
 Warn when the field in a struct is not aligned.
diff --git a/gcc/c/c-decl.cc b/gcc/c/c-decl.cc
index e14f514cb6e..ecd10ebb69c 100644
--- a/gcc/c/c-decl.cc
+++ b/gcc/c/c-decl.cc
@@ -9278,6 +9278,15 @@ finish_struct (location_t loc, tree t, tree fieldlist, 
tree attributes,
TYPE_INCLUDES_FLEXARRAY (t)
  = is_last_field && TYPE_INCLUDES_FLEXARRAY (TREE_TYPE (x));
 
+  if (warn_flex_array_member_not_at_end
+ && !is_last_field
+ && RECORD_OR_UNION_TYPE_P (TREE_TYPE (x))
+ && TYPE_INCLUDES_FLEXARRAY (TREE_TYPE (x)))
+   warning_at (DECL_SOURCE_LOCATION (x),
+   OPT_Wflex_array_member_not_at_end,
+   "structure containing a flexible array member"
+   " is not at the end of another structure");
+
   if (DECL_NAME (x)
  || RECORD_OR_UNION_TYPE_P (TREE_TYPE (x)))
saw_named_field = true;
diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi
index cdbd4b34a35..fbb1d9708ba 100644
--- a/gcc/doc/extend.texi
+++ b/gcc/doc/extend.texi
@@ -1751,7 +1751,49 @@ Flexible array members may only appear as the last 
member of a
 A structure containing a flexible array member, or a union containing
 such a structure (possibly recursively), may not be a member of a
 structure or an element of an array.  (However, these uses are
-permitted by GCC as extensions.)
+permitted by GCC as extensions, see details below.)
+@end itemize
+
+The GCC extension accepts a structure containing an ISO C99 @dfn{flexible array
+member}, or a union containing such a structure (possibly recursively)
+to be a member of a structure.
+
+There are two situations:
+
+@itemize @bullet
+@item
+A structure containing a C99 flexible array member, or a union containing
+such a structure, is the last field of another structure, for example:
+

[V10][PATCH 0/3] Accept and Handle the case when a structure including a FAM nested in another structure

[Qing Zhao via Gcc-patches]

This is the 10th version of the patch, which rebased on the latest trunk.
This is an important patch needed by Linux Kernel security project. 

compared to the 9th patch, the ONLY change is:

I split the 1st patch into the following two patches:

patch 1/3:  Introduce IR bit TYPE_INCLUDES_FLEXARRAY for the GCC extension 
[PR77650]
patch 3/3:  Use TYPE_INCLUDES_FLEXARRAY in __builtin_object_size[PR101832]

so, in patch 1/3 includes all the FE changes, changes in Middle-end to support
the FE changes.
patch 3/3 only includes the change in gcc/tree-object-size.cc and the
testing case to support PR101832. 


Now, there are 3 patches:

patch 1/3:  Introduce IR bit TYPE_INCLUDES_FLEXARRAY for the GCC extension 
[PR77650]
patch 2/3:  Update documentation to clarify a GCC extension [PR77650]
patch 3/3:  Use TYPE_INCLUDES_FLEXARRAY in __builtin_object_size[PR101832]

among the above, the patch 1/3 and the patch 2/3 have been reviewed and
approved.

the patch 3/3 is a very simple change to use the new bit
TYPE_INCLUDES_FLEXARRAY in __builtin_object_size.

I have bootstrapped and regression-tested on both aarch64 and x86. no
issues.

I will commit the patch 1/3 and 2/3 tomorrow since they have been
approved already.

for the patch 3/3, I will wait for several days, if there is no
objection or new comments, I will commit it the end of this week.

Please let me know if you have comments and suggestions.


thanks.
Qing


Qing Zhao (3):
  Introduce IR bit TYPE_INCLUDES_FLEXARRAY for the GCC extension
[PR77650]
  Update documentation to clarify a GCC extension [PR77650]
  Use TYPE_INCLUDES_FLEXARRAY in __builtin_object_size [PR101832]

-- 
2.31.1

Re: [V1][PATCH 1/3] Provide element_count attribute to flexible array member field (PR108896)

[Qing Zhao via Gcc-patches]

> On Jun 16, 2023, at 1:07 PM, Martin Uecker  wrote:
> 
> Am Freitag, dem 16.06.2023 um 16:21 + schrieb Joseph Myers:
>> On Fri, 16 Jun 2023, Martin Uecker via Gcc-patches wrote:
>> 
 Note that no expressions can start with the '.' token at present.  As soon 
 as you invent a new kind of expression that can start with that token, you 
 have syntactic ambiguity.
 
 struct s1 { int c; char a[(struct s2 { int c; char b[.c]; }) {.c=.c}.c]; };
 
 Is ".c=.c" a use of the existing syntax for designated initializers, with 
 the first ".c" being a designator and the second being a use of the new 
 kind of expression, or is it an assignment expression, where both the LHS 
 and the RHS of the assignment use the new kind of expression?  And do 
 those .c, when the use the new kind of expression, refer to the inner or 
 outer struct definition?
>>> 
>>> I would treat this is one integrated feature. Essentially .c is
>>> somthing like this->c for the current struct for designated
>>> initializer *and* size expressions because it is semantically 
>>> so close.In the initializer I would allow only 
>>> the current use for designated initialization for all names of
>>> member of the currently initialized struct,  so .c = .c would 
>>> be invalid.   It should never refer to the outer struct if there
>> 
>> I'm not clear on what the intended disambiguation rule here is, when "." 
>> is seen in initializer list context - does this rule depend on whether the 
>> following identifier is a member of the struct being initialized, so 
>> ".c=.c" would be OK above if the initialized struct didn't have a member 
>> called c but the outer struct definition did? 
> 
> When initializers are parsed it is already clear what
> the names of the members of the inner struct are, so
> one can differentiate between designated initializers 
> and potential other uses in an expression. 
> 
> So the main rule is: if you parse .something in a context
> where a designator is allowed and "something" is a member
> of the current struct, then it is a designator.

So, Limiting the .something ONLY to the CURRENT structure/union might be the 
simple and clean rule.

And I guess that this is also the rule for the current designator initializer 
syntax in C99?

> 
> So for 
> 
> struct foo { int c; int buf[(struct { int d; }){ .d = .c }]; };
> 
> one knows during parsing that the .d is a designator
> and that .c is not.

Therefore, the above should be invalid based on this rule since .c is not a 
member in the current structure.


> For
> 
> struct foo { int c; int buf[(struct { int d; }){ .c = .c }]; };
> 
> one knows that both uses of .c are not.

And this also is invalid since .c is not to a member in the current structure. 

> 
> Whether these different use cases should be allowed or not
> is a different question, but my point is that there does
> not seem to be a problem directly identifying the uses 
> as a designator as usual. To me, this seems to imply that
> it is safe to use the same syntax.
> 
>> That seems like a rather 
>> messy rule.  And does "would allow only" apply other than in the ambiguous 
>> context?  That seems to be implied by ".c=.c" being invalid above, because 
>> to make it invalid you need to disallow the new construct being used for 
>> the second .c, not just make the first .c interpreted as a designator.
> 
> Yes. 
>> 
>> Again, this sort of thing needs a detailed written specification, with 
>> multiple iterations discussed among different implementations. 
> 
> Oh, I agree with this.
> 
>> The above 
>> paragraph doesn't make clear to me any of: the disambiguation rules; what 
>> is allowed in what context; how name lookup works (consider tricky cases 
>> such as a reference to an identifier declared *later* in the same struct, 
>> possibly in the context of C2x tag compatibility where a previous 
>> definition of the struct is visible); when these expressions get 
>> evaluated; what the underlying principles are behind those choices.
> 
> I also agree that all this needs careful consideration and written
> rules.  My point is mereley that there does not seem to be a
> fundamental issue differentiating the new feature from 
> designators during parsing, so there may not be a risk using 
> the same syntax.

Yes, I agree on this. 
Extending the existing designated initializer syntax, .member, for the purpose 
of the argument of the new attribute seems very natural. 
If we can use this syntax in the argument of this new attribute, 
1. it will be easy to extend the argument of this attribute to an expression. 
2. It will also easy to use this syntax later if we accept the following

struct foo {
...
unsigned int count;
...
int data[.count];
};


thanks.

Qing
> 
>> Using a token (existing or new) other than '.' - one that doesn't 
>> introduce ambiguity in any context where expressions can be used - would 
>> help significantly, although some of the 

Re: [PATCH zero-call-used-regs] Add leafy mode for zero-call-used-regs

[Qing Zhao via Gcc-patches]

Hi, Alexandre,

> On Jun 16, 2023, at 3:26 AM, Alexandre Oliva  wrote:
> 
> Hello, Qing,
> 
> On Oct 27, 2022, Qing Zhao  wrote:
> 
> 
>> On Oct 26, 2022, at 5:29 PM, Alexandre Oliva  wrote:
>>> I'm sure there are other scenarios in which keeping at least the
>>> possibility of 'all' is useful.
>> Okay.
> 
> 
>> i.e, instead introducing a new MODE “LEAFY_MODE” and a new user
>> sub-option, for LEAF functions, only
>> Clear its’ used registers even for “ALL”.
> 
>> However, since there is need to clear the un-used registers for leaf
>> functions. It looks like it is needed to provide
>> This new sub-option to users.
> 
>> Is this clear this time?
> 
> Yeah, I guess I understand what you mean.  But since there are cases in
> which clearing all (call-clobbered) registers in a leaf function is
> useful, I suppose it makes sense to offer both possibilities.
agreed.
> 
> If there was a default operation mode for -fzero-call-used-regs, I guess
> it would make sense to consider leafy the default, rather than all, but
> since there isn't, and it always has to be specified explicitly, that's
> not something to be considered.
> 
> So the available choices are:
> 
> 1. introduce 'leafy' as a separate mode, leaving 'all' alone
> 
> 2. change the behavior of 'all' to that of the proposed 'leafy', and either
> 
> 2.a) add another mode that retains the currently-useful behavior of 'all',
>   or
> 
> 2.b) make the current behavior of 'all' no longer available
> 
> Personally, I find 1. the least disruptive to existing users of
> -fzero-call-used-regs.  If we were introducing the option now, maybe 2.a
> would be more sensible, but at this point, changing the meaning of 'all'
> seems to be a disservice to security-sensitive users.
> 
> Those who would prefer the leaner operation on leaf functions can then
> switch to 'leafy' mode, but that's better than finding carefully-crafted
> code relying on the current behavior of 'all' for security suddenly
> changes from under them, isn't it?

Yes, I agree.
> 
> 
> That said, I'm willing to implement the alternate change, if changing
> the expected behavior is preferred over offering a different choice, if
> needed to get closure on this feature.
> 
> For now, I'm just pinging the refreshed and retested patch.

As I mentioned in the previous round of review, I think that the documentation
 might need to add more details on what’s the LEAFY mode,
The purpose of it, and how to use it, provide more details to the end-users.


> Ok to install?
> 
> 
> Add leafy mode for zero-call-used-regs
> 
> Introduce 'leafy' to auto-select between 'used' and 'all' for leaf and
> nonleaf functions, respectively.
> 
> 
> for  gcc/ChangeLog
> 
>   * doc/extend.texi (zero-call-used-regs): Document leafy and
>   variants thereof.
>   * flag-types.h (zero_regs_flags): Add LEAFY_MODE, as well as
>   LEAFY and variants.
>   * function.cc (gen_call_ued_regs_seq): Set only_used for leaf
>   functions in leafy mode.
>   * opts.cc (zero_call_used_regs_opts): Add leafy and variants.
> 
> for  gcc/testsuite/ChangeLog
> 
>   * c-c++-common/zero-scratch-regs-leafy-1.c: New.
>   * c-c++-common/zero-scratch-regs-leafy-2.c: New.
>   * gcc.target/i386/zero-scratch-regs-leafy-1.c: New.
>   * gcc.target/i386/zero-scratch-regs-leafy-2.c: New.
> ---
> gcc/doc/extend.texi|   22 ++--
> gcc/flag-types.h   |5 +
> gcc/function.cc|3 +++
> gcc/opts.cc|4 
> .../c-c++-common/zero-scratch-regs-leafy-1.c   |   15 ++
> .../c-c++-common/zero-scratch-regs-leafy-2.c   |   21 +++
> .../gcc.target/i386/zero-scratch-regs-leafy-1.c|   12 +++
> .../gcc.target/i386/zero-scratch-regs-leafy-2.c|   16 +++
> 8 files changed, 96 insertions(+), 2 deletions(-)
> create mode 100644 gcc/testsuite/c-c++-common/zero-scratch-regs-leafy-1.c
> create mode 100644 gcc/testsuite/c-c++-common/zero-scratch-regs-leafy-2.c
> create mode 100644 gcc/testsuite/gcc.target/i386/zero-scratch-regs-leafy-1.c
> create mode 100644 gcc/testsuite/gcc.target/i386/zero-scratch-regs-leafy-2.c
> 
> diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi
> index 7b5592502734e..f8b0bb53ef5d4 100644
> --- a/gcc/doc/extend.texi
> +++ b/gcc/doc/extend.texi

I think in the documentation of zero_call_used_regs, 

After the description of the 3 basic values: “skip”, “used”, and “all”. 
The description of the new value “leafy” need to be added first.

In addition to the below doc change.

The others LGTM.

Thanks.

Qing

> @@ -4412,10 +4412,28 @@ zeros all call-used registers that pass arguments.
> @item all-gpr-arg
> zeros all call-used general purpose registers that pass
> arguments.
> +
> +@item leafy
> +Same as @samp{used} in a leaf 

Re: [V1][PATCH 1/3] Provide element_count attribute to flexible array member field (PR108896)

[Qing Zhao via Gcc-patches]

> On Jun 16, 2023, at 3:21 AM, Martin Uecker  wrote:
> 
> Am Donnerstag, dem 15.06.2023 um 16:55 + schrieb Joseph Myers:
>> On Thu, 15 Jun 2023, Qing Zhao via Gcc-patches wrote:
>> 
> ...
>>> 1. Update the routine “c_parser_postfix_expression” (is this the right 
>>> place? ) to accept the new designator syntax.
>> 
>> Any design that might work with an expression is the sort of thing that 
>> would likely involve many iterations on the specification (i.e. proposed 
>> wording changes to the C standard) for the interpretation of the new kinds 
>> of expressions, including how to resolve syntactic ambiguities and how 
>> name lookup works, before it could be considered ready to implement, and 
>> then a lot more work on the specification based on implementation 
>> experience.
>> 
>> Note that no expressions can start with the '.' token at present.  As soon 
>> as you invent a new kind of expression that can start with that token, you 
>> have syntactic ambiguity.
>> 
>> struct s1 { int c; char a[(struct s2 { int c; char b[.c]; }) {.c=.c}.c]; };
>> 
>> Is ".c=.c" a use of the existing syntax for designated initializers, with 
>> the first ".c" being a designator and the second being a use of the new 
>> kind of expression, or is it an assignment expression, where both the LHS 
>> and the RHS of the assignment use the new kind of expression?  And do 
>> those .c, when the use the new kind of expression, refer to the inner or 
>> outer struct definition?
> 
> I would treat this is one integrated feature. Essentially .c is
> somthing like this->c for the current struct for designated
> initializer *and* size expressions because it is semantically 
> so close.  

Yes, I think this is reasonable. (Is “this” the immediate containing structure?)

>  In the initializer I would allow only 
> the current use for designated initialization for all names of
> member of the currently initialized struct,  so .c = .c would 
> be invalid.

Given “.c” basically is “this->c”, then .c = .c should be considered as
this->c = this->c, is such self-initialization allowed in C?

>   It should never refer to the outer struct if there
> is a member with the same name in the inner struct, i.e. the
> outside member is then hidden. 

Does the above mean:  if there is NO name conflicting, it could refer to a 
field of an outer struct?

Why this is allowed? Why just disallow all referring to the field of the outer 
structure since .c basically is this->c?
> 
> So this would be ok:
> 
> struct s1 { int d; char a[(struct s2 { int c; char b[.c]; }) {.c=.d}.c]; };
> 
> Here the use of .d would be ok because it is not from the struct
> currently initialized, but from an outside scope.

I think that the above .c=.d should report an error, since .d does not exist in 
the containing structure.

Do I miss anything here?

thanks.

Qing
> 
> Martin
> 
> 
> 
> 

Re: [V1][PATCH 1/3] Provide element_count attribute to flexible array member field (PR108896)

[Qing Zhao via Gcc-patches]

> On Jun 15, 2023, at 6:48 PM, Joseph Myers  wrote:
> 
> On Thu, 15 Jun 2023, Qing Zhao via Gcc-patches wrote:
> 
>> B. The argument of the new attribute “counted_by” is an identifier that can 
>> be
>> accepted by “c_parser_attribute_arguments”:
>> 
>> struct trailing_array_B {
>> Int count;
>> int array_B[] __attribute ((counted_by (count))); 
>> };
>> 
>> 
>> From my current very limited understanding of the C FE source code, it’s 
>> not easy to extend the argument to an expression later for the above. Is 
>> this understanding right?
> 
> It wouldn't be entirely compatible: if you change to interpreting the 
> argument as an expression, then the above would suggest a global variable 
> count is used (as opposed to some other syntax for referring to an element 
> of the containing structure).

Yeah, that’s the reason I tried to introduce the new “.count” syntax for the 
argument 
of the new attribute in the very beginning in order to avoid such incompatible 
issue later.  -:)
> 
> So an attribute that takes an element name might best be a *different* 
> attribute from any potential future one taking an expression (with some 
> new syntax to refer to an element).

So, if we add this “counted_by (identifier)” attribute now, and later we need 
to add another
 new attribute “new_counted_by (expression)”  at that time if needed?

Kees, what’s your opinion on this?

thanks.

Qing
> 
> -- 
> Joseph S. Myers
> jos...@codesourcery.com

Re: [V1][PATCH 1/3] Provide element_count attribute to flexible array member field (PR108896)

[Qing Zhao via Gcc-patches]

> On Jun 15, 2023, at 12:55 PM, Joseph Myers  wrote:
> 
> On Thu, 15 Jun 2023, Qing Zhao via Gcc-patches wrote:
> 
>> Comparing B with A, I don’t see too much benefit, either from 
>> user-interface point of view, or from implementation point of view.
>> 
>> For implementation, both A and B need to search the fields of the 
>> containing structure by the name of the field “count”.
>> 
>> For user interface, I think that A and B are similar.
> 
> But as a language design matter, there are no standard C interfaces that 
> interpret a string as an identifier, so doing so does not fit well with 
> the language.

Okay, makes sense.  So I will choose B over A. -:) 
> 
>> 1. Update the routine “c_parser_postfix_expression” (is this the right 
>> place? ) to accept the new designator syntax.
> 
> Any design that might work with an expression is the sort of thing that 
> would likely involve many iterations on the specification (i.e. proposed 
> wording changes to the C standard) for the interpretation of the new kinds 
> of expressions, including how to resolve syntactic ambiguities and how 
> name lookup works, before it could be considered ready to implement, and 
> then a lot more work on the specification based on implementation 
> experience.

Okay, I see the complication to make such new syntax into C standard…

> 
> Note that no expressions can start with the '.' token at present.  As soon 
> as you invent a new kind of expression that can start with that token, you 
> have syntactic ambiguity.
> 
> struct s1 { int c; char a[(struct s2 { int c; char b[.c]; }) {.c=.c}.c]; };
> 
> Is ".c=.c" a use of the existing syntax for designated initializers, with 
> the first ".c" being a designator and the second being a use of the new 
> kind of expression, or is it an assignment expression, where both the LHS 
> and the RHS of the assignment use the new kind of expression?  And do 
> those .c, when the use the new kind of expression, refer to the inner or 
> outer struct definition?

Okay, I see. Yes, this will be really confusing. 

> 
> There are obvious advantages to using tokens that don't introduce such an 
> ambiguity with designators (i.e., not '.' as the token to start the new 
> kind of expression, but something that cannot start a designator), if such 
> tokens can be found.  But you still have the name lookup question when 
> there are multiple nested structure definitions.  And the question of when 
> expressions are considered to be evaluated, if they have side effects such 
> as ".c=.c" does.
> 
> "Whatever falls out of the implementation" is not a good approach for 
> language design here.  If you want a new kind of expressions here, you 
> need a careful multi-implementation design phase that produces a proper 
> specification and has good reasons for the particular choices made in 
> cases of ambiguity.

Thanks a lot for your detailed explanation on the language design concerns. 
For this new attribute, I was convinced that it might not worth the effort to 
introduce a new syntax at this stage.

Another question,  whether it’s possible to extend such attribute later to 
accept expression as its argument if we take approach B:

B. The argument of the new attribute “counted_by” is an identifier that can be
accepted by “c_parser_attribute_arguments”:

struct trailing_array_B {
 Int count;
 int array_B[] __attribute ((counted_by (count))); 
};


From my current very limited understanding of the C FE source code, it’s not 
easy to extend the argument to an expression later for the above.
Is this understanding right?

(The motivation of accepting expression as the argument for the new attribute 
“counted_by” is 
   from the proposal for LLVM: 
https://discourse.llvm.org/t/rfc-enforcing-bounds-safety-in-c-fbounds-safety/70854:

• __counted_by(N) : The pointer points to memory that contains N 
elements of pointee type. N is an expression of integer type which can be a 
simple reference to declaration, a constant including calls to constant 
functions, or an arithmetic expression that does not have side effect. The 
annotation cannot apply to pointers to incomplete types or types without size 
such as  void *.
)
 
thanks.
Qing

> 
> -- 
> Joseph S. Myers
> jos...@codesourcery.com

Re: [V1][PATCH 1/3] Provide element_count attribute to flexible array member field (PR108896)

[Qing Zhao via Gcc-patches]

Hi, Joseph,

I studied c_parser_attribute_arguments and related source code, 
and also studied PLACEHOLDER_EXPR and related source code. 

Right now, I still cannot decide what’s the best user-interface should be for 
the 
argument of the new attribute “element_count". (Or the new attribute 
name “counted_by” as suggested by Kees). 

There are 3 possible interfaces for the argument of the new attribute 
“counted_by”:

The first 2 interfaces are the following A and B:

A. the argument of the new attribute “counted_by” is a string as the current 
patch:

struct trailing_array_A {
  Int count;
  int array_A[] __attribute ((counted_by (“count"))); 
};

B. The argument of the new attribute “counted_by” is an identifier that can be
 accepted by “c_parser_attribute_arguments”:

struct trailing_array_B {
  Int count;
  int array_B[] __attribute ((counted_by (count))); 
};

To implement this interface,  we need to adjust “attribute_takes_identifier_p” 
to 
accept the new attribute “counted_by” and then interpret this new identifier  
“count” as a 
field of the containing structure by looking at the name.  (Otherwise, the 
identifier “count”
will be treated as an identifier in the current scope, which is not declared 
yet)

Comparing B with A, I don’t see too much benefit, either from user-interface 
point of view, 
or from implementation point of view.  

For implementation, both A and B need to search the fields of the containing 
structure by 
the name of the field “count”.

For user interface, I think that A and B are similar.

In addition to consider the user-interface and implementation, another concern 
is the possibility
 to extend the argument of this new attribute to an expression in the future, 
for example:

struct trailing_array_F {
  Int count;
  int array_F[] __attribute ((counted_by (count * 4))); 
};

In the above struct “trailing_array_F”, the argument of the attribute 
“counted_by” is “count * 4”, which
is an expression.  

If we plan to extend the argument of this new attribute to an expression, then 
neither A nor B is
good, right?

For this purpose, it might be cleaner to introduce a new syntax similar as the 
designator as mentioned in
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108896 as following, i.e, the 
approach C:

C. The argument of the new attribute “counted_by” is a new designator 
identifier that will be parsed as
The field of the containing structure:

struct trailing_array_C {
  Int count;
  int array_C[] __attribute ((counted_by (.count))); 
};

I think that after the C FE accepts this new designator syntax as the argument 
of the attribute, then it’s
very easy to extend the argument to an arbitrary expression later. 

For the implementation of this approach, my current thinking is: 

1. Update the routine “c_parser_postfix_expression” (is this the right place? ) 
to accept the new designator syntax.
2. Use “PLACEHOLDER_EXPR” to represent the containing structure, and build a 
COMPONENT_REF to hold
the argument of the attribute in the IR.
3. When using this attribute in middle-end or sanitizer, use 
SUBSTITUTE_PLACEHOLDER_IN_EXPR(EXP, OBJ)
to get the size info in IR. 

So, right now, I think that we might need to take the approach C?

What’s your opinion and suggestions?

Thanks a lot for your help.

Qing


> On Jun 7, 2023, at 6:05 PM, Joseph Myers  wrote:
> 
> On Wed, 7 Jun 2023, Qing Zhao via Gcc-patches wrote:
> 
>> Are you suggesting to use identifier directly as the argument of the 
>> attribute?
>> I tried this in the beginning, however, the current parser for the attribute 
>> argument can not identify that this identifier is a field identifier inside 
>> the same structure. 
>> 
>> For example:
>> 
>> int count;
>> struct trailing_array_7 {
>>  Int count;
>>  int array_7[] __attribute ((element_count (count))); 
>> };
>> 
>> The identifier “count” inside the attribute will refer to the variable 
>> “int count” outside of the structure.
> 
> c_parser_attribute_arguments is supposed to allow an identifier as an 
> attribute argument - and not look it up (the user of the attribute would 
> later need to look it up in the context of the containing structure).  
> Callers use attribute_takes_identifier_p to determine which attributes 
> take identifiers (versus expressions) as arguments, which would need 
> updating to cover the new attribute.
> 
> There is a ??? comment about the case where the identifier is declared as 
> a type name.  That would simply be one of the cases carried over from the 
> old Bison parser, and it would seem reasonable to remove that 
> special-casing so that the attribute works even when the identifier is 
> declared as a typedef name as an ordinary identifier, since it's fine for 
> structure members to have the same name as a typedef name.
> 
> Certainly taking an identifier directly seems like cleaner syntax than 
> taking a string that then needs reinterpreting as an identifier.
> 
> -- 
> Joseph S. Myers
> jos...@codesourcery.com