[Bug tree-optimization/125295] New: UB shift is hoisted with -fsanitize=undefined

[kristerw at gcc dot gnu.org via Gcc-bugs]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=125295

            Bug ID: 125295
           Summary: UB shift is hoisted with -fsanitize=undefined
           Product: gcc
           Version: 17.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: kristerw at gcc dot gnu.org
            Blocks: 118443
  Target Milestone: ---

The PRE pass hoists a shift instruction before the sanitizer check, which in
principle allows the compiler to eliminate the sanitizer code (although GCC
does not currently take advantage of this UB).

This can be seen by compiling the function below on x86_64 with -O2
-fsanitize=undefined:

char foo(char c, char c2)
{
  c >>= c2;
  c <<= c2;
  return c;
}


The code before the PRE pass starts the function by checking whether the shift
count is too large:

  <bb 2> :
  _17 = (int) c2_16(D);
  _19 = (int) c_18(D);
  _1 = (unsigned int) c2_16(D);
  if (_1 > 31)
    goto <bb 3>;
  else
    goto <bb 4>;

  <bb 3> :
  _2 = (unsigned long) c2_16(D);
  _3 = (unsigned long) c_18(D);
  __builtin___ubsan_handle_shift_out_of_bounds (&*.Lubsan_data0, _3, _2);
  _25 = _19 >> _17;
  goto <bb 5>;

  <bb 4>:
  _4 = _19 >> _17;
  ...

This performs the operation _19 >> _17 on both paths, and the PRE pass hoists
it before the UB check:

  <bb 2> :
  _17 = (int) c2_16(D);
  _19 = (int) c_18(D);
  _1 = (unsigned int) c2_16(D);
  _26 = _19 >> _17;
  if (_1 > 31)
    goto <bb 3>;
  else
    goto <bb 4>;

  <bb 3> :
  _2 = (unsigned long) c2_16(D);
  _3 = (unsigned long) c_18(D);
  __builtin___ubsan_handle_shift_out_of_bounds (&*.Lubsan_data0, _3, _2);
  goto <bb 5>;

  <bb 4> :
  ...


Referenced Bugs:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=118443
[Bug 118443] [Meta bug] Bugs triggered by and blocking more smtgcc testing