> -----Original Message-----
> From: Alfie Richards <[email protected]>
> Sent: 07 October 2026 10:32
> To: Tamar Christina <[email protected]>; [email protected]
> Cc: Alex Coplan <[email protected]>; Alice Carlotti
> <[email protected]>; Andrea Pinski
> <[email protected]>; Richard Biener <[email protected]>;
> Zdenek Dvorak <[email protected]>; Robin Dapp <[email protected]>; Richard
> Sandiford <[email protected]>; Kyrylo Tkachov
> <[email protected]>
> Subject: Re: [PATCH v3 03/16] ifn: optab: Add HSSR ifns and optabs
>
> On 06/10/2026 22:27, Tamar Christina wrote:
> >> -----Original Message-----
> >> From: Alfie Richards <[email protected]>
> >> Sent: 17 September 2026 15:40
> >> To: [email protected]
> >> Cc: Alex Coplan <[email protected]>; Alice Carlotti
> >> <[email protected]>; Andrea Pinski
> >> <[email protected]>; Richard Biener
> <[email protected]>;
> >> Tamar Christina <[email protected]>; Zdenek Dvorak
> <[email protected]>;
> >> Robin Dapp <[email protected]>; Richard Sandiford
> >> <[email protected]>; Kyrylo Tkachov <[email protected]>;
> >> Alfie Richards <[email protected]>
> >> Subject: [PATCH v3 03/16] ifn: optab: Add HSSR ifns and optabs
> >>
> >> Adds 4 new optabs and respective internal functions for hardware
> >> safe speculative reads:
> >>
> >> - mask_ff_hssr_load: a contiguous masked HSSR load
> >> - read_hssr_state: reads the global HSSR state
> >> - set_hssr_state: sets the global HSSR state
> >> - detect_hssr_fault: takes a loaded HSSR state value and returns a
> >> boolean value for if the mask implies a partial read happens.
> >>
> >> As this is initially only for targets that support partial vectors, the
> >> HSSR state is initially assumed to be masks.
> >>
> >> The last optab is required to hide from the gimple representation the
> >> value of the hssr mask. This is because, otherwise later optimizations
> >> try use it when it is a full mask, which introduces a very unfortunate
> >> data dependency.
> >>
> >> gcc/ChangeLog:
> >>
> >> * doc/ifn.texi: Document new internal functions.
> >> * doc/md.texi: Document new optabs.
> >> * gimple-fold.cc (gimple_fold_partial_load_store_mem_ref):
> >> Add handling of IFN_MASK_FF_HSSR_LOAD.
> >> (gimple_fold_partial_load_store): Likewise.
> >> (gimple_fold_call): Likewise.
> >> * internal-fn.cc (read_hssr_state_direct): New macro.
> >> (set_hssr_state_direct): New macro.
> >> (detect_hssr_fault_direct): New macro.
> >> (expand_read_hssr_state_optab_fn): New macro.
> >> (expand_detect_hssr_fault_optab_fn): New macro.
> >> (expand_set_hssr_state_optab_fn): New function.
> >> (direct_read_hssr_state_optab_supported_p): New macro.
> >> (direct_set_hssr_state_optab_supported_p): New macro.
> >> (direct_detect_hssr_fault_optab_supported_p): New macro.
> >> (internal_load_fn_p): Add support for IFN_MASK_FF_HSSR_LOAD.
> >> (internal_hssr_load_fn_p): New function.
> >> (internal_fn_else_index): Add support for IFN_MASK_FF_HSSR_LOAD.
> >> (internal_fn_mask_index): Add support for
> >> IFN_MASK_FF_HSSR_LOAD.
> >> (internal_fn_alias_ptr_index): Add support for
> >> IFN_MASK_FF_HSSR_LOAD.
> >> * internal-fn.def (MASK_FF_HSSR_LOAD): New IFN.
> >> (SET_HSSR_STATE): New IFN.
> >> (READ_HSSR_STATE): New IFN.
> >> (DETECT_HSSR_FAULT): New IFN.
> >> * internal-fn.h (internal_hssr_load_fn_p): New function.
> >> * optabs.def (mask_ff_hssr_load_optab): New optab.
> >> (read_hssr_state_optab): New optab.
> >> (set_hssr_state_optab): New optab.
> >> (detect_hssr_fault_optab): New optab.
> >> * tree-data-ref.cc (get_references_in_stmt):
> >> Add support for IFN_MASK_FF_HSSR_LOAD.
> >> * tree-ssa-alias.cc (ref_maybe_used_by_call_p_1):
> >> Add support for IFN_MASK_FF_HSSR_LOAD.
> >> * tree-ssa-loop-ivopts.cc (get_mem_type_for_internal_fn):
> >> Add support for IFN_MASK_FF_HSSR_LOAD.
> >> (get_alias_ptr_type_for_ptr_address):
> >> Add support for IFN_MASK_FF_HSSR_LOAD.
> >> * tree-vect-slp.cc (vect_get_operand_map):
> >> Add support for IFN_MASK_FF_HSSR_LOAD.
> >>
> >> -- >8 --
> >>
> >> Just to note, I fixed the issue with the test by instead making
> >> gimple_fold_call corectly handle IFN_MASK_FF_HSSR_LOAD and fold it
> >> in this situation.
> >
> > Oro? What issue? Cover letter doesn't mention any :)
> Ah sorry, this was the issue in gcc.target/aarch64/sve/pfalse-load.c
> that we discussed in
> https://gcc.gnu.org/pipermail/gcc-patches/2026-September/730219.html
>
> The issue was that ldff1 loads with an empty mask were no longer being
> removed. You suggested I fix this in the intrinsic folding but in doing
> that I realised this whole issue was because I had messed up some of the
> optimizations in gimple-fold for the new ifns.
Ah I see.
>
> >
> >> ---
> >> gcc/doc/ifn.texi | 29 +++++++++++++++++++++++++
> >> gcc/doc/md.texi | 42
> +++++++++++++++++++++++++++++++++++++
> >> gcc/gimple-fold.cc | 9 ++++++++
> >> gcc/internal-fn.cc | 39 ++++++++++++++++++++++++++++++++++
> >> gcc/internal-fn.def | 18 ++++++++++++++++
> >> gcc/internal-fn.h | 1 +
> >> gcc/optabs.def | 8 +++++++
> >> gcc/tree-data-ref.cc | 2 ++
> >> gcc/tree-ssa-alias.cc | 1 +
> >> gcc/tree-ssa-loop-ivopts.cc | 2 ++
> >> gcc/tree-vect-slp.cc | 3 +++
> >> 11 files changed, 154 insertions(+)
> >>
> >> diff --git a/gcc/doc/ifn.texi b/gcc/doc/ifn.texi
> >> index 7cfc80eecf2..0742803ccd4 100644
> >> --- a/gcc/doc/ifn.texi
> >> +++ b/gcc/doc/ifn.texi
> >> @@ -122,6 +122,35 @@ instruction pattern.
> >> The @code{IFN_MASK_LEN_LOAD} internal function directly maps to the
> >> @code{mask_len_load} instruction pattern.
> >>
> >> +@cindex @code{IFN_MASK_FF_HSSR_LOAD} internal function
> >> +@item @samp{IFN_MASK_FF_HSSR_LOAD}
> >> +The @code{IFN_MASK_FF_HSSR_LOAD} internal function directly maps to
> >> the
> >> +@code{mask_ff_hssr_load} instruction pattern.
> >
> > We've now started documenting IFNs more usefully these days. See e.g.
> >
> > https://patchwork.sourceware.org/project/gcc/patch/patch-20861-rb-rev-
> [email protected]/
>
> Ah okay, will add more here.
>
> >
> > also what does the _ff_ stand for? If it's first_faulting, is that not
> > already
> implicit in the
> > hssr?
>
> The _ff_ means first faulting (eg, guaranteed forward progress), as
> opposed to non-faulting loads, which are both types of hardware safe
> speculative loads.
>
> This is why I think "first faulting loads" is a bad name. But regardless.
>
> I left it in as there is potential that we may later find use for a
> non-faulting HSSR load IFN if doing unrolled HSSR vectorization.
Ok.
>
> Thanks,
> Alfie>
> >> +
> >> +This internal function has a VDEF to represent it changing the global
> >> +HSSR state.
> >> +
> >> +@cindex @code{IFN_READ_HSSR_STATE} internal function
> >> +@item @samp{IFN_READ_HSSR_STATE}
> >> +The @code{IFN_READ_HSSR_STATE} internal function directly maps to the
> >> +@code{read_hssr_state} instruction pattern.
> >> +
> >> +This internal function has a VUSE to represent it reading the global
> >> +HSSR state.
> >> +
> >> +@cindex @code{IFN_SET_HSSR_STATE} internal function
> >> +@item @samp{IFN_SET_HSSR_STATE}
> >> +The @code{IFN_SET_HSSR_STATE} internal function directly maps to the
> >> +@code{set_hssr_state} instruction pattern.
> >> +
> >> +This internal function has a VDEF to represent it reading the global
> >> +HSSR state.
> >> +
> >> +@cindex @code{IFN_DETECT_HSSR_FAULT} internal function
> >> +@item @samp{IFN_DETECT_HSSR_FAULT}
> >> +The @code{IFN_DETECT_HSSR_FAULT} internal function directly maps to
> the
> >> +@code{detect_hssr_fault} instruction pattern.
> >> +
> >> @cindex @code{IFN_SCATTER_STORE} internal function
> >> @item @samp{IFN_SCATTER_STORE}
> >> The @code{IFN_SCATTER_STORE} internal function directly maps to the
> >> diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi
> >> index 80761b0a8a6..1dd9050b6e3 100644
> >> --- a/gcc/doc/md.texi
> >> +++ b/gcc/doc/md.texi
> >> @@ -7823,6 +7823,48 @@ of @code{QI} elements.
> >>
> >> This pattern is not allowed to @code{FAIL}.
> >>
> >> +@mdindex mask_ff_hssr_load@var{m}@var{n}
> >> +@anchor{mask_ff_hssr_load@var{m}@var{n}}
> >> +@item @samp{mask_ff_hssr_load@var{m}@var{n}}
> >> +Perform a masked hardware-safe-speculative load of vector from memory
> >> +operand 1 of mode @var{m} into register operand 0. The mask is
> provided in
> >> +register operand 2 of mode @var{n}. Operand 3 (the ``else value'') is of
> >> mode
> >> +@var{m} and specifies which value is loaded when the mask is unset.
> >> +The predicate of operand 3 must only accept the else values that the
> target
> >> +actually supports. Currently three values are attempted, zero, -1, and
> >> +undefined. GCC handles an else value of zero more efficiently than -1 or
> >> +undefined.
> >> +
> >> +The instruction must also set the global HSSR state so it can be read by
> >> +@samp{read_hssr_state@var{m}}.
> >> +
> >> +This pattern is not allowed to @code{FAIL}.
> >> +
> >> +@mdindex read_hssr_state@var{m}
> >> +@anchor{read_hssr_state@var{m}}
> >> +@item @samp{read_hssr_state@var{m}}
> >> +Reads the global hardware safe speculative read state as a mask of mode
> >> @var{m}
> >> +into register operand 0.
> >> +
> >> +This pattern is not allowed to @code{FAIL}.
> >> +
> >> +@mdindex set_hssr_state@var{m}
> >> +@anchor{set_hssr_state@var{m}}
> >> +@item @samp{set_hssr_state@var{m}}
> >> +Sets the global HSSR state to be the mask in operand 0 of mode @var{m}.
> >> +
> >> +This pattern is not allowed to @code{FAIL}.
> >> +
> >> +@mdindex detect_hssr_fault@var{m}
> >> +@anchor{detect_hssr_fault@var{m}}
> >> +@item @samp{detect_hssr_fault@var{m}}
> >> +Checks if the given hardware safe speculative read mask of mode @var{m}
> in
> >> +operand 1 contains a partial read requiring a fixup path to be taken.
> >> +
> >> +Sets operand 0 of mode @samp{QI} to 1 if there is, and 0 otherwise.
> >> +
> >> +This pattern is not allowed to @code{FAIL}.
> >
> > So this is purely a boolean?
>
> It's not just a boolean, this expands to an operation which takes a HSSR
> state and checks if there is a fault in that state (equivalent to state
> != { -1, ... }).
>
> This is helpful for us as it allows us to expand that directly to
>
> rdffrs
> b.nlast
>
> rather than
>
> rdffrs p1
> nots p1
> b.any
>
> Which requires backend knowledge that after FFR loads, the FFR state is
> "continuous", ie, if the last element is active, they all are.
I see, but is this then not an expansion or instruction selection optimization?
It seems like before expand the additional IFN doesn't gain us much and
requiring
the boolean result back is only because it's still in the if right?
we do a similar optimization on vec_cbranch_*, so I wonder whether this
shouldn't
be similar, so something like a vec_cbranch_hssr or something. Or at the least
that
the transform be done in gimple-isel so the vectorizer just emits the usual
standard
gimple operations?
Doing it in expansion has the benefit of not needing to define the intermediate
results which have to be optimized out later.
I don't feel that strongly about it but given the description above it does
feel more
like a target selection/expansion optimization especially given the comments
below.
Any thoughts on this Richi?
Thanks,
Tamar
>
> There was also another reason explained below.
>
> > Why not make it a vect param? That allows for better
> > testing and experimentation and would allow a backend to set it based on a
> CPU
> > cost model.
> >
> > I assume the branch is on the actual result of the read_hssr_state? So
> codegen is
> > state <- read_hssr_state
> > if (state && param_hssr_avoid_dependency)
> > ...
> > use state.
> > else
> > state = param_hssr_avoid_dependency ? loop_mask : state;
> > ...
> >
> > so that when 0 it folds the branch away?
> >
> > I haven't looked at it's usages yet but I am assuming that's sorta what it
> > looks
> like?
>
> Hmm okay I went to explain the other reason, as I thought you had
> misunderstood and the reason was based off a conversation I had with
> Richard S
> (see https://gcc.gnu.org/pipermail/gcc-patches/2026-June/721275.html)
>
> But in explaining I realised this is now pointless...
>
> It used to be important to "hide" the check of
>
> if (HSSR state != {1,1,...})
>
> as otherwise it got optimized so that in the "good" path there was a
> dependency on "HSSR state" for the next mask, which would have been very
> bad for performance.
>
> But now that I have changed the loop form to not preserve alignment this
> is no longer an issue.
>
> So just the first reason.
>
> Thanks,
> Alfie
> >
> > Thanks,
> > Tamar
> >
> >> +
> >> @mdindex vec_perm@var{m}
> >> @item @samp{vec_perm@var{m}}
> >> Output a (variable) vector permutation. Operand 0 is the destination
> >> diff --git a/gcc/gimple-fold.cc b/gcc/gimple-fold.cc
> >> index e3df0d1163c..fbd06f9dddd 100644
> >> --- a/gcc/gimple-fold.cc
> >> +++ b/gcc/gimple-fold.cc
> >> @@ -6049,6 +6049,10 @@ partial_load_store_mask_state (gcall *call, tree
> >> vectype)
> >> static tree
> >> gimple_fold_partial_load_store_mem_ref (gcall *call, tree vectype)
> >> {
> >> + internal_fn ifn = gimple_call_internal_fn (call);
> >> + /* HSSR loads cannot be folded to MEM_REFs. */
> >> + if (internal_hssr_load_fn_p (ifn))
> >> + return NULL_TREE;
> >> /* Only fold if all lanes are active (unconditional). */
> >> if (partial_load_store_mask_state (call, vectype) != MASK_ALL_ACTIVE)
> >> return NULL_TREE;
> >> @@ -6106,6 +6110,10 @@ gimple_fold_partial_load_store
> >> (gimple_stmt_iterator *gsi, gcall *call)
> >> a store, move the vops from the old stmt to the new one. */
> >> if (!is_gimple_reg (lhs))
> >> gimple_move_vops (new_stmt, call);
> >> + /* HSSR loads have a VDEF in addition to represent changes to the
> >> + global HSSR state. */
> >> + if (internal_hssr_load_fn_p (ifn))
> >> + unlink_stmt_vdef (call);
> >> gsi_replace (gsi, new_stmt, false);
> >> return true;
> >> }
> >> @@ -6387,6 +6395,7 @@ gimple_fold_call (gimple_stmt_iterator *gsi,
> bool
> >> inplace)
> >> case IFN_MASK_LEN_SCATTER_STORE:
> >> case IFN_MASK_STORE_LANES:
> >> case IFN_MASK_LEN_STORE_LANES:
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> changed |= gimple_fold_partial_load_store (gsi, stmt);
> >> break;
> >> default:
> >> diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc
> >> index 3fdf89e0e6e..6f451f9e3ff 100644
> >> --- a/gcc/internal-fn.cc
> >> +++ b/gcc/internal-fn.cc
> >> @@ -197,6 +197,9 @@ init_internal_fns ()
> >> #define crc_direct { 1, -1, true }
> >> #define reduc_sbool_direct { 0, 0, true }
> >> #define select_vl_direct { 2, 0, false }
> >> +#define read_hssr_state_direct { -1, -1, false }
> >> +#define set_hssr_state_direct { 0, 0, false }
> >> +#define detect_hssr_fault_direct { 0, 0, false }
> >>
> >> const direct_internal_fn_info direct_internal_fn_array[IFN_LAST + 1] = {
> >> #define DEF_INTERNAL_FN(CODE, FLAGS, FNSPEC) not_direct,
> >> @@ -4317,6 +4320,21 @@ expand_reduc_sbool_optab_fn (internal_fn
> fn,
> >> gcall *stmt, direct_optab optab)
> >> #define expand_select_vl_optab_fn(FN, STMT, OPTAB) \
> >> expand_convert_optab_fn (FN, STMT, OPTAB, 3)
> >>
> >> +#define expand_read_hssr_state_optab_fn(FN, STMT, OPTAB) \
> >> + expand_direct_optab_fn (FN, STMT, OPTAB, 0)
> >> +
> >> +#define expand_detect_hssr_fault_optab_fn(FN, STMT, OPTAB) \
> >> + expand_direct_optab_fn (FN, STMT, OPTAB, 1)
> >> +
> >> +static void
> >> +expand_set_hssr_state_optab_fn (internal_fn fn, gcall *stmt,
> >> + direct_optab optab)
> >> +{
> >> + tree_pair types = direct_internal_fn_types (fn, stmt);
> >> + insn_code icode = direct_optab_handler (optab, TYPE_MODE
> (types.first));
> >> + expand_fn_using_insn (stmt, icode, 0, 1);
> >> +}
> >> +
> >> /* Expanders for optabs that can use expand_convert_optab_fn. */
> >>
> >> #define expand_unary_convert_optab_fn(FN, STMT, OPTAB) \
> >> @@ -4434,6 +4452,9 @@ multi_vector_optab_supported_p
> (convert_optab
> >> optab, tree_pair types,
> >> #define direct_vec_extract_optab_supported_p
> convert_optab_supported_p
> >> #define direct_reduc_sbool_optab_supported_p
> direct_optab_supported_p
> >> #define direct_select_vl_optab_supported_p convert_optab_supported_p
> >> +#define direct_read_hssr_state_optab_supported_p
> >> direct_optab_supported_p
> >> +#define direct_set_hssr_state_optab_supported_p
> >> direct_optab_supported_p
> >> +#define direct_detect_hssr_fault_optab_supported_p
> >> direct_optab_supported_p
> >>
> >> /* Return the optab used by internal function FN. */
> >>
> >> @@ -5083,6 +5104,7 @@ internal_load_fn_p (internal_fn fn)
> >> case IFN_MASK_LEN_GATHER_LOAD:
> >> case IFN_LEN_LOAD:
> >> case IFN_MASK_LEN_LOAD:
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> return true;
> >>
> >> default:
> >> @@ -5090,6 +5112,20 @@ internal_load_fn_p (internal_fn fn)
> >> }
> >> }
> >>
> >> +/* Return true if IFN is a HSSR load. */
> >> +
> >> +bool
> >> +internal_hssr_load_fn_p (internal_fn fn)
> >> +{
> >> + switch (fn)
> >> + {
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> + return true;
> >> + default:
> >> + return false;
> >> + }
> >> +}
> >> +
> >> /* Return true if IFN is some form of store to memory. */
> >>
> >> bool
> >> @@ -5255,6 +5291,7 @@ internal_fn_else_index (internal_fn fn)
> >> case IFN_MASK_LEN_LOAD:
> >> case IFN_MASK_LOAD_LANES:
> >> case IFN_MASK_LEN_LOAD_LANES:
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> return 3;
> >>
> >> case IFN_COND_FMA:
> >> @@ -5295,6 +5332,7 @@ internal_fn_mask_index (internal_fn fn)
> >> case IFN_MASK_LEN_STORE_LANES:
> >> case IFN_MASK_LEN_LOAD:
> >> case IFN_MASK_LEN_STORE:
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> return 2;
> >>
> >> case IFN_MASK_LEN_STRIDED_LOAD:
> >> @@ -5363,6 +5401,7 @@ internal_fn_alias_ptr_index (internal_fn fn)
> >> case IFN_SCATTER_STORE:
> >> case IFN_MASK_SCATTER_STORE:
> >> case IFN_MASK_LEN_SCATTER_STORE:
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> return 1;
> >>
> >> default:
> >> diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def
> >> index 28ad5b7809d..68397c7bced 100644
> >> --- a/gcc/internal-fn.def
> >> +++ b/gcc/internal-fn.def
> >> @@ -651,6 +651,24 @@ DEF_INTERNAL_FN (BITINTTOFLOAT, ECF_PURE |
> >> ECF_LEAF, ". R . ")
> >> DEF_INTERNAL_OPTAB_FN (BIT_ANDN, ECF_CONST, andn, binary)
> >> DEF_INTERNAL_OPTAB_FN (BIT_IORN, ECF_CONST, iorn, binary)
> >>
> >> +/* Hardware Safe Speculative Reads internal functions. */
> >> +DEF_INTERNAL_OPTAB_FN (MASK_FF_HSSR_LOAD,
> >> + 0,
> >> + mask_ff_hssr_load,
> >> + mask_load)
> >> +DEF_INTERNAL_OPTAB_FN (SET_HSSR_STATE,
> >> + ECF_NOTHROW,
> >> + set_hssr_state,
> >> + set_hssr_state)
> >> +DEF_INTERNAL_OPTAB_FN (READ_HSSR_STATE,
> >> + ECF_NOTHROW,
> >> + read_hssr_state,
> >> + read_hssr_state)
> >> +DEF_INTERNAL_OPTAB_FN (DETECT_HSSR_FAULT,
> >> + ECF_CONST,
> >> + detect_hssr_fault,
> >> + detect_hssr_fault)
> >> +
> >> #undef DEF_INTERNAL_WIDENING_OPTAB_FN
> >> #undef DEF_INTERNAL_SIGNED_COND_FN
> >> #undef DEF_INTERNAL_COND_FN
> >> diff --git a/gcc/internal-fn.h b/gcc/internal-fn.h
> >> index 30930f388b0..d6a28a47d55 100644
> >> --- a/gcc/internal-fn.h
> >> +++ b/gcc/internal-fn.h
> >> @@ -234,6 +234,7 @@ extern bool can_interpret_as_conditional_op_p
> >> (gimple *, tree *,
> >> tree *, tree *, tree *);
> >>
> >> extern bool internal_load_fn_p (internal_fn);
> >> +extern bool internal_hssr_load_fn_p (internal_fn);
> >> extern bool internal_store_fn_p (internal_fn);
> >> extern bool internal_gather_scatter_fn_p (internal_fn);
> >> extern int internal_fn_mask_index (internal_fn);
> >> diff --git a/gcc/optabs.def b/gcc/optabs.def
> >> index d1f70a9b44a..4ae3ef8b78d 100644
> >> --- a/gcc/optabs.def
> >> +++ b/gcc/optabs.def
> >> @@ -118,6 +118,9 @@ OPTAB_CD (usdot_prod_optab,
> >> "usdot_prod$I$a$b")
> >> OPTAB_CD (while_ult_optab, "while_ult$a$b")
> >> OPTAB_CD (select_vl_optab, "select_vl$a$b")
> >>
> >> +/* Hardware Safe Speculative Reads optabs. */
> >> +OPTAB_CD (mask_ff_hssr_load_optab, "mask_ff_hssr_load$a$b")
> >> +
> >> OPTAB_NL(add_optab, "add$P$a3", PLUS, "add", '3',
> >> gen_int_fp_fixed_libfunc)
> >> OPTAB_NX(add_optab, "add$F$a3")
> >> OPTAB_NX(add_optab, "add$Q$a3")
> >> @@ -241,6 +244,11 @@ OPTAB_D (push_optab, "push$a1")
> >> OPTAB_D (reload_in_optab, "reload_in$a")
> >> OPTAB_D (reload_out_optab, "reload_out$a")
> >>
> >> +/* Hardware Safe Speculative Reads optabs. */
> >> +OPTAB_D (read_hssr_state_optab, "read_hssr_state$a")
> >> +OPTAB_D (set_hssr_state_optab, "set_hssr_state$a")
> >> +OPTAB_D (detect_hssr_fault_optab, "detect_hssr_fault$a")
> >> +
> >> OPTAB_DC(cbranch_optab, "cbranch$a4", COMPARE)
> >> OPTAB_D (tbranch_eq_optab, "tbranch_eq$a3")
> >> OPTAB_D (tbranch_ne_optab, "tbranch_ne$a3")
> >> diff --git a/gcc/tree-data-ref.cc b/gcc/tree-data-ref.cc
> >> index d6da0fe58c3..4100c66521b 100644
> >> --- a/gcc/tree-data-ref.cc
> >> +++ b/gcc/tree-data-ref.cc
> >> @@ -5905,6 +5905,7 @@ get_references_in_stmt (gimple *stmt,
> >> vec<data_ref_loc, va_heap> *references)
> >> break;
> >> }
> >> case IFN_MASK_LOAD:
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> case IFN_MASK_STORE:
> >> break;
> >> case IFN_MASK_CALL:
> >> @@ -5962,6 +5963,7 @@ get_references_in_stmt (gimple *stmt,
> >> vec<data_ref_loc, va_heap> *references)
> >> switch (gimple_call_internal_fn (stmt))
> >> {
> >> case IFN_MASK_LOAD:
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> if (gimple_call_lhs (stmt) == NULL_TREE)
> >> break;
> >> ref.is_read = true;
> >> diff --git a/gcc/tree-ssa-alias.cc b/gcc/tree-ssa-alias.cc
> >> index 7fc87b3b944..6122a6505c0 100644
> >> --- a/gcc/tree-ssa-alias.cc
> >> +++ b/gcc/tree-ssa-alias.cc
> >> @@ -2888,6 +2888,7 @@ ref_maybe_used_by_call_p_1 (gcall *call, ao_ref
> >> *ref, bool tbaa_p)
> >> case IFN_MASK_LEN_STORE_LANES:
> >> goto process_args;
> >> case IFN_MASK_LOAD:
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> case IFN_LEN_LOAD:
> >> case IFN_MASK_LEN_LOAD:
> >> case IFN_MASK_LOAD_LANES:
> >> diff --git a/gcc/tree-ssa-loop-ivopts.cc b/gcc/tree-ssa-loop-ivopts.cc
> >> index fa3c2c136fc..3765b7070e9 100644
> >> --- a/gcc/tree-ssa-loop-ivopts.cc
> >> +++ b/gcc/tree-ssa-loop-ivopts.cc
> >> @@ -2371,6 +2371,7 @@ get_mem_type_for_internal_fn (gcall *call, tree
> >> *op_p)
> >> switch (gimple_call_internal_fn (call))
> >> {
> >> case IFN_MASK_LOAD:
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> case IFN_MASK_LOAD_LANES:
> >> case IFN_MASK_LEN_LOAD_LANES:
> >> case IFN_LEN_LOAD:
> >> @@ -7639,6 +7640,7 @@ get_alias_ptr_type_for_ptr_address (iv_use
> *use)
> >> switch (gimple_call_internal_fn (call))
> >> {
> >> case IFN_MASK_LOAD:
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> case IFN_MASK_STORE:
> >> case IFN_MASK_LOAD_LANES:
> >> case IFN_MASK_STORE_LANES:
> >> diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc
> >> index eee02124d5b..e0a32ade79a 100644
> >> --- a/gcc/tree-vect-slp.cc
> >> +++ b/gcc/tree-vect-slp.cc
> >> @@ -587,6 +587,9 @@ vect_get_operand_map (const gimple *stmt, bool
> >> gather_scatter_p,
> >> case IFN_MASK_LOAD:
> >> return gather_scatter_p ? off_arg2_arg3_map : arg2_arg3_map;
> >>
> >> + case IFN_MASK_FF_HSSR_LOAD:
> >> + return arg2_arg3_map;
> >> +
> >> case IFN_GATHER_LOAD:
> >> return arg2_map;
> >>
> >> --
> >> 2.43.0
> >
