I'm implementing the multi-vector load/store support described here:
http://gcc.gnu.org/ml/gcc/2011-03/msg00322.html
For the purposes of this message, the main thing is that the losds
and stores are more naturally represented as GIMPLE_CALLs rather
than GIMPLE_ASSIGNs. That is, we have:
uintWxM_t t[N];
t[0...N-1] = LOAD_LANES (addr[0...M*N-1]);
addr[0...M*N-1] = STORE_LANES (t[0...N-1]);
The question is: how should these LOAD_LANES and STORE_LANES functions
be represented? I'd first tried using built-in functions, but that's
difficult for a number of reasons:
- they mustn't be called by the user. (The interface isn't really
C-compatible, for one thing.)
- the functions are overloaded, and it isn't easy to enumerate
every type instantiation up-front.
I wondered about having a new tree code to represent internal-only
functions. However, Richard suggested that it would be better to
avoid using the GIMPLE_CALL function operand at all, and therefore
avoid the overhead of creating a tree node for the function and
another node for its type. The idea was that we could put the
function opcode in the subcode field (at least for now).
Here's an implementation of that idea. Does it look OK?
I've got the basic interleaved load/store stuff working, and hope to
submit it next week. I thought I'd better post this as an RFC first,
because it might be more controversial.
Tested on arm-linux-gnueabi, with the other patches.
Richard
gcc/
* Makefile.in (INTERNAL_FN_DEF, INTERNAL_FN_H): Define.
(GIMPLE_H): Include $(INTERNAL_FN_H).
(OBJS-common): Add internal-fn.o.
(internal-fn.o): New rule.
* internal-fn.def: New file.
* internal-fn.h: Likewise.
* internal-fn.c: Likewise.
* gimple.h: Include internal-fn.h.
(GF_CALL_INTERNAL_FN): New gf_mask.
(GF_CALL_CANNOT_INLINE, GF_CALL_FROM_THUNK, GF_CALL_RETURN_SLOT_OPT)
(GF_CALL_TAILCALL, GF_CALL_VA_ARG_PACK, GF_CALL_NOTHROW): Bump values.
(gimple_build_call_internal): Declare.
(gimple_build_call_internal_vec): Likewise.
(gimple_call_same_target_p): Likewise.
(gimple_call_fn): Assert that the function is nonnull.
(gimple_call_internal_p): New function.
(gimple_call_internal_fn): Likewise.
(gimple_call_set_internal_fn): Likewise.
(gimple_call_set_fn): Invalidate the GF_CALL_INTERNAL_FN field.
(gimple_call_fndecl): Return null for internal calls.
(gimple_call_return_type): Use the type of the lhs for internal calls.
* gimple.c (gimple_build_call_1): Use gimple_call_set_fn rather
than gimple_set_op.
(gimple_build_call_internal_1): New function.
(gimple_build_call_internal): Likewise.
(gimple_build_call_internal_vec): Likewise.
(walk_gimple_op): Skip the GIMPLE_CALL function for calls to
internal functions.
(gimple_call_same_target_p): New function.
(gimple_call_flags): Handle calls to internal functions.
(gimple_call_arg_flags): Likewise.
(gimple_call_return_flags): Likewise.
(gimple_has_side_effects): Likewise.
(gimple_call_copy_skip_args): Likewise.
* cfgexpand.c (expand_gimple_call_internal): New function.
(expand_call_stmt): Use it.
* expr.c (expand_expr_real_1): Don't call promote_function_mode
for internal functions.
* gimple-fold.c (gimple_fold_stmt_to_constant_1): Don't fold
calls to internal functions.
* gimple-low.c (gimple_check_call_args): Handle calls to internal
functions.
* gimple-pretty-print.c (dump_gimple_call): Likewise.
* ipa-prop.c (ipa_analyze_call_uses): Likewise.
* tree-cfg.c (verify_gimple_call): Likewise.
(do_warn_unused_result): Likewise.
* tree-eh.c (same_handler_p): Use gimple_call_same_target_p.
* tree-ssa-ccp.c (ccp_fold_stmt): Handle calls to internal functions.
* tree-ssa-dom.c (hashable_expr): Use the gimple statement to record
the target of a call.
(initialize_hash_element): Update accordingly.
(hashable_expr_equal_p): Use gimple_call_same_target_p.
(iterative_hash_hashable_expr): Handle calls to internal functions.
(print_expr_hash_elt): Likewise.
* tree-ssa-pre.c (eliminate): Likewise.
* tree-ssa-sccvn.c (copy_reference_ops_from_call): Likewise.
* tree-ssa-structalias.c (get_fi_for_callee): Likewise.
(find_func_aliases): Likewise.
* value-prof.c (gimple_ic_transform): Likewise.
(gimple_indirect_call_to_profile): Likewise.
Index: gcc/Makefile.in
===================================================================
--- gcc/Makefile.in 2011-04-07 09:23:36.000000000 +0100
+++ gcc/Makefile.in 2011-04-07 11:05:54.000000000 +0100
@@ -893,6 +893,8 @@ RTL_ERROR_H = $(RTL_H) $(DIAGNOSTIC_CORE
READ_MD_H = $(OBSTACK_H) $(HASHTAB_H) read-md.h
PARAMS_H = params.h params.def
BUILTINS_DEF = builtins.def sync-builtins.def omp-builtins.def
+INTERNAL_FN_DEF = internal-fn.def
+INTERNAL_FN_H = internal-fn.h $(INTERNAL_FN_DEF)
TREE_H = tree.h all-tree.def tree.def c-family/c-common.def \
$(lang_tree_files) $(MACHMODE_H) tree-check.h $(BUILTINS_DEF) \
$(INPUT_H) statistics.h $(VEC_H) treestruct.def $(HASHTAB_H) \
@@ -901,8 +903,8 @@ TREE_H = tree.h all-tree.def tree.def c-
REGSET_H = regset.h $(BITMAP_H) hard-reg-set.h
BASIC_BLOCK_H = basic-block.h $(PREDICT_H) $(VEC_H) $(FUNCTION_H) cfghooks.h
GIMPLE_H = gimple.h gimple.def gsstruct.def pointer-set.h $(VEC_H) \
- $(GGC_H) $(BASIC_BLOCK_H) $(TARGET_H) tree-ssa-operands.h \
- tree-ssa-alias.h vecir.h
+ vecir.h $(GGC_H) $(BASIC_BLOCK_H) $(TARGET_H) tree-ssa-operands.h \
+ tree-ssa-alias.h $(INTERNAL_FN_H)
GCOV_IO_H = gcov-io.h gcov-iov.h auto-host.h
COVERAGE_H = coverage.h $(GCOV_IO_H)
DEMANGLE_H = $(srcdir)/../include/demangle.h
@@ -1274,6 +1276,7 @@ OBJS-common = \
init-regs.o \
input.o \
integrate.o \
+ internal-fn.o \
intl.o \
ira.o \
ira-build.o \
@@ -2758,6 +2761,8 @@ tree-object-size.o: tree-object-size.c $
$(TM_H) $(TREE_H) $(DIAGNOSTIC_CORE_H) $(DIAGNOSTIC_H) $(TREE_FLOW_H) \
$(TREE_PASS_H) tree-ssa-propagate.h tree-pretty-print.h \
gimple-pretty-print.h
+internal-fn.o : internal-fn.c $(CONFIG_H) $(SYSTEM_H) coretypes.h \
+ $(INTERNAL_FN_H) $(TREE_H) $(EXPR_H) $(OPTABS_H)
gimple.o : gimple.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TREE_H) \
$(GGC_H) $(GIMPLE_H) $(DIAGNOSTIC_CORE_H) $(DIAGNOSTIC_H) gt-gimple.h \
$(TREE_FLOW_H) value-prof.h $(FLAGS_H) $(DEMANGLE_H) \
Index: gcc/internal-fn.def
===================================================================
--- /dev/null 2011-03-23 08:42:11.268792848 +0000
+++ gcc/internal-fn.def 2011-04-08 09:45:32.000000000 +0100
@@ -0,0 +1,34 @@
+/* Internal functions.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+/* This file specifies a list of internal "functions". These functions
+ differ from built-in functions in that they have no linkage and cannot
+ be called directly by the user. They represent operations that are only
+ synthesised by GCC itself.
+
+ Internal functions are used instead of tree codes if the operation
+ and its operands are more naturally represented as a GIMPLE_CALL
+ than a GIMPLE_ASSIGN.
+
+ Each entry in this file has the form:
+
+ DEF_INTERNAL_FN (NAME, FLAGS)
+
+ where NAME is the name of the function and FLAGS is a set of
+ ECF_* flags. */
Index: gcc/internal-fn.h
===================================================================
--- /dev/null 2011-03-23 08:42:11.268792848 +0000
+++ gcc/internal-fn.h 2011-04-07 11:05:54.000000000 +0100
@@ -0,0 +1,51 @@
+/* Internal functions.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+#ifndef GCC_INTERNAL_FN_H
+#define GCC_INTERNAL_FN_H
+
+enum internal_fn {
+#define DEF_INTERNAL_FN(CODE, FLAGS) IFN_##CODE,
+#include "internal-fn.def"
+#undef DEF_INTERNAL_FN
+ IFN_LAST
+};
+
+/* Return the name of internal function FN. The name is only meaningful
+ for dumps; it has no linkage. */
+
+static inline const char *
+internal_fn_name (enum internal_fn fn)
+{
+ extern const char *const internal_fn_name_array[];
+ return internal_fn_name_array[(int) fn];
+}
+
+/* Return the ECF_* flags for function FN. */
+
+static inline int
+internal_fn_flags (enum internal_fn fn)
+{
+ extern const int internal_fn_flags_array[];
+ return internal_fn_flags_array[(int) fn];
+}
+
+extern void expand_internal_call (enum internal_fn, tree, tree *);
+
+#endif
Index: gcc/internal-fn.c
===================================================================
--- /dev/null 2011-03-23 08:42:11.268792848 +0000
+++ gcc/internal-fn.c 2011-04-08 09:47:47.000000000 +0100
@@ -0,0 +1,64 @@
+/* Internal functions.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3. If not see
+<http://www.gnu.org/licenses/>. */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "internal-fn.h"
+#include "tree.h"
+#include "expr.h"
+#include "optabs.h"
+
+/* The names of each internal function, indexed by function number. */
+const char *const internal_fn_name_array[] = {
+#define DEF_INTERNAL_FN(CODE, FLAGS) #CODE,
+#include "internal-fn.def"
+#undef DEF_INTERNAL_FN
+ "<invalid-fn>"
+};
+
+/* The ECF_* flags of each internal function, indexed by function number. */
+const int internal_fn_flags_array[] = {
+#define DEF_INTERNAL_FN(CODE, FLAGS) FLAGS,
+#include "internal-fn.def"
+#undef DEF_INTERNAL_FN
+ 0
+};
+
+/* Routines to expand each internal function, indexed by function number.
+ Each routine has the prototype:
+
+ expand_<NAME> (tree lhs, tree *args)
+
+ where LHS and ARGS are as for expand_internal_call. */
+static void (*const internal_fn_expanders[]) (tree, tree *) = {
+#define DEF_INTERNAL_FN(CODE, FLAGS) expand_##CODE,
+#include "internal-fn.def"
+#undef DEF_INTERNAL_FN
+ 0
+};
+
+/* Expand a call to internal function FN. ARGS is an array of the
+ function's arguments and LHS is where the result should be stored. */
+
+void
+expand_internal_call (enum internal_fn fn, tree lhs, tree *args)
+{
+ internal_fn_expanders[(int) fn] (lhs, args);
+}
Index: gcc/gimple.h
===================================================================
--- gcc/gimple.h 2011-04-07 09:21:45.000000000 +0100
+++ gcc/gimple.h 2011-04-08 09:11:09.000000000 +0100
@@ -30,6 +30,7 @@ #define GCC_GIMPLE_H
#include "basic-block.h"
#include "tree-ssa-operands.h"
#include "tree-ssa-alias.h"
+#include "internal-fn.h"
struct gimple_seq_node_d;
typedef struct gimple_seq_node_d *gimple_seq_node;
@@ -96,12 +97,15 @@ enum gimple_rhs_class
enum gf_mask {
GF_ASM_INPUT = 1 << 0,
GF_ASM_VOLATILE = 1 << 1,
- GF_CALL_CANNOT_INLINE = 1 << 0,
- GF_CALL_FROM_THUNK = 1 << 1,
- GF_CALL_RETURN_SLOT_OPT = 1 << 2,
- GF_CALL_TAILCALL = 1 << 3,
- GF_CALL_VA_ARG_PACK = 1 << 4,
- GF_CALL_NOTHROW = 1 << 5,
+ /* Leave a gap in the middle so that more flags can be added, or the
+ range of the internal function expanded. */
+ GF_CALL_INTERNAL_FN = 0x3f << 0,
+ GF_CALL_CANNOT_INLINE = 1 << 10,
+ GF_CALL_FROM_THUNK = 1 << 11,
+ GF_CALL_RETURN_SLOT_OPT = 1 << 12,
+ GF_CALL_TAILCALL = 1 << 13,
+ GF_CALL_VA_ARG_PACK = 1 << 14,
+ GF_CALL_NOTHROW = 1 << 15,
GF_OMP_PARALLEL_COMBINED = 1 << 0,
/* True on an GIMPLE_OMP_RETURN statement if the return does not require
@@ -817,6 +821,8 @@ #define gimple_build_debug_bind(var,val,
gimple gimple_build_call_vec (tree, VEC(tree, heap) *);
gimple gimple_build_call (tree, unsigned, ...);
+gimple gimple_build_call_internal (enum internal_fn, unsigned, ...);
+gimple gimple_build_call_internal_vec (enum internal_fn, VEC(tree, heap) *);
gimple gimple_build_call_from_tree (tree);
gimple gimplify_assign (tree, tree, gimple_seq *);
gimple gimple_build_cond (enum tree_code, tree, tree, tree, tree);
@@ -861,6 +867,7 @@ gimple_seq gimple_seq_alloc (void);
void gimple_seq_free (gimple_seq);
void gimple_seq_add_seq (gimple_seq *, gimple_seq);
gimple_seq gimple_seq_copy (gimple_seq);
+bool gimple_call_same_target_p (const_gimple, const_gimple);
int gimple_call_flags (const_gimple);
int gimple_call_return_flags (const_gimple);
int gimple_call_arg_flags (const_gimple, unsigned);
@@ -2007,8 +2014,33 @@ gimple_call_set_lhs (gimple gs, tree lhs
static inline tree
gimple_call_fn (const_gimple gs)
{
+ tree op;
+
GIMPLE_CHECK (gs, GIMPLE_CALL);
- return gimple_op (gs, 1);
+ op = gimple_op (gs, 1);
+ gcc_gimple_checking_assert (op != NULL_TREE);
+ return op;
+}
+
+
+/* Return true if call GS calls an internal-only function, as enumerated
+ by internal_fn. */
+
+static inline bool
+gimple_call_internal_p (const_gimple gs)
+{
+ GIMPLE_CHECK (gs, GIMPLE_CALL);
+ return gimple_op (gs, 1) == NULL_TREE;
+}
+
+
+/* Return the target of internal call GS. */
+
+static inline enum internal_fn
+gimple_call_internal_fn (const_gimple gs)
+{
+ gcc_gimple_checking_assert (gimple_call_internal_p (gs));
+ return (enum internal_fn) (gs->gsbase.subcode & GF_CALL_INTERNAL_FN);
}
@@ -2023,12 +2055,26 @@ gimple_call_fn_ptr (const_gimple gs)
}
+/* Set internal function FN to be the function called by call statement GS. */
+
+static inline void
+gimple_call_set_internal_fn (gimple gs, enum internal_fn fn)
+{
+ GIMPLE_CHECK (gs, GIMPLE_CALL);
+ gs->gsbase.subcode &= ~GF_CALL_INTERNAL_FN;
+ gs->gsbase.subcode |= fn;
+ gimple_set_op (gs, 1, NULL_TREE);
+}
+
+
/* Set FN to be the function called by call statement GS. */
static inline void
gimple_call_set_fn (gimple gs, tree fn)
{
GIMPLE_CHECK (gs, GIMPLE_CALL);
+ gs->gsbase.subcode &= ~GF_CALL_INTERNAL_FN;
+ gs->gsbase.subcode |= IFN_LAST;
gimple_set_op (gs, 1, fn);
}
@@ -2050,7 +2096,12 @@ gimple_call_set_fndecl (gimple gs, tree
static inline tree
gimple_call_fndecl (const_gimple gs)
{
- tree addr = gimple_call_fn (gs);
+ tree addr;
+
+ if (gimple_call_internal_p (gs))
+ return NULL_TREE;
+
+ addr = gimple_call_fn (gs);
if (TREE_CODE (addr) == ADDR_EXPR)
{
tree fndecl = TREE_OPERAND (addr, 0);
@@ -2073,8 +2124,14 @@ gimple_call_fndecl (const_gimple gs)
static inline tree
gimple_call_return_type (const_gimple gs)
{
- tree fn = gimple_call_fn (gs);
- tree type = TREE_TYPE (fn);
+ tree fn;
+ tree type;
+
+ if (gimple_call_internal_p (gs))
+ return TREE_TYPE (gimple_call_lhs (gs));
+
+ fn = gimple_call_fn (gs);
+ type = TREE_TYPE (fn);
/* See through the pointer. */
type = TREE_TYPE (type);
Index: gcc/gimple.c
===================================================================
--- gcc/gimple.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/gimple.c 2011-04-07 11:05:54.000000000 +0100
@@ -230,7 +230,7 @@ gimple_build_call_1 (tree fn, unsigned n
gimple s = gimple_build_with_ops (GIMPLE_CALL, ERROR_MARK, nargs + 3);
if (TREE_CODE (fn) == FUNCTION_DECL)
fn = build_fold_addr_expr (fn);
- gimple_set_op (s, 1, fn);
+ gimple_call_set_fn (s, fn);
gimple_call_reset_alias_info (s);
return s;
}
@@ -276,6 +276,58 @@ gimple_build_call (tree fn, unsigned nar
}
+/* Helper for gimple_build_call_internal and gimple_build_call_internal_vec.
+ Build the basic components of a GIMPLE_CALL statement to internal
+ function FN with NARGS arguments. */
+
+static inline gimple
+gimple_build_call_internal_1 (enum internal_fn fn, unsigned nargs)
+{
+ gimple s = gimple_build_with_ops (GIMPLE_CALL, ERROR_MARK, nargs + 3);
+ gimple_call_set_internal_fn (s, fn);
+ gimple_call_reset_alias_info (s);
+ return s;
+}
+
+
+/* Build a GIMPLE_CALL statement to internal function FN. NARGS is
+ the number of arguments. The ... are the arguments. */
+
+gimple
+gimple_build_call_internal (enum internal_fn fn, unsigned nargs, ...)
+{
+ va_list ap;
+ gimple call;
+ unsigned i;
+
+ call = gimple_build_call_internal_1 (fn, nargs);
+ va_start (ap, nargs);
+ for (i = 0; i < nargs; i++)
+ gimple_call_set_arg (call, i, va_arg (ap, tree));
+ va_end (ap);
+
+ return call;
+}
+
+
+/* Build a GIMPLE_CALL statement to internal function FN with the arguments
+ specified in vector ARGS. */
+
+gimple
+gimple_build_call_internal_vec (enum internal_fn fn, VEC(tree, heap) *args)
+{
+ unsigned i, nargs;
+ gimple call;
+
+ nargs = VEC_length (tree, args);
+ call = gimple_build_call_internal_1 (fn, nargs);
+ for (i = 0; i < nargs; i++)
+ gimple_call_set_arg (call, i, VEC_index (tree, args, i));
+
+ return call;
+}
+
+
/* Build a GIMPLE_CALL statement from CALL_EXPR T. Note that T is
assumed to be in GIMPLE form already. Minimal checking is done of
this fact. */
@@ -1398,9 +1450,12 @@ walk_gimple_op (gimple stmt, walk_tree_f
if (ret)
return ret;
- ret = walk_tree (gimple_call_fn_ptr (stmt), callback_op, wi, pset);
- if (ret)
- return ret;
+ if (!gimple_call_internal_p (stmt))
+ {
+ ret = walk_tree (gimple_call_fn_ptr (stmt), callback_op, wi, pset);
+ if (ret)
+ return ret;
+ }
for (i = 0; i < gimple_call_num_args (stmt); i++)
{
@@ -1772,6 +1827,19 @@ gimple_has_body_p (tree fndecl)
return (gimple_body (fndecl) || (fn && fn->cfg));
}
+/* Return true if calls C1 and C2 are known to go to the same function. */
+
+bool
+gimple_call_same_target_p (const_gimple c1, const_gimple c2)
+{
+ if (gimple_call_internal_p (c1))
+ return (gimple_call_internal_p (c2)
+ && gimple_call_internal_fn (c1) == gimple_call_internal_fn (c2));
+ else
+ return (!gimple_call_internal_p (c2)
+ && operand_equal_p (gimple_call_fn (c1), gimple_call_fn (c2), 0));
+}
+
/* Detect flags from a GIMPLE_CALL. This is just like
call_expr_flags, but for gimple tuples. */
@@ -1784,6 +1852,8 @@ gimple_call_flags (const_gimple stmt)
if (decl)
flags = flags_from_decl_or_type (decl);
+ else if (gimple_call_internal_p (stmt))
+ flags = internal_fn_flags (gimple_call_internal_fn (stmt));
else
{
t = TREE_TYPE (gimple_call_fn (stmt));
@@ -1804,8 +1874,14 @@ gimple_call_flags (const_gimple stmt)
int
gimple_call_arg_flags (const_gimple stmt, unsigned arg)
{
- tree type = TREE_TYPE (TREE_TYPE (gimple_call_fn (stmt)));
- tree attr = lookup_attribute ("fn spec", TYPE_ATTRIBUTES (type));
+ tree type;
+ tree attr;
+
+ if (gimple_call_internal_p (stmt))
+ return 0;
+
+ type = TREE_TYPE (TREE_TYPE (gimple_call_fn (stmt)));
+ attr = lookup_attribute ("fn spec", TYPE_ATTRIBUTES (type));
if (!attr)
return 0;
@@ -1845,6 +1921,9 @@ gimple_call_return_flags (const_gimple s
tree type;
tree attr = NULL_TREE;
+ if (gimple_call_internal_p (stmt))
+ return 0;
+
if (gimple_call_flags (stmt) & ECF_MALLOC)
return ERF_NOALIAS;
@@ -2301,7 +2380,8 @@ gimple_has_side_effects (const_gimple s)
return true;
}
- if (TREE_SIDE_EFFECTS (gimple_call_fn (s)))
+ if (!gimple_call_internal_p (s)
+ && TREE_SIDE_EFFECTS (gimple_call_fn (s)))
return true;
for (i = 0; i < nargs; i++)
@@ -2346,8 +2426,9 @@ gimple_rhs_has_side_effects (const_gimpl
/* We cannot use gimple_has_volatile_ops here,
because we must ignore a volatile LHS. */
- if (TREE_SIDE_EFFECTS (gimple_call_fn (s))
- || TREE_THIS_VOLATILE (gimple_call_fn (s)))
+ if (!gimple_call_internal_p (s)
+ && (TREE_SIDE_EFFECTS (gimple_call_fn (s))
+ || TREE_THIS_VOLATILE (gimple_call_fn (s))))
{
gcc_assert (gimple_has_volatile_ops (s));
return true;
@@ -3103,7 +3184,6 @@ canonicalize_cond_expr_cond (tree t)
gimple_call_copy_skip_args (gimple stmt, bitmap args_to_skip)
{
int i;
- tree fn = gimple_call_fn (stmt);
int nargs = gimple_call_num_args (stmt);
VEC(tree, heap) *vargs = VEC_alloc (tree, heap, nargs);
gimple new_stmt;
@@ -3112,7 +3192,11 @@ gimple_call_copy_skip_args (gimple stmt,
if (!bitmap_bit_p (args_to_skip, i))
VEC_quick_push (tree, vargs, gimple_call_arg (stmt, i));
- new_stmt = gimple_build_call_vec (fn, vargs);
+ if (gimple_call_internal_p (stmt))
+ new_stmt = gimple_build_call_internal_vec (gimple_call_internal_fn (stmt),
+ vargs);
+ else
+ new_stmt = gimple_build_call_vec (gimple_call_fn (stmt), vargs);
VEC_free (tree, heap, vargs);
if (gimple_call_lhs (stmt))
gimple_call_set_lhs (new_stmt, gimple_call_lhs (stmt));
Index: gcc/cfgexpand.c
===================================================================
--- gcc/cfgexpand.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/cfgexpand.c 2011-04-08 09:01:28.000000000 +0100
@@ -1830,6 +1830,23 @@ expand_gimple_cond (basic_block bb, gimp
return new_bb;
}
+/* A subroutine of expand_call_stmt. Expand GIMPLE_CALL statement STMT,
+ which is known to be to an internal function. */
+
+static void
+expand_gimple_call_internal (gimple stmt)
+{
+ tree lhs;
+ tree *args;
+ size_t i;
+
+ lhs = gimple_call_lhs (stmt);
+ args = XALLOCAVEC (tree, gimple_call_num_args (stmt));
+ for (i = 0; i < gimple_call_num_args (stmt); i++)
+ args[i] = gimple_call_arg (stmt, i);
+ expand_internal_call (gimple_call_internal_fn (stmt), lhs, args);
+}
+
/* A subroutine of expand_gimple_stmt_1, expanding one GIMPLE_CALL
statement STMT. */
@@ -1837,11 +1854,17 @@ expand_gimple_cond (basic_block bb, gimp
expand_call_stmt (gimple stmt)
{
tree exp;
- tree lhs = gimple_call_lhs (stmt);
+ tree lhs;
size_t i;
bool builtin_p;
tree decl;
+ if (gimple_call_internal_p (stmt))
+ {
+ expand_gimple_call_internal (stmt);
+ return;
+ }
+
exp = build_vl_exp (CALL_EXPR, gimple_call_num_args (stmt) + 3);
CALL_EXPR_FN (exp) = gimple_call_fn (stmt);
@@ -1879,6 +1902,7 @@ expand_call_stmt (gimple stmt)
SET_EXPR_LOCATION (exp, gimple_location (stmt));
TREE_BLOCK (exp) = gimple_block (stmt);
+ lhs = gimple_call_lhs (stmt);
if (lhs)
expand_assignment (lhs, exp, false);
else
Index: gcc/expr.c
===================================================================
--- gcc/expr.c 2011-04-07 11:05:36.000000000 +0100
+++ gcc/expr.c 2011-04-08 09:02:37.000000000 +0100
@@ -8528,10 +8528,15 @@ expand_expr_real_1 (tree exp, rtx target
enum machine_mode pmode;
/* Get the signedness to be used for this variable. Ensure we get
- the same mode we got when the variable was declared. */
+ the same mode we got when the variable was declared.
+
+ Note that calls to internal functions do not result in a
+ call instruction, so promote_function_mode is not meaningful
+ in that case. */
if (code == SSA_NAME
&& (g = SSA_NAME_DEF_STMT (ssa_name))
- && gimple_code (g) == GIMPLE_CALL)
+ && gimple_code (g) == GIMPLE_CALL
+ && !gimple_call_internal_p (g))
pmode = promote_function_mode (type, mode, &unsignedp,
TREE_TYPE
(TREE_TYPE (gimple_call_fn (g))),
Index: gcc/gimple-fold.c
===================================================================
--- gcc/gimple-fold.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/gimple-fold.c 2011-04-07 11:05:54.000000000 +0100
@@ -2856,7 +2856,13 @@ gimple_fold_stmt_to_constant_1 (gimple s
case GIMPLE_CALL:
{
- tree fn = (*valueize) (gimple_call_fn (stmt));
+ tree fn;
+
+ if (gimple_call_internal_p (stmt))
+ /* No folding yet for these functions. */
+ return NULL_TREE;
+
+ fn = (*valueize) (gimple_call_fn (stmt));
if (TREE_CODE (fn) == ADDR_EXPR
&& TREE_CODE (TREE_OPERAND (fn, 0)) == FUNCTION_DECL
&& DECL_BUILT_IN (TREE_OPERAND (fn, 0)))
Index: gcc/gimple-low.c
===================================================================
--- gcc/gimple-low.c 2011-04-07 09:23:36.000000000 +0100
+++ gcc/gimple-low.c 2011-04-07 11:05:54.000000000 +0100
@@ -225,7 +225,8 @@ gimple_check_call_args (gimple stmt)
parms = NULL_TREE;
if (fndecl)
parms = TYPE_ARG_TYPES (TREE_TYPE (fndecl));
- else if (POINTER_TYPE_P (TREE_TYPE (gimple_call_fn (stmt))))
+ else if (!gimple_call_internal_p (stmt)
+ && POINTER_TYPE_P (TREE_TYPE (gimple_call_fn (stmt))))
parms = TYPE_ARG_TYPES (TREE_TYPE (TREE_TYPE (gimple_call_fn (stmt))));
/* Verify if the type of the argument matches that of the function
Index: gcc/gimple-pretty-print.c
===================================================================
--- gcc/gimple-pretty-print.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/gimple-pretty-print.c 2011-04-07 11:05:54.000000000 +0100
@@ -616,8 +616,12 @@ dump_gimple_call (pretty_printer *buffer
if (flags & TDF_RAW)
{
- dump_gimple_fmt (buffer, spc, flags, "%G <%T, %T",
- gs, gimple_call_fn (gs), lhs);
+ if (gimple_call_internal_p (gs))
+ dump_gimple_fmt (buffer, spc, flags, "%G <%s, %T", gs,
+ internal_fn_name (gimple_call_internal_fn (gs)), lhs);
+ else
+ dump_gimple_fmt (buffer, spc, flags, "%G <%T, %T",
+ gs, gimple_call_fn (gs), lhs);
if (gimple_call_num_args (gs) > 0)
{
pp_string (buffer, ", ");
@@ -637,7 +641,10 @@ dump_gimple_call (pretty_printer *buffer
pp_space (buffer);
}
- print_call_name (buffer, gimple_call_fn (gs), flags);
+ if (gimple_call_internal_p (gs))
+ pp_string (buffer, internal_fn_name (gimple_call_internal_fn (gs)));
+ else
+ print_call_name (buffer, gimple_call_fn (gs), flags);
pp_string (buffer, " (");
dump_gimple_call_args (buffer, gs, flags);
pp_character (buffer, ')');
Index: gcc/ipa-prop.c
===================================================================
--- gcc/ipa-prop.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/ipa-prop.c 2011-04-07 11:05:54.000000000 +0100
@@ -1416,8 +1416,12 @@ ipa_analyze_call_uses (struct cgraph_nod
struct ipa_node_params *info,
struct param_analysis_info *parms_info, gimple call)
{
- tree target = gimple_call_fn (call);
+ tree target;
+
+ if (gimple_call_internal_p (call))
+ return;
+ target = gimple_call_fn (call);
if (TREE_CODE (target) == SSA_NAME)
ipa_analyze_indirect_call_uses (node, info, parms_info, call, target);
else if (TREE_CODE (target) == OBJ_TYPE_REF)
Index: gcc/tree-cfg.c
===================================================================
--- gcc/tree-cfg.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/tree-cfg.c 2011-04-07 11:05:54.000000000 +0100
@@ -3042,23 +3042,43 @@ valid_fixed_convert_types_p (tree type1,
static bool
verify_gimple_call (gimple stmt)
{
- tree fn = gimple_call_fn (stmt);
+ tree fn;
tree fntype, fndecl;
unsigned i;
- if (!is_gimple_call_addr (fn))
+ if (!gimple_call_internal_p (stmt))
{
- error ("invalid function in gimple call");
- debug_generic_stmt (fn);
- return true;
- }
-
- if (!POINTER_TYPE_P (TREE_TYPE (fn))
- || (TREE_CODE (TREE_TYPE (TREE_TYPE (fn))) != FUNCTION_TYPE
- && TREE_CODE (TREE_TYPE (TREE_TYPE (fn))) != METHOD_TYPE))
- {
- error ("non-function in gimple call");
- return true;
+ fn = gimple_call_fn (stmt);
+ if (!is_gimple_call_addr (fn))
+ {
+ error ("invalid function in gimple call");
+ debug_generic_stmt (fn);
+ return true;
+ }
+ if (!POINTER_TYPE_P (TREE_TYPE (fn))
+ || (TREE_CODE (TREE_TYPE (TREE_TYPE (fn))) != FUNCTION_TYPE
+ && TREE_CODE (TREE_TYPE (TREE_TYPE (fn))) != METHOD_TYPE))
+ {
+ error ("non-function in gimple call");
+ return true;
+ }
+ fntype = TREE_TYPE (TREE_TYPE (fn));
+ if (gimple_call_lhs (stmt)
+ && !useless_type_conversion_p (TREE_TYPE (gimple_call_lhs (stmt)),
+ TREE_TYPE (fntype))
+ /* ??? At least C++ misses conversions at assignments from
+ void * call results.
+ ??? Java is completely off. Especially with functions
+ returning java.lang.Object.
+ For now simply allow arbitrary pointer type conversions. */
+ && !(POINTER_TYPE_P (TREE_TYPE (gimple_call_lhs (stmt)))
+ && POINTER_TYPE_P (TREE_TYPE (fntype))))
+ {
+ error ("invalid conversion in gimple call");
+ debug_generic_stmt (TREE_TYPE (gimple_call_lhs (stmt)));
+ debug_generic_stmt (TREE_TYPE (fntype));
+ return true;
+ }
}
fndecl = gimple_call_fndecl (stmt);
@@ -3086,24 +3106,6 @@ verify_gimple_call (gimple stmt)
return true;
}
- fntype = TREE_TYPE (TREE_TYPE (fn));
- if (gimple_call_lhs (stmt)
- && !useless_type_conversion_p (TREE_TYPE (gimple_call_lhs (stmt)),
- TREE_TYPE (fntype))
- /* ??? At least C++ misses conversions at assignments from
- void * call results.
- ??? Java is completely off. Especially with functions
- returning java.lang.Object.
- For now simply allow arbitrary pointer type conversions. */
- && !(POINTER_TYPE_P (TREE_TYPE (gimple_call_lhs (stmt)))
- && POINTER_TYPE_P (TREE_TYPE (fntype))))
- {
- error ("invalid conversion in gimple call");
- debug_generic_stmt (TREE_TYPE (gimple_call_lhs (stmt)));
- debug_generic_stmt (TREE_TYPE (fntype));
- return true;
- }
-
if (gimple_call_chain (stmt)
&& !is_gimple_val (gimple_call_chain (stmt)))
{
@@ -3121,7 +3123,7 @@ verify_gimple_call (gimple stmt)
error ("static chain in indirect gimple call");
return true;
}
- fn = TREE_OPERAND (fn, 0);
+ fn = TREE_OPERAND (gimple_call_fn (stmt), 0);
if (!DECL_STATIC_CHAIN (fn))
{
@@ -7435,6 +7437,8 @@ do_warn_unused_result (gimple_seq seq)
case GIMPLE_CALL:
if (gimple_call_lhs (g))
break;
+ if (gimple_call_internal_p (g))
+ break;
/* This is a naked call, as opposed to a GIMPLE_CALL with an
LHS. All calls whose value is ignored should be
Index: gcc/tree-eh.c
===================================================================
--- gcc/tree-eh.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/tree-eh.c 2011-04-07 11:05:54.000000000 +0100
@@ -2743,7 +2743,7 @@ same_handler_p (gimple_seq oneh, gimple_
|| gimple_call_lhs (twos)
|| gimple_call_chain (ones)
|| gimple_call_chain (twos)
- || !operand_equal_p (gimple_call_fn (ones), gimple_call_fn (twos), 0)
+ || !gimple_call_same_target_p (ones, twos)
|| gimple_call_num_args (ones) != gimple_call_num_args (twos))
return false;
Index: gcc/tree-ssa-ccp.c
===================================================================
--- gcc/tree-ssa-ccp.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/tree-ssa-ccp.c 2011-04-07 11:05:54.000000000 +0100
@@ -1723,11 +1723,18 @@ ccp_fold_stmt (gimple_stmt_iterator *gsi
return true;
}
+ /* Internal calls provide no argument types, so the extra laxity
+ for normal calls does not apply. */
+ if (gimple_call_internal_p (stmt))
+ return false;
+
+ callee = gimple_call_fn (stmt);
+
/* Propagate into the call arguments. Compared to replace_uses_in
this can use the argument slot types for type verification
instead of the current argument type. We also can safely
drop qualifiers here as we are dealing with constants anyway. */
- argt = TYPE_ARG_TYPES (TREE_TYPE (TREE_TYPE (gimple_call_fn (stmt))));
+ argt = TYPE_ARG_TYPES (TREE_TYPE (TREE_TYPE (callee)));
for (i = 0; i < gimple_call_num_args (stmt) && argt;
++i, argt = TREE_CHAIN (argt))
{
@@ -1743,7 +1750,6 @@ ccp_fold_stmt (gimple_stmt_iterator *gsi
}
}
- callee = gimple_call_fn (stmt);
if (TREE_CODE (callee) == OBJ_TYPE_REF
&& TREE_CODE (OBJ_TYPE_REF_EXPR (callee)) == SSA_NAME)
{
Index: gcc/tree-ssa-dom.c
===================================================================
--- gcc/tree-ssa-dom.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/tree-ssa-dom.c 2011-04-07 11:05:54.000000000 +0100
@@ -64,7 +64,7 @@ struct hashable_expr
struct { enum tree_code op; tree opnd; } unary;
struct { enum tree_code op; tree opnd0, opnd1; } binary;
struct { enum tree_code op; tree opnd0, opnd1, opnd2; } ternary;
- struct { tree fn; bool pure; size_t nargs; tree *args; } call;
+ struct { gimple fn_from; bool pure; size_t nargs; tree *args; } call;
} ops;
};
@@ -258,7 +258,7 @@ initialize_hash_element (gimple stmt, tr
expr->type = TREE_TYPE (gimple_call_lhs (stmt));
expr->kind = EXPR_CALL;
- expr->ops.call.fn = gimple_call_fn (stmt);
+ expr->ops.call.fn_from = stmt;
if (gimple_call_flags (stmt) & (ECF_CONST | ECF_PURE))
expr->ops.call.pure = true;
@@ -422,8 +422,8 @@ hashable_expr_equal_p (const struct hash
/* If the calls are to different functions, then they
clearly cannot be equal. */
- if (! operand_equal_p (expr0->ops.call.fn,
- expr1->ops.call.fn, 0))
+ if (!gimple_call_same_target_p (expr0->ops.call.fn_from,
+ expr1->ops.call.fn_from))
return false;
if (! expr0->ops.call.pure)
@@ -503,9 +503,15 @@ iterative_hash_hashable_expr (const stru
{
size_t i;
enum tree_code code = CALL_EXPR;
+ gimple fn_from;
val = iterative_hash_object (code, val);
- val = iterative_hash_expr (expr->ops.call.fn, val);
+ fn_from = expr->ops.call.fn_from;
+ if (gimple_call_internal_p (fn_from))
+ val = iterative_hash_hashval_t
+ ((hashval_t) gimple_call_internal_fn (fn_from), val);
+ else
+ val = iterative_hash_expr (gimple_call_fn (fn_from), val);
for (i = 0; i < expr->ops.call.nargs; i++)
val = iterative_hash_expr (expr->ops.call.args[i], val);
}
@@ -565,8 +571,14 @@ print_expr_hash_elt (FILE * stream, cons
{
size_t i;
size_t nargs = element->expr.ops.call.nargs;
+ gimple fn_from;
- print_generic_expr (stream, element->expr.ops.call.fn, 0);
+ fn_from = element->expr.ops.call.fn_from;
+ if (gimple_call_internal_p (fn_from))
+ fputs (internal_fn_name (gimple_call_internal_fn (fn_from)),
+ stream);
+ else
+ print_generic_expr (stream, gimple_call_fn (fn_from), 0);
fprintf (stream, " (");
for (i = 0; i < nargs; i++)
{
Index: gcc/tree-ssa-pre.c
===================================================================
--- gcc/tree-ssa-pre.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/tree-ssa-pre.c 2011-04-07 11:05:54.000000000 +0100
@@ -4381,6 +4381,7 @@ eliminate (void)
/* Visit indirect calls and turn them into direct calls if
possible. */
if (is_gimple_call (stmt)
+ && !gimple_call_internal_p (stmt)
&& TREE_CODE (gimple_call_fn (stmt)) == SSA_NAME)
{
tree orig_fn = gimple_call_fn (stmt);
Index: gcc/tree-ssa-sccvn.c
===================================================================
--- gcc/tree-ssa-sccvn.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/tree-ssa-sccvn.c 2011-04-07 11:05:54.000000000 +0100
@@ -911,7 +911,10 @@ copy_reference_ops_from_call (gimple cal
memset (&temp, 0, sizeof (temp));
temp.type = gimple_call_return_type (call);
temp.opcode = CALL_EXPR;
- temp.op0 = gimple_call_fn (call);
+ if (gimple_call_internal_p (call))
+ temp.op0 = NULL_TREE;
+ else
+ temp.op0 = gimple_call_fn (call);
temp.op1 = gimple_call_chain (call);
temp.off = -1;
VEC_safe_push (vn_reference_op_s, heap, *result, &temp);
Index: gcc/tree-ssa-structalias.c
===================================================================
--- gcc/tree-ssa-structalias.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/tree-ssa-structalias.c 2011-04-07 11:05:54.000000000 +0100
@@ -4026,6 +4026,8 @@ get_fi_for_callee (gimple call)
{
tree decl;
+ gcc_assert (!gimple_call_internal_p (call));
+
/* If we can directly resolve the function being called, do so.
Otherwise, it must be some sort of indirect expression that
we should still be able to handle. */
@@ -4319,6 +4321,7 @@ find_func_aliases (gimple origt)
/* Fallthru to general call handling. */;
}
if (!in_ipa_mode
+ || gimple_call_internal_p (t)
|| (fndecl
&& (!(fi = lookup_vi_for_tree (fndecl))
|| !fi->is_fn_info)))
Index: gcc/value-prof.c
===================================================================
--- gcc/value-prof.c 2011-04-07 09:21:45.000000000 +0100
+++ gcc/value-prof.c 2011-04-07 11:05:54.000000000 +0100
@@ -1237,6 +1237,9 @@ gimple_ic_transform (gimple stmt)
if (gimple_code (stmt) != GIMPLE_CALL)
return false;
+ if (gimple_call_internal_p (stmt))
+ return false;
+
callee = gimple_call_fn (stmt);
if (TREE_CODE (callee) == FUNCTION_DECL)
@@ -1276,7 +1279,7 @@ gimple_ic_transform (gimple stmt)
if (dump_file)
{
fprintf (dump_file, "Indirect call -> direct call ");
- print_generic_expr (dump_file, gimple_call_fn (stmt), TDF_SLIM);
+ print_generic_expr (dump_file, callee, TDF_SLIM);
fprintf (dump_file, "=> ");
print_generic_expr (dump_file, direct_call->decl, TDF_SLIM);
fprintf (dump_file, " transformation on insn ");
@@ -1630,6 +1633,7 @@ gimple_indirect_call_to_profile (gimple
tree callee;
if (gimple_code (stmt) != GIMPLE_CALL
+ || gimple_call_internal_p (stmt)
|| gimple_call_fndecl (stmt) != NULL_TREE)
return;
