This patch makes the middle-end adjustments needed to allow a MEM to take a
vector of addresses in order to read/write a non-contiguous vector of data.
Mostly the changes don't have to do anything besides letting the vectors pass.
There's a new MEM flag: MEM_INDEPENDENT_P that asserts that the addresses in a
vector can be treated as if they do not conflict. Nothing uses it, yet, but it
will allow future optimizations that rely on left-to-right semantics to do the
right thing. GCN instructions do not have such a guarantee, so they will
require that the flag is set.
These changes, alone, are not enough to actually allow such MEMs, since it's
unlikely that any back-end reports the vector modes "legitimate", so there
should be no change of behaviour on any existing target (unconfirmed).
I've modified the type of only the TARGET_ADDR_SPACE_VALID_POINTER_MODE hook,
which is currently not being used by any target (amdgcn will use it in the
following patch), and left the TARGET_VALID_POINTER_MODE alone so as not to
have to touch all the other back-ends. If another back-end wants to use
vectors of addresses then the updated documentation suggests to switch
to using TARGET_ADDR_SPACE_VALID_POINTER_MODE.
I have not made any changes to LRA code. I believe this is not necessary as
long as the target uses "define_special_memory_constraint". I failed to
figure out how to make the regular memory constraints work, and as it's not
necessary for my use case I'm loath to invest any more time into that.
Hopefully the documentation patches should describe the intended semantics.
I've tried to keep it very simple.
There's one "TODO" in the code; I'm intentionally applying the principle of
"don't add it if you don't need it" here (not least because I don't know how
to test it). There's an assert to make sure it never bites anyone silently.
gcc/ChangeLog:
* doc/rtl.texi: Document MEM_INDEPENDENT_P.
Document new MEM semantics.
* doc/tm.texi: Likewise.
* emit-rtl.cc (gen_int_mode): Extend to DTRT for vector modes.
(adjust_address_1): Permit vectors of addresses.
* explow.cc (convert_memory_address_addr_space_1): Permit vectors of
addresses, for no-op conversions only.
* recog.cc (address_operand): Permit vectors of addresses.
* rtl.h (MEM_INDEPENDENT_P): New macro.
(get_address_mode): Change return type to machine_mode.
* rtlanal.cc (get_address_mode): Likewise.
* simplify-rtx.cc (simplify_context::simplify_subreg): Handle vectors
of addresses.
* target.def (pointer_mode): Document vectors of addresses.
(address_mode): Likewise.
(valid_pointer_mode): Change scalar_int_mode to machine_mode, and
document the vector use case.
* targhooks.cc (default_addr_space_valid_pointer_mode): Change
scalar_int_mode to machine_mode, but reject non-scalar inputs.
* targhooks.h (default_addr_space_valid_pointer_mode): Likewise.
---
gcc/doc/rtl.texi | 45 +++++++++++++++++++++++++++++++++++++++++++++
gcc/doc/tm.texi | 28 +++++++++++++++++++++-------
gcc/emit-rtl.cc | 30 ++++++++++++++++++++++++------
gcc/explow.cc | 12 ++++++++++--
gcc/recog.cc | 3 ++-
gcc/rtl.h | 7 ++++++-
gcc/rtlanal.cc | 4 ++--
gcc/simplify-rtx.cc | 36 +++++++++++++++++++++++++++++++++++-
gcc/target.def | 28 +++++++++++++++++++---------
gcc/targhooks.cc | 7 +++++--
gcc/targhooks.h | 3 +--
11 files changed, 170 insertions(+), 33 deletions(-)
diff --git a/gcc/doc/rtl.texi b/gcc/doc/rtl.texi
index f606b97389f..16476e29fad 100644
--- a/gcc/doc/rtl.texi
+++ b/gcc/doc/rtl.texi
@@ -703,6 +703,42 @@ application, this memory will never be subsequently
modified.
Stored in the @code{unchanging} field and printed as @samp{/u}.
+@findex MEM_INDEPENDENT_P
+@cindex @code{mem} and @samp{/i}
+@item MEM_INDEPENDENT_P (@var{x})
+This flag only applies to @code{mem}s that have vector addresses (that is,
+to @code{mem}s whose addresses have a vector mode). The flag is nonzero
+if the creator asserts that the @code{mem}'s address vector specifies
+independent, non-overlapping memory locations, so that the @code{mem}
+does not refer to the same byte twice.
+
+This assertion does not mean that overlap is numerically impossible.
+It instead means that, if the instruction containing the @code{mem}
+is reached, the semantics of the source program require the addresses
+to be independent. The behavior in other cases is undefined.
+
+RTL semantics apply the following restrictions:
+
+@itemize
+@item
+If the assertion made by @code{MEM_INDEPENDENT_P} does not hold at runtime,
+reading from the @code{mem} will return an undefined value, but will have
+no side effects other than reading from the specified bytes.
+
+@item
+Similarly, writing to such a @code{mem} will set the specified bytes
+to undefined values, but will have no side effects other than that.
+@end itemize
+
+@code{MEM_INDEPENDENT_P} can be useful when writing to a @code{mem}
+that has a vector address. RTL semantics require such writes to behave
+@emph{as if} they processed the vector in left-to-right order, with later
+writes winning over earlier writes. However, when @code{MEM_INDEPENDENT_P}
+is true, an instruction can adhere to these semantics by writing to the
+bytes in any order, or in parallel.
+
+Stored in the @code{return_val} field and printed as @samp{/i}.
+
@findex PREFETCH_SCHEDULE_BARRIER_P
@cindex @code{prefetch} and @samp{/v}
@cindex @code{volatile}, in @code{prefetch}
@@ -2369,6 +2405,15 @@ a unit of memory is accessed. @var{alias} specifies an
alias set for the
reference. In general two items are in different alias sets if they cannot
reference the same memory address.
+The @var{addr} expression can represent either a scalar base address, or a
+vector of addresses (using an appropriate vector mode). In the vector case,
+the number of lanes in the address mode must match the number of lanes in the
+data mode @var{m}. Each vector lane can be viewed as an independent memory
+access of the corresponding scalar mode, performed in left-to-right order. The
+@code{MEM_INDEPENDENT_P} flag can be used to assert that the addresses will not
+conflict. All lanes are assumed to have the same attributes (address space,
+volatile, etc.).
+
The construct @code{(mem:BLK (scratch))} is considered to alias all
other memories. Thus it may be used as a memory barrier in epilogue
stack deallocation patterns.
diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi
index 97afd017522..17fae2e3a0e 100644
--- a/gcc/doc/tm.texi
+++ b/gcc/doc/tm.texi
@@ -11542,23 +11542,37 @@ c_register_addr_space ("__ea", ADDR_SPACE_EA);
@deftypefn {Target Hook} scalar_int_mode TARGET_ADDR_SPACE_POINTER_MODE
(addr_space_t @var{address_space})
Define this to return the machine mode to use for pointers to
@var{address_space} if the target supports named address spaces.
+
+Targets that support vectors of pointers should return only the scalar base
+mode here, but permit the vector modes in
+@code{TARGET_ADDR_SPACE_VALID_POINTER_MODE}.
+
The default version of this hook returns @code{ptr_mode}.
@end deftypefn
@deftypefn {Target Hook} scalar_int_mode TARGET_ADDR_SPACE_ADDRESS_MODE
(addr_space_t @var{address_space})
Define this to return the machine mode to use for addresses in
@var{address_space} if the target supports named address spaces.
+
+Targets that support vectors of addresses should return only the scalar base
+mode here, but permit the vector modes in
+@code{TARGET_ADDR_SPACE_LEGITIMATE_ADDRESS_P} and
+@code{TARGET_ADDR_SPACE_LEGITIMIZE_ADDRESS}.
+
The default version of this hook returns @code{Pmode}.
@end deftypefn
-@deftypefn {Target Hook} bool TARGET_ADDR_SPACE_VALID_POINTER_MODE
(scalar_int_mode @var{mode}, addr_space_t @var{as})
+@deftypefn {Target Hook} bool TARGET_ADDR_SPACE_VALID_POINTER_MODE
(machine_mode @var{mode}, addr_space_t @var{as})
Define this to return nonzero if the port can handle pointers
-with machine mode @var{mode} to address space @var{as}. This target
-hook is the same as the @code{TARGET_VALID_POINTER_MODE} target hook,
-except that it includes explicit named address space support. The default
-version of this hook returns true for the modes returned by either the
-@code{TARGET_ADDR_SPACE_POINTER_MODE} or @code{TARGET_ADDR_SPACE_ADDRESS_MODE}
-target hooks for the given address space.
+with machine mode @var{mode} to address space @var{as}. This hook can
+also be used to allow pointers to use vector modes (e.g. to permit
+@samp{(mem:V4SI (reg:V4DI))}). This target hook is the same as the
+@code{TARGET_VALID_POINTER_MODE} target hook, except that it includes
+explicit named address space support, and does not mandate
+@code{scalar_int_mode}. The default version of this hook returns true for
+the modes returned by either the @code{TARGET_ADDR_SPACE_POINTER_MODE}
+or @code{TARGET_ADDR_SPACE_ADDRESS_MODE} target hooks for the given address
+space.
@end deftypefn
@deftypefn {Target Hook} bool TARGET_ADDR_SPACE_LEGITIMATE_ADDRESS_P
(machine_mode @var{mode}, rtx @var{exp}, bool @var{strict}, addr_space_t
@var{as}, code_helper @var{ch})
diff --git a/gcc/emit-rtl.cc b/gcc/emit-rtl.cc
index d064b2a1064..a88ffbbc2cb 100644
--- a/gcc/emit-rtl.cc
+++ b/gcc/emit-rtl.cc
@@ -540,14 +540,29 @@ gen_rtx_CONST_INT (machine_mode mode ATTRIBUTE_UNUSED,
HOST_WIDE_INT arg)
return *slot;
}
+/* Return a CONST_INT, CONST_POLY_INT, or CONST_VECTOR containing the given
+ value. In the case of a vector, the value will be duplicated. */
+
rtx
gen_int_mode (poly_int64 c, machine_mode mode)
{
- c = trunc_int_for_mode (c, mode);
+ machine_mode s_mode = (VECTOR_MODE_P (mode) ? GET_MODE_INNER (mode) : mode);
+ rtx val;
+
+ c = trunc_int_for_mode (c, s_mode);
if (c.is_constant ())
- return GEN_INT (c.coeffs[0]);
- unsigned int prec = GET_MODE_PRECISION (as_a <scalar_mode> (mode));
- return immed_wide_int_const (poly_wide_int::from (c, prec, SIGNED), mode);
+ val = GEN_INT (c.coeffs[0]);
+ else
+ {
+ unsigned int prec = GET_MODE_PRECISION (as_a <scalar_mode> (s_mode));
+ val = immed_wide_int_const (poly_wide_int::from (c, prec, SIGNED),
+ s_mode);
+ }
+
+ if (VECTOR_MODE_P (mode))
+ val = gen_const_vec_duplicate (mode, val);
+
+ return val;
}
/* CONST_DOUBLEs might be created from pairs of integers, or from
@@ -2383,7 +2398,7 @@ adjust_address_1 (rtx memref, machine_mode mode,
poly_int64 offset,
{
rtx addr = XEXP (memref, 0);
rtx new_rtx;
- scalar_int_mode address_mode;
+ machine_mode address_mode;
class mem_attrs attrs (*get_mem_attrs (memref)), *defattrs;
unsigned HOST_WIDE_INT max_align;
#ifdef POINTERS_EXTEND_UNSIGNED
@@ -2417,7 +2432,10 @@ adjust_address_1 (rtx memref, machine_mode mode,
poly_int64 offset,
/* Convert a possibly large offset to a signed value within the
range of the target address space. */
address_mode = get_address_mode (memref);
- offset = trunc_int_for_mode (offset, address_mode);
+ if (!VECTOR_MODE_P (address_mode))
+ offset = trunc_int_for_mode (offset, address_mode);
+ else
+ offset = trunc_int_for_mode (offset, GET_MODE_INNER (address_mode));
if (adjust_address)
{
diff --git a/gcc/explow.cc b/gcc/explow.cc
index ec2bc9bc9b6..9c7c834aaab 100644
--- a/gcc/explow.cc
+++ b/gcc/explow.cc
@@ -298,8 +298,13 @@ convert_memory_address_addr_space_1 (scalar_int_mode
to_mode ATTRIBUTE_UNUSED,
bool in_const ATTRIBUTE_UNUSED,
bool no_emit ATTRIBUTE_UNUSED)
{
+ machine_mode xmode = GET_MODE (x);
+ bool vecaddr_p = VECTOR_MODE_P (GET_MODE (x));
+ if (vecaddr_p)
+ xmode = GET_MODE_INNER (xmode);
+
#ifndef POINTERS_EXTEND_UNSIGNED
- gcc_assert (GET_MODE (x) == to_mode || GET_MODE (x) == VOIDmode);
+ gcc_assert (xmode == to_mode || xmode == VOIDmode);
return x;
#else /* defined(POINTERS_EXTEND_UNSIGNED) */
scalar_int_mode pointer_mode, address_mode, from_mode;
@@ -307,9 +312,12 @@ convert_memory_address_addr_space_1 (scalar_int_mode
to_mode ATTRIBUTE_UNUSED,
enum rtx_code code;
/* If X already has the right mode, just return it. */
- if (GET_MODE (x) == to_mode)
+ if (xmode == to_mode)
return x;
+ /* TODO: support vector conversions. */
+ gcc_assert (!vecaddr_p);
+
pointer_mode = targetm.addr_space.pointer_mode (as);
address_mode = targetm.addr_space.address_mode (as);
from_mode = to_mode == pointer_mode ? address_mode : pointer_mode;
diff --git a/gcc/recog.cc b/gcc/recog.cc
index dfce5df743d..ddfffdac0ff 100644
--- a/gcc/recog.cc
+++ b/gcc/recog.cc
@@ -1633,7 +1633,8 @@ address_operand (rtx op, machine_mode mode)
{
/* Wrong mode for an address expr. */
if (GET_MODE (op) != VOIDmode
- && ! SCALAR_INT_MODE_P (GET_MODE (op)))
+ && ! SCALAR_INT_MODE_P (GET_MODE (op))
+ && ! (GET_MODE_CLASS (GET_MODE (op)) == MODE_VECTOR_INT))
return false;
return memory_address_p (mode, op);
diff --git a/gcc/rtl.h b/gcc/rtl.h
index d61e576c940..d87ce41d915 100644
--- a/gcc/rtl.h
+++ b/gcc/rtl.h
@@ -2657,6 +2657,11 @@ do {
\
#define MEM_READONLY_P(RTX) \
(RTL_FLAG_CHECK1 ("MEM_READONLY_P", (RTX), MEM)->unchanging)
+/* 1 if RTX is a mem whose address vector elements all point to
+ independent memory locations, without overlapping each other. */
+#define MEM_INDEPENDENT_P(RTX) \
+ (RTL_FLAG_CHECK1 ("MEM_INDEPENDENT_P", (RTX), MEM)->return_val)
+
/* 1 if RTX is a mem and we should keep the alias set for this mem
unchanged when we access a component. Set to 1, or example, when we
are already in a non-addressable component of an aggregate. */
@@ -3703,7 +3708,7 @@ inline rtx single_set (const rtx_insn *insn)
return single_set_2 (insn, PATTERN (insn));
}
-extern scalar_int_mode get_address_mode (rtx mem);
+extern machine_mode get_address_mode (rtx mem);
extern bool rtx_addr_can_trap_p (const_rtx);
extern bool nonzero_address_p (const_rtx);
extern bool rtx_unstable_p (const_rtx);
diff --git a/gcc/rtlanal.cc b/gcc/rtlanal.cc
index df4a69d180a..7f170cb7dfc 100644
--- a/gcc/rtlanal.cc
+++ b/gcc/rtlanal.cc
@@ -6295,7 +6295,7 @@ low_bitmask_len (machine_mode mode, unsigned
HOST_WIDE_INT m)
/* Return the mode of MEM's address. */
-scalar_int_mode
+machine_mode
get_address_mode (rtx mem)
{
machine_mode mode;
@@ -6303,7 +6303,7 @@ get_address_mode (rtx mem)
gcc_assert (MEM_P (mem));
mode = GET_MODE (XEXP (mem, 0));
if (mode != VOIDmode)
- return as_a <scalar_int_mode> (mode);
+ return mode;
return targetm.addr_space.address_mode (MEM_ADDR_SPACE (mem));
}
diff --git a/gcc/simplify-rtx.cc b/gcc/simplify-rtx.cc
index 38c8abfe3e7..67f3307d2b0 100644
--- a/gcc/simplify-rtx.cc
+++ b/gcc/simplify-rtx.cc
@@ -8777,9 +8777,43 @@ simplify_context::simplify_subreg (machine_mode
outermode, rtx op,
&& (! MEM_VOLATILE_P (op)
|| ! have_insn_for (SET, innermode))
&& !(STRICT_ALIGNMENT && MEM_ALIGN (op) < GET_MODE_ALIGNMENT (outermode))
- && known_le (outersize, innersize))
+ && known_le (outersize, innersize)
+ /* You can't get the Nth element of a vector of addresses (i.e.
+ gather/scatter) by adding an offset to the base. */
+ && !VECTOR_MODE_P (GET_MODE (XEXP (op, 0))))
return adjust_address_nv (op, outermode, byte);
+ /* Simplify (subreg:<m> (mem:<vec_m> (reg:<vec_p> addr)) x)
+ to (mem:<m> (subreg:<p> (reg:<vec_p>)) y) */
+ if (MEM_P (op)
+ && VECTOR_MODE_P (innermode)
+ && VECTOR_MODE_P (GET_MODE (XEXP (op, 0)))
+ && GET_MODE_INNER (innermode) == outermode)
+ {
+ unsigned int index;
+ if (constant_multiple_p (byte, GET_MODE_SIZE (outermode), &index))
+ {
+ rtx addr = XEXP (op, 0);
+ machine_mode addrmode = GET_MODE (addr);
+ machine_mode addr_elt_mode = GET_MODE_INNER (addrmode);
+ poly_uint64 addr_byte = index * GET_MODE_SIZE (addr_elt_mode);
+
+ rtx new_addr
+ = (REG_P (addr)
+ ? simplify_gen_subreg (addr_elt_mode, addr, addrmode,
+ addr_byte)
+ : simplify_subreg (addr_elt_mode, addr, addrmode, addr_byte));
+ if (new_addr)
+ {
+ rtx new_mem = gen_rtx_MEM (outermode, new_addr);
+ set_mem_addr_space (new_mem, MEM_ADDR_SPACE (op));
+ MEM_VOLATILE_P (new_mem) = MEM_VOLATILE_P (op);
+ MEM_NOTRAP_P (new_mem) = MEM_NOTRAP_P (op);
+ return new_mem;
+ }
+ }
+ }
+
/* Handle complex or vector values represented as CONCAT or VEC_CONCAT
of two parts. */
if (GET_CODE (op) == CONCAT
diff --git a/gcc/target.def b/gcc/target.def
index 11334f9d443..4c4f91827b2 100644
--- a/gcc/target.def
+++ b/gcc/target.def
@@ -3451,7 +3451,10 @@ HOOK_VECTOR (TARGET_ADDR_SPACE_HOOKS, addr_space)
DEFHOOK
(pointer_mode,
"Define this to return the machine mode to use for pointers to\n\
-@var{address_space} if the target supports named address spaces.\n\
+@var{address_space} if the target supports named address spaces.\n\n\
+Targets that support vectors of pointers should return only the scalar base\n\
+mode here, but permit the vector modes in\n\
+@code{TARGET_ADDR_SPACE_VALID_POINTER_MODE}.\n\n\
The default version of this hook returns @code{ptr_mode}.",
scalar_int_mode, (addr_space_t address_space),
default_addr_space_pointer_mode)
@@ -3460,7 +3463,11 @@ The default version of this hook returns
@code{ptr_mode}.",
DEFHOOK
(address_mode,
"Define this to return the machine mode to use for addresses in\n\
-@var{address_space} if the target supports named address spaces.\n\
+@var{address_space} if the target supports named address spaces.\n\n\
+Targets that support vectors of addresses should return only the scalar base\n\
+mode here, but permit the vector modes in\n\
+@code{TARGET_ADDR_SPACE_LEGITIMATE_ADDRESS_P} and\n\
+@code{TARGET_ADDR_SPACE_LEGITIMIZE_ADDRESS}.\n\n\
The default version of this hook returns @code{Pmode}.",
scalar_int_mode, (addr_space_t address_space),
default_addr_space_address_mode)
@@ -3470,13 +3477,16 @@ The default version of this hook returns @code{Pmode}.",
DEFHOOK
(valid_pointer_mode,
"Define this to return nonzero if the port can handle pointers\n\
-with machine mode @var{mode} to address space @var{as}. This target\n\
-hook is the same as the @code{TARGET_VALID_POINTER_MODE} target hook,\n\
-except that it includes explicit named address space support. The default\n\
-version of this hook returns true for the modes returned by either the\n\
-@code{TARGET_ADDR_SPACE_POINTER_MODE} or
@code{TARGET_ADDR_SPACE_ADDRESS_MODE}\n\
-target hooks for the given address space.",
- bool, (scalar_int_mode mode, addr_space_t as),
+with machine mode @var{mode} to address space @var{as}. This hook can\n\
+also be used to allow pointers to use vector modes (e.g. to permit\n\
+@samp{(mem:V4SI (reg:V4DI))}). This target hook is the same as the\n\
+@code{TARGET_VALID_POINTER_MODE} target hook, except that it includes\n\
+explicit named address space support, and does not mandate\n\
+@code{scalar_int_mode}. The default version of this hook returns true for\n\
+the modes returned by either the @code{TARGET_ADDR_SPACE_POINTER_MODE}\n\
+or @code{TARGET_ADDR_SPACE_ADDRESS_MODE} target hooks for the given address\n\
+space.",
+ bool, (machine_mode mode, addr_space_t as),
default_addr_space_valid_pointer_mode)
/* True if an address is a valid memory address to a given named address
diff --git a/gcc/targhooks.cc b/gcc/targhooks.cc
index 7123827cfd7..01defd7dbb3 100644
--- a/gcc/targhooks.cc
+++ b/gcc/targhooks.cc
@@ -1728,10 +1728,13 @@ default_addr_space_address_mode (addr_space_t addrspace
ATTRIBUTE_UNUSED)
To match the above, the same modes apply to all address spaces. */
bool
-default_addr_space_valid_pointer_mode (scalar_int_mode mode,
+default_addr_space_valid_pointer_mode (machine_mode mode,
addr_space_t as ATTRIBUTE_UNUSED)
{
- return targetm.valid_pointer_mode (mode);
+ if (! SCALAR_INT_MODE_P (mode))
+ return false;
+
+ return targetm.valid_pointer_mode (as_a <scalar_int_mode> (mode));
}
/* Some places still assume that all pointer or address modes are the
diff --git a/gcc/targhooks.h b/gcc/targhooks.h
index ffeaeb2a772..2a429e7f4a4 100644
--- a/gcc/targhooks.h
+++ b/gcc/targhooks.h
@@ -206,8 +206,7 @@ extern bool default_valid_pointer_mode (scalar_int_mode);
extern bool default_ref_may_alias_errno (class ao_ref *);
extern scalar_int_mode default_addr_space_pointer_mode (addr_space_t);
extern scalar_int_mode default_addr_space_address_mode (addr_space_t);
-extern bool default_addr_space_valid_pointer_mode (scalar_int_mode,
- addr_space_t);
+extern bool default_addr_space_valid_pointer_mode (machine_mode, addr_space_t);
extern bool default_addr_space_legitimate_address_p (machine_mode, rtx, bool,
addr_space_t, code_helper);
extern rtx default_addr_space_legitimize_address (rtx, rtx, machine_mode,
--
2.54.0
