Re: [PATCH v2 26/28] arm64/sve: Add documentation
On Mon, Oct 09, 2017 at 03:07:23PM +0100, Alex Bennée wrote: > > Dave Martin writes: > > > On Mon, Oct 09, 2017 at 10:34:25AM +0100, Alex Bennée wrote: > >> > >> Dave Martin writes: > >> > >> > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote: > >> >> On 31/08/17 18:00, Dave Martin wrote: > >> >> > +9. System runtime configuration > >> >> > + > >> >> > + > >> >> > +* To mitigate the ABI impact of expansion of the signal frame, a > >> >> > policy > >> >> > + mechanism is provided for administrators, distro maintainers and > >> >> > developers > >> >> > + to set the default vector length for userspace processes: > >> >> > + > >> >> > +/proc/cpu/sve_default_vector_length > >> >> > >> >> > >> >> elsewhere in the patch series i see > >> >> > >> >> /proc/sys/abi/sve_default_vector_length > >> >> > >> >> is this supposed to be the same? > >> > > >> > Good spot, thanks! > >> > > >> > /proc/cpu/ was the old location: they should both say /proc/abi/. > >> > I'll fix it. > >> > >> Isn't /sys (or rather sysfs) the preferred location for modern control > >> knobs that mirror the kernels object model or is SVE a special case for > >> extending /proc? > > > > I couldn't figure out which kernel object this maps to. There's no > > device, no driver. This isn't even per-cpu. > > Hmm I can see: > > /sys/devices/system/cpu > > On both my x86 and arm64 systems - but I guess this is more ABIish than > CPU feature related. > > > sysctl is already used for similar knobs to this one, so I followed that > > precedent -- though if someone argues strongly enough it could be > > changed. > > > > Are there already examples of arch controls like this in sysfs? I > > wasn't aware of any, but I didn't look all that hard... > > Given the paucity of the /proc/sys/abi on both systems I guess this sort > of knob is rare enough that people haven't expressed a strong preference > for sysfs here. I have no objection to staying with /proc/sys/abi/. That was my thinking: sysctls tend to control the kernel, especially process behaviour, whereas /sys/ controls devices and subsystems. That's not a concrete rule though and not written down, and doubtless a major new set of sysctls would be shot down regardless of what they do. Part of the problem with /proc is that people historically put things in there that have random ad-hoc behaviour and semantics. The sysctl framework at least imposes some sanity here. There is also some support for specialising sysctls to user namespaces, which makes some sense in that /proc/sys/abi/* should probably be per- container -- though whether it's ever considered important enough to actually be implemented is another question. I certainly don't attempt to do it today. I don't know how sysfs interacts with namespaces, but probably it can. I guess I'll wait for someone to object loudly... Cheers ---Dave ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v2 26/28] arm64/sve: Add documentation
Dave Martin writes: > On Mon, Oct 09, 2017 at 10:34:25AM +0100, Alex Bennée wrote: >> >> Dave Martin writes: >> >> > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote: >> >> On 31/08/17 18:00, Dave Martin wrote: >> >> > +9. System runtime configuration >> >> > + >> >> > + >> >> > +* To mitigate the ABI impact of expansion of the signal frame, a policy >> >> > + mechanism is provided for administrators, distro maintainers and >> >> > developers >> >> > + to set the default vector length for userspace processes: >> >> > + >> >> > +/proc/cpu/sve_default_vector_length >> >> >> >> >> >> elsewhere in the patch series i see >> >> >> >> /proc/sys/abi/sve_default_vector_length >> >> >> >> is this supposed to be the same? >> > >> > Good spot, thanks! >> > >> > /proc/cpu/ was the old location: they should both say /proc/abi/. >> > I'll fix it. >> >> Isn't /sys (or rather sysfs) the preferred location for modern control >> knobs that mirror the kernels object model or is SVE a special case for >> extending /proc? > > I couldn't figure out which kernel object this maps to. There's no > device, no driver. This isn't even per-cpu. Hmm I can see: /sys/devices/system/cpu On both my x86 and arm64 systems - but I guess this is more ABIish than CPU feature related. > sysctl is already used for similar knobs to this one, so I followed that > precedent -- though if someone argues strongly enough it could be > changed. > > Are there already examples of arch controls like this in sysfs? I > wasn't aware of any, but I didn't look all that hard... Given the paucity of the /proc/sys/abi on both systems I guess this sort of knob is rare enough that people haven't expressed a strong preference for sysfs here. I have no objection to staying with /proc/sys/abi/. -- Alex Bennée ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v2 26/28] arm64/sve: Add documentation
On Mon, Oct 09, 2017 at 10:34:25AM +0100, Alex Bennée wrote: > > Dave Martin writes: > > > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote: > >> On 31/08/17 18:00, Dave Martin wrote: > >> > +9. System runtime configuration > >> > + > >> > + > >> > +* To mitigate the ABI impact of expansion of the signal frame, a policy > >> > + mechanism is provided for administrators, distro maintainers and > >> > developers > >> > + to set the default vector length for userspace processes: > >> > + > >> > +/proc/cpu/sve_default_vector_length > >> > >> > >> elsewhere in the patch series i see > >> > >> /proc/sys/abi/sve_default_vector_length > >> > >> is this supposed to be the same? > > > > Good spot, thanks! > > > > /proc/cpu/ was the old location: they should both say /proc/abi/. > > I'll fix it. > > Isn't /sys (or rather sysfs) the preferred location for modern control > knobs that mirror the kernels object model or is SVE a special case for > extending /proc? I couldn't figure out which kernel object this maps to. There's no device, no driver. This isn't even per-cpu. sysctl is already used for similar knobs to this one, so I followed that precedent -- though if someone argues strongly enough it could be changed. Are there already examples of arch controls like this in sysfs? I wasn't aware of any, but I didn't look all that hard... Cheers ---Dave ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v2 26/28] arm64/sve: Add documentation
Dave Martin writes: > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote: >> On 31/08/17 18:00, Dave Martin wrote: >> > +9. System runtime configuration >> > + >> > + >> > +* To mitigate the ABI impact of expansion of the signal frame, a policy >> > + mechanism is provided for administrators, distro maintainers and >> > developers >> > + to set the default vector length for userspace processes: >> > + >> > +/proc/cpu/sve_default_vector_length >> >> >> elsewhere in the patch series i see >> >> /proc/sys/abi/sve_default_vector_length >> >> is this supposed to be the same? > > Good spot, thanks! > > /proc/cpu/ was the old location: they should both say /proc/abi/. > I'll fix it. Isn't /sys (or rather sysfs) the preferred location for modern control knobs that mirror the kernels object model or is SVE a special case for extending /proc? > > ---Dave -- Alex Bennée ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v2 26/28] arm64/sve: Add documentation
On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote: > On 31/08/17 18:00, Dave Martin wrote: > > +9. System runtime configuration > > + > > + > > +* To mitigate the ABI impact of expansion of the signal frame, a policy > > + mechanism is provided for administrators, distro maintainers and > > developers > > + to set the default vector length for userspace processes: > > + > > +/proc/cpu/sve_default_vector_length > > > elsewhere in the patch series i see > > /proc/sys/abi/sve_default_vector_length > > is this supposed to be the same? Good spot, thanks! /proc/cpu/ was the old location: they should both say /proc/abi/. I'll fix it. ---Dave ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v2 26/28] arm64/sve: Add documentation
On 31/08/17 18:00, Dave Martin wrote: > +9. System runtime configuration > + > + > +* To mitigate the ABI impact of expansion of the signal frame, a policy > + mechanism is provided for administrators, distro maintainers and developers > + to set the default vector length for userspace processes: > + > +/proc/cpu/sve_default_vector_length elsewhere in the patch series i see /proc/sys/abi/sve_default_vector_length is this supposed to be the same? ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v2 26/28] arm64/sve: Add documentation
On Thu, Oct 05, 2017 at 05:39:24PM +0100, Szabolcs Nagy wrote:
> On 31/08/17 18:00, Dave Martin wrote:
[...]
> > + PR_SVE_SET_VL_ONEXEC
> > +
> > + Defer the requested vector length change until the next execve().
> > + This allows launching of a new program with a different vector
> > + length, while avoiding runtime side effects in the caller.
> > +
> > + This also overrides the effect of PR_SVE_SET_VL_INHERIT for the
> > + first execve().
> > +
> > + Without PR_SVE_SET_VL_ONEXEC, any outstanding deferred vector
> > + length change is cancelled.
> > +
>
> based on later text it seems this works if exeve is
> called in the same thread as prctl was called in.
>
> this is a bit weird from user-space pov so it may
> make sense to state it here explicitly.
True. Looking at the prctl(2) man page it looks like other per-thread
properties are inherited across execve() in a similar way, but it's at
least worth a mention. PR_SET_SECCOMP seems to work like this, for
example.
So, the intention is that you do a prctl(...ONEXEC) in the run up to
execve(), rather than doing it at other random times. The primary
reason for ONEXEC is to avoid the side-effects of actually changing
the VL.
Looking at this though...
I wonder whether PR_SVE_SET_VL(... PR_SVE_SET_VL_ONEXEC) should return
the VL set for exec, rather than the current VL (which is unchanged by
definition in this case, thus uninteresting).
This would allow the ONEXEC flag to be used to probe for available VLs
without the other side-effects of changing VL, something like:
int old = prctl(PR_SVE_GET_VL);
int ret;
ret = prctl(PR_SVE_SET_VL, 144 | PR_SVE_SET_VL_ONEXEC);
if (ret == -1) {
perror("PR_SVE_SET_VL");
goto error;
}
if ((ret & PR_SVE_VL_LEN_MASK) == 144)
have_vl_144 = true;
if (prctl(PR_SVE_SET_VL, old | PR_SVE_SET_VL_ONEXEC) == -1) {
perror("PR_SVE_SET_VL");
goto error;
}
This does _not_ do the expected thing right now, since there's no
direct way to retrieve thread.sve_vl_onexec directly from the kernel
(and it didn't really seem justified to add one).
Thoughts?
Cheers
---Dave
___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v2 26/28] arm64/sve: Add documentation
On 31/08/17 18:00, Dave Martin wrote: > +prctl(PR_SVE_SET_VL, unsigned long arg) > + > +Sets the vector length of the calling thread and related flags, where > +arg == vl | flags. > + > +vl is the desired vector length, where sve_vl_valid(vl) must be true. > + > +flags: > + > + PR_SVE_SET_VL_INHERIT > + > + Inherit the current vector length across execve(). Otherwise, the > + vector length is reset to the system default at execve(). (See > + Section 9.) > + > + PR_SVE_SET_VL_ONEXEC > + > + Defer the requested vector length change until the next execve(). > + This allows launching of a new program with a different vector > + length, while avoiding runtime side effects in the caller. > + > + This also overrides the effect of PR_SVE_SET_VL_INHERIT for the > + first execve(). > + > + Without PR_SVE_SET_VL_ONEXEC, any outstanding deferred vector > + length change is cancelled. > + based on later text it seems this works if exeve is called in the same thread as prctl was called in. this is a bit weird from user-space pov so it may make sense to state it here explicitly. > +Return value: a nonnegative on success, or a negative value on error: > + EINVAL: SVE not supported, invalid vector length requested, or > + invalid flags. > + > +On success, the calling thread's vector length is changed to the largest > +value supported by the system that is less than or equal to vl. > +If vl == SVE_VL_MAX, the calling thread's vector length is changed to the > +largest value supported by the system. > + > +The returned value describes the resulting configuration, encoded as for > +PR_SVE_GET_VL. > + > +Changing the vector length causes all of P0..P15, FFR and all bits of > +Z0..V31 except for Z0 bits [127:0] .. Z31 bits [127:0] to become > +unspecified. Calling PR_SVE_SET_VL with vl equal to the thread's current > +vector length does not constitute a change to the vector length for this > +purpose. ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
[PATCH v2 26/28] arm64/sve: Add documentation
This patch adds basic documentation of the user/kernel interface provided by the for SVE. Signed-off-by: Dave Martin Cc: Alex Bennée Cc: Mark Rutland --- Changes since v1 Requested by Alex Bennée: * Add cross-reference from sigcontext.h to the description of VL/VQ. Other: * [ABI fix] Bail out with -EIO if attempting to set the SVE regs for an unsupported VL, rather than misparsing the regset data. * Document detection of SVE via CPUID MRS. * Remove comment about coredump notes being padded for NT_ARM_SVE. This is no longer the case with variable-size regsets supported in the core code. * Fix a missing bullet char --- Documentation/arm64/sve.txt | 477 +++ arch/arm64/include/uapi/asm/sigcontext.h | 3 + 2 files changed, 480 insertions(+) create mode 100644 Documentation/arm64/sve.txt diff --git a/Documentation/arm64/sve.txt b/Documentation/arm64/sve.txt new file mode 100644 index 000..3c0f97f --- /dev/null +++ b/Documentation/arm64/sve.txt @@ -0,0 +1,477 @@ +Scalable Vector Extension support for AArch64 Linux +=== + +Author: Dave Martin +Date: 4 August 2017 + +This document outlines briefly the interface provided to userspace by Linux in +order to support use of the ARM Scalable Vector Extension (SVE). + +This is an outline of the most important features and issues only and not +intended to be exhaustive. + +This document does not aim to describe the SVE architecture or programmer's +model. To aid understanding, a minimal description of relevant programmer's +model features for SVE is included in Appendix A. + + +1. General +--- + +* SVE registers Z0..Z31, P0..P15 and FFR and the current vector length VL, are + tracked per-thread. + +* The presence of SVE is reported to userspace via HWCAP_SVE in the aux vector + AT_HWCAP entry. Presence of this flag implies the presence of the SVE + instructions and registers, and the Linux-specific system interfaces + described in this document. SVE is reported in /proc/cpuinfo as "sve". + +* Support for the execution of SVE instructions in userspace can also be + detected by reading the CPU ID register ID_AA64PFR0_EL1 using an MRS + instruction, and checking that the value of the SVE field is nonzero. [3] + + It does not guarantee the presence of the system interfaces described in the + following sections: software that needs to verify that those interfaces are + present must check for HWCAP_SVE instead. + +* Debuggers should restrict themselves to interacting with the target via the + NT_ARM_SVE regset. The recommended way of detecting support for this regset + is to connect to a target process first and then attempt a + ptrace(PTRACE_GETREGSET, pid, NT_ARM_SVE, &iov). + + +2. Vector length terminology +- + +The size of an SVE vector (Z) register is referred to as the "vector length". + +To avoid confusion about the units used to express vector length, the kernel +adopts the following conventions: + +* Vector length (VL) = size of a Z-register in bytes + +* Vector quadwords (VQ) = size of a Z-register in units of 128 bits + +(So, VL = 16 * VQ.) + +The VQ convention is used where the underlying granularity is important, such +as in data structure definitions. In most other situations, the VL convention +is used. This is consistent with the meaning of the "VL" pseudo-register in +the SVE instruction set architecture. + + +3. System call behaviour +- + +* On syscall, V0..V31 are preserved (as without SVE). Thus, bits [127:0] of + Z0..Z31 are preserved. All other bits of Z0..Z31, and all of P0..P15 and FFR + become unspecified on return from a syscall. + +* The SVE registers are not used to pass arguments to or receive results from + any syscall. + +* In practice the affected registers/bits will be preserved or will be replaced + with zeros on return from a syscall, but userspace should not make + assumptions about this. The kernel behaviour may vary on a case-by-case + basis. + + +4. Signal handling +--- + +* A new signal frame record sve_context encodes the SVE registers on signal + delivery. [1] + +* This record is supplementary to fpsimd_context. The FPSR and FPCR registers + are only present in fpsimd_context. For convenience, the content of V0..V31 + is duplicated between sve_context and fpsimd_context. + +* The signal frame record for SVE always contains basic metadata, in particular + the thread's vector length (in sve_context.vl). + +* The SVE registers may or may not be included in the record, depending on + whether the registers are live for the thread. The registers are present if + and only if: + sve_context.head.size >= SVE_SIG_CONTEXT_SIZE(sve_vq_from_vl(sve_context.vl)). + +* If the registers are present, the remainder of the record has a vl-dependent + size and layout. Macros SIG_SVE_* a
