Re: [PATCH v2] drm/i915: Document the Virtual Engine uAPI
On 17/06/2021 18:17, Daniel Vetter wrote:
On Mon, Jun 14, 2021 at 10:09:59AM +0100, Tvrtko Ursulin wrote:
From: Tvrtko Ursulin
A little bit of documentation covering the topics of engine discovery,
context engine maps and virtual engines. It is not very detailed but
supposed to be a starting point of giving a brief high level overview of
general principles and intended use cases.
v2:
* Have the text in uapi header and link from there.
Signed-off-by: Tvrtko Ursulin
Cc: Daniel Vetter
What I meant was the kerneldoc directly as kerneldoc for the uapi structs,
like Matt has done for e.g. drm_i915_gem_create_ext_memory_regions.
Hm I wanted to add some commentary to give a high level picture of this
area and not necessarily focus on uapi structs details. Some of them (at
least one I think) already have their own documentation and the rest
could be added in detail. But I do think a short "story" in the order of
chapters I added to i915.rst makes sense as reading material.
But then I also realized that Matt hasn't set up the include for this, so
it's not automatic at all yet :-/
No idea what where how you mean. The fact i915_drm.h docs are not pulled
in anywhere?
Regards,
Tvrtko
-Daniel
---
Documentation/gpu/i915.rst | 18
include/uapi/drm/i915_drm.h | 188
2 files changed, 206 insertions(+)
diff --git a/Documentation/gpu/i915.rst b/Documentation/gpu/i915.rst
index 42ce0196930a..00aa55bbe0fd 100644
--- a/Documentation/gpu/i915.rst
+++ b/Documentation/gpu/i915.rst
@@ -335,6 +335,24 @@ for execution also include a list of all locations within
buffers that
refer to GPU-addresses so that the kernel can edit the buffer correctly.
This process is dubbed relocation.
+Engine Discovery uAPI
+-
+
+.. kernel-doc:: include/uapi/drm/i915_drm.h
+ :doc: Engine Discovery uAPI
+
+Context Engine Map uAPI
+---
+
+.. kernel-doc:: include/uapi/drm/i915_drm.h
+ :doc: Context Engine Map uAPI
+
+Virtual Engine uAPI
+---
+
+.. kernel-doc:: include/uapi/drm/i915_drm.h
+ :doc: Virtual Engine uAPI
+
Locking Guidelines
--
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index a1cb4aa035a9..2f70c48567c0 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -1806,6 +1806,69 @@ struct drm_i915_gem_context_param_sseu {
__u32 rsvd;
};
+/**
+ * DOC: Virtual Engine uAPI
+ *
+ * Virtual engine is a concept where userspace is able to configure a set of
+ * physical engines, submit a batch buffer, and let the driver execute it on
any
+ * engine from the set as it sees fit.
+ *
+ * This is primarily useful on parts which have multiple instances of a same
+ * class engine, like for example GT3+ Skylake parts with their two VCS
engines.
+ *
+ * For instance userspace can enumerate all engines of a certain class using
the
+ * previously described `Engine Discovery uAPI`_. After that userspace can
+ * create a GEM context with a placeholder slot for the virtual engine (using
+ * `I915_ENGINE_CLASS_INVALID` and `I915_ENGINE_CLASS_INVALID_NONE` for class
+ * and instance respectively) and finally using the
+ * `I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE` extension place a virtual engine in
+ * the same reserved slot.
+ *
+ * Example of creating a virtual engine and submitting a batch buffer to it:
+ *
+ * .. code-block:: C
+ *
+ * I915_DEFINE_CONTEXT_ENGINES_LOAD_BALANCE(virtual, 2) = {
+ * .base.name = I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE,
+ * .engine_index = 0, // Place this virtual engine into engine map
slot 0
+ * .num_siblings = 2,
+ * .engines = { { I915_ENGINE_CLASS_VIDEO, 0 },
+ * { I915_ENGINE_CLASS_VIDEO, 1 }, },
+ * };
+ * I915_DEFINE_CONTEXT_PARAM_ENGINES(engines, 1) = {
+ * .engines = { { I915_ENGINE_CLASS_INVALID,
+ *I915_ENGINE_CLASS_INVALID_NONE } },
+ * .extensions = to_user_pointer(), // Chains after
load_balance extension
+ * };
+ * struct drm_i915_gem_context_create_ext_setparam p_engines = {
+ * .base = {
+ * .name = I915_CONTEXT_CREATE_EXT_SETPARAM,
+ * },
+ * .param = {
+ * .param = I915_CONTEXT_PARAM_ENGINES,
+ * .value = to_user_pointer(),
+ * .size = sizeof(engines),
+ * },
+ * };
+ * struct drm_i915_gem_context_create_ext create = {
+ * .flags = I915_CONTEXT_CREATE_FLAGS_USE_EXTENSIONS,
+ * .extensions = to_user_pointer(_engines);
+ * };
+ *
+ * ctx_id = gem_context_create_ext(drm_fd, );
+ *
+ * // Now we have created a GEM context with its engine map containing a
+ * // single virtual engine. Submissions to this slot can go either to
+ * // vcs0 or vcs1, depending on the load
Re: [PATCH v2] drm/i915: Document the Virtual Engine uAPI
On Mon, Jun 14, 2021 at 10:09:59AM +0100, Tvrtko Ursulin wrote:
> From: Tvrtko Ursulin
>
> A little bit of documentation covering the topics of engine discovery,
> context engine maps and virtual engines. It is not very detailed but
> supposed to be a starting point of giving a brief high level overview of
> general principles and intended use cases.
>
> v2:
> * Have the text in uapi header and link from there.
>
> Signed-off-by: Tvrtko Ursulin
> Cc: Daniel Vetter
What I meant was the kerneldoc directly as kerneldoc for the uapi structs,
like Matt has done for e.g. drm_i915_gem_create_ext_memory_regions.
But then I also realized that Matt hasn't set up the include for this, so
it's not automatic at all yet :-/
-Daniel
> ---
> Documentation/gpu/i915.rst | 18
> include/uapi/drm/i915_drm.h | 188
> 2 files changed, 206 insertions(+)
>
> diff --git a/Documentation/gpu/i915.rst b/Documentation/gpu/i915.rst
> index 42ce0196930a..00aa55bbe0fd 100644
> --- a/Documentation/gpu/i915.rst
> +++ b/Documentation/gpu/i915.rst
> @@ -335,6 +335,24 @@ for execution also include a list of all locations
> within buffers that
> refer to GPU-addresses so that the kernel can edit the buffer correctly.
> This process is dubbed relocation.
>
> +Engine Discovery uAPI
> +-
> +
> +.. kernel-doc:: include/uapi/drm/i915_drm.h
> + :doc: Engine Discovery uAPI
> +
> +Context Engine Map uAPI
> +---
> +
> +.. kernel-doc:: include/uapi/drm/i915_drm.h
> + :doc: Context Engine Map uAPI
> +
> +Virtual Engine uAPI
> +---
> +
> +.. kernel-doc:: include/uapi/drm/i915_drm.h
> + :doc: Virtual Engine uAPI
> +
> Locking Guidelines
> --
>
> diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
> index a1cb4aa035a9..2f70c48567c0 100644
> --- a/include/uapi/drm/i915_drm.h
> +++ b/include/uapi/drm/i915_drm.h
> @@ -1806,6 +1806,69 @@ struct drm_i915_gem_context_param_sseu {
> __u32 rsvd;
> };
>
> +/**
> + * DOC: Virtual Engine uAPI
> + *
> + * Virtual engine is a concept where userspace is able to configure a set of
> + * physical engines, submit a batch buffer, and let the driver execute it on
> any
> + * engine from the set as it sees fit.
> + *
> + * This is primarily useful on parts which have multiple instances of a same
> + * class engine, like for example GT3+ Skylake parts with their two VCS
> engines.
> + *
> + * For instance userspace can enumerate all engines of a certain class using
> the
> + * previously described `Engine Discovery uAPI`_. After that userspace can
> + * create a GEM context with a placeholder slot for the virtual engine (using
> + * `I915_ENGINE_CLASS_INVALID` and `I915_ENGINE_CLASS_INVALID_NONE` for class
> + * and instance respectively) and finally using the
> + * `I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE` extension place a virtual engine
> in
> + * the same reserved slot.
> + *
> + * Example of creating a virtual engine and submitting a batch buffer to it:
> + *
> + * .. code-block:: C
> + *
> + * I915_DEFINE_CONTEXT_ENGINES_LOAD_BALANCE(virtual, 2) = {
> + * .base.name = I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE,
> + * .engine_index = 0, // Place this virtual engine into engine map
> slot 0
> + * .num_siblings = 2,
> + * .engines = { { I915_ENGINE_CLASS_VIDEO, 0 },
> + *{ I915_ENGINE_CLASS_VIDEO, 1 }, },
> + * };
> + * I915_DEFINE_CONTEXT_PARAM_ENGINES(engines, 1) = {
> + * .engines = { { I915_ENGINE_CLASS_INVALID,
> + * I915_ENGINE_CLASS_INVALID_NONE } },
> + * .extensions = to_user_pointer(), // Chains after
> load_balance extension
> + * };
> + * struct drm_i915_gem_context_create_ext_setparam p_engines = {
> + * .base = {
> + * .name = I915_CONTEXT_CREATE_EXT_SETPARAM,
> + * },
> + * .param = {
> + * .param = I915_CONTEXT_PARAM_ENGINES,
> + * .value = to_user_pointer(),
> + * .size = sizeof(engines),
> + * },
> + * };
> + * struct drm_i915_gem_context_create_ext create = {
> + * .flags = I915_CONTEXT_CREATE_FLAGS_USE_EXTENSIONS,
> + * .extensions = to_user_pointer(_engines);
> + * };
> + *
> + * ctx_id = gem_context_create_ext(drm_fd, );
> + *
> + * // Now we have created a GEM context with its engine map containing a
> + * // single virtual engine. Submissions to this slot can go either to
> + * // vcs0 or vcs1, depending on the load balancing algorithm used inside
> + * // the driver. The load balancing is dynamic from one batch buffer to
> + * // another and transparent to userspace.
> + *
> + * ...
> + * execbuf.rsvd1 = ctx_id;
> + * execbuf.flags = 0; // Submits to index 0 which is the virtual engine
> + * gem_execbuf(drm_fd, );
> + */
> +
> /*
> *
[PATCH v2] drm/i915: Document the Virtual Engine uAPI
From: Tvrtko Ursulin
A little bit of documentation covering the topics of engine discovery,
context engine maps and virtual engines. It is not very detailed but
supposed to be a starting point of giving a brief high level overview of
general principles and intended use cases.
v2:
* Have the text in uapi header and link from there.
Signed-off-by: Tvrtko Ursulin
Cc: Daniel Vetter
---
Documentation/gpu/i915.rst | 18
include/uapi/drm/i915_drm.h | 188
2 files changed, 206 insertions(+)
diff --git a/Documentation/gpu/i915.rst b/Documentation/gpu/i915.rst
index 42ce0196930a..00aa55bbe0fd 100644
--- a/Documentation/gpu/i915.rst
+++ b/Documentation/gpu/i915.rst
@@ -335,6 +335,24 @@ for execution also include a list of all locations within
buffers that
refer to GPU-addresses so that the kernel can edit the buffer correctly.
This process is dubbed relocation.
+Engine Discovery uAPI
+-
+
+.. kernel-doc:: include/uapi/drm/i915_drm.h
+ :doc: Engine Discovery uAPI
+
+Context Engine Map uAPI
+---
+
+.. kernel-doc:: include/uapi/drm/i915_drm.h
+ :doc: Context Engine Map uAPI
+
+Virtual Engine uAPI
+---
+
+.. kernel-doc:: include/uapi/drm/i915_drm.h
+ :doc: Virtual Engine uAPI
+
Locking Guidelines
--
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index a1cb4aa035a9..2f70c48567c0 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -1806,6 +1806,69 @@ struct drm_i915_gem_context_param_sseu {
__u32 rsvd;
};
+/**
+ * DOC: Virtual Engine uAPI
+ *
+ * Virtual engine is a concept where userspace is able to configure a set of
+ * physical engines, submit a batch buffer, and let the driver execute it on
any
+ * engine from the set as it sees fit.
+ *
+ * This is primarily useful on parts which have multiple instances of a same
+ * class engine, like for example GT3+ Skylake parts with their two VCS
engines.
+ *
+ * For instance userspace can enumerate all engines of a certain class using
the
+ * previously described `Engine Discovery uAPI`_. After that userspace can
+ * create a GEM context with a placeholder slot for the virtual engine (using
+ * `I915_ENGINE_CLASS_INVALID` and `I915_ENGINE_CLASS_INVALID_NONE` for class
+ * and instance respectively) and finally using the
+ * `I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE` extension place a virtual engine in
+ * the same reserved slot.
+ *
+ * Example of creating a virtual engine and submitting a batch buffer to it:
+ *
+ * .. code-block:: C
+ *
+ * I915_DEFINE_CONTEXT_ENGINES_LOAD_BALANCE(virtual, 2) = {
+ * .base.name = I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE,
+ * .engine_index = 0, // Place this virtual engine into engine map
slot 0
+ * .num_siblings = 2,
+ * .engines = { { I915_ENGINE_CLASS_VIDEO, 0 },
+ * { I915_ENGINE_CLASS_VIDEO, 1 }, },
+ * };
+ * I915_DEFINE_CONTEXT_PARAM_ENGINES(engines, 1) = {
+ * .engines = { { I915_ENGINE_CLASS_INVALID,
+ *I915_ENGINE_CLASS_INVALID_NONE } },
+ * .extensions = to_user_pointer(), // Chains after
load_balance extension
+ * };
+ * struct drm_i915_gem_context_create_ext_setparam p_engines = {
+ * .base = {
+ * .name = I915_CONTEXT_CREATE_EXT_SETPARAM,
+ * },
+ * .param = {
+ * .param = I915_CONTEXT_PARAM_ENGINES,
+ * .value = to_user_pointer(),
+ * .size = sizeof(engines),
+ * },
+ * };
+ * struct drm_i915_gem_context_create_ext create = {
+ * .flags = I915_CONTEXT_CREATE_FLAGS_USE_EXTENSIONS,
+ * .extensions = to_user_pointer(_engines);
+ * };
+ *
+ * ctx_id = gem_context_create_ext(drm_fd, );
+ *
+ * // Now we have created a GEM context with its engine map containing a
+ * // single virtual engine. Submissions to this slot can go either to
+ * // vcs0 or vcs1, depending on the load balancing algorithm used inside
+ * // the driver. The load balancing is dynamic from one batch buffer to
+ * // another and transparent to userspace.
+ *
+ * ...
+ * execbuf.rsvd1 = ctx_id;
+ * execbuf.flags = 0; // Submits to index 0 which is the virtual engine
+ * gem_execbuf(drm_fd, );
+ */
+
/*
* i915_context_engines_load_balance:
*
@@ -1882,6 +1945,61 @@ struct i915_context_engines_bond {
struct i915_engine_class_instance engines[N__]; \
} __attribute__((packed)) name__
+/**
+ * DOC: Context Engine Map uAPI
+ *
+ * Context engine map is a new way of addressing engines when submitting batch-
+ * buffers, replacing the existing way of using identifiers like
`I915_EXEC_BLT`
+ * inside the flags field of `struct drm_i915_gem_execbuffer2`.
+ *
+ * To use it created GEM contexts need to be configured with a
