Re: [PATCH v5 6/6] powerpc/pseries: Add support for FORM2 associativity
On Thu, Jul 22, 2021 at 01:04:42PM +0530, Aneesh Kumar K.V wrote:
> David Gibson writes:
>
> > On Mon, Jun 28, 2021 at 08:41:17PM +0530, Aneesh Kumar K.V wrote:
> >> PAPR interface currently supports two different ways of communicating
> >> resource
> >> grouping details to the OS. These are referred to as Form 0 and Form 1
> >> associativity grouping. Form 0 is the older format and is now considered
> >> deprecated. This patch adds another resource grouping named FORM2.
> >>
> >> Signed-off-by: Daniel Henrique Barboza
> >> Signed-off-by: Aneesh Kumar K.V
> >> ---
> >> Documentation/powerpc/associativity.rst | 103 ++
> >> arch/powerpc/include/asm/firmware.h | 3 +-
> >> arch/powerpc/include/asm/prom.h | 1 +
> >> arch/powerpc/kernel/prom_init.c | 3 +-
> >> arch/powerpc/mm/numa.c| 157 ++
> >> arch/powerpc/platforms/pseries/firmware.c | 1 +
> >> 6 files changed, 242 insertions(+), 26 deletions(-)
> >> create mode 100644 Documentation/powerpc/associativity.rst
> >>
> >> diff --git a/Documentation/powerpc/associativity.rst
> >> b/Documentation/powerpc/associativity.rst
> >> new file mode 100644
> >> index ..31cc7da2c7a6
> >> --- /dev/null
> >> +++ b/Documentation/powerpc/associativity.rst
> >> @@ -0,0 +1,103 @@
> >> +
> >> +NUMA resource associativity
> >> +=
> >> +
> >> +Associativity represents the groupings of the various platform resources
> >> into
> >> +domains of substantially similar mean performance relative to resources
> >> outside
> >> +of that domain. Resources subsets of a given domain that exhibit better
> >> +performance relative to each other than relative to other resources
> >> subsets
> >> +are represented as being members of a sub-grouping domain. This
> >> performance
> >> +characteristic is presented in terms of NUMA node distance within the
> >> Linux kernel.
> >> +From the platform view, these groups are also referred to as domains.
> >
> > Pretty hard to decipher, but that's typical for PAPR.
> >
> >> +PAPR interface currently supports different ways of communicating these
> >> resource
> >> +grouping details to the OS. These are referred to as Form 0, Form 1 and
> >> Form2
> >> +associativity grouping. Form 0 is the older format and is now considered
> >> deprecated.
> >
> > Nit: s/older/oldest/ since there are now >2 forms.
>
> updated.
>
> >
> >> +Hypervisor indicates the type/form of associativity used via
> >> "ibm,architecture-vec-5 property".
> >> +Bit 0 of byte 5 in the "ibm,architecture-vec-5" property indicates usage
> >> of Form 0 or Form 1.
> >> +A value of 1 indicates the usage of Form 1 associativity. For Form 2
> >> associativity
> >> +bit 2 of byte 5 in the "ibm,architecture-vec-5" property is used.
> >> +
> >> +Form 0
> >> +-
> >> +Form 0 associativity supports only two NUMA distances (LOCAL and REMOTE).
> >> +
> >> +Form 1
> >> +-
> >> +With Form 1 a combination of ibm,associativity-reference-points, and
> >> ibm,associativity
> >> +device tree properties are used to determine the NUMA distance between
> >> resource groups/domains.
> >> +
> >> +The “ibm,associativity” property contains a list of one or more numbers
> >> (domainID)
> >> +representing the resource’s platform grouping domains.
> >> +
> >> +The “ibm,associativity-reference-points” property contains a list of one
> >> or more numbers
> >> +(domainID index) that represents the 1 based ordinal in the associativity
> >> lists.
> >> +The list of domainID indexes represents an increasing hierarchy of
> >> resource grouping.
> >> +
> >> +ex:
> >> +{ primary domainID index, secondary domainID index, tertiary domainID
> >> index.. }
> >> +
> >> +Linux kernel uses the domainID at the primary domainID index as the NUMA
> >> node id.
> >> +Linux kernel computes NUMA distance between two domains by recursively
> >> comparing
> >> +if they belong to the same higher-level domains. For mismatch at every
> >> higher
> >> +level of the resource group, the kernel doubles the NUMA distance between
> >> the
> >> +comparing domains.
> >> +
> >> +Form 2
> >> +---
> >> +Form 2 associativity format adds separate device tree properties
> >> representing NUMA node distance
> >> +thereby making the node distance computation flexible. Form 2 also allows
> >> flexible primary
> >> +domain numbering. With numa distance computation now detached from the
> >> index value in
> >> +"ibm,associativity-reference-points" property, Form 2 allows a large
> >> number of primary domain
> >> +ids at the same domainID index representing resource groups of different
> >> performance/latency
> >> +characteristics.
> >> +
> >> +Hypervisor indicates the usage of FORM2 associativity using bit 2 of byte
> >> 5 in the
> >> +"ibm,architecture-vec-5" property.
> >> +
> >> +"ibm,numa-lookup-index-table" property contains a list of one or more
> >>
Re: [PATCH v5 6/6] powerpc/pseries: Add support for FORM2 associativity
David Gibson writes:
> On Mon, Jun 28, 2021 at 08:41:17PM +0530, Aneesh Kumar K.V wrote:
>> PAPR interface currently supports two different ways of communicating
>> resource
>> grouping details to the OS. These are referred to as Form 0 and Form 1
>> associativity grouping. Form 0 is the older format and is now considered
>> deprecated. This patch adds another resource grouping named FORM2.
>>
>> Signed-off-by: Daniel Henrique Barboza
>> Signed-off-by: Aneesh Kumar K.V
>> ---
>> Documentation/powerpc/associativity.rst | 103 ++
>> arch/powerpc/include/asm/firmware.h | 3 +-
>> arch/powerpc/include/asm/prom.h | 1 +
>> arch/powerpc/kernel/prom_init.c | 3 +-
>> arch/powerpc/mm/numa.c| 157 ++
>> arch/powerpc/platforms/pseries/firmware.c | 1 +
>> 6 files changed, 242 insertions(+), 26 deletions(-)
>> create mode 100644 Documentation/powerpc/associativity.rst
>>
>> diff --git a/Documentation/powerpc/associativity.rst
>> b/Documentation/powerpc/associativity.rst
>> new file mode 100644
>> index ..31cc7da2c7a6
>> --- /dev/null
>> +++ b/Documentation/powerpc/associativity.rst
>> @@ -0,0 +1,103 @@
>> +
>> +NUMA resource associativity
>> +=
>> +
>> +Associativity represents the groupings of the various platform resources
>> into
>> +domains of substantially similar mean performance relative to resources
>> outside
>> +of that domain. Resources subsets of a given domain that exhibit better
>> +performance relative to each other than relative to other resources subsets
>> +are represented as being members of a sub-grouping domain. This performance
>> +characteristic is presented in terms of NUMA node distance within the Linux
>> kernel.
>> +From the platform view, these groups are also referred to as domains.
>
> Pretty hard to decipher, but that's typical for PAPR.
>
>> +PAPR interface currently supports different ways of communicating these
>> resource
>> +grouping details to the OS. These are referred to as Form 0, Form 1 and
>> Form2
>> +associativity grouping. Form 0 is the older format and is now considered
>> deprecated.
>
> Nit: s/older/oldest/ since there are now >2 forms.
updated.
>
>> +Hypervisor indicates the type/form of associativity used via
>> "ibm,architecture-vec-5 property".
>> +Bit 0 of byte 5 in the "ibm,architecture-vec-5" property indicates usage of
>> Form 0 or Form 1.
>> +A value of 1 indicates the usage of Form 1 associativity. For Form 2
>> associativity
>> +bit 2 of byte 5 in the "ibm,architecture-vec-5" property is used.
>> +
>> +Form 0
>> +-
>> +Form 0 associativity supports only two NUMA distances (LOCAL and REMOTE).
>> +
>> +Form 1
>> +-
>> +With Form 1 a combination of ibm,associativity-reference-points, and
>> ibm,associativity
>> +device tree properties are used to determine the NUMA distance between
>> resource groups/domains.
>> +
>> +The “ibm,associativity” property contains a list of one or more numbers
>> (domainID)
>> +representing the resource’s platform grouping domains.
>> +
>> +The “ibm,associativity-reference-points” property contains a list of one or
>> more numbers
>> +(domainID index) that represents the 1 based ordinal in the associativity
>> lists.
>> +The list of domainID indexes represents an increasing hierarchy of resource
>> grouping.
>> +
>> +ex:
>> +{ primary domainID index, secondary domainID index, tertiary domainID
>> index.. }
>> +
>> +Linux kernel uses the domainID at the primary domainID index as the NUMA
>> node id.
>> +Linux kernel computes NUMA distance between two domains by recursively
>> comparing
>> +if they belong to the same higher-level domains. For mismatch at every
>> higher
>> +level of the resource group, the kernel doubles the NUMA distance between
>> the
>> +comparing domains.
>> +
>> +Form 2
>> +---
>> +Form 2 associativity format adds separate device tree properties
>> representing NUMA node distance
>> +thereby making the node distance computation flexible. Form 2 also allows
>> flexible primary
>> +domain numbering. With numa distance computation now detached from the
>> index value in
>> +"ibm,associativity-reference-points" property, Form 2 allows a large number
>> of primary domain
>> +ids at the same domainID index representing resource groups of different
>> performance/latency
>> +characteristics.
>> +
>> +Hypervisor indicates the usage of FORM2 associativity using bit 2 of byte 5
>> in the
>> +"ibm,architecture-vec-5" property.
>> +
>> +"ibm,numa-lookup-index-table" property contains a list of one or more
>> numbers representing
>> +the domainIDs present in the system. The offset of the domainID in this
>> property is
>> +used as an index while computing numa distance information via
>> "ibm,numa-distance-table".
>> +
>> +prop-encoded-array: The number N of the domainIDs encoded as with
>> encode-int, followed by
>> +N
Re: [PATCH v5 6/6] powerpc/pseries: Add support for FORM2 associativity
On Mon, Jun 28, 2021 at 08:41:17PM +0530, Aneesh Kumar K.V wrote:
> PAPR interface currently supports two different ways of communicating resource
> grouping details to the OS. These are referred to as Form 0 and Form 1
> associativity grouping. Form 0 is the older format and is now considered
> deprecated. This patch adds another resource grouping named FORM2.
>
> Signed-off-by: Daniel Henrique Barboza
> Signed-off-by: Aneesh Kumar K.V
> ---
> Documentation/powerpc/associativity.rst | 103 ++
> arch/powerpc/include/asm/firmware.h | 3 +-
> arch/powerpc/include/asm/prom.h | 1 +
> arch/powerpc/kernel/prom_init.c | 3 +-
> arch/powerpc/mm/numa.c| 157 ++
> arch/powerpc/platforms/pseries/firmware.c | 1 +
> 6 files changed, 242 insertions(+), 26 deletions(-)
> create mode 100644 Documentation/powerpc/associativity.rst
>
> diff --git a/Documentation/powerpc/associativity.rst
> b/Documentation/powerpc/associativity.rst
> new file mode 100644
> index ..31cc7da2c7a6
> --- /dev/null
> +++ b/Documentation/powerpc/associativity.rst
> @@ -0,0 +1,103 @@
> +
> +NUMA resource associativity
> +=
> +
> +Associativity represents the groupings of the various platform resources into
> +domains of substantially similar mean performance relative to resources
> outside
> +of that domain. Resources subsets of a given domain that exhibit better
> +performance relative to each other than relative to other resources subsets
> +are represented as being members of a sub-grouping domain. This performance
> +characteristic is presented in terms of NUMA node distance within the Linux
> kernel.
> +From the platform view, these groups are also referred to as domains.
Pretty hard to decipher, but that's typical for PAPR.
> +PAPR interface currently supports different ways of communicating these
> resource
> +grouping details to the OS. These are referred to as Form 0, Form 1 and Form2
> +associativity grouping. Form 0 is the older format and is now considered
> deprecated.
Nit: s/older/oldest/ since there are now >2 forms.
> +Hypervisor indicates the type/form of associativity used via
> "ibm,architecture-vec-5 property".
> +Bit 0 of byte 5 in the "ibm,architecture-vec-5" property indicates usage of
> Form 0 or Form 1.
> +A value of 1 indicates the usage of Form 1 associativity. For Form 2
> associativity
> +bit 2 of byte 5 in the "ibm,architecture-vec-5" property is used.
> +
> +Form 0
> +-
> +Form 0 associativity supports only two NUMA distances (LOCAL and REMOTE).
> +
> +Form 1
> +-
> +With Form 1 a combination of ibm,associativity-reference-points, and
> ibm,associativity
> +device tree properties are used to determine the NUMA distance between
> resource groups/domains.
> +
> +The “ibm,associativity” property contains a list of one or more numbers
> (domainID)
> +representing the resource’s platform grouping domains.
> +
> +The “ibm,associativity-reference-points” property contains a list of one or
> more numbers
> +(domainID index) that represents the 1 based ordinal in the associativity
> lists.
> +The list of domainID indexes represents an increasing hierarchy of resource
> grouping.
> +
> +ex:
> +{ primary domainID index, secondary domainID index, tertiary domainID
> index.. }
> +
> +Linux kernel uses the domainID at the primary domainID index as the NUMA
> node id.
> +Linux kernel computes NUMA distance between two domains by recursively
> comparing
> +if they belong to the same higher-level domains. For mismatch at every higher
> +level of the resource group, the kernel doubles the NUMA distance between the
> +comparing domains.
> +
> +Form 2
> +---
> +Form 2 associativity format adds separate device tree properties
> representing NUMA node distance
> +thereby making the node distance computation flexible. Form 2 also allows
> flexible primary
> +domain numbering. With numa distance computation now detached from the index
> value in
> +"ibm,associativity-reference-points" property, Form 2 allows a large number
> of primary domain
> +ids at the same domainID index representing resource groups of different
> performance/latency
> +characteristics.
> +
> +Hypervisor indicates the usage of FORM2 associativity using bit 2 of byte 5
> in the
> +"ibm,architecture-vec-5" property.
> +
> +"ibm,numa-lookup-index-table" property contains a list of one or more
> numbers representing
> +the domainIDs present in the system. The offset of the domainID in this
> property is
> +used as an index while computing numa distance information via
> "ibm,numa-distance-table".
> +
> +prop-encoded-array: The number N of the domainIDs encoded as with
> encode-int, followed by
> +N domainID encoded as with encode-int
> +
> +For ex:
> +"ibm,numa-lookup-index-table" = {4, 0, 8, 250, 252}. The offset of domainID
> 8 (2) is used when
> +computing the
[PATCH v5 6/6] powerpc/pseries: Add support for FORM2 associativity
PAPR interface currently supports two different ways of communicating resource
grouping details to the OS. These are referred to as Form 0 and Form 1
associativity grouping. Form 0 is the older format and is now considered
deprecated. This patch adds another resource grouping named FORM2.
Signed-off-by: Daniel Henrique Barboza
Signed-off-by: Aneesh Kumar K.V
---
Documentation/powerpc/associativity.rst | 103 ++
arch/powerpc/include/asm/firmware.h | 3 +-
arch/powerpc/include/asm/prom.h | 1 +
arch/powerpc/kernel/prom_init.c | 3 +-
arch/powerpc/mm/numa.c| 157 ++
arch/powerpc/platforms/pseries/firmware.c | 1 +
6 files changed, 242 insertions(+), 26 deletions(-)
create mode 100644 Documentation/powerpc/associativity.rst
diff --git a/Documentation/powerpc/associativity.rst
b/Documentation/powerpc/associativity.rst
new file mode 100644
index ..31cc7da2c7a6
--- /dev/null
+++ b/Documentation/powerpc/associativity.rst
@@ -0,0 +1,103 @@
+
+NUMA resource associativity
+=
+
+Associativity represents the groupings of the various platform resources into
+domains of substantially similar mean performance relative to resources outside
+of that domain. Resources subsets of a given domain that exhibit better
+performance relative to each other than relative to other resources subsets
+are represented as being members of a sub-grouping domain. This performance
+characteristic is presented in terms of NUMA node distance within the Linux
kernel.
+From the platform view, these groups are also referred to as domains.
+
+PAPR interface currently supports different ways of communicating these
resource
+grouping details to the OS. These are referred to as Form 0, Form 1 and Form2
+associativity grouping. Form 0 is the older format and is now considered
deprecated.
+
+Hypervisor indicates the type/form of associativity used via
"ibm,architecture-vec-5 property".
+Bit 0 of byte 5 in the "ibm,architecture-vec-5" property indicates usage of
Form 0 or Form 1.
+A value of 1 indicates the usage of Form 1 associativity. For Form 2
associativity
+bit 2 of byte 5 in the "ibm,architecture-vec-5" property is used.
+
+Form 0
+-
+Form 0 associativity supports only two NUMA distances (LOCAL and REMOTE).
+
+Form 1
+-
+With Form 1 a combination of ibm,associativity-reference-points, and
ibm,associativity
+device tree properties are used to determine the NUMA distance between
resource groups/domains.
+
+The “ibm,associativity” property contains a list of one or more numbers
(domainID)
+representing the resource’s platform grouping domains.
+
+The “ibm,associativity-reference-points” property contains a list of one or
more numbers
+(domainID index) that represents the 1 based ordinal in the associativity
lists.
+The list of domainID indexes represents an increasing hierarchy of resource
grouping.
+
+ex:
+{ primary domainID index, secondary domainID index, tertiary domainID index.. }
+
+Linux kernel uses the domainID at the primary domainID index as the NUMA node
id.
+Linux kernel computes NUMA distance between two domains by recursively
comparing
+if they belong to the same higher-level domains. For mismatch at every higher
+level of the resource group, the kernel doubles the NUMA distance between the
+comparing domains.
+
+Form 2
+---
+Form 2 associativity format adds separate device tree properties representing
NUMA node distance
+thereby making the node distance computation flexible. Form 2 also allows
flexible primary
+domain numbering. With numa distance computation now detached from the index
value in
+"ibm,associativity-reference-points" property, Form 2 allows a large number of
primary domain
+ids at the same domainID index representing resource groups of different
performance/latency
+characteristics.
+
+Hypervisor indicates the usage of FORM2 associativity using bit 2 of byte 5 in
the
+"ibm,architecture-vec-5" property.
+
+"ibm,numa-lookup-index-table" property contains a list of one or more numbers
representing
+the domainIDs present in the system. The offset of the domainID in this
property is
+used as an index while computing numa distance information via
"ibm,numa-distance-table".
+
+prop-encoded-array: The number N of the domainIDs encoded as with encode-int,
followed by
+N domainID encoded as with encode-int
+
+For ex:
+"ibm,numa-lookup-index-table" = {4, 0, 8, 250, 252}. The offset of domainID 8
(2) is used when
+computing the distance of domain 8 from other domains present in the system.
For the rest of
+this document, this offset will be referred to as domain distance offset.
+
+"ibm,numa-distance-table" property contains a list of one or more numbers
representing the NUMA
+distance between resource groups/domains present in the system.
+
+prop-encoded-array: The number N of the distance values encoded as with
encode-int,
