Re: [PATCH v2 3/6] irqchip/gic-v4.1: Ensure L2 vPE table is allocated at RD level
Hi Marc,
On 2020/2/13 22:22, Marc Zyngier wrote:
Hi Zenghui,
On 2020-02-06 07:57, Zenghui Yu wrote:
In GICv4, we will ensure that level2 vPE table memory is allocated
for the specified vpe_id on all v4 ITS, in its_alloc_vpe_table().
This still works well for the typical GICv4.1 implementation, where
the new vPE table is shared between the ITSs and the RDs.
To make it explicit, let us introduce allocate_vpe_l2_table() to
make sure that the L2 tables are allocated on all v4.1 RDs. We're
likely not need to allocate memory in it because the vPE table is
shared and (L2 table is) already allocated at ITS level, except
for the case where the ITS doesn't share anything (say SVPET == 0,
practically unlikely but architecturally allowed).
The implementation of allocate_vpe_l2_table() is mostly copied from
its_alloc_table_entry().
Signed-off-by: Zenghui Yu
---
drivers/irqchip/irq-gic-v3-its.c | 80
1 file changed, 80 insertions(+)
diff --git a/drivers/irqchip/irq-gic-v3-its.c
b/drivers/irqchip/irq-gic-v3-its.c
index 0f1fe56ce0af..ae4e7b355b46 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -2443,6 +2443,72 @@ static u64
inherit_vpe_l1_table_from_rd(cpumask_t **mask)
return 0;
}
+static bool allocate_vpe_l2_table(int cpu, u32 id)
+{
+ void __iomem *base = gic_data_rdist_cpu(cpu)->rd_base;
+ u64 val, gpsz, npg;
+ unsigned int psz, esz, idx;
+ struct page *page;
+ __le64 *table;
+
+ if (!gic_rdists->has_rvpeid)
+ return true;
+
+ val = gits_read_vpropbaser(base + SZ_128K + GICR_VPROPBASER);
Having rebased the rest of the GICv4.1 series on top of -rc1, I've hit a
small
issue right here. I run a FVP model that only spawns 4 CPUs, while the
DT has
8 of them. This means that online_cpus = 4, and possible_cpus = 8.
So in my case, half of the RDs have base == NULL, and things stop quickly.
Ah, so this may also be why we check '!base' for each possible CPU in
inherit_vpe_l1_table_from_rd(). I didn't think about it at that time.
I plan to queue the following:
diff --git a/drivers/irqchip/irq-gic-v3-its.c
b/drivers/irqchip/irq-gic-v3-its.c
index d85dc8dcb0ad..7656b353a95f 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -2526,6 +2526,10 @@ static bool allocate_vpe_l2_table(int cpu, u32 id)
if (!gic_rdists->has_rvpeid)
return true;
+ /* Skip non-present CPUs */
+ if (!base)
+ return true;
+
val = gicr_read_vpropbaser(base + SZ_128K + GICR_VPROPBASER);
esz = FIELD_GET(GICR_VPROPBASER_4_1_ENTRY_SIZE, val) + 1;
Thanks for fixing the second bug for this patch :-)
Zenghui
___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v2 3/6] irqchip/gic-v4.1: Ensure L2 vPE table is allocated at RD level
Hi Zenghui,
On 2020-02-06 07:57, Zenghui Yu wrote:
In GICv4, we will ensure that level2 vPE table memory is allocated
for the specified vpe_id on all v4 ITS, in its_alloc_vpe_table().
This still works well for the typical GICv4.1 implementation, where
the new vPE table is shared between the ITSs and the RDs.
To make it explicit, let us introduce allocate_vpe_l2_table() to
make sure that the L2 tables are allocated on all v4.1 RDs. We're
likely not need to allocate memory in it because the vPE table is
shared and (L2 table is) already allocated at ITS level, except
for the case where the ITS doesn't share anything (say SVPET == 0,
practically unlikely but architecturally allowed).
The implementation of allocate_vpe_l2_table() is mostly copied from
its_alloc_table_entry().
Signed-off-by: Zenghui Yu
---
drivers/irqchip/irq-gic-v3-its.c | 80
1 file changed, 80 insertions(+)
diff --git a/drivers/irqchip/irq-gic-v3-its.c
b/drivers/irqchip/irq-gic-v3-its.c
index 0f1fe56ce0af..ae4e7b355b46 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -2443,6 +2443,72 @@ static u64
inherit_vpe_l1_table_from_rd(cpumask_t **mask)
return 0;
}
+static bool allocate_vpe_l2_table(int cpu, u32 id)
+{
+ void __iomem *base = gic_data_rdist_cpu(cpu)->rd_base;
+ u64 val, gpsz, npg;
+ unsigned int psz, esz, idx;
+ struct page *page;
+ __le64 *table;
+
+ if (!gic_rdists->has_rvpeid)
+ return true;
+
+ val = gits_read_vpropbaser(base + SZ_128K + GICR_VPROPBASER);
Having rebased the rest of the GICv4.1 series on top of -rc1, I've hit a
small
issue right here. I run a FVP model that only spawns 4 CPUs, while the
DT has
8 of them. This means that online_cpus = 4, and possible_cpus = 8.
So in my case, half of the RDs have base == NULL, and things stop
quickly.
I plan to queue the following:
diff --git a/drivers/irqchip/irq-gic-v3-its.c
b/drivers/irqchip/irq-gic-v3-its.c
index d85dc8dcb0ad..7656b353a95f 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -2526,6 +2526,10 @@ static bool allocate_vpe_l2_table(int cpu, u32
id)
if (!gic_rdists->has_rvpeid)
return true;
+ /* Skip non-present CPUs */
+ if (!base)
+ return true;
+
val = gicr_read_vpropbaser(base + SZ_128K + GICR_VPROPBASER);
esz = FIELD_GET(GICR_VPROPBASER_4_1_ENTRY_SIZE, val) + 1;
Thanks,
M.
--
Jazz is not dead. It just smells funny...
___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
[PATCH v2 3/6] irqchip/gic-v4.1: Ensure L2 vPE table is allocated at RD level
In GICv4, we will ensure that level2 vPE table memory is allocated
for the specified vpe_id on all v4 ITS, in its_alloc_vpe_table().
This still works well for the typical GICv4.1 implementation, where
the new vPE table is shared between the ITSs and the RDs.
To make it explicit, let us introduce allocate_vpe_l2_table() to
make sure that the L2 tables are allocated on all v4.1 RDs. We're
likely not need to allocate memory in it because the vPE table is
shared and (L2 table is) already allocated at ITS level, except
for the case where the ITS doesn't share anything (say SVPET == 0,
practically unlikely but architecturally allowed).
The implementation of allocate_vpe_l2_table() is mostly copied from
its_alloc_table_entry().
Signed-off-by: Zenghui Yu
---
drivers/irqchip/irq-gic-v3-its.c | 80
1 file changed, 80 insertions(+)
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 0f1fe56ce0af..ae4e7b355b46 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -2443,6 +2443,72 @@ static u64 inherit_vpe_l1_table_from_rd(cpumask_t **mask)
return 0;
}
+static bool allocate_vpe_l2_table(int cpu, u32 id)
+{
+ void __iomem *base = gic_data_rdist_cpu(cpu)->rd_base;
+ u64 val, gpsz, npg;
+ unsigned int psz, esz, idx;
+ struct page *page;
+ __le64 *table;
+
+ if (!gic_rdists->has_rvpeid)
+ return true;
+
+ val = gits_read_vpropbaser(base + SZ_128K + GICR_VPROPBASER);
+
+ esz = FIELD_GET(GICR_VPROPBASER_4_1_ENTRY_SIZE, val) + 1;
+ gpsz = FIELD_GET(GICR_VPROPBASER_4_1_PAGE_SIZE, val);
+ npg = FIELD_GET(GICR_VPROPBASER_4_1_SIZE, val) + 1;
+
+ switch (gpsz) {
+ default:
+ WARN_ON(1);
+ /* fall through */
+ case GIC_PAGE_SIZE_4K:
+ psz = SZ_4K;
+ break;
+ case GIC_PAGE_SIZE_16K:
+ psz = SZ_16K;
+ break;
+ case GIC_PAGE_SIZE_64K:
+ psz = SZ_64K;
+ break;
+ }
+
+ /* Don't allow vpe_id that exceeds single, flat table limit */
+ if (!(val & GICR_VPROPBASER_4_1_INDIRECT))
+ return (id < (npg * psz / (esz * SZ_8)));
+
+ /* Compute 1st level table index & check if that exceeds table limit */
+ idx = id >> ilog2(psz / (esz * SZ_8));
+ if (idx >= (npg * psz / GITS_LVL1_ENTRY_SIZE))
+ return false;
+
+ table = gic_data_rdist_cpu(cpu)->vpe_l1_base;
+
+ /* Allocate memory for 2nd level table */
+ if (!table[idx]) {
+ page = alloc_pages(GFP_KERNEL | __GFP_ZERO, get_order(psz));
+ if (!page)
+ return false;
+
+ /* Flush Lvl2 table to PoC if hw doesn't support coherency */
+ if (!(val & GICR_VPROPBASER_SHAREABILITY_MASK))
+ gic_flush_dcache_to_poc(page_address(page), psz);
+
+ table[idx] = cpu_to_le64(page_to_phys(page) | GITS_BASER_VALID);
+
+ /* Flush Lvl1 entry to PoC if hw doesn't support coherency */
+ if (!(val & GICR_VPROPBASER_SHAREABILITY_MASK))
+ gic_flush_dcache_to_poc(table + idx,
GITS_LVL1_ENTRY_SIZE);
+
+ /* Ensure updated table contents are visible to RD hardware */
+ dsb(sy);
+ }
+
+ return true;
+}
+
static int allocate_vpe_l1_table(void)
{
void __iomem *vlpi_base = gic_data_rdist_vlpi_base();
@@ -2957,6 +3023,7 @@ static bool its_alloc_device_table(struct its_node *its,
u32 dev_id)
static bool its_alloc_vpe_table(u32 vpe_id)
{
struct its_node *its;
+ int cpu;
/*
* Make sure the L2 tables are allocated on *all* v4 ITSs. We
@@ -2979,6 +3046,19 @@ static bool its_alloc_vpe_table(u32 vpe_id)
return false;
}
+ /* Non v4.1? No need to iterate RDs and go back early. */
+ if (!gic_rdists->has_rvpeid)
+ return true;
+
+ /*
+* Make sure the L2 tables are allocated for all copies of
+* the L1 table on *all* v4.1 RDs.
+*/
+ for_each_possible_cpu(cpu) {
+ if (!allocate_vpe_l2_table(cpu, vpe_id))
+ return false;
+ }
+
return true;
}
--
2.19.1
___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
