Re: [U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[York Sun]

On 12/01/2015 10:38 PM, Hou Zhiqiang-B48286 wrote:



>>  /* Invalidate all table entries */
>>  memset(level0_table, 0, PGTABLE_SIZE); @@ -269,6 +344,22 @@ static
>> inline void final_mmu_setup(void)
>> _mmu_table[i]);
>>  }
>>  }
>> +/* Set the secure memory to secure in MMU */ #ifdef
>> +CONFIG_SYS_MEM_RESERVE_SECURE
>> +if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) { #ifdef
>> +CONFIG_FSL_LSCH3
>> +level2_table_secure = level2_table1 + 512; #elif
>> +defined(CONFIG_FSL_LSCH2)
>> +level2_table_secure = level2_table2 + 512; #endif
>> +/* update tlb pointer */
>> +gd->arch.tlb_addr = gd->secure_ram & ~0x3;
> 
> The memory reserved for mmu table was lost? If it's better to record it and 
> use for non-sec mmu table?

That can be arranged.

> 
> If this func is called from non-secure state, for example EL2, the secure 
> memory cannot be accessed and the PMD_SECT_NS bit should be cleared.

We need to setup secure memory at EL3 to have secure memory to begin with. But
you have a point here. Will look into it.

York
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

Re: [U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[York Sun]

On 12/02/2015 06:48 PM, Hou Zhiqiang-B48286 wrote:
> 
> 
>> -Original Message-
>> From: York Sun [mailto:york...@freescale.com]
>> Sent: 2015年12月3日 0:00
>> To: Hou Zhiqiang-B48286; U-Boot Mailing List
>> Cc: Pan Lijun-B44306; Hu Mingkai-B21284; Sharma Bhupesh-B45370; Gong
>> Qianyu-B52263; Tom Rini; Li Yang-Leo-R58472; Albert Aribaud; Kushwaha
>> Prabhakar-B32579; Wang Huan-B18965
>> Subject: Re: [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in
>> MMU tables
>>
>>
>>
>> On 12/01/2015 10:38 PM, Hou Zhiqiang-B48286 wrote:
>>
>> 
>>
/* Invalidate all table entries */
memset(level0_table, 0, PGTABLE_SIZE); @@ -269,6 +344,22 @@ static
 inline void final_mmu_setup(void)
   _mmu_table[i]);
}
}
 +  /* Set the secure memory to secure in MMU */ #ifdef
 +CONFIG_SYS_MEM_RESERVE_SECURE
 +  if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) { #ifdef
 +CONFIG_FSL_LSCH3
 +  level2_table_secure = level2_table1 + 512; #elif
 +defined(CONFIG_FSL_LSCH2)
 +  level2_table_secure = level2_table2 + 512; #endif
 +  /* update tlb pointer */
 +  gd->arch.tlb_addr = gd->secure_ram & ~0x3;
>>>
>>> The memory reserved for mmu table was lost? If it's better to record it
>> and use for non-sec mmu table?
>>
>> That can be arranged.
>>
>>>
>>> If this func is called from non-secure state, for example EL2, the
>> secure memory cannot be accessed and the PMD_SECT_NS bit should be
>> cleared.
>>
>> We need to setup secure memory at EL3 to have secure memory to begin with.
>> But you have a point here. Will look into it.
>>
> 
> Yes, this func is called at EL3 for now, it is a assumption for this func.
> I mean we'd better make it a API that can be invoked at both EL3 and EL2 
> to setup the MMU table. And do you know if the EL1 should be taken into 
> account in u-boot?

We don't need to differentiate this function for EL2 or EL1 if we ever need to
run at. The only difference is EL3 can handle secure access. So adding a
detection of exception level before setting secure bit will suffice.

York

___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

Re: [U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[Hou Zhiqiang]

> -Original Message-
> From: York Sun [mailto:york...@freescale.com]
> Sent: 2015年12月3日 0:00
> To: Hou Zhiqiang-B48286; U-Boot Mailing List
> Cc: Pan Lijun-B44306; Hu Mingkai-B21284; Sharma Bhupesh-B45370; Gong
> Qianyu-B52263; Tom Rini; Li Yang-Leo-R58472; Albert Aribaud; Kushwaha
> Prabhakar-B32579; Wang Huan-B18965
> Subject: Re: [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in
> MMU tables
> 
> 
> 
> On 12/01/2015 10:38 PM, Hou Zhiqiang-B48286 wrote:
> 
> 
> 
> >>/* Invalidate all table entries */
> >>memset(level0_table, 0, PGTABLE_SIZE); @@ -269,6 +344,22 @@ static
> >> inline void final_mmu_setup(void)
> >>   _mmu_table[i]);
> >>}
> >>}
> >> +  /* Set the secure memory to secure in MMU */ #ifdef
> >> +CONFIG_SYS_MEM_RESERVE_SECURE
> >> +  if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) { #ifdef
> >> +CONFIG_FSL_LSCH3
> >> +  level2_table_secure = level2_table1 + 512; #elif
> >> +defined(CONFIG_FSL_LSCH2)
> >> +  level2_table_secure = level2_table2 + 512; #endif
> >> +  /* update tlb pointer */
> >> +  gd->arch.tlb_addr = gd->secure_ram & ~0x3;
> >
> > The memory reserved for mmu table was lost? If it's better to record it
> and use for non-sec mmu table?
> 
> That can be arranged.
> 
> >
> > If this func is called from non-secure state, for example EL2, the
> secure memory cannot be accessed and the PMD_SECT_NS bit should be
> cleared.
> 
> We need to setup secure memory at EL3 to have secure memory to begin with.
> But you have a point here. Will look into it.
> 

Yes, this func is called at EL3 for now, it is a assumption for this func.
I mean we'd better make it a API that can be invoked at both EL3 and EL2 
to setup the MMU table. And do you know if the EL1 should be taken into 
account in u-boot?

Thanks,
Zhiqiang
___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

Re: [U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[Hou Zhiqiang]

> -Original Message-
> From: York Sun [mailto:york...@freescale.com]
> Sent: 2015年12月3日 11:05
> To: Hou Zhiqiang-B48286; U-Boot Mailing List
> Cc: Pan Lijun-B44306; Hu Mingkai-B21284; Sharma Bhupesh-B45370; Gong
> Qianyu-B52263; Tom Rini; Li Yang-Leo-R58472; Albert Aribaud; Kushwaha
> Prabhakar-B32579; Wang Huan-B18965
> Subject: Re: [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in
> MMU tables
> 
> 
> 
> On 12/02/2015 06:48 PM, Hou Zhiqiang-B48286 wrote:
> >
> >
> >> -Original Message-
> >> From: York Sun [mailto:york...@freescale.com]
> >> Sent: 2015年12月3日 0:00
> >> To: Hou Zhiqiang-B48286; U-Boot Mailing List
> >> Cc: Pan Lijun-B44306; Hu Mingkai-B21284; Sharma Bhupesh-B45370; Gong
> >> Qianyu-B52263; Tom Rini; Li Yang-Leo-R58472; Albert Aribaud; Kushwaha
> >> Prabhakar-B32579; Wang Huan-B18965
> >> Subject: Re: [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non
> >> secure in MMU tables
> >>
> >>
> >>
> >> On 12/01/2015 10:38 PM, Hou Zhiqiang-B48286 wrote:
> >>
> >> 
> >>
>   /* Invalidate all table entries */
>   memset(level0_table, 0, PGTABLE_SIZE); @@ -269,6 +344,22 @@
>  static inline void final_mmu_setup(void)
>  _mmu_table[i]);
>   }
>   }
>  +/* Set the secure memory to secure in MMU */ #ifdef
>  +CONFIG_SYS_MEM_RESERVE_SECURE
>  +if (gd->secure_ram & MEM_RESERVE_SECURE_MAINTAINED) { #ifdef
>  +CONFIG_FSL_LSCH3
>  +level2_table_secure = level2_table1 + 512; #elif
>  +defined(CONFIG_FSL_LSCH2)
>  +level2_table_secure = level2_table2 + 512; #endif
>  +/* update tlb pointer */
>  +gd->arch.tlb_addr = gd->secure_ram & ~0x3;
> >>>
> >>> The memory reserved for mmu table was lost? If it's better to record
> >>> it
> >> and use for non-sec mmu table?
> >>
> >> That can be arranged.
> >>
> >>>
> >>> If this func is called from non-secure state, for example EL2, the
> >> secure memory cannot be accessed and the PMD_SECT_NS bit should be
> >> cleared.
> >>
> >> We need to setup secure memory at EL3 to have secure memory to begin
> with.
> >> But you have a point here. Will look into it.
> >>
> >
> > Yes, this func is called at EL3 for now, it is a assumption for this
> func.
> > I mean we'd better make it a API that can be invoked at both EL3 and
> > EL2 to setup the MMU table. And do you know if the EL1 should be taken
> > into account in u-boot?
> 
> We don't need to differentiate this function for EL2 or EL1 if we ever
> need to run at. The only difference is EL3 can handle secure access. So
> adding a detection of exception level before setting secure bit will
> suffice.

ok.

Thanks,
Zhiqiang

___
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

Re: [U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[Hou Zhiqiang]

Hi York,

> -Original Message-
> From: York Sun [mailto:york...@freescale.com]
> Sent: 2015年11月19日 2:02
> To: U-Boot Mailing List
> Cc: Sun York-R58495; Pan Lijun-B44306; Hu Mingkai-B21284; Sharma Bhupesh-
> B45370; Gong Qianyu-B52263; Tom Rini; Li Yang-Leo-R58472; Albert Aribaud;
> Hou Zhiqiang-B48286; Kushwaha Prabhakar-B32579; Wang Huan-B18965
> Subject: [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU
> tables
> 
> DDR has been set as secure in MMU tables. Non-secure master such as SDHC
> DMA cannot access data correctly. Mixing secure and non- secure MMU
> entries requirs the MMU tables themselves in secure memory. This patch
> moves MMU tables into a secure DDR area.
> 
> Early MMU tables are changed to set DDR as non-secure. A new table is
> added into final MMU tables so secure memory can have 2MB granuality.
> 
> gd->secure_ram tracks the location of this secure memory. For
> ARMv8 SoCs, the RAM base is not zero and RAM is divided into several
> banks. gd->secure_ram needs to be maintained before using. This
> maintenance is board-specific, depending on the SoC and memory bank of
> the secure memory falls into.
> 
> Signed-off-by: York Sun 
> 
> fixup
> ---
> 
> Changes in v6:
>   Move cmd_bdinfo change to 1st patch in this set
>   Rearrange #ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> 
> Changes in v5:
>   Put ifdef where gd->secure_ram is used
> 
> Changes in v4:
>   Drop RFC from patch prefix
>   Drop excessive mmu table for secure ram for early MMU
>   Update commit message accordingly
>   Mark QBMan cacheable portal memory non-secure
> 
> Changes in v3:
>   Replace CONFIG_FSL_PPA_RESERVED_DRAM_SIZE with
> CONFIG_SYS_MEM_RESERVE_SECURE
>   Sanity check gd->secure_ram before using
>   Define CONFIG_SYS_MEM_RESERVE_SECURE in SoC header file
>   Include ls1043ardb
>   Modified commit message.
> 
> Changes in v2:
>   Move gd->arch.secure_ram to gd->secure_ram.
>   Change the calculation of gd->secure_ram accordingly.
>   Chnage commit message slightly accordingly.
> 
> Changes in v1: None
> 
>  arch/arm/cpu/armv8/fsl-layerscape/cpu.c   |  111
> +++--
>  arch/arm/include/asm/arch-fsl-layerscape/config.h |6 ++
>  arch/arm/include/asm/arch-fsl-layerscape/cpu.h|   14 ++-
>  board/freescale/ls1043ardb/ddr.c  |4 +
>  board/freescale/ls2085a/ddr.c |   15 +++
>  board/freescale/ls2085aqds/ddr.c  |   15 +++
>  board/freescale/ls2085ardb/ddr.c  |   15 +++
>  7 files changed, 165 insertions(+), 15 deletions(-)
> 
> diff --git a/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> b/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> index 9d1c70f..501feb3 100644
> --- a/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> +++ b/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
> @@ -206,11 +206,65 @@ static inline void early_mmu_setup(void)
>   set_sctlr(get_sctlr() | CR_M);
>  }
> 
> +#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
> +/*
> + * Called from final mmu setup. The phys_addr is new, non-existing
> + * address. A new sub table is created @level2_table_secure to cover
> + * size of CONFIG_SYS_MEM_RESERVE_SECURE memory.
> + */
> +static inline int final_secure_ddr(u64 *level0_table,
> +u64 *level2_table_secure,
> +phys_addr_t phys_addr)
> +{
> + int ret = -EINVAL;
> + struct table_info table = {};
> + struct sys_mmu_table ddr_entry = {
> + 0, 0, BLOCK_SIZE_L1, MT_NORMAL,
> + PMD_SECT_OUTER_SHARE | PMD_SECT_NS
> + };
> + u64 index;
> +
> + /* Need to create a new table */
> + ddr_entry.virt_addr = phys_addr & ~(BLOCK_SIZE_L1 - 1);
> + ddr_entry.phys_addr = phys_addr & ~(BLOCK_SIZE_L1 - 1);
> + ret = find_table(_entry, , level0_table);
> + if (ret)
> + return ret;
> + index = (ddr_entry.virt_addr - table.table_base) >>
> SECTION_SHIFT_L1;
> + set_pgtable_table(table.ptr, index, level2_table_secure);
> + table.ptr = level2_table_secure;
> + table.table_base = ddr_entry.virt_addr;
> + table.entry_size = BLOCK_SIZE_L2;
> + ret = set_block_entry(_entry, );
> + if (ret) {
> + printf("MMU error: could not fill non-secure ddr block
> entries\n");
> + return ret;
> + }
> + ddr_entry.virt_addr = phys_addr;
> + ddr_entry.phys_addr = phys_addr;
> + ddr_entry.size = CONFIG_SYS_MEM_RESERVE_SECURE;
> + ddr_entry.attribute = PMD_SECT_OUTER_SHARE;
> + ret = find_table(_entry, , level0_table);
> + if (ret) {
> + printf("MMU error: could not find secure ddr table\n");
> + return ret;
> + }
> + ret = set_block_entry(_entry, );
> + if (ret)
> + printf("MMU error: could not set secure ddr block entry\n");
> +
> + return ret;
> +}
> +#endif
> +
>  /*
>   * The final tables look similar to early tables, but different in
> detail.
>   * These tables are in DRAM. 

[U-Boot] [PATCH v6 2/2] armv8: fsl-layerscape: Make DDR non secure in MMU tables

[York Sun]

DDR has been set as secure in MMU tables. Non-secure master such
as SDHC DMA cannot access data correctly. Mixing secure and non-
secure MMU entries requirs the MMU tables themselves in secure
memory. This patch moves MMU tables into a secure DDR area.

Early MMU tables are changed to set DDR as non-secure. A new
table is added into final MMU tables so secure memory can have
2MB granuality.

gd->secure_ram tracks the location of this secure memory. For
ARMv8 SoCs, the RAM base is not zero and RAM is divided into several
banks. gd->secure_ram needs to be maintained before using. This
maintenance is board-specific, depending on the SoC and memory
bank of the secure memory falls into.

Signed-off-by: York Sun 

fixup
---

Changes in v6:
  Move cmd_bdinfo change to 1st patch in this set
  Rearrange #ifdef CONFIG_SYS_MEM_RESERVE_SECURE

Changes in v5:
  Put ifdef where gd->secure_ram is used

Changes in v4:
  Drop RFC from patch prefix
  Drop excessive mmu table for secure ram for early MMU
  Update commit message accordingly
  Mark QBMan cacheable portal memory non-secure

Changes in v3:
  Replace CONFIG_FSL_PPA_RESERVED_DRAM_SIZE with CONFIG_SYS_MEM_RESERVE_SECURE
  Sanity check gd->secure_ram before using
  Define CONFIG_SYS_MEM_RESERVE_SECURE in SoC header file
  Include ls1043ardb
  Modified commit message.

Changes in v2:
  Move gd->arch.secure_ram to gd->secure_ram.
  Change the calculation of gd->secure_ram accordingly.
  Chnage commit message slightly accordingly.

Changes in v1: None

 arch/arm/cpu/armv8/fsl-layerscape/cpu.c   |  111 +++--
 arch/arm/include/asm/arch-fsl-layerscape/config.h |6 ++
 arch/arm/include/asm/arch-fsl-layerscape/cpu.h|   14 ++-
 board/freescale/ls1043ardb/ddr.c  |4 +
 board/freescale/ls2085a/ddr.c |   15 +++
 board/freescale/ls2085aqds/ddr.c  |   15 +++
 board/freescale/ls2085ardb/ddr.c  |   15 +++
 7 files changed, 165 insertions(+), 15 deletions(-)

diff --git a/arch/arm/cpu/armv8/fsl-layerscape/cpu.c 
b/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
index 9d1c70f..501feb3 100644
--- a/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
+++ b/arch/arm/cpu/armv8/fsl-layerscape/cpu.c
@@ -206,11 +206,65 @@ static inline void early_mmu_setup(void)
set_sctlr(get_sctlr() | CR_M);
 }
 
+#ifdef CONFIG_SYS_MEM_RESERVE_SECURE
+/*
+ * Called from final mmu setup. The phys_addr is new, non-existing
+ * address. A new sub table is created @level2_table_secure to cover
+ * size of CONFIG_SYS_MEM_RESERVE_SECURE memory.
+ */
+static inline int final_secure_ddr(u64 *level0_table,
+  u64 *level2_table_secure,
+  phys_addr_t phys_addr)
+{
+   int ret = -EINVAL;
+   struct table_info table = {};
+   struct sys_mmu_table ddr_entry = {
+   0, 0, BLOCK_SIZE_L1, MT_NORMAL,
+   PMD_SECT_OUTER_SHARE | PMD_SECT_NS
+   };
+   u64 index;
+
+   /* Need to create a new table */
+   ddr_entry.virt_addr = phys_addr & ~(BLOCK_SIZE_L1 - 1);
+   ddr_entry.phys_addr = phys_addr & ~(BLOCK_SIZE_L1 - 1);
+   ret = find_table(_entry, , level0_table);
+   if (ret)
+   return ret;
+   index = (ddr_entry.virt_addr - table.table_base) >> SECTION_SHIFT_L1;
+   set_pgtable_table(table.ptr, index, level2_table_secure);
+   table.ptr = level2_table_secure;
+   table.table_base = ddr_entry.virt_addr;
+   table.entry_size = BLOCK_SIZE_L2;
+   ret = set_block_entry(_entry, );
+   if (ret) {
+   printf("MMU error: could not fill non-secure ddr block 
entries\n");
+   return ret;
+   }
+   ddr_entry.virt_addr = phys_addr;
+   ddr_entry.phys_addr = phys_addr;
+   ddr_entry.size = CONFIG_SYS_MEM_RESERVE_SECURE;
+   ddr_entry.attribute = PMD_SECT_OUTER_SHARE;
+   ret = find_table(_entry, , level0_table);
+   if (ret) {
+   printf("MMU error: could not find secure ddr table\n");
+   return ret;
+   }
+   ret = set_block_entry(_entry, );
+   if (ret)
+   printf("MMU error: could not set secure ddr block entry\n");
+
+   return ret;
+}
+#endif
+
 /*
  * The final tables look similar to early tables, but different in detail.
  * These tables are in DRAM. Sub tables are added to enable cache for
  * QBMan and OCRAM.
  *
+ * Put the MMU table in secure memory if gd->secure_ram is valid.
+ * OCRAM will be not used for this purpose so gd->secure_ram can't be 0.
+ *
  * Level 1 table 0 contains 512 entries for each 1GB from 0 to 512GB.
  * Level 1 table 1 contains 512 entries for each 1GB from 512GB to 1TB.
  * Level 2 table 0 contains 512 entries for each 2MB from 0 to 1GB.
@@ -224,17 +278,38 @@ static inline void early_mmu_setup(void)
 static inline void final_mmu_setup(void)
 {
unsigned int el, i;
-   u64 *level0_table = (u64