Re: [edk2-devel] Clarification of Memory management in PEI phase

[vincent zimmer]

sounds like a good idea.  As a next step, perhaps a candidate activity
for 
https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Code-First-Process
in anticipation of future inclusion in PI?
Vincent

On Wed, Jun 22, 2022 at 2:41 PM Brian J. Johnson  wrote:
>
> Andrew,
>
> Yes, adding a new HOB type to represent free pool would probably be the 
> easiest.
>
> Or we could write or borrow a traditional malloc() implementation, similar to 
> DXE's pool allocator, and back it with memory from AllocatePages().  That 
> would probably have better performance, and would avoid fragmenting the pool 
> memory when non-pool HOBs are added.  It would probably be more code, though.
>
> Brian J. Johnson
> 
> From: Andrew Fish [mailto:af...@apple.com]
> Sent: Wednesday, June 22, 2022, 3:54 PM
> To: Brian J. Johnson 
> Cc: devel@edk2.groups.io, ayushdevel1...@gmail.com
> Subject: [edk2-devel] Clarification of Memory management in PEI phase
>
> Brian,
>
> I think all the PEI Allocate Pool does is make a HOB [1]. I don’t think we 
> can remove HOBs when memory is freed as that would change  pointers to pool. 
> It may be possible to mark a region as free and allocate from that list batch 
> 1st, and just over allocate space if needed. It could be as simple as just 
> adding a new HOB type to represent free pool.
>
>
> [1] 
> https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878
>
> Thanks,
>
> Andrew Fish
>
> On Jun 22, 2022, at 12:39 PM, Brian J. Johnson 
> Sorry for the late response to this thread...
>
> PEI has grown greatly in complexity, as Andrew pointed out.  Do we (TianoCore 
> community) think it's time to add a real pool manager to PEI?  There's 
> getting to be quite a bit of post-DRAM-initialization PEI code on some 
> platforms.  And really, having a limited amount of pre-DRAM memory available 
> is an even better reason for having an effective pool allocator, so the 
> memory can be freed and reused.
>
> FWIW we (HPE) needed to add a private pool allocator to manage memory for a 
> proprietary h/w initialization module which needs to run in post-DRAM PEI, 
> and depends on allocating and freeing memory in different phases of its 
> operation.
>
> Brian J. Johnson
>
> ____
> From: Ayush Singh [mailto:ayushdevel1...@gmail.com]
> Sent: Friday, June 10, 2022, 12:22 AM
> To: Andrew Fish 
> Cc: devel@edk2.groups.io
> Subject: [edk2-devel] Clarification of Memory management in PEI phase
>
> Thanks for the wonderful answer.
>
> Ayush Singh
>
> On Thu, Jun 9 2022 at 01:26:58 PM -0700, Andrew Fish  wrote:
>
> On Jun 9, 2022, at 10:28 AM, Ayush Singh  Hello everyone, Can anyone help me with understanding dynamic memory 
> management in PEI phase? In the UEFI Platform Integration Specification, 
> version 1.7 Errata A, Section 4.6, PEI Memory services are given which 
> include: 1. InstallPeiMemory()
>
> This is basically: (*PeiServices)->InstallPeiMemory (PeiServices, 
> MemoryBegin, MemoryLength); This is how you tell the PEI Core the location of 
> the memory that will can be used in PEI.
>
> 2. AllocatePages() 3. AllocatePool() 4. CopyMem() 5. SetMem() 6. FreePages() 
> However, no `FreePool()` service seems to be present. So how is the memory 
> allocated using `AllocatePool()` freed?
>
> It basically gets Freed when you transition to the DXE phase. To step back 
> for a minute I think it is important to remember that the main job of PEI is 
> to initialize DRAM, and deal with S3 (resuming from suspend to RAM). So as 
> soon as you have DRAM you are kind done and ready for the DXE IPL so you can 
> load the DXE Phase and start up EFI. Remember PEI is Pre EFI. The reality is 
> programming DRAM is complex and lots of code got written, then lots more code 
> got written and PEI has become large for some ports. That was never the 
> intent. PEI is designed as a way to run C code when you code is running from 
> ROM and you donā��t have any DRAM. For x86 not having DRAM means you are 
> using the cache as RAM. For some SoCs there is actually an SRAM you can use. 
> Thus the PEI memory allocation scheme is designed to deal with this very 
> constrained environment. You start PEI with a heap and stack. You can also 
> allocate HOBs (Hand Off Blocks). A pool allocation in PEI is just a HOB. See 
> [1]. There is no way to free a HOB. So the AllocatePool() kind of leaks into 
> DXE too as an entry in the HOB list. But when the OS called 
> gBS->ExitBootServices() that frees all non runtime memory back to the OS. If 
> you look a AllocatePages/FreePages you will see AllocatePages creates a HOB 
> that points to the mem

Re: [edk2-devel] Clarification of Memory management in PEI phase

[Ayush Singh]

Hello everyone, Thanks for the helpful answers.

After discussion with the mentors, it was decided that we should first
get the Rust std to work in DXE phase. So for now, that is the goal.
Once DXE is complete and there is still time left (or I can just do it
after GSoC as well), then I might look into implementing std for the
PEI.

Yours Sincerely
Ayush Singh

On Thu, Jun 23, 2022 at 4:28 AM Nate DeSimone
 wrote:
>
> Hi Ayush,
>
>
>
> For your work to make Rust run in PEI I would recommend writing a generic 
> heap manager that uses the PEI services AllocatePage() and FreePage(). PEI 
> does allow you to truly free up memory but only when allocated in 4KB 
> increments. Your heap manager can allow the Rust program to go to a smaller 
> granularity.
>
>
>
> In parallel, I can see merit in the argument for adding proper heap 
> management to PEI, but that would be a PI specification update that is way 
> outside the scope of your GSoC project and won’t happen fast enough for the 
> work you need to do this summer .
>
>
>
> Thanks,
>
> Nate
>
>
>
> From: devel@edk2.groups.io  On Behalf Of Ayush Singh
> Sent: Thursday, June 9, 2022 10:23 PM
> To: Andrew Fish 
> Cc: devel@edk2.groups.io
> Subject: Re: [edk2-devel] Clarification of Memory management in PEI phase
>
>
>
> Thanks for the wonderful answer.
>
>
>
> Ayush Singh
>
>
> On Thu, Jun 9 2022 at 01:26:58 PM -0700, Andrew Fish  wrote:
>
> On Jun 9, 2022, at 10:28 AM, Ayush Singh  wrote: 
> Hello everyone, Can anyone help me with understanding dynamic memory 
> management in PEI phase? In the UEFI Platform Integration Specification, 
> version 1.7 Errata A, Section 4.6, PEI Memory services are given which 
> include: 1. InstallPeiMemory()
>
> This is basically: (*PeiServices)->InstallPeiMemory (PeiServices, 
> MemoryBegin, MemoryLength); This is how you tell the PEI Core the location of 
> the memory that will can be used in PEI.
>
> 2. AllocatePages() 3. AllocatePool() 4. CopyMem() 5. SetMem() 6. FreePages() 
> However, no `FreePool()` service seems to be present. So how is the memory 
> allocated using `AllocatePool()` freed?
>
> It basically gets Freed when you transition to the DXE phase. To step back 
> for a minute I think it is important to remember that the main job of PEI is 
> to initialize DRAM, and deal with S3 (resuming from suspend to RAM). So as 
> soon as you have DRAM you are kind done and ready for the DXE IPL so you can 
> load the DXE Phase and start up EFI. Remember PEI is Pre EFI. The reality is 
> programming DRAM is complex and lots of code got written, then lots more code 
> got written and PEI has become large for some ports. That was never the 
> intent. PEI is designed as a way to run C code when you code is running from 
> ROM and you donā��t have any DRAM. For x86 not having DRAM means you are 
> using the cache as RAM. For some SoCs there is actually an SRAM you can use. 
> Thus the PEI memory allocation scheme is designed to deal with this very 
> constrained environment. You start PEI with a heap and stack. You can also 
> allocate HOBs (Hand Off Blocks). A pool allocation in PEI is just a HOB. See 
> [1]. There is no way to free a HOB. So the AllocatePool() kind of leaks into 
> DXE too as an entry in the HOB list. But when the OS called 
> gBS->ExitBootServices() that frees all non runtime memory back to the OS. If 
> you look a AllocatePages/FreePages you will see AllocatePages creates a HOB 
> that points to the memory region, and FreePages just marks that HOB as not 
> used. That code is also in this file [1]. TL;DR there is no pool manager in 
> PEI. [1] 
> https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878
>  Thanks, Andrew Fish
>
> Ayush Singh
>
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#90710): https://edk2.groups.io/g/devel/message/90710
Mute This Topic: https://groups.io/mt/91651322/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Clarification of Memory management in PEI phase

[Michael Kubacki]

I agree with other comments here. I mainly want to add that if you're 
new to PI/UEFI, you might find the following resource helpful for tying 
together some pieces from the code and documents like the PI Spec.


https://github.com/tianocore-docs/Docs/raw/master/White_Papers/A_Tour_Beyond_BIOS_Memory_Map_And_Practices_in_UEFI_BIOS_V2.pdf

Thanks,
Michael

On 6/22/2022 6:58 PM, Nate DeSimone wrote:

Hi Ayush,

For your work to make Rust run in PEI I would recommend writing a 
generic heap manager that uses the PEI services AllocatePage() and 
FreePage(). PEI does allow you to truly free up memory but only when 
allocated in 4KB increments. Your heap manager can allow the Rust 
program to go to a smaller granularity.


In parallel, I can see merit in the argument for adding proper heap 
management to PEI, but that would be a PI specification update that is 
way outside the scope of your GSoC project and won’t happen fast enough 
for the work you need to do this summer .


Thanks,

Nate

*From:* devel@edk2.groups.io  *On Behalf Of *Ayush 
Singh

*Sent:* Thursday, June 9, 2022 10:23 PM
*To:* Andrew Fish 
*Cc:* devel@edk2.groups.io
*Subject:* Re: [edk2-devel] Clarification of Memory management in PEI phase

Thanks for the wonderful answer.

Ayush Singh


On Thu, Jun 9 2022 at 01:26:58 PM -0700, Andrew Fish <mailto:af...@apple.com>> wrote:


On Jun 9, 2022, at 10:28 AM, Ayush Singh
mailto:ayushdevel1...@gmail.com>>
wrote: Hello everyone, Can anyone help me with understanding
dynamic memory management in PEI phase? In the UEFI Platform
Integration Specification, version 1.7 Errata A, Section 4.6,
PEI Memory services are given which include: 1. InstallPeiMemory()

This is basically: (*PeiServices)->InstallPeiMemory (PeiServices,
MemoryBegin, MemoryLength); This is how you tell the PEI Core the
location of the memory that will can be used in PEI.

2. AllocatePages() 3. AllocatePool() 4. CopyMem() 5. SetMem() 6.
FreePages() However, no `FreePool()` service seems to be
present. So how is the memory allocated using `AllocatePool()`
freed?

It basically gets Freed when you transition to the DXE phase. To
step back for a minute I think it is important to remember that the
main job of PEI is to initialize DRAM, and deal with S3 (resuming
from suspend to RAM). So as soon as you have DRAM you are kind done
and ready for the DXE IPL so you can load the DXE Phase and start up
EFI. Remember PEI is Pre EFI. The reality is programming DRAM is
complex and lots of code got written, then lots more code got
written and PEI has become large for some ports. That was never the
intent. PEI is designed as a way to run C code when you code is
running from ROM and you donā��t have any DRAM. For x86 not having
DRAM means you are using the cache as RAM. For some SoCs there is
actually an SRAM you can use. Thus the PEI memory allocation scheme
is designed to deal with this very constrained environment. You
start PEI with a heap and stack. You can also allocate HOBs (Hand
Off Blocks). A pool allocation in PEI is just a HOB. See [1]. There
is no way to free a HOB. So the AllocatePool() kind of leaks into
DXE too as an entry in the HOB list. But when the OS called
gBS->ExitBootServices() that frees all non runtime memory back to
the OS. If you look a AllocatePages/FreePages you will see
AllocatePages creates a HOB that points to the memory region, and
FreePages just marks that HOB as not used. That code is also in this
file [1]. TL;DR there is no pool manager in PEI. [1]

https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878

<https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878>
Thanks, Andrew Fish

Ayush Singh





-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#90703): https://edk2.groups.io/g/devel/message/90703
Mute This Topic: https://groups.io/mt/91651322/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Clarification of Memory management in PEI phase

[Nate DeSimone]

Hi Ayush,

For your work to make Rust run in PEI I would recommend writing a generic heap 
manager that uses the PEI services AllocatePage() and FreePage(). PEI does 
allow you to truly free up memory but only when allocated in 4KB increments. 
Your heap manager can allow the Rust program to go to a smaller granularity.

In parallel, I can see merit in the argument for adding proper heap management 
to PEI, but that would be a PI specification update that is way outside the 
scope of your GSoC project and won’t happen fast enough for the work you need 
to do this summer .

Thanks,
Nate

From: devel@edk2.groups.io  On Behalf Of Ayush Singh
Sent: Thursday, June 9, 2022 10:23 PM
To: Andrew Fish 
Cc: devel@edk2.groups.io
Subject: Re: [edk2-devel] Clarification of Memory management in PEI phase

Thanks for the wonderful answer.

Ayush Singh

On Thu, Jun 9 2022 at 01:26:58 PM -0700, Andrew Fish 
mailto:af...@apple.com>> wrote:

On Jun 9, 2022, at 10:28 AM, Ayush Singh 
mailto:ayushdevel1...@gmail.com>> wrote: Hello 
everyone, Can anyone help me with understanding dynamic memory management in 
PEI phase? In the UEFI Platform Integration Specification, version 1.7 Errata 
A, Section 4.6, PEI Memory services are given which include: 1. 
InstallPeiMemory()
This is basically: (*PeiServices)->InstallPeiMemory (PeiServices, MemoryBegin, 
MemoryLength); This is how you tell the PEI Core the location of the memory 
that will can be used in PEI.
2. AllocatePages() 3. AllocatePool() 4. CopyMem() 5. SetMem() 6. FreePages() 
However, no `FreePool()` service seems to be present. So how is the memory 
allocated using `AllocatePool()` freed?
It basically gets Freed when you transition to the DXE phase. To step back for 
a minute I think it is important to remember that the main job of PEI is to 
initialize DRAM, and deal with S3 (resuming from suspend to RAM). So as soon as 
you have DRAM you are kind done and ready for the DXE IPL so you can load the 
DXE Phase and start up EFI. Remember PEI is Pre EFI. The reality is programming 
DRAM is complex and lots of code got written, then lots more code got written 
and PEI has become large for some ports. That was never the intent. PEI is 
designed as a way to run C code when you code is running from ROM and you 
donā��t have any DRAM. For x86 not having DRAM means you are using the cache as 
RAM. For some SoCs there is actually an SRAM you can use. Thus the PEI memory 
allocation scheme is designed to deal with this very constrained environment. 
You start PEI with a heap and stack. You can also allocate HOBs (Hand Off 
Blocks). A pool allocation in PEI is just a HOB. See [1]. There is no way to 
free a HOB. So the AllocatePool() kind of leaks into DXE too as an entry in the 
HOB list. But when the OS called gBS->ExitBootServices() that frees all non 
runtime memory back to the OS. If you look a AllocatePages/FreePages you will 
see AllocatePages creates a HOB that points to the memory region, and FreePages 
just marks that HOB as not used. That code is also in this file [1]. TL;DR 
there is no pool manager in PEI. [1] 
https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878
 Thanks, Andrew Fish
Ayush Singh



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#90702): https://edk2.groups.io/g/devel/message/90702
Mute This Topic: https://groups.io/mt/91651322/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Clarification of Memory management in PEI phase

[Brian J. Johnson]

Andrew,

Yes, adding a new HOB type to represent free pool would probably be the 
easiest.


Or we could write or borrow a traditional malloc() implementation, 
similar to DXE's pool allocator, and back it with memory from 
AllocatePages().  That would probably have better performance, and would 
avoid fragmenting the pool memory when non-pool HOBs are added.  It 
would probably be more code, though.


Brian J. Johnson

*From:* Andrew Fish [mailto:af...@apple.com]
*Sent:* Wednesday, June 22, 2022, 3:54 PM
*To:* Brian J. Johnson 
*Cc:* devel@edk2.groups.io, ayushdevel1...@gmail.com
*Subject:* [edk2-devel] Clarification of Memory management in PEI phase


Brian,

I think all the PEI Allocate Pool does is make a HOB [1]. I don’t 
think we can remove HOBs when memory is freed as that would change 
 pointers to pool. It may be possible to mark a region as free and 
allocate from that list batch 1st, and just over allocate space if 
needed. It could be as simple as just adding a new HOB type to 
represent free pool.



[1] 
https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878


Thanks,

Andrew Fish

On Jun 22, 2022, at 12:39 PM, Brian J. Johnson 


Sorry for the late response to this thread...

PEI has grown greatly in complexity, as Andrew pointed out.  Do we 
(TianoCore community) think it's time to add a real pool manager to 
PEI? There's getting to be quite a bit of post-DRAM-initialization 
PEI code on some platforms.  And really, having a limited amount of 
pre-DRAM memory available is an even better reason for having an 
effective pool allocator, so the memory can be freed and reused.


FWIW we (HPE) needed to add a private pool allocator to manage memory 
for a proprietary h/w initialization module which needs to run in 
post-DRAM PEI, and depends on allocating and freeing memory in 
different phases of its operation.


Brian J. Johnson


*From:* Ayush Singh [mailto:ayushdevel1...@gmail.com]
*Sent:* Friday, June 10, 2022, 12:22 AM
*To:* Andrew Fish 
*Cc:* devel@edk2.groups.io
*Subject:* [edk2-devel] Clarification of Memory management in PEI phase


Thanks for the wonderful answer.

Ayush Singh

On Thu, Jun 9 2022 at 01:26:58 PM -0700, Andrew Fish 
 wrote:


On Jun 9, 2022, at 10:28 AM, Ayush Singh
include: 1. InstallPeiMemory() 

This is basically: (*PeiServices)->InstallPeiMemory (PeiServices, 
MemoryBegin, MemoryLength); This is how you tell the PEI Core the 
location of the memory that will can be used in PEI.


2. AllocatePages() 3. AllocatePool() 4. CopyMem() 5. SetMem()
6. FreePages() However, no `FreePool()` service seems to be
present. So how is the memory allocated using `AllocatePool()`
freed? 

It basically gets Freed when you transition to the DXE phase. To 
step back for a minute I think it is important to remember that the 
main job of PEI is to initialize DRAM, and deal with S3 (resuming 
from suspend to RAM). So as soon as you have DRAM you are kind done 
and ready for the DXE IPL so you can load the DXE Phase and start 
up EFI. Remember PEI is Pre EFI. The reality is programming DRAM is 
complex and lots of code got written, then lots more code got 
written and PEI has become large for some ports. That was never the 
intent. PEI is designed as a way to run C code when you code is 
running from ROM and you donā��t have any DRAM. For x86 not having 
DRAM means you are using the cache as RAM. For some SoCs there is 
actually an SRAM you can use. Thus the PEI memory allocation scheme 
is designed to deal with this very constrained environment. You 
start PEI with a heap and stack. You can also allocate HOBs (Hand 
Off Blocks). A pool allocation in PEI is just a HOB. See [1]. There 
is no way to free a HOB. So the AllocatePool() kind of leaks into 
DXE too as an entry in the HOB list. But when the OS called 
gBS->ExitBootServices() that frees all non runtime memory back to 
the OS. If you look a AllocatePages/FreePages you will see 
AllocatePages creates a HOB that points to the memory region, and 
FreePages just marks that HOB as not used. That code is also in 
this file [1]. TL;DR there is no pool manager in PEI. [1] 
https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878 
Thanks, Andrew Fish


Ayush Singh 






Brian

--

   "There is the greatest practical benefit in making a few failures
early in life."
   -- Thomas Henry Huxley



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#90700): https://edk2.groups.io/g/devel/message/90700
Mute This Topic: https://groups.io/mt/91651322/21656
Group

Re: [edk2-devel] Clarification of Memory management in PEI phase

[Andrew Fish via groups.io]

Brian,

I think all the PEI Allocate Pool does is make a HOB [1]. I don’t think we can 
remove HOBs when memory is freed as that would change  pointers to pool. It may 
be possible to mark a region as free and allocate from that list batch 1st, and 
just over allocate space if needed. It could be as simple as just adding a new 
HOB type to represent free pool. 


[1] 
https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878

Thanks,

Andrew Fish

> On Jun 22, 2022, at 12:39 PM, Brian J. Johnson  wrote:
> 
> Sorry for the late response to this thread...
> 
> PEI has grown greatly in complexity, as Andrew pointed out.  Do we (TianoCore 
> community) think it's time to add a real pool manager to PEI?  There's 
> getting to be quite a bit of post-DRAM-initialization PEI code on some 
> platforms.  And really, having a limited amount of pre-DRAM memory available 
> is an even better reason for having an effective pool allocator, so the 
> memory can be freed and reused.
> 
> FWIW we (HPE) needed to add a private pool allocator to manage memory for a 
> proprietary h/w initialization module which needs to run in post-DRAM PEI, 
> and depends on allocating and freeing memory in different phases of its 
> operation.
> 
> Brian J. Johnson
> 
> From: Ayush Singh [mailto:ayushdevel1...@gmail.com 
> <mailto:ayushdevel1...@gmail.com>]
> Sent: Friday, June 10, 2022, 12:22 AM
> To: Andrew Fish  <mailto:af...@apple.com>
> Cc: devel@edk2.groups.io <mailto:devel@edk2.groups.io>
> Subject: [edk2-devel] Clarification of Memory management in PEI phase
> 
>> Thanks for the wonderful answer. 
>> 
>> Ayush Singh
>> 
>> On Thu, Jun 9 2022 at 01:26:58 PM -0700, Andrew Fish  
>> <mailto:af...@apple.com> wrote:
>>> 
>>>  On Jun 9, 2022, at 10:28 AM, Ayush Singh >> <mailto:ayushdevel1...@gmail.com>wrote:
>>>  
>>>  Hello everyone,
>>>  
>>>  Can anyone help me with understanding dynamic memory management in PEI
>>>  phase? In the UEFI Platform Integration Specification, version 1.7
>>>  Errata A, Section 4.6, PEI Memory services are given which include:
>>>  
>>>  1. InstallPeiMemory()
>>> 
>>> This is basically:
>>> (*PeiServices)->InstallPeiMemory (PeiServices, MemoryBegin, MemoryLength);
>>> 
>>> This is how you tell the PEI Core the location of the memory that will can 
>>> be used in PEI. 
>>> 
>>>  2. AllocatePages()
>>>  3. AllocatePool()
>>>  4. CopyMem()
>>>  5. SetMem()
>>>  6. FreePages()
>>>  
>>>  However, no `FreePool()` service seems to be present. So how is the
>>>  memory allocated using `AllocatePool()` freed?
>>>  
>>> 
>>> It basically gets Freed when you transition to the DXE phase. 
>>> 
>>> To step back for a minute I think it is important to remember that the main 
>>> job of PEI is to initialize DRAM, and deal with S3 (resuming from suspend 
>>> to RAM). So as soon as you have DRAM you are kind done and ready for the 
>>> DXE IPL so you can load the DXE Phase and start up EFI. Remember PEI is Pre 
>>> EFI. The reality is programming DRAM is complex and lots of code got 
>>> written, then lots more code got written and PEI has become large for some 
>>> ports. That was never the intent. PEI is designed as a way to run C code 
>>> when you code is running from ROM and you donā��t have any DRAM. For x86 
>>> not having DRAM means you are using the cache as RAM. For some SoCs there 
>>> is actually an SRAM you can use. Thus the PEI memory allocation scheme is 
>>> designed to deal with this very constrained environment. 
>>> 
>>> You start PEI with a heap and stack. You can also allocate HOBs (Hand Off 
>>> Blocks). A pool allocation in PEI is just a HOB. See [1]. There is no way 
>>> to free a HOB. So the AllocatePool() kind of leaks into DXE too as an entry 
>>> in the HOB list. But when the OS called gBS->ExitBootServices() that frees 
>>> all non runtime memory back to the OS. 
>>> 
>>> If you look a AllocatePages/FreePages you will see AllocatePages creates a 
>>> HOB that points to the memory region, and FreePages just marks that HOB as 
>>> not used. That code is also in this file [1]. 
>>> 
>>> TL;DR there is no pool manager in PEI. 
>>> 
>>> [1] 
>>> https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878
>>>  
>>> <https://github.com/tianocore/edk2/blob/master/Md

Re: [edk2-devel] Clarification of Memory management in PEI phase

[Brian J. Johnson]

Sorry for the late response to this thread...

PEI has grown greatly in complexity, as Andrew pointed out. Do we 
(TianoCore community) think it's time to add a real pool manager to 
PEI?  There's getting to be quite a bit of post-DRAM-initialization PEI 
code on some platforms.  And really, having a limited amount of pre-DRAM 
memory available is an even better reason for having an effective pool 
allocator, so the memory can be freed and reused.


FWIW we (HPE) needed to add a private pool allocator to manage memory 
for a proprietary h/w initialization module which needs to run in 
post-DRAM PEI, and depends on allocating and freeing memory in different 
phases of its operation.


Brian J. Johnson


*From:* Ayush Singh [mailto:ayushdevel1...@gmail.com]
*Sent:* Friday, June 10, 2022, 12:22 AM
*To:* Andrew Fish 
*Cc:* devel@edk2.groups.io
*Subject:* [edk2-devel] Clarification of Memory management in PEI phase


Thanks for the wonderful answer.

Ayush Singh

On Thu, Jun 9 2022 at 01:26:58 PM -0700, Andrew Fish  
wrote:


On Jun 9, 2022, at 10:28 AM, Ayush Singh
InstallPeiMemory() 

This is basically: (*PeiServices)->InstallPeiMemory (PeiServices, 
MemoryBegin, MemoryLength); This is how you tell the PEI Core the 
location of the memory that will can be used in PEI.


2. AllocatePages() 3. AllocatePool() 4. CopyMem() 5. SetMem() 6.
FreePages() However, no `FreePool()` service seems to be present.
So how is the memory allocated using `AllocatePool()` freed? 

It basically gets Freed when you transition to the DXE phase. To step 
back for a minute I think it is important to remember that the main 
job of PEI is to initialize DRAM, and deal with S3 (resuming from 
suspend to RAM). So as soon as you have DRAM you are kind done and 
ready for the DXE IPL so you can load the DXE Phase and start up EFI. 
Remember PEI is Pre EFI. The reality is programming DRAM is complex 
and lots of code got written, then lots more code got written and PEI 
has become large for some ports. That was never the intent. PEI is 
designed as a way to run C code when you code is running from ROM and 
you donā��t have any DRAM. For x86 not having DRAM means you are 
using the cache as RAM. For some SoCs there is actually an SRAM you 
can use. Thus the PEI memory allocation scheme is designed to deal 
with this very constrained environment. You start PEI with a heap and 
stack. You can also allocate HOBs (Hand Off Blocks). A pool 
allocation in PEI is just a HOB. See [1]. There is no way to free a 
HOB. So the AllocatePool() kind of leaks into DXE too as an entry in 
the HOB list. But when the OS called gBS->ExitBootServices() that 
frees all non runtime memory back to the OS. If you look a 
AllocatePages/FreePages you will see AllocatePages creates a HOB that 
points to the memory region, and FreePages just marks that HOB as not 
used. That code is also in this file [1]. TL;DR there is no pool 
manager in PEI. [1] 
https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878 
Thanks, Andrew Fish


Ayush Singh 




Brian

--

   "It's OK to be stuck.  99% of the time in your own [research] work,
you're stuck."
   -- Mark Lawrence



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#90695): https://edk2.groups.io/g/devel/message/90695
Mute This Topic: https://groups.io/mt/91651322/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Clarification of Memory management in PEI phase

[Ayush Singh]

Thanks for the wonderful answer.

Ayush Singh

On Thu, Jun 9 2022 at 01:26:58 PM -0700, Andrew Fish  
wrote:


 On Jun 9, 2022, at 10:28 AM, Ayush Singh > wrote:


 Hello everyone,

 Can anyone help me with understanding dynamic memory management in 
PEI

 phase? In the UEFI Platform Integration Specification, version 1.7
 Errata A, Section 4.6, PEI Memory services are given which include:

 1. InstallPeiMemory()


This is basically:
(*PeiServices)->InstallPeiMemory (PeiServices, MemoryBegin, 
MemoryLength);


This is how you tell the PEI Core the location of the memory that 
will can be used in PEI.



 2. AllocatePages()
 3. AllocatePool()
 4. CopyMem()
 5. SetMem()
 6. FreePages()

 However, no `FreePool()` service seems to be present. So how is the
 memory allocated using `AllocatePool()` freed?



It basically gets Freed when you transition to the DXE phase.

To step back for a minute I think it is important to remember that 
the main job of PEI is to initialize DRAM, and deal with S3 (resuming 
from suspend to RAM). So as soon as you have DRAM you are kind done 
and ready for the DXE IPL so you can load the DXE Phase and start up 
EFI. Remember PEI is Pre EFI. The reality is programming DRAM is 
complex and lots of code got written, then lots more code got written 
and PEI has become large for some ports. That was never the intent. 
PEI is designed as a way to run C code when you code is running from 
ROM and you donāt have any DRAM. For x86 not having DRAM means you 
are using the cache as RAM. For some SoCs there is actually an SRAM 
you can use. Thus the PEI memory allocation scheme is designed to 
deal with this very constrained environment.


You start PEI with a heap and stack. You can also allocate HOBs (Hand 
Off Blocks). A pool allocation in PEI is just a HOB. See [1]. There 
is no way to free a HOB. So the AllocatePool() kind of leaks into DXE 
too as an entry in the HOB list. But when the OS called 
gBS->ExitBootServices() that frees all non runtime memory back to the 
OS.


If you look a AllocatePages/FreePages you will see AllocatePages 
creates a HOB that points to the memory region, and FreePages just 
marks that HOB as not used. That code is also in this file [1].


TL;DR there is no pool manager in PEI.

[1] 




Thanks,

Andrew Fish


 Ayush Singh


 









-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#90435): https://edk2.groups.io/g/devel/message/90435
Mute This Topic: https://groups.io/mt/91651322/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Clarification of Memory management in PEI phase

[Andrew Fish via groups.io]

> On Jun 9, 2022, at 10:28 AM, Ayush Singh  wrote:
> 
> Hello everyone,
> 
> Can anyone help me with understanding dynamic memory management in PEI
> phase? In the UEFI Platform Integration Specification, version 1.7
> Errata A, Section 4.6, PEI Memory services are given which include:
> 
> 1. InstallPeiMemory()

This is basically:
(*PeiServices)->InstallPeiMemory (PeiServices, MemoryBegin, MemoryLength);

This is how you tell the PEI Core the location of the memory that will can be 
used in PEI. 

> 2. AllocatePages()
> 3. AllocatePool()
> 4. CopyMem()
> 5. SetMem()
> 6. FreePages()
> 
> However, no `FreePool()` service seems to be present. So how is the
> memory allocated using `AllocatePool()` freed?
> 

It basically gets Freed when you transition to the DXE phase. 

To step back for a minute I think it is important to remember that the main job 
of PEI is to initialize DRAM, and deal with S3 (resuming from suspend to RAM). 
So as soon as you have DRAM you are kind done and ready for the DXE IPL so you 
can load the DXE Phase and start up EFI. Remember PEI is Pre EFI. The reality 
is programming DRAM is complex and lots of code got written, then lots more 
code got written and PEI has become large for some ports. That was never the 
intent. PEI is designed as a way to run C code when you code is running from 
ROM and you don’t have any DRAM. For x86 not having DRAM means you are using 
the cache as RAM. For some SoCs there is actually an SRAM you can use. Thus the 
PEI memory allocation scheme is designed to deal with this very constrained 
environment. 

You start PEI with a heap and stack. You can also allocate HOBs (Hand Off 
Blocks). A pool allocation in PEI is just a HOB. See [1]. There is no way to 
free a HOB. So the AllocatePool() kind of leaks into DXE too as an entry in the 
HOB list. But when the OS called gBS->ExitBootServices() that frees all non 
runtime memory back to the OS. 

If you look a AllocatePages/FreePages you will see AllocatePages creates a HOB 
that points to the memory region, and FreePages just marks that HOB as not 
used. That code is also in this file [1]. 

TL;DR there is no pool manager in PEI. 

[1] 
https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Pei/Memory/MemoryServices.c#L878


Thanks,

Andrew Fish

> Ayush Singh
> 
> 
> 
> 
> 



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#90419): https://edk2.groups.io/g/devel/message/90419
Mute This Topic: https://groups.io/mt/91651322/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] Clarification of Memory management in PEI phase

[Ayush Singh]

Hello everyone,

Can anyone help me with understanding dynamic memory management in PEI
phase? In the UEFI Platform Integration Specification, version 1.7
Errata A, Section 4.6, PEI Memory services are given which include:

1. InstallPeiMemory()
2. AllocatePages()
3. AllocatePool()
4. CopyMem()
5. SetMem()
6. FreePages()

However, no `FreePool()` service seems to be present. So how is the
memory allocated using `AllocatePool()` freed?

Ayush Singh


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#90409): https://edk2.groups.io/g/devel/message/90409
Mute This Topic: https://groups.io/mt/91651322/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-