Hi Norman,
> as I noted in my other posting, I'm afraid that the support of mappings
> via mmap at fixed addresses can never be robust across kernels because
> one cannot make assumptions about the virtual address space layout.
>
> To create such a mechanism in a reasonably robust way, I'd go for the
> use of a managed dataspace. In the libc configuration, one could define
> the virtual address range where mmap mappings with fixed addresses are
> supposed to go. Upon start of the program. The libc would try to
> preserve this area by creating an appropriately sized Region_map (via
> the RM service) and attaching it at the base of the configured range. If
> this step fails (this can happen, depending on the kernel or
> architecture - think of 32bit), it will fail right at this early point.
>
Problem I see here is that current implementation of Libc alloc() do not have
notation of "memory allocated, but not committed".
In particular, I do not want to allocate during libc init all physical memory
which I can use in the future; I want to allocate virtual memory range without
backened physical ones, and populate it by demand.
This is a way how golang own runtime memory allocator implemented: they
reserve single continuous memory range for so called «arena» where they store
go data and some aux structures.
They can arrange on different OS up to 544Gb (on some like linux smaller).
Later, when necessary, they really fill parts of this range by some real memory.
They use mmap with _PROT_NONE, _MAP_ANON|_MAP_PRIVATE to reserve and mmap with
_PROT_READ|_PROT_WRITE, _MAP_ANON|_MAP_FIXED|_MAP_PRIVATE to commit.
In your implementation of mmap I see that _PROT_NONE is not supported.
as I see, current interface of libc alloc() assume that all memory stored
inside is committed (has been called _ds_pool.expand()), so, I can’t reserve
memory and populate it by demand naturally, without libc alloc changes?
By the way, I see a couple of references in genode docs to the way to "use
managed dataspace" and "populate it by phys memory by demand", while no
references to code samples provided?
How I can reserve virtual memory range in component without backend storage, in
what allocator?
> Once the managed dataspace is attached to the component's virtual
> address space, the libc can manually manage its layout via 'attach_at'
> without fearing conflicts. For an example, please have a look at
> base/src/test/sub_rm/.
as I understand, this approach will work only if I alloc whole range of virtual
memory _backend by physical memory_? In that case I can use standardised in
alloc approach and do not need additional call to expand_at() interface?
by the way, from the recommended example I found the following code:
/*
* Create region cap to be used for the actual test
*/
log("create managed dataspace");
Region_map_client sub_rm(rm.create(SUB_RM_SIZE));
enum { DS_SIZE = 4*4096 };
Ram_dataspace_capability ds = env.ram().alloc(DS_SIZE);
As I understand, we separately alloc DS_SIZE of physical memory and SUB_RM_SIZE
of virtual memory?
How I can specify here what exact range of virtual addresses I need to preserve?
later I see 2 different way to attach to the same ds (physical memory portion):
log("attach RAM ds to a fixed position at sub rm");
enum { DS_SUB_OFFSET = 4096 };
if ((addr_t)sub_rm.attach_at(ds, DS_SUB_OFFSET, 0, 0) != DS_SUB_OFFSET)
fail("attach_at return-value mismatch»);
and
char *sub_rm_base = env.rm().attach_at(sub_rm.dataspace(),
local_attach_addr);
What is a difference between them (except difference in DS_SUB_OFFSET vs
local_attach_addr)?
More specifically, between returned addresses from sub_rm.attach_at() and
env.rm().attach_at()?
>
> This feature would work independently from the 'Libc::Mem_alloc'. I
> think that the use of 'Libc::Mem_alloc' by 'mmap' is not the right way
> to go when taking mmap beyond it current scope. In particular,
We need to register our virtual address range where mmap mappings with fixed
addresses are supposed to go inside allocator of libc, as I understand.
so, it works only if we pre-load Libc::Mem_alloc by our own area backend by
real memory?
otherwise how this alloc will take memory from it?
> 'Libc::Mem_alloc' will eventually be swapped out by a better allocator
> (such as jemalloc). Extending its interface (as suggested by your patch)
> will make this step more complicated.
Sincerely,
Alexander
_______________________________________________
Genode users mailing list
[email protected]
https://lists.genode.org/listinfo/users
