[osv-dev] Re: Demand paging for malloc

2019-09-09 Thread pusnow . kaist
It only fails on firecracker whose default memory size is 128M.

For QEMU, if I set VM's memory to 128M, it also fails.

./scripts/build -j24 image=native-example && ./scripts/run.py -m 128M 

trace:

OSv v0.53.0-87-gf7b6bee5
eth0: 192.168.122.15
Booted up in 345.76 ms
Hello from main
allocation 2000 start
Unreasonable allocation attempt, larger than memory. Aborting.
[backtrace]
0x403e29b4 
0x403e5da1 
0x403e60b7 
0x403e63f6 
0x1140094f 
0x4042d39c 
0x4020dfe3 
0x4042d568 
0x40461be5 
0x403f9fb6 
0x40399e52 
0x9f01e98d66991fff 
0x403f997f 
0x4156415741e58947 



Wonsup


2019년 9월 10일 화요일 오전 12시 35분 7초 UTC+9, Waldek Kozaczuk 님의 말:
>
> Interesting. I cannot reproduce the malloc() problem. I have no issues 
> running your example with uncommented malloc:
>
> #include 
> #include 
> #include 
>
> int main()
> {
> size_t size = 512 * 1024 * 1024;
> printf("Hello from main\n");
> printf("allocation %lx start\n", size);
> int *p = (int *)malloc(size); // FAIL
> //int *p = (int *)mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | 
> MAP_ANONYMOUS, -1, 0); // OK
> printf("allocation %lx = %p\n", size, p);
> *(p) = 512;
> printf("access done\n");
>
> return 0;
> }
>
> OSv v0.53.0-107-g4f59f284
>
> eth0: 192.168.122.15
>
> Booted up in 135.90 ms
>
> Cmdline: /test_large --help
>
> Hello from main
>
> allocation 2000 start
>
> allocation 2000 = 0x81b0d040
>
> access done
>
> As you can see the address returned is a virtual one. Can you email your 
> thread stack trace when it crashes in your case?
>
> As far as why we commit the memory for all malloc I can not answer this 
> and others might be able to address why it was designed like this. 
>
> This might be a relevant issue - 
> https://github.com/cloudius-systems/osv/issues/854. I guess we might 
> change implementation of malloc_large() and for large sizes create a VMA 
> like for mmap. But then malloc is used all over the place in kernel code so 
> I wonder if we would not have issues like nested faults - see this - 
> https://github.com/cloudius-systems/osv/issues/143 - it has some 
> background between malloc() and mmap() handling in OSv.
>
> Waldek
>
> On Monday, September 9, 2019 at 7:41:31 AM UTC-4, pusno...@gmail.com 
> wrote:
>>
>> Hi, 
>> I found malloc returns physical address in mempool area and does not 
>> perform demand paging (only mmap does).
>> Is there any reason for the design choice?
>> OSv fails, even if it only uses small portion of allocated memory.
>>
>>
>> #include 
>> #include 
>> #include 
>>
>> int main()
>> {
>> size_t size = 512 * 1024 * 1024;
>> printf("Hello from main\n");
>> printf("allocation %x start\n", size);
>> //int *p = (int *)malloc(size); // FAIL
>> int *p = (int *)mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | 
>> MAP_ANONYMOUS, -1, 0); // OK
>> printf("allocation %x = %p\n", size, p);
>> *(p) = 512;
>> printf("access done\n");
>>
>> return 0;
>> }
>>
>>
>> Thanks.
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/27528a92-b136-4f85-a914-647d55656613%40googlegroups.com.


[osv-dev] Re: Demand paging for malloc

2019-09-09 Thread Waldek Kozaczuk
Interesting. I cannot reproduce the malloc() problem. I have no issues 
running your example with uncommented malloc:

#include 
#include 
#include 

int main()
{
size_t size = 512 * 1024 * 1024;
printf("Hello from main\n");
printf("allocation %lx start\n", size);
int *p = (int *)malloc(size); // FAIL
//int *p = (int *)mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | 
MAP_ANONYMOUS, -1, 0); // OK
printf("allocation %lx = %p\n", size, p);
*(p) = 512;
printf("access done\n");

return 0;
}

OSv v0.53.0-107-g4f59f284

eth0: 192.168.122.15

Booted up in 135.90 ms

Cmdline: /test_large --help

Hello from main

allocation 2000 start

allocation 2000 = 0x81b0d040

access done

As you can see the address returned is a virtual one. Can you email your 
thread stack trace when it crashes in your case?

As far as why we commit the memory for all malloc I can not answer this and 
others might be able to address why it was designed like this. 

This might be a relevant issue - 
https://github.com/cloudius-systems/osv/issues/854. I guess we might change 
implementation of malloc_large() and for large sizes create a VMA like for 
mmap. But then malloc is used all over the place in kernel code so I wonder 
if we would not have issues like nested faults - see this - 
https://github.com/cloudius-systems/osv/issues/143 - it has some background 
between malloc() and mmap() handling in OSv.

Waldek

On Monday, September 9, 2019 at 7:41:31 AM UTC-4, pusno...@gmail.com wrote:
>
> Hi, 
> I found malloc returns physical address in mempool area and does not 
> perform demand paging (only mmap does).
> Is there any reason for the design choice?
> OSv fails, even if it only uses small portion of allocated memory.
>
>
> #include 
> #include 
> #include 
>
> int main()
> {
> size_t size = 512 * 1024 * 1024;
> printf("Hello from main\n");
> printf("allocation %x start\n", size);
> //int *p = (int *)malloc(size); // FAIL
> int *p = (int *)mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE | 
> MAP_ANONYMOUS, -1, 0); // OK
> printf("allocation %x = %p\n", size, p);
> *(p) = 512;
> printf("access done\n");
>
> return 0;
> }
>
>
> Thanks.
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/e43978ce-23a0-4023-ae7e-17dba12ef379%40googlegroups.com.