On Fri, Mar 13, 2020 at 03:43:21AM +0530, Neeraj Pal wrote:
> On Tue, Mar 10, 2020 at 4:03 PM Otto Moerbeek <[email protected]> wrote:
> > There's an off by one in your question :-)
> Yeah, sorry about that, actually in flow of writing the mail forgot to notice.
>
> > Fo single threaded programs, two malloc_dir pools are maintained.
> > One for MAP_CONCEALED memory (#0) and one for regular (#1).
> > For multi-threaded porgram more pools are created. This is to avoid
> > contention,
> > accesses to diffrent pools can run concurently.
> okay, thanks for the information. So, likewise, for multi threaded
> applications, by default the malloc_mutexes is 8, (#0 for
> MAP_CONCEALED and other 7 for regular) as mentioned in the below code:
Please indent your code snippets.
>
> static void
> omalloc_init(void)
> {
> char *p, *q, b[16];
> int i, j, mib[2];
> size_t sb;
> /*
> * Default options
> */
> mopts.malloc_mutexes = 8;
> mopts.def_malloc_junk = 1;
> ...
> ...
> ...
>
>
>
> > yes. That way both underflow and oveflow has a chance to be caught.
> yeah, it's good. but I am not sure about it from the code. I mean from
> the below code snippet it seems that by default (means vm.malloc_conf
> != G) it is not on both sides?
> <guard page> dir_info
di_info is special. Having a guard page on both sides for regular
allocation can be done, but would waste more pages. Note that
allocations are already spread thrrougout the address space, so it is
very likely that an allocation is surrounded by unmapped pages.
>
> static void
> omalloc_poolinit(struct dir_info **dp, int mmap_flag)
> {
> char *p;
> size_t d_avail, regioninfo_size;
> struct dir_info *d;
> int i, j;
> /*
> * Allocate dir_info with a guard page on either side. Also
> * randomise offset inside the page at which the dir_info
> * lies (subject to alignment by 1 << MALLOC_MINSHIFT)
> */
> if ((p = MMAPNONE(DIR_INFO_RSZ + (MALLOC_PAGESIZE * 2), mmap_flag)) ==
> MAP_FAILED)
> wrterror(NULL, "malloc init mmap failed");
> mprotect(p + MALLOC_PAGESIZE, DIR_INFO_RSZ, PROT_READ | PROT_WRITE);
> d_avail = (DIR_INFO_RSZ - sizeof(*d)) >> MALLOC_MINSHIFT;
> d = (struct dir_info *)(p + MALLOC_PAGESIZE +
> (arc4random_uniform(d_avail) << MALLOC_MINSHIFT));
> ...
> ...
> ...
>
> From the above code, my observations are
> sizeof(*d) = 4824
> MALLOC_PAGEMASK = 4095
> DIR_INFO_RSZ = (4284 + 4095) & ~4095 = 8192
We need two pages to store dir_info.
>
> Now, MMAPNONE maps up to len (8192 + (4096 * 2)) = 16384
We allocate 4 pages prot none.
> then, mprotecting the pages through p + MALLOC_PAGESIZE + DIR_INFO_RSZ - 1
the two middle pages are r/w.
> d_avail = (8192 - 4824) >> 4 = 3368 >> 4 = 210
>
> Now, d = (p + MALLOC_PAGESIZE + (random_no_under_210 << 4)
di_info ends up on an aligned address somewhere in the middle pages on
an offset between 0 and (210<<4) = 0..3360, counting from the start of
the two middle pages.
>
> where d is the randomized offset inside the page at which dir_info lies,
> So, lets suppose p is 1000 then 1000 + 4096 + (100 << 4) , then d will be
> 6696.
> So, it means [p + MALLOC_PAGESIZE] can be treated as guard page before
> dir_info offset and if yes then after that there is no guard page by
> default, right?
No, there will be a guard page on each side.
>
>
> > The second index of chunk_dir has size MALLOC_CHUNK_LISTS which is 4,
> > not 32.
> Yeah, sorry for incorrect values.
> > More than one list of free chunk pages per chunk size is maintained to
> > allow for more randomization.
> Okay, so in short it means below code will create 12 chunk_info_list
> where i is 0 to 11 and for each and every ith index there is j, so as
> per that,
> chunk_dir[0][0]
> chunk_dir[0][1]
> chunk_dir[0][2]
> chunk_dir[0][3]
> ...
> ...
> ...
> chunk_dir[11][0]
> chunk_dir[11][1]
> chunk_dir[11][2]
> chunk_dir[11][3]
>
> ...
> ...
> ...
> for (i = 0; i <= MALLOC_MAXSHIFT; i++) {
> LIST_INIT(&d->chunk_info_list[i]);
> for (j = 0; j < MALLOC_CHUNK_LISTS; j++)
> LIST_INIT(&d->chunk_dir[i][j]);
> ...
> ....
> ....
> So, these many lists simply means it allows more randomization,
> wherever it is used, like also in case of allocating chunk using
> omalloc_make_chunks() in malloc_bytes()
> ...
> ...
> ...
> j = find_chunksize(size);
> r = ((u_int)getrbyte(d) << 8) | getrbyte(d);
> listnum = r % MALLOC_CHUNK_LISTS;
> /* If it's empty, make a page more of that size chunks */
> if ((bp = LIST_FIRST(&d->chunk_dir[j][listnum])) == NULL) {
> bp = omalloc_make_chunks(d, j, listnum);
> if (bp == NULL)
> return NULL;
> }
> ...
> ...
> ...
> And, then also it means that it maybe possible that one can increase
> the MALLOC_MAXSHIFT and MALLOC_CHUNK_LISTS to increase more
> randomization?
MALLOC_CHUNK_LISTS could be increased at the cost of overhead.
MALLOC_MAXSHIFT cannot, it is the shift of the max chunk size that fits in
a page.
>
> Also, may I know the use of structure "struct region_info", is it used
> to keep track of mmap’ed regions by storing their address and size
> into a hash table as mentioned in the
> https://www.openbsd.com.au/papers/eurobsdcon2009/otto-malloc.pdf
yes.
>
>
> Please confirm whether my understanding is correct or not.
>
> Regards,
> Neeraj
>
