This sounds great. A secondary concern is that we'll want to properly account for this stuff in memory reports. I think we can do this if we have a way to ask the kernel "How many copies exist of each page within this span of virtual memory?"
That definitely doesn't need to be in place when we land your first pass at this, but it's something you may want to keep it in mind. On Sat, May 4, 2013 at 12:03 AM, Thinker K.F. Li <[email protected]> wrote: > Calling free() is enough. You can imagine that memdup() is a function > calling malloc() before calling memcow to map content of a source > memory to the new address. The memory returned from memdup() is still > a block of annonymous memory managed by memory allocator; like jemalloc. > > It looks like > > void *memdup(void *src, unsigned long sz) { > void *dst = malloc(sz); > if (!dst) return NULL; > if (src is aligned && sz is big enough) { > memcow(dst, src, sz); /* map content pages of src at dst too. */ > } else { > memcpy(dst, src, sz); > } > return dst; > } > > I had checked that jemalloc always return a page aligned memory for > big sizes. But, for nsStringBuffer, there is a prefixed > header/struct, so string is not page aligned with a small offset from > the page boundary. For this kind of sources, if the size is big enough > and the offset is small enough, we can use fallback memcpy() for first > and last page but keep using memcow for in-between pages, and return > the address with the small offset from the page boundary. > > From: Justin Lebar <[email protected]> > Subject: Re: [b2g] Introduce COW for B2G > Date: Fri, 3 May 2013 15:30:45 -0400 > >> How would one free memdup()'ed memory? Would free() be sufficient, or >> would one need to call something else? >> >> On Fri, May 3, 2013 at 6:36 AM, Thinker K.F. Li <[email protected]> wrote: >>> With memcow, checking size and alignment for source and targeet in >>> memcpy() is most simple one of here purposed ways to apply COW. A >>> wrapper for memcpy may be enough. >>> >>> From: Ting-Yuan Huang <[email protected]> >>> Subject: Re: Introduce COW for B2G >>> Date: Fri, 3 May 2013 03:04:11 -0700 (PDT) >>> >>>>> After some discussions, Ting-Yuan is trying to implement a COW >>>>> mechanism at userspace for sharing pages of source and target memory >>>>> blocks passed to memcpy. He expect to use a memdup() function to >>>>> replace all malloc() & memcpy() paired function calls. So, we do some >>>>> tricky magic to make COW applied. (I believe he will explain it >>>>> later.) >>>> >>>> I'd like to propose a new API to jemalloc, called memdup(). memdup() >>>> behaves >>>> similar to the standard strdup(), except that it duplicates memory instead >>>> of null-terminated strings. It's quite often that a memcpy() immediately >>>> follows an malloc() and the contents in source and destination then are >>>> identical. In this case we can point the virtual pages of the destination >>>> to the >>>> physical frames of the source and defer allocating and copying memory by >>>> marking them copy-on-write. If there is a non-trivial ratio of pages not >>>> altered eventually, memory and CPU usages are saved. >>>> >>>>> In another word, you can use memcow to map pages of A >>>>> block to the address range of B block, A and B are allocated through >>>>> malloc(), instead of calling memcpy. It saves a lot of memory, >>>>> especially for the cases like bug 850175. >>>> >>>> Getting anonymous pages by mmap() and immediately calling memcow() is >>>> equivalent to memdup() which is semantically better I think :) >>>> >>>> Another possible approach might be, as an optimization, to implement COW in >>>> memcpy(). One of the difficulty could be how to identify safe sources and >>>> destinations. >>>> >>>> Any ideas? >>> _______________________________________________ >>> dev-b2g mailing list >>> [email protected] >>> https://lists.mozilla.org/listinfo/dev-b2g _______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
