That's correct. The runtime has a simple heuristic for avoiding zeroing but it's far from perfect. As a result, a ballast is inherently always going to be a little risky. This is especially true on some platforms, such as Windows, since there's no way to avoid marking the memory as committed (Windows is free to use demand paging for memory in the range, so overall system memory pressure may increase, but you can't avoid it being *counted* as committed for a particular process).
Taking a step back: why a ballast? What about your application makes a ballast a better idea than, for example, setting GOMEMLIMIT=<something> and GOGC=off? (For additional context, back when the memory limit was proposed, so was a memory target (https://go.googlesource.com/proposal/+/master/design/44309-user-configurable-memory-target.md) which more directly replaces a ballast. I found very little interest in this feature.) On Monday, November 7, 2022 at 9:08:23 AM UTC-5 hit.zh...@gmail.com wrote: > Hi, guys, I know what happened. > > When we write `ballast := make([]byte, 1<<30)`, it will call makeslice to > create a new slice. It's a large object. The memory will be allocated > directly via allocLarge() function from heap. > > Actually, after allocating a mspan for it, it will check the address range > whether it should be zeroed. Please check function `func > allocNeedsZero(base, npage uintptr) bool`. When this function returns true, > it means the slice underlying memory will be written to zero, otherwise it > won't write it to zero. > > When we make a ballast as mentioned before, maybe it succeed (no RSS taken > up) or fail (RSS taken up), it's relevant with the return value of function > allocNeedsZero(...). And this function return true or false is relevant > with the arena's state which is affected by previous object allocation and > recycle. > > On Monday, November 7, 2022 at 1:26:41 PM UTC+8 tapi...@gmail.com wrote: > >> I ever also found this problem: global ballast doesn't work. >> So I always use local ballast instead now. >> >> On Sunday, November 6, 2022 at 8:37:55 PM UTC+8 hit.zh...@gmail.com >> wrote: >> >>> Before, I think the memory is allocated by mmap anon, which Linux OS >>> guarantees reading will return zero value and no physical pages allocated. >>> When writing happens, the physical pages will be allocated. Then I think >>> the create a byte slice maybe the same. >>> >>> Your idea is clear. I agree with it. >>> >>> Just now, I use gdb to dump the ballast anon memory and use hexdump to >>> check its dirty content, all data is zero (Maybe zeroing happens). >>> But after turning GC off, it works as expected (no RSS is taken, no >>> DIRTY). >>> I think there must be something I didn't get it. >>> >>> On Sunday, November 6, 2022 at 8:11:51 PM UTC+8 Jan Mercl wrote: >>> >>>> On Sun, Nov 6, 2022 at 12:54 PM Kn (Kn) <hit.zh...@gmail.com> wrote: >>>> >>>> > Now the problem begins. I expect the ballast like `ballast := >>>> make([]byte, 1<<30)` shouldn't take up any physical memory because there's >>>> no any writing to it. >>>> >>>> The backing array is specified to be zeroed, so we cannot say there's >>>> no writing to it. Depending on the size of the backing array and the >>>> operating system it may not get written as an optimization if backed >>>> by memory the OS can guarantee to be zero filled. Only then it may >>>> remain not yet bound to physical memory. >>>> >>>> A simple implementation will just zero it, meaning the opposite >>>> happens - every byte of the backing array gets written and backing >>>> pages for it get allocated. >>>> >>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/e3deee3a-5382-4828-bfea-de221fff2175n%40googlegroups.com.
