Actually, we used go1.16.5 before go1.19 released. Now we're consider using go1.19 GOMEMLIMIT instead.
But GOMEMLIMIT is a soft limit, if you deploy multiple services in the same host (not per container per service), there will be problems. The GC is delayed, and if combined with GOGC=off, the memory usage will be higher. Maybe manually trigger forced GC, and combine GOMEMLIMIT will be better. I'm testing... 在2022年11月11日星期五 UTC+8 05:58:54<mkny...@google.com> 写道： > That's correct. The runtime has a simple heuristic for avoiding zeroing > but it's far from perfect. As a result, a ballast is inherently always > going to be a little risky. This is especially true on some platforms, such > as Windows, since there's no way to avoid marking the memory as committed > (Windows is free to use demand paging for memory in the range, so overall > system memory pressure may increase, but you can't avoid it being > *counted* as committed for a particular process). > > Taking a step back: why a ballast? What about your application makes a > ballast a better idea than, for example, setting GOMEMLIMIT=<something> and > GOGC=off? > > (For additional context, back when the memory limit was proposed, so was a > memory target ( > https://go.googlesource.com/proposal/+/master/design/44309-user-configurable-memory-target.md) > > which more directly replaces a ballast. I found very little interest in > this feature.) > > On Monday, November 7, 2022 at 9:08:23 AM UTC-5 hit.zh...@gmail.com wrote: > >> Hi, guys, I know what happened. >> >> When we write `ballast := make([]byte, 1<<30)`, it will call makeslice to >> create a new slice. It's a large object. The memory will be allocated >> directly via allocLarge() function from heap. >> >> Actually, after allocating a mspan for it, it will check the address >> range whether it should be zeroed. Please check function `func >> allocNeedsZero(base, npage uintptr) bool`. When this function returns true, >> it means the slice underlying memory will be written to zero, otherwise it >> won't write it to zero. >> >> When we make a ballast as mentioned before, maybe it succeed (no RSS >> taken up) or fail (RSS taken up), it's relevant with the return value of >> function allocNeedsZero(...). And this function return true or false is >> relevant with the arena's state which is affected by previous object >> allocation and recycle. >> >> On Monday, November 7, 2022 at 1:26:41 PM UTC+8 tapi...@gmail.com wrote: >> >>> I ever also found this problem: global ballast doesn't work. >>> So I always use local ballast instead now. >>> >>> On Sunday, November 6, 2022 at 8:37:55 PM UTC+8 hit.zh...@gmail.com >>> wrote: >>> >>>> Before, I think the memory is allocated by mmap anon, which Linux OS >>>> guarantees reading will return zero value and no physical pages allocated. >>>> When writing happens, the physical pages will be allocated. Then I think >>>> the create a byte slice maybe the same. >>>> >>>> Your idea is clear. I agree with it. >>>> >>>> Just now, I use gdb to dump the ballast anon memory and use hexdump to >>>> check its dirty content, all data is zero (Maybe zeroing happens). >>>> But after turning GC off, it works as expected (no RSS is taken, no >>>> DIRTY). >>>> I think there must be something I didn't get it. >>>> >>>> On Sunday, November 6, 2022 at 8:11:51 PM UTC+8 Jan Mercl wrote: >>>> >>>>> On Sun, Nov 6, 2022 at 12:54 PM Kn (Kn) <hit.zh...@gmail.com> wrote: >>>>> >>>>> > Now the problem begins. I expect the ballast like `ballast := >>>>> make([]byte, 1<<30)` shouldn't take up any physical memory because >>>>> there's >>>>> no any writing to it. >>>>> >>>>> The backing array is specified to be zeroed, so we cannot say there's >>>>> no writing to it. Depending on the size of the backing array and the >>>>> operating system it may not get written as an optimization if backed >>>>> by memory the OS can guarantee to be zero filled. Only then it may >>>>> remain not yet bound to physical memory. >>>>> >>>>> A simple implementation will just zero it, meaning the opposite >>>>> happens - every byte of the backing array gets written and backing >>>>> pages for it get allocated. >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/858a4a4d-d049-40d7-9efe-ee6cfae0fa2dn%40googlegroups.com.
