Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-23 Thread Robert Engels
See https://github.com/golang/go/issues/9849 Go has no limit, you use ulimit to control. -Original Message- >From: Kevin Chadwick >Sent: Jan 23, 2020 10:26 AM >To: golang-nuts@googlegroups.com >Subject: Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-23 Thread Kevin Chadwick
On 2020-01-23 14:18, robert engels wrote: > There is nothing “special” about it - generally the Go process calls > “malloc()” and fails with OOM (unable to expand the process memory size), but > the OS can kill processes in a low system memory condition without them > calling malloc (OOM killer

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-23 Thread robert engels
There is nothing “special” about it - generally the Go process calls “malloc()” and fails with OOM (unable to expand the process memory size), but the OS can kill processes in a low system memory condition without them calling malloc (OOM killer kills the hogs). If you process is dying due to th

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-23 Thread Kevin Chadwick
On 2020-01-20 18:57, Robert Engels wrote: > This is solved pretty easily in Java using soft references and a hard memory > cap. > > Similar techniques may work here. One of the only things I dislike about GO compared to C is the arbitrary memory allocation but it has great benefits in coding ti

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-20 Thread robert engels
This is actually SoftRef - which are like a WeakRef but they are only collected under memory pressure. If the WeakRef package works, I assume it could be modified to enable “soft ref” like functionality. It was my understanding that you need GC/runtime support to truly make this work, but maybe

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-20 Thread Eric S. Raymond
Robert Engels : > This is solved pretty easily in Java using soft references and a hard memory > cap. That'd be nice, but the onnly weak-references package I've found doesn't seem to allow more than one weakref per target. That's really annoying, because my use case is a target object for a man

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-20 Thread Robert Engels
This is solved pretty easily in Java using soft references and a hard memory cap. Similar techniques may work here. > On Jan 20, 2020, at 11:22 AM, Christian Mauduit wrote: > > Hi, > > That is a generic question and I think that if you want to keep an approach > with a "global indicator o

Re: [go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-20 Thread Christian Mauduit
Hi, That is a generic question and I think that if you want to keep an approach with a "global indicator of how much memory is used", your approach is OK. You might also want to store this information of "should I throttle" in a cache or something, the cache could be just a shared atomic flag

[go-nuts] Is there some kind of a MaxHeapAlarm implementation?

2020-01-20 Thread Urjit Singh Bhatia
Hi folks, I am trying to figure out if someone has a decent solution for max memory usage/mem-pressure so far. I went through some of the github issues related to this (SetMaxHeap proposals and related discussions) but most of them are still under review: - https://go-review.googlesource.co