Pushkar - based on my understanding of Fei's experiments, the issue was doing the DONT_DUMP marking, which would cause problems with jemalloc. That's part of the effort of getting jemalloc ready.
Walt - I've read through your code and I don't really the benefits. It's definitely cleaner code, but I don't see the implementation advantage, particularly with regard to reducing pressure on heaps. E.g. "f there are a lot of smaller dynamic objects with short lifetimes, it will reduce thread blocking on the heap mutex" - I don't see that. With the current implementation, in those circumstances there is extremely little pressure on the heap because the objects are popping on and off thread local free lists and not hitting the underlying allocation mechanism. Also, AFAICT from the code, objects that are bigger than a pointer are allocated directly from the heap. Given that few, if any, of the ATS objects in question are that small, it seems like this effectively disables freelists. How is that better than just calling new and delete directly? Another major problem is that, due the current free list implementation, there is not much concern about constructors and destructors and depending on those to keep object state correct is likely to be bug prone. This is going to be an issue for jemalloc as well, but if we're going to do the work we should move all the way to no free lists at all. On Tue, Dec 11, 2018 at 6:05 PM Bryan Call <bc...@apache.org> wrote: > The freelist can be used with jemalloc, but the thought/theory is that you > can turn off the freelist and use jemalloc and get similar performance. > This needs to be validated. > > -Bryan > > > On Dec 11, 2018, at 3:18 PM, Walt Karas <wka...@oath.com.INVALID> wrote: > > > > I thought jemalloc is used as a drop-in replacement for the standard lib > > heap functions / operators. So how can the freelist stuff not work with > it? > > > > On Tue, Dec 11, 2018 at 4:48 PM Bryan Call <bc...@apache.org> wrote: > > > >> There is no point in cleaning up the code if the plan is to not use it > and > >> remove it from our codebase. Work should be done on proving that > jemalloc > >> is valid alternative. > >> > >> If jemalloc doesn’t prove to workout, then we might look at cleaning up > >> the freelist. > >> > >> -Bryan > >> > >>> On Dec 10, 2018, at 5:42 PM, Walt Karas <wka...@oath.com.INVALID> > wrote: > >>> > >>> As far as one can tell is a big limitation with code like: > >>> > >>> #if (defined(__i386__) || defined(__arm__) || defined(__mips__)) && > >>>> (SIZEOF_VOIDP == 4) > >>>> > >>>> #define FREELIST_POINTER(_x) (_x).s.pointer > >>>> > >>>> #define FREELIST_VERSION(_x) (_x).s.version > >>>> > >>>> #define SET_FREELIST_POINTER_VERSION(_x, _p, _v) \ > >>>> > >>>> (_x).s.pointer = _p; \ > >>>> > >>>> (_x).s.version = _v > >>>> > >>>> #elif TS_HAS_128BIT_CAS > >>>> > >>>> #define FREELIST_POINTER(_x) (_x).s.pointer > >>>> > >>>> #define FREELIST_VERSION(_x) (_x).s.version > >>>> > >>>> #define SET_FREELIST_POINTER_VERSION(_x, _p, _v) \ > >>>> > >>>> (_x).s.pointer = _p; \ > >>>> > >>>> (_x).s.version = _v > >>>> > >>>> #elif defined(__x86_64__) || defined(__ia64__) || > defined(__powerpc64__) > >>>> || defined(__aarch64__) || defined(__mips64) > >>>> > >>>> #define FREELIST_POINTER(_x) \ > >>>> > >>>> ((void *)(((((intptr_t)(_x).data) << 16) >> 16) | > >>>> (((~((((intptr_t)(_x).data) << 16 >> 63) - 1)) >> 48) << 48))) // sign > >>>> extend > >>>> > >>>> #define FREELIST_VERSION(_x) (((intptr_t)(_x).data) >> 48) > >>>> > >>>> #define SET_FREELIST_POINTER_VERSION(_x, _p, _v) (_x).data = > >>>> ((((intptr_t)(_p)) & 0x0000FFFFFFFFFFFFULL) | (((_v)&0xFFFFULL) << > 48)) > >>>> > >>>> #else > >>>> > >>>> #error "unsupported processor" > >>>> > >>>> #endif > >>>> > >>> > >>> > >>> > >>> > >>> On Mon, Dec 10, 2018 at 5:02 PM Leif Hedstrom <zw...@apache.org> > wrote: > >>> > >>>> > >>>> > >>>>> On Dec 10, 2018, at 10:29 AM, SUSAN HINRICHS <shinr...@ieee.org> > >> wrote: > >>>>> > >>>>> Based on Fei's measurements, the ATS freelists provide no benefit > over > >>>>> jemalloc. We are now in a position to do larger tests over our > >>>> production > >>>>> installs. > >>>> > >>>> > >>>> Agreed, that was generally what I noticed too, except, I could not get > >> ATS > >>>> to be stable with just jemalloc. It’d eventually get unhappy, but I > >> didn’t > >>>> investigate further. But this is my point, lets focus the efforts on > >> moving > >>>> us forward, to jemalloc, and not mess around with freelist as it is, > >>>> because it works fine as far as I can tell. > >>>> > >>>> — leif > >>>> > >>>> > >> > >> > > -- *Beware the fisherman who's casting out his line in to a dried up riverbed.* *Oh don't try to tell him 'cause he won't believe. Throw some bread to the ducks instead.* *It's easier that way. *- Genesis : Duke : VI 25-28
