Hi, inline
----- "Malahal Naineni" <mala...@us.ibm.com> wrote: > > perf record shows that too much time is spent in malloc/free > functions. > Reported functions are alloc_nfs_request, alloc_nfs_res, and few > objects > in src/xdr_ioq.c file. alloc_nfs_res seems thread specific, so could > be > allocated one per thread. If we can make other pools lockless > (instead > of malloc), that would be great! We have work-in-progress changes on former-cohortfs Ceph branches that uses, MCAS lock-free dictionaries and their lock-free allocator caches. The allocators can be used standalone, but we were interested in exploring both potentially for use in Ganesha. Our Ceph-based inode cache simulator which used these could sustain over 2M create/s, IIRC. Marcus did some work on the pool allocator to upgrade its epoch-based reclamation logic, to NEBR (discussed here http://csng.cs.toronto.edu/publication_files/0000/0159/jpdc07.pdf TLS would be probably work for some things, as you say. Other things we've used in the last year or so: 1. pool interface exposed in Mellanox Accelio library--initially was special purpose, but has been generalized (partly in our Accelio Ceph work); It has special capabilities (registration) for use with RDMA. I don't know if this is mature as MCAS, but would be work a look. 2. for some key allocations in our Ceph OSD, we used a lock-free bounded queue of structures--this appeared to very work well, for large objects. For small allocations, jemalloc was much faster. Regards, Matt > > Regards, Malahal. > > -- Matt Benjamin CohortFS, LLC. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://cohortfs.com tel. 734-761-4689 fax. 734-769-8938 cel. 734-216-5309 ------------------------------------------------------------------------------ Monitor Your Dynamic Infrastructure at Any Scale With Datadog! Get real-time metrics from all of your servers, apps and tools in one place. SourceForge users - Click here to start your Free Trial of Datadog now! http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140 _______________________________________________ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
