I'm thinking there's something wrong here. The handle cache is limited (to 32k by default, unless you raised the limit). The dirent cache is currently unlimited (limits coming in 2.5), but a dirent is only 80 bytes + sizeof(filename), so that can't do it. This can be tested, if you want to upgrade to at least 2.4.2, as we added the ability to put a limit on the size of cached directories. Setting that to low effectively stops dirent caching.
I can't come up with a use of more than ~2k/inode based on code, so something wrong since that's off by almost 2 orders of magnitude. It's possible that some of that missing memory is taken by gfapi structures (I assume you're using FSAL_GLUSTER?) that aren't referenced in Ganesha proper, but it's unlikely to be that much. The only other thing I can think of is the DRC fixes. I believe that removed leaks, and since those are related to operations, the longer the run, the more it leaks, I believe. Some of those fixes are in later 2.4.x, so upgrading to 2.4.4 and re-testing might be useful. Daniel On 03/24/2017 02:16 AM, sriram patil wrote: > Hi, > > We have been able to reproduce the OOM killer error consistently. NFS > Ganesha memory consumption goes as high as 85-90 gigs (>60% of total > memory). The test case is very simple, > just keep creating files (with dd, bs=64k count=1). > > The setup has 1 NFS Ganesha server and 8 clients which are > continuously firing dds. After about 1M files, the memory reaches to > 85 gigs. Which is like every inode consuming 85k of memory. > > I see that there are already a couple of issues opened on glusterfs, > > https://bugzilla.redhat.com/show_bug.cgi?id=1240172 > https://bugzilla.redhat.com/show_bug.cgi?id=1337867 > > Is this a known issue for nfs-ganesha? Can we work around this problem > with some inode cache configuration (cache size, entries hw, cache > fds, etc)? > > Thanks, > Sriram > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Nfs-ganesha-devel mailing list > Nfs-ganesha-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel > ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
