Hi, Sorry for the confusion--I had forgotten or never noticed that Ganesha updates these itself.
Clearly, the number of fds is partially independent of the number of cache entries--for FSALs which in fact have OS fds. Among other things, the lru thread is working to close entries when they drop to L2. Matt ----- "Malahal Naineni" <mala...@us.ibm.com> wrote: > It is probably distro specific, but 4096 is the default NOFILE limit > on > RHEL6/RHEL7 systems. It can be set to as high as 1024*1024. See > 14447420966b9dfcdeb9975ac134a36613ade3c4 for further details on how > it > is set to 1M by default for ganesha daemon. This is available in > 2.2-stable. > > Regards, Malahal. > > Krishna Harathi [khara...@exablox.com] wrote: > > Matt, > > Yes we are using VFS FSAL. I will re-run the workload after > increasing the > > limits. > > Thanks. > > Regards. > > Krishna Harathi > > On Sat, May 16, 2015 at 8:36 AM, Matt W. Benjamin > <[1]m...@cohortfs.com> > > wrote: > > > > Hi Krishna, > > > > If you're using a VFS-like FSAL, I wouldn't think you'd want to > be > > hard limited to 4K open files. > > > > It could be useful to have more flexibility in (choice of?) > LRU > > reclaim strategy, but to start with, I'd raise this limit (on > > Linux, /proc/sys/fs/nr_open, etc). > > > > Matt > > ----- "Krishna Harathi" <[2]khara...@exablox.com> wrote: > > > > > To be clear, this comment cache_inode_lru.c did not help > much. > > > > > > > > > /* Set high and low watermark for cache entries. This seems > a > > > bit fishy, so come back and revisit this. */ > > > lru_state.entries_hiwat = cache_param.entries_hwmark; > > > lru_state.entries_used = 0 ; > > > > > > > > > > > > > > > Regards. > > > Krishna Harathi > > > > > > On Fri, May 15, 2015 at 7:44 PM, Krishna Harathi < > > > [3]khara...@exablox.com > wrote: > > > > > > > > > > > > > > > I am trying to determine the relation between the system > imposed limit > > > of max FDs (4096 now) and > > > the Cache Inode HW Mark (100000 default). > > > > > > > > > In a test work load, I am getting errno 24 (too many open > files) error > > > returned, > > > and here is our current settings. > > > > > > > > > > > > 15/05/2015 18:48:23 : epoch 5556a1e7 : OneBlox-0333 : > > > nfs-ganesha-12814[main] cache_inode_lru_pkginit :INODE LRU > :INFO > > > :Attempting to increase soft limit from 1024 to hard limit of > 4096 > > > 15/05/2015 18:48:23 : epoch 5556a1e7 : OneBlox-0333 : > > > nfs-ganesha-12814[main] cache_inode_lru_pkginit :INODE LRU > :INFO > > > :Setting the system-imposed limit on FDs to 4096 > > > > > > > > > > > > > > > I looked at the related code in Cache_inode but could not > determine > > > the relationship between > > > the system limit and cache_inode or how many file FDs Ganesha > will try > > > to open. > > > > > > > > > Any help is (1) What is the system limit to be set, given the > default > > > cache inode HWmark? > > > (2) For the given system limit, what is the cache inode > setting. We > > > are using Ganesha 2.1. > > > > > > > > > > > > > > > > > > Regards. > > > Krishna Harathi > > > > > > > > > ------------------------------------------------------------------------------ > > > One dashboard for servers and applications across > > > Physical-Virtual-Cloud > > > Widest out-of-the-box monitoring support with 50+ > applications > > > Performance metrics, stats and reports that give you > Actionable > > > Insights > > > Deep dive visibility with transaction tracing using APM > Insight. > > > [4]http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > > > _______________________________________________ > > > Nfs-ganesha-devel mailing list > > > [5]Nfs-ganesha-devel@lists.sourceforge.net > > > > [6]https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel > > -- > > Matt Benjamin > > CohortFS, LLC. > > 315 West Huron Street, Suite 140A > > Ann Arbor, Michigan 48103 > > > > [7]http://cohortfs.com > > > > tel. [8]734-761-4689 > > fax. [9]734-769-8938 > > cel. [10]734-216-5309 > > > > References > > > > Visible links > > 1. mailto:m...@cohortfs.com > > 2. mailto:khara...@exablox.com > > 3. mailto:khara...@exablox.com > > 4. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > > 5. mailto:Nfs-ganesha-devel@lists.sourceforge.net > > 6. > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel > > 7. http://cohortfs.com/ > > 8. file:///tmp/tel:734-761-4689 > > 9. file:///tmp/tel:734-769-8938 > > 10. file:///tmp/tel:734-216-5309 > > > > ------------------------------------------------------------------------------ > > One dashboard for servers and applications across > Physical-Virtual-Cloud > > Widest out-of-the-box monitoring support with 50+ applications > > Performance metrics, stats and reports that give you Actionable > Insights > > Deep dive visibility with transaction tracing using APM Insight. > > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > > > _______________________________________________ > > Nfs-ganesha-devel mailing list > > Nfs-ganesha-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel -- Matt Benjamin CohortFS, LLC. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://cohortfs.com tel. 734-761-4689 fax. 734-769-8938 cel. 734-216-5309 ------------------------------------------------------------------------------ One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
