Re: [lustre-discuss] Memory Management in Lustre

[Jeff Johnson]

Ellis,

I haven't messed with it much personally but if you look at some of
the Lustre module parameters, like in the case of module obdclass, you
will see some options that could be of interest like lu_cache_percent.

I'm sure a Whamcloud person might chime in with more detail.

# modinfo obdclass
filename:   /lib/modules/3.10.0-957.27.2.el7.DPC.x86_64/extra/obdclass.ko.xz
license:GPL
version:2.12.2
description:Lustre Class Driver
author: OpenSFS, Inc. 
alias:  fs-lustre
retpoline:  Y
rhelversion:7.6
srcversion: 3D7126D7BB611F089C67867
depends:libcfs,lnet,crc-t10dif
vermagic:   3.10.0-957.27.2.el7.DPC.x86_64 SMP mod_unload modversions
parm:   lu_cache_percent:Percentage of memory to be used as
lu_object cache (int)
parm:   lu_cache_nr:Maximum number of objects in lu_object cache (long)
parm:   lprocfs_no_percpu_stats:Do not alloc percpu data for
lprocfs stats (int)

--Jeff


On Wed, Jan 19, 2022 at 6:35 PM Ellis Wilson via lustre-discuss
 wrote:
>
> Hi folks,
>
> Broader (but related) question than my current malaise with OOM issues on 
> 2.14/2.15:  Is there any documentation or can somebody point me at some code 
> that explains memory management within Lustre?  I've hunted through Lustre 
> manuals, the Lustre internals doc, and a bunch of code, but can find nothing 
> that documents the memory architecture in place.  I'm specifically looking at 
> PTLRPC and OBD code right now, and I can't seem to find anywhere that 
> explicitly limits the amount of allocations Lustre will perform.  On other 
> filesystems I've worked on there are memory pools that you can explicitly 
> size with maxes, and while these may be discrete between areas or reference 
> counters used to leverage a system-shared pool, I expected to see /something/ 
> that might bake in limits of some kind.  I'm sure I'm just not finding it.  
> Any help is greatly appreciated.
>
> Best,
>
> ellis
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org



-- 
--
Jeff Johnson
Co-Founder
Aeon Computing

jeff.john...@aeoncomputing.com
www.aeoncomputing.com
t: 858-412-3810 x1001   f: 858-412-3845
m: 619-204-9061

4170 Morena Boulevard, Suite C - San Diego, CA 92117

High-Performance Computing / Lustre Filesystems / Scale-out Storage
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] Memory Management in Lustre

[Ellis Wilson via lustre-discuss]

Hi folks,

Broader (but related) question than my current malaise with OOM issues on 
2.14/2.15:  Is there any documentation or can somebody point me at some code 
that explains memory management within Lustre?  I've hunted through Lustre 
manuals, the Lustre internals doc, and a bunch of code, but can find nothing 
that documents the memory architecture in place.  I'm specifically looking at 
PTLRPC and OBD code right now, and I can't seem to find anywhere that 
explicitly limits the amount of allocations Lustre will perform.  On other 
filesystems I've worked on there are memory pools that you can explicitly size 
with maxes, and while these may be discrete between areas or reference counters 
used to leverage a system-shared pool, I expected to see /something/ that might 
bake in limits of some kind.  I'm sure I'm just not finding it.  Any help is 
greatly appreciated.

Best,

ellis
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org