I'm certainly not Andreas. That said... You're running an MPI simulation, assumingly across most or all of your 34 compute nodes. Lustre server operations, their lnet activity and the backend storage I/O will create a profound imbalance on the few compute nodes you designate to do both server and client operation. That and you expose yourself to deadlocks and other potentials mentioned earlier. I do not know how performant your login server is, but depending on the file operations of your simulations you could cavitate your login server. Also, you generally don't want users logging in on a node as critical as an MDS.
You would be better served by allocating two of your compute nodes to just be Lustre servers, one mds/oss, the other an oss and run 32 clean client nodes. More stable, clean and in the end probably more workflow productivity over time. Fewer technical incidents. Just my opinion...others may differ. --Jeff On Fri, Oct 13, 2023 at 12:43 PM Fedele Stabile < fedele.stab...@fis.unical.it> wrote: > I believe in Linux is possible to limit the memory used by a user and also > it is possible to limit the amount of cpu used so I can limit resources for > group user and also if i put oss server in a vm i suppose i can limit cpu > and memory usage. > My scenario is: i have 34 compute nodes 512 GB RAM and 34 HD 16 TB each > that I can arrange in 9 nodes, i have also a management node that can be > used for LUSTRE metadata server, infiniband is 200 Gb/s > We make mhd simulations. > What Lustre configuration do you suggest? > > ------------------------------ > *Da:* Andreas Dilger <adil...@whamcloud.com> > *Inviato:* Venerdì, Ottobre 13, 2023 7:19:11 PM > *A:* Fedele Stabile <fedele.stab...@fis.unical.it> > *Cc:* lustre-discuss@lists.lustre.org <lustre-discuss@lists.lustre.org> > *Oggetto:* Re: [lustre-discuss] OSS on compute node > > On Oct 13, 2023, at 20:58, Fedele Stabile <fedele.stab...@fis.unical.it> > wrote: > > > Hello everyone, > We are in progress to integrate Lustre on our little HPC Cluster and we > would like to know if it is possible to use the same node in a cluster to > act as an OSS with disks and to also use it as a Compute Node and then > install a Lustre Client. > I know that the OSS server require a modified kernel so I suppose it can > be installed in a virtual machine using kvm on a compute node. > > > There isn't really a problem with running a client + OSS on the same node > anymore, nor is there a problem with an OSS running inside a VM (if you > have SR-IOV and enough CPU+RAM to run the server). > > *HOWEVER*, I don't think it would be good to have the client mounted on > the *VM host*, and then run the OSS on a *VM guest*. That could lead to > deadlocks and priority inversion if the client becomes busy, but depends on > the local OSS to flush dirty data from RAM and the OSS cannot run in the VM > because it doesn't have any RAM... > > If the client and OSS are BOTH run in VMs, or neither run in VMs, or only > the client run in a VM, then that should be OK, but may have reduced > performance due to the server contending with the client application. > > Cheers, Andreas > -- > Andreas Dilger > Lustre Principal Architect > Whamcloud > > > > > > > > > _______________________________________________ > lustre-discuss mailing list > lustre-discuss@lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > -- ------------------------------ Jeff Johnson Co-Founder Aeon Computing jeff.john...@aeoncomputing.com www.aeoncomputing.com t: 858-412-3810 x1001 f: 858-412-3845 m: 619-204-9061 4170 Morena Boulevard, Suite C - San Diego, CA 92117 High-Performance Computing / Lustre Filesystems / Scale-out Storage
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
