[lustre-discuss] DUE TODAY - LUG 2024 Call for Presentations

2024-03-11 Thread OpenSFS Administration via lustre-discuss 
Hello Lustre Community,

 

Today, March 11th (23:59, AoE "Anywhere on Earth") is the deadline to submit
an abstract for LUG 2024. We look forward to featuring technical
presentations from the Lustre Community! 

 

The Lustre User Group (LUG) conference is the high performance computing
industry's primary venue for discussion on the open source Lustre file
system and other technologies. The conference focuses on the latest Lustre
developments and allows attendees to network with peers.  

 

The 2024 Lustre User Group (LUG) conference
  will be held on May 6-9, 2024 at Texas Tech
University in Lubbock, Texas. LUG Registration is now open
 . 

 

Call for Presentations

 

The LUG Program Committee is particularly seeking presentations on:

- Experiences running the newer Lustre releases

- Feedback on existing Lustre features (Client Data Encryption, OST Pool
Quotas, DNEv3, FLR, PCC, SSK, DoM, Multi-Rail LNet, HSM, etc.)

- Upcoming Lustre features

- Best practices and practical experiences in deploying, monitoring, and
operating Lustre

- Pushing the boundaries with non-traditional deployments

- Performance & feature comparison with other filesystems

 

Submission Guidelines

 

You only need an abstract for the submission process; we will request
presentation materials once abstracts are reviewed and selected. Abstracts
should be a minimum of 250 words and should provide a clear description of
the planned presentation and its goals. All LUG presentations will be 30
minutes (including questions).  All presentations must be original and not
simultaneously submitted to another conference. The abstract should mention
the branch and version of Lustre used (or derived from).

 

The submission details web page for LUG 2024 is available at:
https://easychair.org/cfp/LUG2024
 

 

You will need to create a user account on EasyChair if you don't already
have one. https://easychair.org/account/signin
 

 

Interested in sponsoring LUG? Please contact OpenSFS Administration
  for sponsorship options and pricing.

 

We look forward to seeing you at LUG 2024!

 

The LUG 2024 Program Committee

 ---

 

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] mkfs.lustre Undefined symbol: zfs_enable_quota

2024-03-11 Thread damir chanyshev 

Hello!

Looks like you are using a dev build, long story short these builds only 
for testing purposes.


Regarding issue, check this ticket 
https://jira.whamcloud.com/browse/LU-17611 and patch


https://review.whamcloud.com/c/fs/lustre-release/+/54293/


11/03/2024 13:08, Jacob Schwartz via lustre-discuss пишет:

Hi,
My name is Jacob and I work with Mobileye.
After successfully compiling lustre 2.15.61 with openzfs 2.1.15, I 
encountered the following problem while running mkfs.lustre:
mkfs.lustre FATAL: /usr/lib64/lustre/mount_osd_zfs.so: undefined 
symbol: zfs_enable_quota
I checked the zfs symbols in the library 
/usr/lib64/lustre/mount_osd_zfs.so and I could not find the symbol 
zfs_enable_quota

nm -D /usr/lib64/lustre/mount_osd_zfs.so|grep zfs
              U zfs_destroy
20b0 T zfs_fini
               U zfs_get_pool_handle
               U zfs_get_user_props
               U zfs_name_valid
               U zfs_open
002051c0 D zfs_ops
               U zfs_prop_inherit
               U zfs_prop_set
               U zpool_close
               U zpool_get_prop_int
               U zpool_open
               U zpool_set_propcd

The OS I am working with is AlmaLinux8.9 (kernel version 
4.18.0-513.5.1.el8_9.x86_64).


OpenZFS 2.1.15 was compiles as follow:
cd zfs-2.1.15
 sh autogen.sh
 ./configure --with-spec=redhat
 make -j1 rpm-utils rpm-kmod

Lustre 2.15.61 was compiled as follow:
sh ./autogen.sh
./configure --enable-server --enable-modules --disable-ldiskfs 
--disable-gss-keyring --with-mds-max-threads=2048 --enable-quota 
--with-zfs=/usr/src/zfs-2.1.15 --with-o2ib=yes

make rpms

Can someone please assist.
Thanks,
Jacob.

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


--
Thanks,
Damir.

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

[lustre-discuss] mkfs.lustre Undefined symbol: zfs_enable_quota

2024-03-11 Thread Jacob Schwartz via lustre-discuss 
Hi,
My name is Jacob and I work with Mobileye.
After successfully compiling lustre 2.15.61 with openzfs 2.1.15, I encountered 
the following problem while running mkfs.lustre:
mkfs.lustre FATAL: /usr/lib64/lustre/mount_osd_zfs.so: undefined symbol: 
zfs_enable_quota
I checked the zfs symbols in the library /usr/lib64/lustre/mount_osd_zfs.so and 
I could not find the symbol zfs_enable_quota
nm -D /usr/lib64/lustre/mount_osd_zfs.so|grep zfs
U zfs_destroy
20b0 T zfs_fini
 U zfs_get_pool_handle
 U zfs_get_user_props
 U zfs_name_valid
 U zfs_open
002051c0 D zfs_ops
 U zfs_prop_inherit
 U zfs_prop_set
 U zpool_close
 U zpool_get_prop_int
 U zpool_open
 U zpool_set_propcd

The OS I am working with is AlmaLinux8.9 (kernel version 
4.18.0-513.5.1.el8_9.x86_64).

OpenZFS 2.1.15 was compiles as follow:
cd zfs-2.1.15
 sh autogen.sh
 ./configure --with-spec=redhat
 make -j1 rpm-utils rpm-kmod

Lustre 2.15.61 was compiled as follow:
sh ./autogen.sh
./configure --enable-server --enable-modules --disable-ldiskfs 
--disable-gss-keyring --with-mds-max-threads=2048 --enable-quota 
--with-zfs=/usr/src/zfs-2.1.15 --with-o2ib=yes
make rpms

Can someone please assist.
Thanks,
Jacob.
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] The confusion for mds hardware requirement

2024-03-11 Thread Andreas Dilger via lustre-discuss 
All of the numbers in this example are estimates/approximations to give an idea 
about the amount of memory that the MDS may need under normal operating 
circumstances.  However, the MDS will also continue to function with more or 
less memory.  The actual amount of memory in use will change very significantly 
based on application type, workload, etc. and the numbers "256" and "100,000" 
are purely examples of how many files might be in use.

I'm not sure you can "test" those numbers, because whatever number of files you 
test with will be the number of files actually in use.  You could potentially 
_measure_ the number of files/locks in use on a large cluster, but again this 
will be highly site and application dependent.

Cheers, Andreas

On Mar 11, 2024, at 01:24, Amin Brick Mover 
mailto:aminbrickmo...@gmail.com>> wrote:

Hi,  Andreas.

Thank you for your reply.

Can I consider 256 files per core as an empirical parameter? And does the 
parameter '256' need testing based on hardware conditions? Additionally, in the 
calculation formula "12 interactive clients * 100,000 files * 2KB = 2400 MB," 
is the number '100,000' files also an empirical parameter? Do I need to test 
it. Can I directly use the values '256' and '100,000'?

Andreas Dilger mailto:adil...@whamcloud.com>> 
于2024年3月11日周一 05:47写道：
These numbers are just estimates, you can use values more suitable to your 
workload.

Similarly, 32-core clients may be on the low side these days.  NVIDIA DGX nodes 
have 256 cores, though you may not have 1024 of them.

The net answer is that having 64GB+ of RAM is inexpensive these days and 
improves MDS performance, especially if you compare it to the cost of client 
nodes that would sit waiting for filesystem access if the MDS is short of RAM.  
Better to have too much RAM on the MDS than too little.

Cheers, Andreas

On Mar 4, 2024, at 00:56, Amin Brick Mover via lustre-discuss 
mailto:lustre-discuss@lists.lustre.org>> wrote:

In the Lustre Manual 5.5.2.1 section, the examples mentioned:
For example, for a single MDT on an MDS with 1,024 compute nodes, 12 
interactive login nodes, and a
20 million file working set (of which 9 million files are cached on the clients 
at one time):
Operating system overhead = 4096 MB (RHEL8)
File system journal = 4096 MB
1024 * 32-core clients * 256 files/core * 2KB = 16384 MB
12 interactive clients * 100,000 files * 2KB = 2400 MB
20 million file working set * 1.5KB/file = 30720 MB
I'm curious, how were the two numbers, 256 files/core and 100,000 files, 
determined? Why?

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud








Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] The confusion for mds hardware requirement

2024-03-11 Thread Amin Brick Mover via lustre-discuss 
Hi,  Andreas.

Thank you for your reply.

Can I consider 256 files per core as an empirical parameter? And does the
parameter '256' need testing based on hardware conditions? Additionally, in
the calculation formula "12 interactive clients * 100,000 files * 2KB =
2400 MB," is the number '100,000' files also an empirical parameter? Do I
need to test it. Can I directly use the values '256' and '100,000'?

Andreas Dilger  于2024年3月11日周一 05:47写道：

> These numbers are just estimates, you can use values more suitable to your
> workload.
>
> Similarly, 32-core clients may be on the low side these days.  NVIDIA DGX
> nodes have 256 cores, though you may not have 1024 of them.
>
> The net answer is that having 64GB+ of RAM is inexpensive these days and
> improves MDS performance, especially if you compare it to the cost of
> client nodes that would sit waiting for filesystem access if the MDS is
> short of RAM.  Better to have too much RAM on the MDS than too little.
>
> Cheers, Andreas
>
> On Mar 4, 2024, at 00:56, Amin Brick Mover via lustre-discuss <
> lustre-discuss@lists.lustre.org> wrote:
>
> In the Lustre Manual 5.5.2.1 section, the examples mentioned:
>
> *For example, for a single MDT on an MDS with 1,024 compute nodes, 12
> interactive login nodes, and a*
> *20 million file working set (of which 9 million files are cached on the
> clients at one time):*
> *Operating system overhead = 4096 MB (RHEL8)*
> *File system journal = 4096 MB*
> *1024 * 32-core clients * 256 files/core * 2KB = 16384 MB*
> *12 interactive clients * 100,000 files * 2KB = 2400 MB*
> *20 million file working set * 1.5KB/file = 30720 MB*
>
> I'm curious, how were the two numbers, 256 files/core and 100,000 files,
> determined? Why?
>
> ___
> lustre-discuss mailing list
> lustre-discuss@lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Whamcloud
>
>
>
>
>
>
>
>
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org