I ran the latest command you provided and it does not show the parameter, like you see. I can do screenshare.
[opc@lustre-client-1 ~]$ df -h Filesystem Size Used Avail Use% Mounted on /dev/sda3 39G 2.5G 36G 7% / devtmpfs 158G 0 158G 0% /dev tmpfs 158G 0 158G 0% /dev/shm tmpfs 158G 17M 158G 1% /run tmpfs 158G 0 158G 0% /sys/fs/cgroup /dev/sda1 512M 12M 501M 3% /boot/efi 10.0.3.6@tcp1:/lfsbv 50T 89M 48T 1% /mnt/mdt_bv 10.0.3.6@tcp1:/lfsnvme 185T 8.7M 176T 1% /mnt/mdt_nvme tmpfs 32G 0 32G 0% /run/user/1000 [opc@lustre-client-1 ~]$ lctl list_param -R llite | grep max_read_ahead [opc@lustre-client-1 ~]$ So I ran this: [opc@lustre-client-1 ~]$ lctl list_param -R llite > llite_parameters.txt There are other parameters under llite. I attached the complete list. From: "Moreno Diego (ID SIS)" <diego.mor...@id.ethz.ch> Date: Friday, December 13, 2019 at 8:36 AM To: Pinkesh Valdria <pinkesh.vald...@oracle.com>, "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> Subject: Re: [lustre-discuss] Degraded read performance with Large Bulk IO (16MB RPC) >From what I can see I think you just ran the wrong command (lctl list_param -R >* ) or it doesn’t work as you expected on 2.12.3. But llite params are sure there on a *mounted* Lustre client. This will give you the parameters you’re looking for and need to modify to have, likely, better read performance: lctl list_param -R llite | grep max_read_ahead From: Pinkesh Valdria <pinkesh.vald...@oracle.com> Date: Friday, 13 December 2019 at 17:33 To: "Moreno Diego (ID SIS)" <diego.mor...@id.ethz.ch>, "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> Subject: Re: [lustre-discuss] Degraded read performance with Large Bulk IO (16MB RPC) This is how I installed lustre clients (only showing packages installed steps). cat > /etc/yum.repos.d/lustre.repo << EOF [hpddLustreserver] name=CentOS- - Lustre baseurl=https://downloads.whamcloud.com/public/lustre/latest-release/el7/server/ gpgcheck=0 [e2fsprogs] name=CentOS- - Ldiskfs baseurl=https://downloads.whamcloud.com/public/e2fsprogs/latest/el7/ gpgcheck=0 [hpddLustreclient] name=CentOS- - Lustre baseurl=https://downloads.whamcloud.com/public/lustre/latest-release/el7/client/ gpgcheck=0 EOF yum install lustre-client -y reboot From: "Moreno Diego (ID SIS)" <diego.mor...@id.ethz.ch> Date: Friday, December 13, 2019 at 2:55 AM To: Pinkesh Valdria <pinkesh.vald...@oracle.com>, "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> Subject: Re: [lustre-discuss] Degraded read performance with Large Bulk IO (16MB RPC) >From what I can see they exist on my 2.12.3 client node: [root@rufus4 ~]# lctl list_param -R llite | grep max_read_ahead llite.reprofs-ffff9f7c3b4a8800.max_read_ahead_mb llite.reprofs-ffff9f7c3b4a8800.max_read_ahead_per_file_mb llite.reprofs-ffff9f7c3b4a8800.max_read_ahead_whole_mb Regards, Diego From: Pinkesh Valdria <pinkesh.vald...@oracle.com> Date: Wednesday, 11 December 2019 at 17:46 To: "Moreno Diego (ID SIS)" <diego.mor...@id.ethz.ch>, "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> Subject: Re: [lustre-discuss] Degraded read performance with Large Bulk IO (16MB RPC) I was not able to find those parameters on my client nodes, OSS or MGS nodes. Here is how I was extracting all parameters . mkdir -p lctl_list_param_R/ cd lctl_list_param_R/ lctl list_param -R * > lctl_list_param_R [opc@lustre-client-1 lctl_list_param_R]$ less lctl_list_param_R | grep ahead llite.lfsbv-ffff98231c3bc000.statahead_agl llite.lfsbv-ffff98231c3bc000.statahead_max llite.lfsbv-ffff98231c3bc000.statahead_running_max llite.lfsnvme-ffff98232c30e000.statahead_agl llite.lfsnvme-ffff98232c30e000.statahead_max llite.lfsnvme-ffff98232c30e000.statahead_running_max [opc@lustre-client-1 lctl_list_param_R]$ I also tried these commands: Not working: On client nodes lctl get_param llite.lfsbv-*.max_read_ahead_mb error: get_param: param_path 'llite/lfsbv-*/max_read_ahead_mb': No such file or directory [opc@lustre-client-1 lctl_list_param_R]$ Works On client nodes lctl get_param llite.*.statahead_agl llite.lfsbv-ffff98231c3bc000.statahead_agl=1 llite.lfsnvme-ffff98232c30e000.statahead_agl=1 [opc@lustre-client-1 lctl_list_param_R]$ From: "Moreno Diego (ID SIS)" <diego.mor...@id.ethz.ch> Date: Tuesday, December 10, 2019 at 2:06 AM To: Pinkesh Valdria <pinkesh.vald...@oracle.com>, "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> Subject: Re: [lustre-discuss] Degraded read performance with Large Bulk IO (16MB RPC) With that kind of degradation performance on read I would immediately think on llite’s max_read_ahead parameters on the client. Specifically these 2: max_read_ahead_mb: total amount of MB allocated for read ahead, usually quite low for bandwidth benchmarking purposes and when there’re several files per client max_read_ahead_per_file_mb: the default is quite low for 16MB RPCs (only a few RPCs per file) You probably need to check the effect increasing both of them. Regards, Diego From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of Pinkesh Valdria <pinkesh.vald...@oracle.com> Date: Tuesday, 10 December 2019 at 09:40 To: "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> Subject: [lustre-discuss] Degraded read performance with Large Bulk IO (16MB RPC) I was expecting better or same read performance with Large Bulk IO (16MB RPC), but I see degradation in performance. Do I need to tune any other parameter to benefit from Large Bulk IO? Appreciate if I can get any pointers to troubleshoot further. Throughput before - Read: 2563 MB/s - Write: 2585 MB/s Throughput after - Read: 1527 MB/s. (down by ~1025) - Write: 2859 MB/s Changes I did are: On oss - lctl set_param obdfilter.lfsbv-*.brw_size=16 On clients - unmounted and remounted - lctl set_param osc.lfsbv-OST*.max_pages_per_rpc=4096 (got auto-updated after re-mount) - lctl set_param osc.*.max_rpcs_in_flight=64 (Had to manually increase this to 64, since after re-mount, it was auto-set to 8, but read/write performance was poor) - lctl set_param osc.*.max_dirty_mb=2040. (setting the value to 2048 was failing with : Numerical result out of range error. Previously it was set to 2000 when I got good performance. My other settings: - lnetctl net add --net tcp1 --if $interface –peer-timeout 180 –peer-credits 128 –credits 1024 - echo "options ksocklnd nscheds=10 sock_timeout=100 credits=2560 peer_credits=63 enable_irq_affinity=0" > /etc/modprobe.d/ksocklnd.conf - lfs setstripe -c 1 -S 1M /mnt/mdt_bv/test1
llite llite.lfsbv-ffff98231c3bc000 llite.lfsbv-ffff98231c3bc000.blocksize llite.lfsbv-ffff98231c3bc000.checksum_pages llite.lfsbv-ffff98231c3bc000.checksums llite.lfsbv-ffff98231c3bc000.client_type llite.lfsbv-ffff98231c3bc000.default_easize llite.lfsbv-ffff98231c3bc000.fast_read llite.lfsbv-ffff98231c3bc000.filesfree llite.lfsbv-ffff98231c3bc000.filestotal llite.lfsbv-ffff98231c3bc000.fstype llite.lfsbv-ffff98231c3bc000.kbytesavail llite.lfsbv-ffff98231c3bc000.kbytesfree llite.lfsbv-ffff98231c3bc000.kbytestotal llite.lfsbv-ffff98231c3bc000.lazystatfs llite.lfsbv-ffff98231c3bc000.lmv llite.lfsbv-ffff98231c3bc000.lmv.activeobd llite.lfsbv-ffff98231c3bc000.lmv.desc_uuid llite.lfsbv-ffff98231c3bc000.lmv.numobd llite.lfsbv-ffff98231c3bc000.lmv.target_obds llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000 llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.active llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.blocksize llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.conn_uuid llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.contention_seconds llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.filesfree llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.filestotal llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.kbytesavail llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.kbytesfree llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.kbytestotal llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.max_mod_rpcs_in_flight llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.max_rpcs_in_flight llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.mds_conn_uuid llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.ping llite.lfsbv-ffff98231c3bc000.lmv.target_obds.lfsbv-MDT0000-mdc-ffff98231c3bc000.uuid llite.lfsbv-ffff98231c3bc000.lmv.uuid llite.lfsbv-ffff98231c3bc000.lov llite.lfsbv-ffff98231c3bc000.lov.activeobd llite.lfsbv-ffff98231c3bc000.lov.blocksize llite.lfsbv-ffff98231c3bc000.lov.desc_uuid llite.lfsbv-ffff98231c3bc000.lov.filesfree llite.lfsbv-ffff98231c3bc000.lov.filestotal llite.lfsbv-ffff98231c3bc000.lov.kbytesavail llite.lfsbv-ffff98231c3bc000.lov.kbytesfree llite.lfsbv-ffff98231c3bc000.lov.kbytestotal llite.lfsbv-ffff98231c3bc000.lov.numobd llite.lfsbv-ffff98231c3bc000.lov.stripecount llite.lfsbv-ffff98231c3bc000.lov.stripeoffset llite.lfsbv-ffff98231c3bc000.lov.stripetype llite.lfsbv-ffff98231c3bc000.lov.target_obds llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000 llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.active llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.blocksize llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.checksum_dump llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.checksums llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.conn_uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.contention_seconds llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.cur_dirty_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.cur_lost_grant_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.destroys_in_flight llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.filesfree llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.filestotal llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.grant_shrink llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.grant_shrink_interval llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.idle_connect llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.idle_timeout llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.kbytesavail llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.kbytesfree llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.kbytestotal llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.lockless_truncate llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.max_dirty_mb llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.max_rpcs_in_flight llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.ost_conn_uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.ping llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.resend_count llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.short_io_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST000f-osc-ffff98231c3bc000.uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000 llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.active llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.blocksize llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.checksum_dump llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.checksums llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.conn_uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.contention_seconds llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.cur_dirty_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.cur_lost_grant_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.destroys_in_flight llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.filesfree llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.filestotal llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.grant_shrink llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.grant_shrink_interval llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.idle_connect llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.idle_timeout llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.kbytesavail llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.kbytesfree llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.kbytestotal llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.lockless_truncate llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.max_dirty_mb llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.max_rpcs_in_flight llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.ost_conn_uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.ping llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.resend_count llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.short_io_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST001f-osc-ffff98231c3bc000.uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000 llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.active llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.blocksize llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.checksum_dump llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.checksums llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.conn_uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.contention_seconds llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.cur_dirty_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.cur_lost_grant_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.destroys_in_flight llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.filesfree llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.filestotal llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.grant_shrink llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.grant_shrink_interval llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.idle_connect llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.idle_timeout llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.kbytesavail llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.kbytesfree llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.kbytestotal llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.lockless_truncate llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.max_dirty_mb llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.max_rpcs_in_flight llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.ost_conn_uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.ping llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.resend_count llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.short_io_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST002f-osc-ffff98231c3bc000.uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000 llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.active llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.blocksize llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.checksum_dump llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.checksums llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.conn_uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.contention_seconds llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.cur_dirty_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.cur_lost_grant_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.destroys_in_flight llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.filesfree llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.filestotal llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.grant_shrink llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.grant_shrink_interval llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.idle_connect llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.idle_timeout llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.kbytesavail llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.kbytesfree llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.kbytestotal llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.lockless_truncate llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.max_dirty_mb llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.max_rpcs_in_flight llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.ost_conn_uuid llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.ping llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.resend_count llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.short_io_bytes llite.lfsbv-ffff98231c3bc000.lov.target_obds.lfsbv-OST003f-osc-ffff98231c3bc000.uuid llite.lfsbv-ffff98231c3bc000.lov.uuid llite.lfsbv-ffff98231c3bc000.max_easize llite.lfsbv-ffff98231c3bc000.stat_blocksize llite.lfsbv-ffff98231c3bc000.statahead_agl llite.lfsbv-ffff98231c3bc000.statahead_max llite.lfsbv-ffff98231c3bc000.statahead_running_max llite.lfsbv-ffff98231c3bc000.statfs_max_age llite.lfsbv-ffff98231c3bc000.stats_track_gid llite.lfsbv-ffff98231c3bc000.stats_track_pid llite.lfsbv-ffff98231c3bc000.stats_track_ppid llite.lfsbv-ffff98231c3bc000.tiny_write llite.lfsbv-ffff98231c3bc000.uuid llite.lfsbv-ffff98231c3bc000.xattr_cache llite.lfsnvme-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.checksum_pages llite.lfsnvme-ffff98232c30e000.checksums llite.lfsnvme-ffff98232c30e000.client_type llite.lfsnvme-ffff98232c30e000.default_easize llite.lfsnvme-ffff98232c30e000.fast_read llite.lfsnvme-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.fstype llite.lfsnvme-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lazystatfs llite.lfsnvme-ffff98232c30e000.lmv llite.lfsnvme-ffff98232c30e000.lmv.activeobd llite.lfsnvme-ffff98232c30e000.lmv.desc_uuid llite.lfsnvme-ffff98232c30e000.lmv.numobd llite.lfsnvme-ffff98232c30e000.lmv.target_obds llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.max_mod_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.mds_conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0000-mdc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.max_mod_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.mds_conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0001-mdc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.max_mod_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.mds_conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0002-mdc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.max_mod_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.mds_conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0003-mdc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.max_mod_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.mds_conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0004-mdc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.max_mod_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.mds_conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0005-mdc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.max_mod_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.mds_conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0006-mdc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.max_mod_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.mds_conn_uuid llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lmv.target_obds.lfsnvme-MDT0007-mdc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lmv.uuid llite.lfsnvme-ffff98232c30e000.lov llite.lfsnvme-ffff98232c30e000.lov.activeobd llite.lfsnvme-ffff98232c30e000.lov.blocksize llite.lfsnvme-ffff98232c30e000.lov.desc_uuid llite.lfsnvme-ffff98232c30e000.lov.filesfree llite.lfsnvme-ffff98232c30e000.lov.filestotal llite.lfsnvme-ffff98232c30e000.lov.kbytesavail llite.lfsnvme-ffff98232c30e000.lov.kbytesfree llite.lfsnvme-ffff98232c30e000.lov.kbytestotal llite.lfsnvme-ffff98232c30e000.lov.numobd llite.lfsnvme-ffff98232c30e000.lov.stripecount llite.lfsnvme-ffff98232c30e000.lov.stripeoffset llite.lfsnvme-ffff98232c30e000.lov.stripetype llite.lfsnvme-ffff98232c30e000.lov.target_obds llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.checksum_dump llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.checksums llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.cur_dirty_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.cur_lost_grant_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.destroys_in_flight llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.grant_shrink llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.grant_shrink_interval llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.idle_connect llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.idle_timeout llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.lockless_truncate llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.max_dirty_mb llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.ost_conn_uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.resend_count llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.short_io_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0007-osc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.checksum_dump llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.checksums llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.cur_dirty_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.cur_lost_grant_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.destroys_in_flight llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.grant_shrink llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.grant_shrink_interval llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.idle_connect llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.idle_timeout llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.lockless_truncate llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.max_dirty_mb llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.ost_conn_uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.resend_count llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.short_io_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST000f-osc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.checksum_dump llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.checksums llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.cur_dirty_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.cur_lost_grant_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.destroys_in_flight llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.grant_shrink llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.grant_shrink_interval llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.idle_connect llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.idle_timeout llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.lockless_truncate llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.max_dirty_mb llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.ost_conn_uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.resend_count llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.short_io_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST0017-osc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000 llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.active llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.blocksize llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.checksum_dump llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.checksums llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.conn_uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.contention_seconds llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.cur_dirty_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.cur_lost_grant_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.destroys_in_flight llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.filesfree llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.filestotal llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.grant_shrink llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.grant_shrink_interval llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.idle_connect llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.idle_timeout llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.kbytesavail llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.kbytesfree llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.kbytestotal llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.lockless_truncate llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.max_dirty_mb llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.max_rpcs_in_flight llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.ost_conn_uuid llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.ping llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.resend_count llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.short_io_bytes llite.lfsnvme-ffff98232c30e000.lov.target_obds.lfsnvme-OST001f-osc-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.lov.uuid llite.lfsnvme-ffff98232c30e000.max_easize llite.lfsnvme-ffff98232c30e000.stat_blocksize llite.lfsnvme-ffff98232c30e000.statahead_agl llite.lfsnvme-ffff98232c30e000.statahead_max llite.lfsnvme-ffff98232c30e000.statahead_running_max llite.lfsnvme-ffff98232c30e000.statfs_max_age llite.lfsnvme-ffff98232c30e000.stats_track_gid llite.lfsnvme-ffff98232c30e000.stats_track_pid llite.lfsnvme-ffff98232c30e000.stats_track_ppid llite.lfsnvme-ffff98232c30e000.tiny_write llite.lfsnvme-ffff98232c30e000.uuid llite.lfsnvme-ffff98232c30e000.xattr_cache
_______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
