Hallo All, a question whats happening here: We are on GPFS 5.0.1.1 and host a TSM-Server-Cluster. A colleague from me want to add new nsd’s to grow its tsm-storagepool (filedevice class volumes). The tsmpool fs has before 45TB of space after that 128TB. We create new 50 GB tsm-volumes with define volume cmd, but the cmd goes in error after the allocating of 89TB. Following Outputs here: [root@node_a tsmpool]# df -hT Filesystem Type Size Used Avail Use% Mounted on tsmpool gpfs 128T 128T 44G 100% /gpfs/tsmpool
root@node_a tsmpool]# mmdf tsmpool --block-size auto disk disk size failure holds holds free free name group metadata data in full blocks in fragments --------------- ------------- -------- -------- ----- -------------------- ------------------- Disks in storage pool: system (Maximum disk size allowed is 839.99 GB) nsd_r2g8f_tsmpool_001 100G 0 Yes No 88G ( 88%) 10.4M ( 0%) nsd_c4g8f_tsmpool_001 100G 1 Yes No 88G ( 88%) 10.4M ( 0%) nsd_g4_tsmpool 256M 2 No No 0 ( 0%) 0 ( 0%) ------------- -------------------- ------------------- (pool total) 200.2G 176G ( 88%) 20.8M ( 0%) Disks in storage pool: data01 (Maximum disk size allowed is 133.50 TB) nsd_r2g8d_tsmpool_016 8T 0 No Yes 3.208T ( 40%) 7.867M ( 0%) nsd_r2g8d_tsmpool_015 8T 0 No Yes 3.205T ( 40%) 7.867M ( 0%) nsd_r2g8d_tsmpool_014 8T 0 No Yes 3.208T ( 40%) 7.867M ( 0%) nsd_r2g8d_tsmpool_013 8T 0 No Yes 3.206T ( 40%) 7.867M ( 0%) nsd_r2g8d_tsmpool_012 8T 0 No Yes 3.208T ( 40%) 7.867M ( 0%) nsd_r2g8d_tsmpool_011 8T 0 No Yes 3.205T ( 40%) 7.867M ( 0%) nsd_r2g8d_tsmpool_001 8T 0 No Yes 1.48G ( 0%) 14.49M ( 0%) nsd_r2g8d_tsmpool_002 8T 0 No Yes 1.582G ( 0%) 16.12M ( 0%) nsd_r2g8d_tsmpool_003 8T 0 No Yes 1.801G ( 0%) 14.7M ( 0%) nsd_r2g8d_tsmpool_004 8T 0 No Yes 1.629G ( 0%) 15.21M ( 0%) nsd_r2g8d_tsmpool_005 8T 0 No Yes 1.609G ( 0%) 14.22M ( 0%) nsd_r2g8d_tsmpool_006 8T 0 No Yes 1.453G ( 0%) 17.4M ( 0%) nsd_r2g8d_tsmpool_010 8T 0 No Yes 3.208T ( 40%) 7.867M ( 0%) nsd_r2g8d_tsmpool_009 8T 0 No Yes 3.197T ( 40%) 7.867M ( 0%) nsd_r2g8d_tsmpool_007 8T 0 No Yes 3.194T ( 40%) 7.875M ( 0%) nsd_r2g8d_tsmpool_008 8T 0 No Yes 3.195T ( 40%) 7.867M ( 0%) nsd_c4g8d_tsmpool_016 8T 1 No Yes 3.195T ( 40%) 7.867M ( 0%) nsd_c4g8d_tsmpool_006 8T 1 No Yes 888M ( 0%) 21.63M ( 0%) nsd_c4g8d_tsmpool_005 8T 1 No Yes 996M ( 0%) 18.22M ( 0%) nsd_c4g8d_tsmpool_004 8T 1 No Yes 920M ( 0%) 11.21M ( 0%) nsd_c4g8d_tsmpool_003 8T 1 No Yes 984M ( 0%) 14.7M ( 0%) nsd_c4g8d_tsmpool_002 8T 1 No Yes 1.082G ( 0%) 11.89M ( 0%) nsd_c4g8d_tsmpool_001 8T 1 No Yes 1.035G ( 0%) 14.49M ( 0%) nsd_c4g8d_tsmpool_007 8T 1 No Yes 3.281T ( 41%) 7.867M ( 0%) nsd_c4g8d_tsmpool_008 8T 1 No Yes 3.199T ( 40%) 7.867M ( 0%) nsd_c4g8d_tsmpool_009 8T 1 No Yes 3.195T ( 40%) 7.867M ( 0%) nsd_c4g8d_tsmpool_010 8T 1 No Yes 3.195T ( 40%) 7.867M ( 0%) nsd_c4g8d_tsmpool_011 8T 1 No Yes 3.195T ( 40%) 7.867M ( 0%) nsd_c4g8d_tsmpool_012 8T 1 No Yes 3.195T ( 40%) 7.867M ( 0%) nsd_c4g8d_tsmpool_013 8T 1 No Yes 3.195T ( 40%) 7.867M ( 0%) nsd_c4g8d_tsmpool_014 8T 1 No Yes 3.195T ( 40%) 7.875M ( 0%) nsd_c4g8d_tsmpool_015 8T 1 No Yes 3.194T ( 40%) 7.867M ( 0%) ------------- -------------------- ------------------- (pool total) 256T 64.09T ( 25%) 341.6M ( 0%) ============= ==================== =================== (data) 256T 64.09T ( 25%) 341.6M ( 0%) (metadata) 200G 176G ( 88%) 20.8M ( 0%) ============= ==================== =================== (total) 256.2T 64.26T ( 25%) 362.4M ( 0%) In GPFS we had already space but the above df seems to be wrong and that make TSM unhappy. If we manually wrote a 50GB File in this FS like: [root@sap00733 tsmpool]# dd if=/dev/zero of=/gpfs/tsmpool/output bs=2M count=25600 25600+0 records in 25600+0 records out 53687091200 bytes (54 GB) copied, 30.2908 s, 1.8 GB/s We see at df level now these: [root@sap00733 tsmpool]# df -hT Filesystem Type Size Used Avail Use% Mounted on tsmpool gpfs 128T 96T 33T 75% /gpfs/tsmpool if we delete these file we see already the first output of 44G free space only. This seems to be the os df Interface seems to be brocken here. What I also must mentioned we use some ignore parameters: root @node_a(rhel7.4)> mmfsadm dump config |grep ignore ignoreNonDioInstCount 0 ! ignorePrefetchLUNCount 1 ignoreReplicaSpaceOnStat 0 ignoreReplicationForQuota 0 ! ignoreReplicationOnStatfs 1 ignoreSync 0 the fs has the -S relatime option. Are there any Known bug here existend ? Any hints on that? Regards Renar Renar Grunenberg Abteilung Informatik – Betrieb HUK-COBURG Bahnhofsplatz 96444 Coburg Telefon: 09561 96-44110 Telefax: 09561 96-44104 E-Mail: renar.grunenb...@huk-coburg.de Internet: www.huk.de ________________________________ HUK-COBURG Haftpflicht-Unterstützungs-Kasse kraftfahrender Beamter Deutschlands a. G. in Coburg Reg.-Gericht Coburg HRB 100; St.-Nr. 9212/101/00021 Sitz der Gesellschaft: Bahnhofsplatz, 96444 Coburg Vorsitzender des Aufsichtsrats: Prof. Dr. Heinrich R. Schradin. Vorstand: Klaus-Jürgen Heitmann (Sprecher), Stefan Gronbach, Dr. Hans Olav Herøy, Dr. Jörg Rheinländer (stv.), Sarah Rössler, Daniel Thomas. ________________________________ Diese Nachricht enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese Nachricht irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Nachricht. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Nachricht ist nicht gestattet. This information may contain confidential and/or privileged information. If you are not the intended recipient (or have received this information in error) please notify the sender immediately and destroy this information. Any unauthorized copying, disclosure or distribution of the material in this information is strictly forbidden. ________________________________
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss