Hi All. I'm trying to configure lustre CFS on 2 servers (1 MDS, 1 OST - both SUSE enterprise 10). Everything seems to be fine, it's startup and I could see /mnt/lustre mounted properly. However when I try to copy file after few seconds it says "No space left on device", even so there is a bunch of free space.
Here is what I have on those servers: foma - MDS, susen - OST -------------------------------------------------------------- susen:/mnt/lustre # ls bigfile info2.txt test.info susen:/mnt/lustre # ls -la total 80280 drwxr-xr-x 2 root root 4096 2007-03-21 11:18 . drwxr-xr-x 5 root root 120 2007-03-21 11:15 .. -rw-r--r-- 1 root root 82108416 2007-03-21 11:18 bigfile -rw-r--r-- 1 root root 14 2007-03-21 11:17 info2.txt -rw-r--r-- 1 root root 197 2007-03-21 11:16 test.info susen:/mnt/lustre # susen:/mnt/lustre # susen:/mnt/lustre # dd of=/mnt/lustre/bigfile2 if=/dev/zero bs=1048576 dd: writing /mnt/lustre/bigfile2': No space left on device 81+0 records in 80+0 records out 84201472 bytes (84 MB) copied, 0.89681 seconds, 93.9 MB/s susen:/mnt/lustre # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda2 74G 3.7G 70G 5% / udev 500M 108K 500M 1% /dev foma:/mds-test/foma 275M 172M 89M 66% /mnt/lustre susen:/mnt/lustre # dd of=/mnt/lustre/bigfile3 if=/dev/zero bs=1048576 dd: writing /mnt/lustre/bigfile3': No space left on device 81+0 records in 80+0 records out 84205568 bytes (84 MB) copied, 0.930681 seconds, 90.5 MB/s susen:/mnt/lustre # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda2 74G 3.8G 70G 6% / udev 500M 108K 500M 1% /dev foma:/mds-test/foma 275M 252M 8.1M 97% /mnt/lustre ----------------------------------------------------------------- >From the log file: Mar 21 11:39:37 susen syslog-ng[1972]: STATS: dropped 0 Mar 21 11:40:29 susen kernel: LustreError: 3416:0:(client.c:576:ptlrpc_check_status()) @@@ type == PTL_RPC_MSG_ERR, err == -28 [EMAIL PROTECTED] x1272/t0 o4->o [EMAIL PROTECTED]:28 lens 328/288 ref 2 fl Rpc:R/0/0 rc 0/-28 Mar 21 11:40:29 susen kernel: LustreError: 3416:0:(client.c:576:ptlrpc_check_status()) Skipped 1 previous similar message Mar 21 11:40:29 susen kernel: LustreError: 3416:0:(client.c:576:ptlrpc_check_status()) @@@ type == PTL_RPC_MSG_ERR, err == -28 [EMAIL PROTECTED] x1273/t0 o4->o [EMAIL PROTECTED]:28 lens 328/288 ref 2 fl Rpc:R/0/0 rc 0/-28 Mar 21 11:40:40 susen kernel: LustreError: 3416:0:(client.c:576:ptlrpc_check_status()) @@@ type == PTL_RPC_MSG_ERR, err == -28 [EMAIL PROTECTED] x1422/t0 o4->o [EMAIL PROTECTED]:28 lens 328/288 ref 2 fl Rpc:R/0/0 rc 0/-28 Here is my config: #!/bin/sh # config.sh # SuSen - MDS,OTS1 and client1 # Foma - OTS2 and client2 # Create nodes rm -f config.xml lmc -m config.xml --add node --node susen lmc -m config.xml --add net --node susen --nid susen --nettype tcp lmc -m config.xml --add node --node foma lmc -m config.xml --add net --node foma --nid foma --nettype tcp #lmc -m config.xml --add net --node client --nid '*' --nettype tcp # Cofigure MDS lmc -m config.xml --add mds --node foma --mds mds-test --fstype ext3 --dev /tmp/mds-test --size 50000 # Configures OSTs lmc -m config.xml --add lov --lov lov-test --mds mds-test --stripe_sz 1048576 --stripe_cnt 0 --stripe_pattern 0 lmc -m config.xml --add ost --node susen --lov lov-test --ost ost1-test --fstype ext3 --dev /tmp/ost1-test --size 100000 lmc -m config.xml --add ost --node susen --lov lov-test --ost ost2-test --fstype ext3 --dev /tmp/ost2-test --size 100000 lmc -m config.xml --add ost --node susen --lov lov-test --ost ost3-test --fstype ext3 --dev /tmp/ost3-test --size 100000 # Configure client (this is a 'generic' client used for all client mounts) lmc -m config.xml --add mtpt --node foma --path /mnt/lustre --mds mds-test --lov lov-test Any suggestion ? _______________________________________________ Lustre-discuss mailing list [email protected] https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
