Hey Christian, HDFS does not support the full set of POSIX semantics. I suspect bonnie++ is exercising things which HDFS doesn't support (such as writes to a file which is already closed or seeks during writes).
Additionally, I would warn that, as bonnie++ is designed to test the performance of harddrives / network file systems / cluster file systems, you will find it doesn't capture well the performance aspects of a distributed file system. In other words, you are performance testing an apple with a test suite designed for oranges. Brian On Jul 6, 2010, at 1:14 PM, Christian Baun wrote: > Hi, > > After some time, bonnie++ exits too with this error message: > > Can't write block.: Invalid argument > Bonnie: drastic I/O error (re write(2)): Invalid argument > [1]+ Exit 1 bonnie++ -s 3584 -d /hdfs/ -x5 -u hadoop -q -b > -m HDFS_test > /tmp/ausgabe.txt 2>&1 > > This looks not so good. :-( > > Best Regards, > Christian > > > Am Dienstag, 6. Juli 2010 schrieb Christian Baun: >> Hi, >> >> I installed a small Hadoop-Cluster with Cloudera CDH3. >> >> 2 Server are running. One is running the NameNode and DataNode. The other >> server is just a DataNode. >> A dedicated Client exists too. >> >> I want to run some performance tests with bonnie++ and iozone. >> bonnie++ works but when I start iozone, it interrups. >> >> Is this a known issue? >> >> $ mount | grep hdfs >> fuse_dfs on /hdfs type fuse.fuse_dfs >> (rw,nosuid,nodev,allow_other,default_permissions) >> >> $ iozone -Rac -g 3584M -f /hdfs/testfile | tee -a /tmp/iozone_results.txt & >> [1] 6283 >> had...@ip-10-243-13-191:/home/ubuntu$ Iozone: Performance Test of >> File I/O >> Version $Revision: 3.308 $ >> Compiled for 32 bit mode. >> Build: linux >> >> Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins >> Al Slater, Scott Rhine, Mike Wisner, Ken Goss >> Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, >> Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, >> Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, >> Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root. >> >> Run began: Tue Jul 6 16:40:35 2010 >> >> Excel chart generation enabled >> Auto Mode >> Include close in write timing >> Using maximum file size of 3670016 kilobytes. >> Command line used: iozone -Rac -g 3584M -f /hdfs/testfile >> Output is in Kbytes/sec >> Time Resolution = 0.000001 seconds. >> Processor cache size set to 1024 Kbytes. >> Processor cache line size set to 32 bytes. >> File stride size set to 17 * record size. >> random random >> bkwd record stride >> KB reclen write rewrite read reread read write >> read rewrite read fwrite frewrite fread freread >> 64 4write: Invalid argument >> >> Error writing block 0, fd= 3 >> >> iozone: interrupted >> >> >> Best Regards, >> Christian >> >
smime.p7s
Description: S/MIME cryptographic signature
