I am evaluating ceph on a cluster of three machines. Each time I try to
copy a 4.6G file using the kernel client (2.6.24.4) all ceph data
becomes corrupted.
The following messages are found in the log:
080923.115611.857658 ebofs.oc.rx_finish bad csum wanted f8692852eac0562d
got 0 for object block 1017
080923.115611.857678 ebofs.oc.rx_finish bad csum wanted ec3261c5d13c5be2
got 0 for object block 1018
080923.115611.857698 ebofs.oc.rx_finish bad csum wanted deeaf1ca5338d346
got 0 for object block 1019
080923.115611.857717 ebofs.oc.rx_finish bad csum wanted a7dd1efcd77f52ec
got 0 for object block 1020
080923.115611.857737 ebofs.oc.rx_finish bad csum wanted 70590fbcb3c89879
got 0 for object block 1021
080923.115611.857756 ebofs.oc.rx_finish bad csum wanted c4b32d796070269b
got 0 for object block 1022
080923.115611.857776 ebofs.oc.rx_finish bad csum wanted a342f2b87c2bab44
got 0 for object block 1023
080923.115611.857790 ebofs.oc.rx_finish bad csum in
0/0/10000000007.0000000d over 0~32
080923.115611.857856 ebofs.oc.rx_finish rx -> corrupt on
bufferhead(992~32 v0/0 rx 0xab78770)
Here is the process to reproduce the failure:
I have the ceph/src directory nfs mounted on nodes x11 and x12 from x10.
x10 ./monmaptool --create --clobber --add 10.1.28.10:12345 --print
.ceph_monmap
x10 ./mkmonfs --clobber mondata/mon0 --mon 0 --monmap .ceph_monmap
x10 ./cmon mondata/mon0 -d --debug_mon 10 --debug_ms 1
x10 ./osdmaptool --clobber --createsimple .ceph_monmap 3 --print
.ceph_osdmap
x10 ./cmonctl osd setmap -i .ceph_osdmap
x10 ./cosd --mkfs_for_osd 0 /dev/md0
x10 ./cosd /dev/md0 -d --debug_osd 10
x11 ./cosd --mkfs_for_osd 1 /dev/md0
x11 ./cosd /dev/md0 -d --debug_osd 10
x12 ./cosd --mkfs_for_osd 2 /dev/sda9
x12 ./cosd /dev/sda9 -d --debug_osd 10
x10 ./cmds --debug_ms 1 --debug_mds 10 -d
x10 mount -t ceph 10.1.28.10:/ /mnt/ceph
x11 mount -t ceph 10.1.28.10:/ /mnt/ceph
x11 echo 11 > /mnt/ceph/11
x11 cat /mnt/ceph/11
11
x12 mount -t ceph 10.1.28.10:/ /mnt/ceph
x12 echo 12 > /mnt/ceph/12
x12 cat /mnt/ceph/12
12
x11 dd if=mandriva-linux-2008.0-pwp-dvd-i586.iso of=/mnt/ceph/11.iso
bs=1024k
4383+1 records in
4383+1 records out
x12 dd if=mandriva-linux-2008.0-pwp-dvd-i586.iso of=/mnt/ceph/12.iso
bs=1024k
4383+1 records in
4383+1 records out
# diff /mnt/ceph/11.iso /mnt/ceph/12.iso
Files /mnt/ceph/11.iso and /mnt/ceph/12.iso differ
# cat /mnt/ceph/11
c
# cat /mnt/ceph/12
w?
# dd if=/mnt/ceph/11.iso of=/dev/null bs=1024k
dd: reading `/mnt/ceph/11.iso': Input/output error
71+1 records in
71+1 records out
The 11.iso and 12.iso are expected to be the same and the 11 and 12
files created earlier should contain 11 and 12.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Ceph-devel mailing list
Ceph-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ceph-devel
