Hi! I've just experienced a disk crash in my ceph cluster. This seems to have caused an error in the kernel since every I/O command sent to any mapped RBD is defered indefinitely, even an hour later, the OSD is already taken out of the cluster and ceph -s says HEALTH_OK.
A kernel dump is provided at this pastebin link: http://pastebin.com/WMqmiUsM What did I do prior the crash? I mapped one clean rbd device at /dev/rbd0 and one rbd device that had a partition and ext4 filesystem at 20GB each I started to dd rbd1 to rbd0 (dd if=/dev/rbd1 of=/dev/rbd0 bs=4M) It copied a few GB, then deep-scrub started and at that moment one of the disks failed. and dd got stuck (disk defered). And it still is. I think that only a reboot of the server can fix this problem now. This is not normal behavior when a disk crashes, right? I'm running the linux kernel 3.15.7 and are using layered rbd devices. Thank you for your time. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
