Hi,
I rebooted a failed server, which is now showing a rogue filesystem mount.
Actually, there were also several disks missing in the node, all reported as
"prepared" by ceph-disk, but not activated.
[root@ceph2 ~]# grep /var/lib/ceph/tmp /etc/mtab
/dev/sdo1 /var/lib/ceph/tmp/mnt.usVRe8 xfs rw,noatime,attr2,inode64,noquota 0 0
This path does not exist, and after having to "ceph-disk activate-all", I can
now see the OSD under it's correct path (and missing ones got mounted too) :
[root@ceph2 ~]# grep sdo1 /etc/mtab
/dev/sdo1 /var/lib/ceph/tmp/mnt.usVRe8 xfs rw,noatime,attr2,inode64,noquota 0 0
/dev/sdo1 /var/lib/ceph/osd/ceph-53 xfs rw,noatime,attr2,inode64,noquota 0 0
[root@ceph2 ~]# ll /var/lib/ceph/tmp/mnt.usVRe8
ls: cannot access /var/lib/ceph/tmp/mnt.usVRe8: No such file or directory
I just looked at the logs, and it appears that this sdo disk performed an XFS
recovery at boot :
Mar 2 11:33:45 ceph2 kernel: [ 21.479747] XFS (sdo1): Mounting Filesystem
Mar 2 11:33:45 ceph2 kernel: [ 21.641263] XFS (sdo1): Starting recovery
(logdev: internal)
Mar 2 11:33:45 ceph2 kernel: [ 21.674451] XFS (sdo1): Ending recovery
(logdev: internal)
I do not see any "Ending clean mount" line for this disk.
If I check the syslogs, I can see OSDs are usually mounted twice, but not
always, and sometimes they even aren't mounted at all :
[root@ceph2 ~]# zegrep 'XFS.*Ending clean' /var/log/messages.1.gz |sed -e
s/.*XFS/XFS/|sort |uniq -c
2 XFS (sdb1): Ending clean mount
2 XFS (sdd1): Ending clean mount
1 XFS (sde1): Ending clean mount
2 XFS (sdg1): Ending clean mount
1 XFS (sdh1): Ending clean mount
1 XFS (sdi1): Ending clean mount
2 XFS (sdj1): Ending clean mount
3 XFS (sdk1): Ending clean mount
3 XFS (sdl1): Ending clean mount
4 XFS (sdm1): Ending clean mount
So : would there be an issue with disks that perform an XFS recovery at boot ?
I know that reboot will cleanup things, but rebooting isn't the cleanest thing
to do...
Thanks
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
