Hi,
I am currently running a ceph cluster running in CEPHFS with 3 nodes each
have 6 osd's except 1 who got 5. I got 3 mds : 2 active and 1 standby, 3
mon.
[root@ceph-n1 ~]# ceph -s
cluster:
id: 1d97aa70-2029-463a-b6fa-20e98f3e21fb
health: HEALTH_WARN
3 clients failing to respond to capability release
2 MDSs report slow requests
services:
mon: 3 daemons, quorum ceph-n1,ceph-n2,ceph-n3
mgr: ceph-n1(active), standbys: ceph-n2, ceph-n3
mds: cephfs-2/2/2 up {0=ceph-n1=up:active,1=ceph-n2=up:active}, 1
up:standby
osd: 17 osds: 17 up, 17 in
data:
pools: 2 pools, 1024 pgs
objects: 541k objects, 42006 MB
usage: 143 GB used, 6825 GB / 6969 GB avail
pgs: 1024 active+clean
io:
client: 32980 B/s rd, 77295 B/s wr, 5 op/s rd, 14 op/s wr
I'm using the cephFs as a mail storage. I currently have 3500 mailbox some
of them are IMAP the others are POP3 the goal is to be able to migrate all
mailbox from my old
infrastructure so around 30 000 mailbox.
I'm now facing a problem :
MDS_CLIENT_LATE_RELEASE 3 clients failing to respond to capability release
mdsceph-n1(mds.0): Client mda3.sogetel.net failing to respond to
capability releaseclient_id: 1134426
mdsceph-n1(mds.0): Client mda2.sogetel.net failing to respond to
capability releaseclient_id: 1172391
mdsceph-n2(mds.1): Client mda3.sogetel.net failing to respond to
capability releaseclient_id: 1134426
MDS_SLOW_REQUEST 2 MDSs report slow requests
mdsceph-n1(mds.0): 112 slow requests are blocked > 30 sec
mdsceph-n2(mds.1): 323 slow requests are blocked > 30 sec
I can't figure out how to fix this.
Here some information's about my cluster :
I'm running ceph luminous 12.2.5 on my 3 ceph nodes : ceph-n1, ceph-n2,
ceph-n3.
I have 3 client identical :
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: Fedora
Description: Fedora release 25 (Twenty Five)
Release: 25
Codename: TwentyFive
My ceph nodes :
CentOS Linux release 7.5.1804 (Core)
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/";
BUG_REPORT_URL="https://bugs.centos.org/";
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
CentOS Linux release 7.5.1804 (Core)
CentOS Linux release 7.5.1804 (Core)
ceph daemon mds.ceph-n1 perf dump mds :
"mds": {
"request": 21968558,
"reply": 21954801,
"reply_latency": {
"avgcount": 21954801,
"sum": 100879.560315258,
"avgtime": 0.004594874
},
"forward": 13627,
"dir_fetch": 3327,
"dir_commit": 162830,
"dir_split": 1,
"dir_merge": 0,
"inode_max": 2147483647,
"inodes": 68767,
"inodes_top": 4524,
"inodes_bottom": 56697,
"inodes_pin_tail": 7546,
"inodes_pinned": 62304,
"inodes_expired": 1640159,
"inodes_with_caps": 62192,
"caps": 114126,
"subtrees": 14,
"traverse": 38309963,
"traverse_hit": 37606227,
"traverse_forward": 12189,
"traverse_discover": 6634,
"traverse_dir_fetch": 1769,
"traverse_remote_ino": 6,
"traverse_lock": 7731,
"load_cent": 2196856701,
"q": 0,
"exported": 143,
"exported_inodes": 291372,
"imported": 125,
"imported_inodes": 176509
Thanks for your help.
Regards
Marc-Antoine
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
