Hi Cephers,
On Sunday evening we are upgraded Ceph form 0.87 to 0.94. After upgrade VM's
running on Proxmox, freezes for 3-4s in 10min periods (application is not
responding on Windows). Before upgrade everything was working fine. On
/proc/diskstats at field 7 (time spent reading (ms) ) and 11 (time spent
writing (ms)) there are peaks from 90ms to 2000ms on osd disks.
Is there any default settings changed? Every server have one ssd for journal
and 4-6 osd. BBU are ok on controllers. Our ceph.conf below.
[global]
fsid={UUID}
mon initial members = ceph35, ceph30, ceph20, ceph15, ceph10
mon host = 10.20.8.35, 10.20.8.30, 10.20.8.20, 10.20.8.15, 10.20.8.10
public network = 10.20.8.0/22
cluster network = 10.20.4.0/22
filestore xattr use omap = true
filestore max sync interval = 30
osd journal size = 10240
osd mount options xfs = "rw,noatime,inode64,allocsize=4M"
osd pool default size = 3
osd pool default min size = 1
osd pool default pg num = 2048
osd pool default pgp num = 2048
osd disk thread ioprio class = idle
osd disk thread ioprio priority = 7
osd crush chooseleaf type = 1
osd recovery max active = 1
osd recovery op priority = 1
osd max backfills = 1
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
rbd default format = 2
##ceph35 osds
[osd.0]
cluster addr = 10.20.4.35
[osd.1]
cluster addr = 10.20.4.35
[osd.2]
cluster addr = 10.20.4.35
[osd.3]
cluster addr = 10.20.4.35
[osd.4]
cluster addr = 10.20.4.35
[osd.5]
cluster addr = 10.20.4.35
##ceph25 osds
[osd.6]
cluster addr = 10.20.4.25
[osd.7]
cluster addr = 10.20.4.25
[osd.8]
cluster addr = 10.20.4.25
[osd.9]
cluster addr = 10.20.4.25
[osd.10]
cluster addr = 10.20.4.25
[osd.11]
cluster addr = 10.20.4.25
##ceph15 osds
[osd.12]
cluster addr = 10.20.4.15
[osd.13]
cluster addr = 10.20.4.15
[osd.14]
cluster addr = 10.20.4.15
[osd.15]
cluster addr = 10.20.4.15
##ceph30 osds
[osd.16]
cluster addr = 10.20.4.30
[osd.17]
cluster addr = 10.20.4.30
[osd.18]
cluster addr = 10.20.4.30
[osd.19]
cluster addr = 10.20.4.30
[osd.20]
cluster addr = 10.20.4.30
[osd.21]
cluster addr = 10.20.4.30
##ceph20 osds
[osd.22]
cluster addr = 10.20.4.20
[osd.23]
cluster addr = 10.20.4.20
[osd.24]
cluster addr = 10.20.4.20
[osd.25]
cluster addr = 10.20.4.20
[osd.26]
cluster addr = 10.20.4.20
[osd.27]
cluster addr = 10.20.4.20
##ceph10 osd
[osd.28]
cluster addr = 10.20.4.10
[osd.29]
cluster addr = 10.20.4.10
[osd.30]
cluster addr = 10.20.4.10
[osd.31]
cluster addr = 10.20.4.10
#adresy monitorów
[mon.ceph35]
host = ceph35
mon addr = 10.20.8.35:6789
[mon.ceph30]
host = ceph30
mon addr = 10.20.8.30:6789
[mon.ceph20]
host = ceph20
mon addr = 10.20.8.20:6789
[mon.ceph15]
host = ceph15
mon addr = 10.20.8.15:6789
[mon.ceph10]
host = ceph10
mon addr = 10.20.8.10:6789
Thanks for help.
Regards
Mateusz
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
