On Tue, Jan 19, 2016 at 12:03:24PM +1000, Piers Rowan wrote:
> sda = a usb backup drive
> dm-0 = / (MySQL) RAID
> dm-1 = / (MySQL) RAID
>
> dm-2-5 = /home (also where VM's Live) [1 x hot spare]
1. what kind of disks are these?
can you run:
find /dev/disk/by-id/ -iname ata-* -o -iname usb-* |
grep -v -- -part |
while read disk ; do
echo $(basename $(readlink $disk)) $(basename $disk)
done | \
sed -re "s/(usb|ata)-// ; s/(SATA|Generic)_//" |
sort
this will show the brand and model of all disks in the system like this:
$ list_disks
sda WDC_WD10EACS-00ZJB0_WD-WCASJ2114122
sdb ST4000DX001-1CE168_Z303PTHA
sdc ST4000DX001-1CE168_Z302ZSGB
sdd ST4000DX001-1CE168_Z303PVZ9
sde WDC_WD10EACS-00ZJB0_WD-WCASJ2195141
sdf WDC_WD10EARS-00Y5B1_WD-WMAV50933036
sdg ST31000528AS_9VP18CCV
sdh OCZ-VECTOR_OCZ-0974C023I4P2G1B8
sdi ST4000DX001-1CE168_Z303PSH6
sdj OCZ-VECTOR_OCZ-8RL5XW08536INH7R
2. is /home RAID-5? i'm guessing it is since RAID-10 with 3 drives and
a hot-spare doesn't make any sense. RAID-5 can be dreadfully slow,
especially on random writes.
what kind of virtual disks are you using for the VMś? cow2 image files?
raw or lvm partitions? partitions are much faster than qcow2 files.
3. is /home where users' mail lives?
or does dovecot move it ~user/Mail or similar from /var/mail?
Do the users use their /home directories for anything else? samba or
nfs exports for example? would you class that as light or heavy usage?
> iostat -x 10
> Linux 2.6.32-573.7.1.el6.x86_64 19/01/16 _x86_64_ (12 CPU)
>
> avg-cpu: %user %nice %system %iowait %steal %idle
> 5.27 0.00 1.74 3.98 0.00 89.01
>
> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz
> avgqu-sz await svctm %util
> sda 144.40 398.34 92.54 95.71 5790.73 4493.15 54.63
> 0.32 1.71 1.38 25.92
so you're running a backup at the moment? is that when the slowdown occurs, or
does it happen any old time?
what kind of backup software? rsync?
> dm-0 0.00 0.00 154.36 81.59 1234.90 652.68 8.00
> 0.04 0.16 0.44 10.43
> dm-1 0.00 0.00 154.36 81.59 1234.90 652.68 8.00
> 0.04 0.17 0.44 10.44
mysql drives seem fine.
> dm-2 0.00 0.00 0.00 0.00 0.00 0.00 8.00
> 0.00 3.36 3.07 0.00
> dm-3 0.00 0.00 0.00 0.00 0.00 0.00 8.00
> 0.00 0.00 0.00 0.00
> dm-4 0.00 0.00 9.63 8.24 77.01 65.93 8.00
> 0.22 12.41 1.52 2.72
> dm-5 0.00 0.00 73.20 404.46 4478.78 3774.52 17.28
> 0.24 0.27 0.46 21.75
it's odd that most of the I/O is on just one of these /home drives.
craig
--
craig sanders <c...@taz.net.au>
BOFH excuse #94:
Internet outage
_______________________________________________
luv-main mailing list
luv-main@luv.asn.au
http://lists.luv.asn.au/listinfo/luv-main
