My xfs inode size is 265 Bytes.
When I increase the memory and change vm.vfs_cache_pressure to 1, is it
possible to store the inode tree in the memory?
Maybe the random disk seeks are the problem.
My disks are all Hitachi SATA 3 or 4 TB with 7.200 rpm.
The movement of 1250 partitions should go faster then 2-3 weeks, even if there
are more then 20.000 objects stored in each partition. In our case that's only
2,5 GB in each partition.
Here is a iostat snapshot:
avg-cpu: %user %nice %system %iowait %steal %idle
16.78 0.00 19.75 8.28 0.00 55.19
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz
avgqu-sz await r_await w_await svctm %util
sda 0.00 1.40 10.20 2.80 220.00 115.20 51.57
0.10 7.51 9.57 0.00 7.26 9.44
sdb 0.00 0.00 16.40 3.60 300.00 187.20 48.72
0.18 8.84 10.78 0.00 7.08 14.16
sdc 0.00 0.00 16.00 4.80 303.20 273.40 55.44
0.30 14.19 18.45 0.00 8.88 18.48
sdd 0.00 0.20 494.80 137.60 3693.60 1074.30 15.08
0.12 0.19 0.23 0.05 0.10 6.32
sde 0.00 0.00 143.80 2.60 1028.80 123.90 15.75
1.15 7.84 7.97 0.62 6.49 95.04
sdf 0.00 0.00 11.40 1.00 270.40 42.40 50.45
0.14 11.29 12.28 0.00 8.06 10.00
sdg 0.00 0.00 16.60 5.40 226.40 316.10 49.32
0.23 10.51 13.78 0.44 8.25 18.16
sdh 0.00 0.00 94.80 3.80 681.60 156.70 17.00
1.27 12.51 13.00 0.21 9.89 97.52
sdi 0.00 0.00 43.40 4.60 1741.60 210.30 81.33
1.15 24.70 27.32 0.00 11.18 53.68
sdj 0.00 1.20 17.00 135.40 194.40 1482.30 22.00
0.67 4.39 18.26 2.65 1.42 21.68
sdk 0.00 0.00 26.20 4.20 1382.40 183.30 103.01
0.24 7.84 9.04 0.38 7.34 22.32
sdl 0.00 1.20 14.40 139.80 223.20 779.10 13.00
0.55 3.58 12.67 2.64 0.98 15.04
-----Ursprüngliche Nachricht-----
Von: Robert van Leeuwen [mailto:[email protected]]
Gesendet: Dienstag, 10. Februar 2015 12:23
An: Klaus Schürmann; '[email protected]'
Betreff: RE: [Openstack] [SWIFT] Bad replication performance after adding new
drives
> I set the vfs_cache_pressure to 10 and moved container- and account-server to
> SSD harddrives.
> The normal performance for object writes and reads are quite ok.
> But why takes moving some partions to only two new harddisks so much time?
> Will it be faster if I add more memory?
My guess: Probably the source disks/servers are slow.
When the inode tree is not in memory it will do a lot of random reads to the
disks (for both the inode tree and the actual file).
An rsync of any directory will become slow on the source side ( iirc you can
see this in the replicator log) You should be able to see in e.g. atop if the
source or destination disks are the limiting factor.
If the source is the issue it might help to increase the maximum number of
simultaneous rsync processes so you have more parallel slow processes ;) Note
that this can have impact on the general speed of the Swift cluster.
More memory will probably help a bit.
Cheers,
Robert
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : [email protected]
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
