This is my article :). To patch to the kernel (http://www.theirek.com/downloads/code/CMD_FLUSH.diff ). After rebooting, run the following commands: echo temporary write through > /sys/class/scsi_disk/<disk>/cache_type
2014-04-28 15:44 GMT+04:00 Indra Pramana <[email protected]>: > Hi Irek, > > Thanks for the article. Do you have any other web sources pertaining to > the same issue, which is in English? > > Looking forward to your reply, thank you. > > Cheers. > > > On Mon, Apr 28, 2014 at 7:40 PM, Irek Fasikhov <[email protected]> wrote: > >> Most likely you need to apply a patch to the kernel. >> >> >> http://www.theirek.com/blog/2014/02/16/patch-dlia-raboty-s-enierghoniezavisimym-keshiem-ssd-diskov >> >> >> 2014-04-28 15:20 GMT+04:00 Indra Pramana <[email protected]>: >> >> Hi Udo and Irek, >>> >>> Good day to you, and thank you for your emails. >>> >>> >>> >perhaps due IOs from the journal? >>> >You can test with iostat (like "iostat -dm 5 sdg"). >>> >>> Yes, I have shared the iostat result earlier on this same thread. At >>> times the utilisation of the 2 journal drives will hit 100%, especially >>> when I simulate writing data using rados bench command. Any suggestions >>> what could be the cause of the I/O issue? >>> >>> >>> ==== >>> avg-cpu: %user %nice %system %iowait %steal %idle >>> 1.85 0.00 1.65 3.14 0.00 93.36 >>> >>> >>> Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s >>> avgrq-sz avgqu-sz await r_await w_await svctm %util >>> sdg 0.00 0.00 0.00 55.00 0.00 25365.33 >>> 922.38 34.22 568.90 0.00 568.90 17.82 98.00 >>> sdf 0.00 0.00 0.00 55.67 0.00 25022.67 >>> 899.02 29.76 500.57 0.00 500.57 17.60 98.00 >>> >>> >>> avg-cpu: %user %nice %system %iowait %steal %idle >>> 2.10 0.00 1.37 2.07 0.00 94.46 >>> >>> >>> Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s >>> avgrq-sz avgqu-sz await r_await w_await svctm %util >>> sdg 0.00 0.00 0.00 56.67 0.00 25220.00 >>> 890.12 23.60 412.14 0.00 412.14 17.62 99.87 >>> sdf 0.00 0.00 0.00 52.00 0.00 24637.33 >>> 947.59 33.65 587.41 0.00 587.41 19.23 100.00 >>> >>> >>> avg-cpu: %user %nice %system %iowait %steal %idle >>> 2.21 0.00 1.77 6.75 0.00 89.27 >>> >>> >>> Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s >>> avgrq-sz avgqu-sz await r_await w_await svctm %util >>> sdg 0.00 0.00 0.00 54.33 0.00 24802.67 >>> 912.98 25.75 486.36 0.00 486.36 18.40 100.00 >>> sdf 0.00 0.00 0.00 53.00 0.00 24716.00 >>> 932.68 35.26 669.89 0.00 669.89 18.87 100.00 >>> >>> >>> avg-cpu: %user %nice %system %iowait %steal %idle >>> 1.87 0.00 1.67 5.25 0.00 91.21 >>> >>> >>> Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s >>> avgrq-sz avgqu-sz await r_await w_await svctm %util >>> sdg 0.00 0.00 0.00 94.33 0.00 26257.33 >>> 556.69 18.29 208.44 0.00 208.44 10.50 99.07 >>> sdf 0.00 0.00 0.00 51.33 0.00 24470.67 >>> 953.40 32.75 684.62 0.00 684.62 19.51 100.13 >>> >>> >>> avg-cpu: %user %nice %system %iowait %steal %idle >>> 1.51 0.00 1.34 7.25 0.00 89.89 >>> >>> >>> Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s >>> avgrq-sz avgqu-sz await r_await w_await svctm %util >>> sdg 0.00 0.00 0.00 52.00 0.00 22565.33 >>> 867.90 24.73 446.51 0.00 446.51 19.10 99.33 >>> sdf 0.00 0.00 0.00 64.67 0.00 24892.00 >>> 769.86 19.50 330.02 0.00 330.02 15.32 99.07 >>> ==== >>> >>> >You what model SSD? >>> >>> For this one, I am using Seagate 100GB SSD, model: HDS-2TM-ST100FM0012 >>> >>> >Which version of the kernel? >>> >>> Ubuntu 13.04, Linux kernel version: 3.8.0-19-generic #30-Ubuntu SMP Wed >>> May 1 16:35:23 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux >>> >>> Looking forward to your reply, thank you. >>> >>> Cheers. >>> >>> >>> >>> On Mon, Apr 28, 2014 at 4:45 PM, Irek Fasikhov <[email protected]>wrote: >>> >>>> You what model SSD? >>>> Which version of the kernel? >>>> >>>> >>>> >>>> 2014-04-28 12:35 GMT+04:00 Udo Lembke <[email protected]>: >>>> >>>>> Hi, >>>>> perhaps due IOs from the journal? >>>>> You can test with iostat (like "iostat -dm 5 sdg"). >>>>> >>>>> on debian iostat is in the package sysstat. >>>>> >>>>> Udo >>>>> >>>>> Am 28.04.2014 07:38, schrieb Indra Pramana: >>>>> > Hi Craig, >>>>> > >>>>> > Good day to you, and thank you for your enquiry. >>>>> > >>>>> > As per your suggestion, I have created a 3rd partition on the SSDs >>>>> and did >>>>> > the dd test directly into the device, and the result is very slow. >>>>> > >>>>> > ==== >>>>> > root@ceph-osd-08:/mnt# dd bs=1M count=128 if=/dev/zero of=/dev/sdg3 >>>>> > conv=fdatasync oflag=direct >>>>> > 128+0 records in >>>>> > 128+0 records out >>>>> > 134217728 bytes (134 MB) copied, 19.5223 s, 6.9 MB/s >>>>> > >>>>> > root@ceph-osd-08:/mnt# dd bs=1M count=128 if=/dev/zero of=/dev/sdf3 >>>>> > conv=fdatasync oflag=direct >>>>> > 128+0 records in >>>>> > 128+0 records out >>>>> > 134217728 bytes (134 MB) copied, 5.34405 s, 25.1 MB/s >>>>> > ==== >>>>> > >>>>> > I did a test onto another server with exactly similar specification >>>>> and >>>>> > similar SSD drive (Seagate SSD 100 GB) but not added into the >>>>> cluster yet >>>>> > (thus no load), and the result is fast: >>>>> > >>>>> > ==== >>>>> > root@ceph-osd-09:/home/indra# dd bs=1M count=128 if=/dev/zero >>>>> of=/dev/sdf1 >>>>> > conv=fdatasync oflag=direct >>>>> > 128+0 records in >>>>> > 128+0 records out >>>>> > 134217728 bytes (134 MB) copied, 0.742077 s, 181 MB/s >>>>> > ==== >>>>> > >>>>> > Is the Ceph journal load really takes up a lot of the SSD resources? >>>>> I >>>>> > don't understand how come the performance can drop significantly. >>>>> > Especially since the two Ceph journals are only taking the first 20 >>>>> GB out >>>>> > of the 100 GB of the SSD total capacity. >>>>> > >>>>> > Any advice is greatly appreciated. >>>>> > >>>>> > Looking forward to your reply, thank you. >>>>> > >>>>> > Cheers. >>>>> > >>>>> > >>>>> > >>>>> _______________________________________________ >>>>> ceph-users mailing list >>>>> [email protected] >>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>> >>>> >>>> >>>> >>>> -- >>>> С уважением, Фасихов Ирек Нургаязович >>>> Моб.: +79229045757 >>>> >>> >>> >> >> >> -- >> С уважением, Фасихов Ирек Нургаязович >> Моб.: +79229045757 >> > > -- С уважением, Фасихов Ирек Нургаязович Моб.: +79229045757
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
