Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-05-01 Thread Christian Balzer
Levesque aleves...@gtcomm.net À: Christian Balzer ch...@gol.com Cc: ceph-users ceph-users@lists.ceph.com Envoyé: Vendredi 24 Avril 2015 22:00:44 Objet: Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals) Hi Christian, We tested some DC

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-30 Thread Christian Balzer
Levesque aleves...@gtcomm.net À: Christian Balzer ch...@gol.com Cc: ceph-users ceph-users@lists.ceph.com Envoyé: Vendredi 24 Avril 2015 22:00:44 Objet: Re: [ceph-users] Possible improvements for a slow write speed(excluding independent SSD journals) Hi Christian, We tested

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-29 Thread Anthony Levesque
. - Mail original - De: Anthony Levesque aleves...@gtcomm.net À: Christian Balzer ch...@gol.com Cc: ceph-users ceph-users@lists.ceph.com Envoyé: Vendredi 24 Avril 2015 22:00:44 Objet: Re: [ceph-users] Possible improvements for a slow write speed(excluding independent SSD journals

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-29 Thread Christian Balzer
: Anthony Levesque aleves...@gtcomm.net À: Christian Balzer ch...@gol.com Cc: ceph-users ceph-users@lists.ceph.com Envoyé: Vendredi 24 Avril 2015 22:00:44 Objet: Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals) Hi Christian, We

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-26 Thread Andrei Mikhailovsky
-users ceph-users@lists.ceph.com Sent: Saturday, 25 April, 2015 5:32:30 PM Subject: Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals) I'm able to reach around 2-25000iops with 4k block with s3500 (with o_dsync) (so yes, around 80-100MB/S). I'l

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-25 Thread Christian Balzer
Objet: Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals) Hi Christian, We tested some DC S3500 300GB using dd if=randfile of=/dev/sda bs=4k count=10 oflag=direct,dsync we got 96 MB/s which is far from the 315 MB/s from the website

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-25 Thread Alexandre DERUMIER
Envoyé: Vendredi 24 Avril 2015 22:00:44 Objet: Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals) Hi Christian, We tested some DC S3500 300GB using dd if=randfile of=/dev/sda bs=4k count=10 oflag=direct,dsync we got 96 MB/s which is far from

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-24 Thread Anthony Levesque
Hi Christian, We tested some DC S3500 300GB using dd if=randfile of=/dev/sda bs=4k count=10 oflag=direct,dsync we got 96 MB/s which is far from the 315 MB/s from the website. Can I ask you or anyone on the mailing list how you are testing the write speed for journals? Thanks --- Anthony

[ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-23 Thread Anthony Levesque
To update you on the current test in our lab: 1.We tested the Samsung OSD in Recovery mode and the speed was able to maxout 2x 10GbE port(transferring data at 2200+ MB/s during recovery). So for normal write operation without O_DSYNC writes Samsung drives seem ok. 2.We then tested a couple of

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-23 Thread Christian Balzer
Hello, On Thu, 23 Apr 2015 18:40:38 -0400 Anthony Levesque wrote: To update you on the current test in our lab: 1.We tested the Samsung OSD in Recovery mode and the speed was able to maxout 2x 10GbE port(transferring data at 2200+ MB/s during recovery). So for normal write operation

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-21 Thread Eneko Lacunza
Hi, I'm just writing to you to stress out what others have already said, because it is very important that you take it very seriously. On 20/04/15 19:17, J-P Methot wrote: On 4/20/2015 11:01 AM, Christian Balzer wrote: This is similar to another thread running right now, but since our

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-21 Thread Mark Nelson
*From: *Eneko Lacunza elacu...@binovo.es *To: *J-P Methot jpmet...@gtcomm.net, Christian Balzer ch...@gol.com, ceph-users@lists.ceph.com *Sent: *Tuesday, 21 April, 2015 8:18:20 AM *Subject: *Re: [ceph-users] Possible improvements for a slow write speed (excluding independent

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-21 Thread J-P Methot
*From: *Eneko Lacunza elacu...@binovo.es *To: *J-P Methot jpmet...@gtcomm.net, Christian Balzer ch...@gol.com, ceph-users@lists.ceph.com *Sent: *Tuesday, 21 April, 2015 8:18:20 AM *Subject: *Re: [ceph-users] Possible improvements for a slow write speed (excluding

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-21 Thread Andrei Mikhailovsky
: Tuesday, 21 April, 2015 8:18:20 AM Subject: Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals) Hi, I'm just writing to you to stress out what others have already said, because it is very important that you take it very seriously. On 20/04/15 19:17

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-21 Thread Alex Moore
*From: *Eneko Lacunza elacu...@binovo.es *To: *J-P Methot jpmet...@gtcomm.net, Christian Balzer ch...@gol.com, ceph-users@lists.ceph.com *Sent: *Tuesday, 21 April, 2015 8:18:20 AM *Subject: *Re: [ceph-users] Possible improvements for a slow write

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-20 Thread Christian Balzer
On Mon, 20 Apr 2015 13:17:18 -0400 J-P Methot wrote: On 4/20/2015 11:01 AM, Christian Balzer wrote: Hello, On Mon, 20 Apr 2015 10:30:41 -0400 J-P Methot wrote: Hi, This is similar to another thread running right now, but since our current setup is completely different from the

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-20 Thread Barclay Jameson
Are your journals on separate disks? What is your ratio of journal disks to data disks? Are you doing replication size 3 ? On Mon, Apr 20, 2015 at 9:30 AM, J-P Methot jpmet...@gtcomm.net wrote: Hi, This is similar to another thread running right now, but since our current setup is completely

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-20 Thread J-P Methot
My journals are on-disk, each disk being a SSD. The reason I didn't go with dedicated drives for journals is that when designing the setup, I was told that having dedicated journal SSDs on a full-SSD setup would not give me performance increases. So that makes the journal disk to data disk

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-20 Thread Mark Nelson
The big question is how fast these drives can do O_DSYNC writes. The basic gist of this is that for every write to the journal, an ATA_CMD_FLUSH call is made to ensure that the device (or potentially the controller) know that this data really needs to be stored safely before the flush is

[ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-20 Thread J-P Methot
Hi, This is similar to another thread running right now, but since our current setup is completely different from the one described in the other thread, I thought it may be better to start a new one. We are running Ceph Firefly 0.80.8 (soon to be upgraded to 0.80.9). We have 6 OSD hosts

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-20 Thread Alexandre DERUMIER
original - De: J-P Methot jpmet...@gtcomm.net À: ceph-users ceph-users@lists.ceph.com Envoyé: Lundi 20 Avril 2015 16:30:41 Objet: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals) Hi, This is similar to another thread running right now, but since our

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-20 Thread Christian Balzer
Hello, On Mon, 20 Apr 2015 10:30:41 -0400 J-P Methot wrote: Hi, This is similar to another thread running right now, but since our current setup is completely different from the one described in the other thread, I thought it may be better to start a new one. We are running Ceph

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-20 Thread Barclay Jameson
Using rados benchmark. It's just a test pool anyway. I will stick with my current OSD setup (16HDDs 4 SSDs for a 1:4 ration of SSD to HDD). I can get 800 MB/s write and about 1GB read. On Mon, Apr 20, 2015 at 11:19 AM, Mark Nelson mnel...@redhat.com wrote: How are you measuring the 300MB/s and

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-20 Thread Barclay Jameson
I have a SSD pool for testing (only 8 Drives) but when I do a 1 SSD with journal and 1 SSD with Data I get 300 MB/s write. When I change all 8 Disks to house the journal I get 184MB/s write. On Mon, Apr 20, 2015 at 10:16 AM, Mark Nelson mnel...@redhat.com wrote: The big question is how fast

Re: [ceph-users] Possible improvements for a slow write speed (excluding independent SSD journals)

2015-04-20 Thread Mark Nelson
How are you measuring the 300MB/s and 184MB/s? IE is it per drive, or the client throughput? Also what controller do you have? We've seen some controllers from certain manufacturers start to top out at around 1-2GB/s with write cache enabled. Mark On 04/20/2015 11:15 AM, Barclay Jameson