Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Dimitri Maziuk
On 1/15/2014 9:16 AM, Mark Nelson wrote: On 01/15/2014 09:14 AM, Alexandre DERUMIER wrote: For the system disk, do you use some kind of internal flash memory disk ? We probably should have, but ended up with I think just a 500GB 7200rpm disk, whatever was cheapest. :) If your system has to

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Mark Nelson
On 01/15/2014 09:35 AM, Cedric Lemarchand wrote: Le 15/01/2014 16:25, Mark Nelson a écrit : On 01/15/2014 09:22 AM, Cedric Lemarchand wrote: Hello guys, What about arm hardware ? did someone already use some like Viridis ? http://www.boston.co.uk/solutions/viridis/viridis-4u.aspx Afaik, th

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Cedric Lemarchand
Le 15/01/2014 16:25, Mark Nelson a écrit : On 01/15/2014 09:22 AM, Cedric Lemarchand wrote: Hello guys, What about arm hardware ? did someone already use some like Viridis ? http://www.boston.co.uk/solutions/viridis/viridis-4u.aspx Afaik, the boston solution was based on Calxeda gear... ht

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Cedric Lemarchand
ceph-users@lists.ceph.com Envoyé: Mercredi 15 Janvier 2014 15:40:12 Objet: Re: [ceph-users] servers advise (dell r515 or supermicro ) On 01/15/2014 08:04 AM, Alexandre DERUMIER wrote: We are using Supermicro 2uTwin nodes. These have 2 nodes in 2u with each 12 disks. We use X9DRT-HF+ mainboards, 2

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Mark Nelson
: "Mark Nelson" À: ceph-users@lists.ceph.com Envoyé: Mercredi 15 Janvier 2014 15:40:12 Objet: Re: [ceph-users] servers advise (dell r515 or supermicro ) On 01/15/2014 08:04 AM, Alexandre DERUMIER wrote: We are using Supermicro 2uTwin nodes. These have 2 nodes in 2u with each 12 disk

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Mark Nelson
to see how well these do! Mark - Mail original - De: "Mark Nelson" À: ceph-users@lists.ceph.com Envoyé: Mercredi 15 Janvier 2014 15:40:12 Objet: Re: [ceph-users] servers advise (dell r515 or supermicro ) On 01/15/2014 08:04 AM, Alexandre DERUMIER wrote: We are using

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Alexandre DERUMIER
hat cpu frequency/number of core do you use ? For the system disk, do you use some kind of internal flash memory disk ? - Mail original - De: "Mark Nelson" À: ceph-users@lists.ceph.com Envoyé: Mercredi 15 Janvier 2014 15:40:12 Objet: Re: [ceph-users] servers advise (dell r

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Sebastien Han
However you have to get > 480GB which ridiculously large for a journal. I believe they are pretty expensive too. Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien@enovance.com Address : 10, rue de la Victoire

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Mark Nelson
On 01/15/2014 08:50 AM, Sebastien Han wrote: However you have to get > 480GB which ridiculously large for a journal. I believe they are pretty expensive too. Looks like the M500 in 480GB capacity is around $300 on amazon right now VS about $300 for a 200GB DC S3700. The M500 has more capacit

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Michael J. Kidd
actually, they're very inexpensive as far as SSD's go. The 960gb m500 can be had on Amazon for $499 US on prime (as of yesterday anyway). Sent from my mobile device. Please excuse brevity and typographical errors. On Jan 15, 2014 9:50 AM, "Sebastien Han" wrote: > However you have to get > 480G

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Stefan Priebe
Am 15.01.2014 15:50, schrieb Sebastien Han: However you have to get > 480GB which ridiculously large for a journal. I believe they are pretty expensive too. that's correct it just them instead of SATA or SAS disks in ceph ;-) so 960GB makes sense. Sébastien Han Cloud Engineer "Alwa

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Stefan Priebe
The S3700 does not need stuff like this. It internally ignores flushes. Also there is an upstream one: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=39c60a0948cc06139e2fbfe084f83cb7e7deae3b Stefan Am 15.01.2014 15:47, schrieb Ирек Фасихов: Kernel Patch for In

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Ирек Фасихов
Kernel Patch for Intel S3700, Intel 530... diff --git a/drivers/scsi/sd.c b/drivers//scsi/sd.c --- a/drivers/scsi/sd.c 2013-09-14 12:53:21.0 +0400 +++ b/drivers//scsi/sd.c2013-12-19 21:43:29.0 +0400 @@ -137,6 +137,7 @@ char *buffer_data; struct scsi_mode_

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Sebastien Han
Sorry I was only looking at the 4K aligned results. Sébastien Han Cloud Engineer "Always give 100%. Unless you're giving blood.” Phone: +33 (0)1 49 70 99 72 Mail: sebastien@enovance.com Address : 10, rue de la Victoire - 75009 Paris Web : www.enovance.com - Twitter : @enovance

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Mark Nelson
On 01/15/2014 08:39 AM, Stefan Priebe wrote: Am 15.01.2014 15:34, schrieb Sebastien Han: Hum the Crucial m500 is pretty slow. The biggest one doesn’t even reach 300MB/s. Intel DC S3700 100G showed around 200MB/sec for us. where did you get this values from? I've some 960GB and they all have >

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Stefan Priebe
Am 15.01.2014 15:44, schrieb Mark Nelson: On 01/15/2014 08:39 AM, Stefan Priebe wrote: Am 15.01.2014 15:34, schrieb Sebastien Han: Hum the Crucial m500 is pretty slow. The biggest one doesn’t even reach 300MB/s. Intel DC S3700 100G showed around 200MB/sec for us. where did you get this value

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Mark Nelson
ata center said the thing weighs about 260lbs and hangs out the back of the rack. :D Mark - Mail original - De: "Robert van Leeuwen" À: "Alexandre DERUMIER" , ceph-users@lists.ceph.com Envoyé: Mercredi 15 Janvier 2014 14:36:32 Objet: RE: [ceph-users] servers a

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Ирек Фасихов
I use H700 on Dell R815, 4 nodes. No problem performance. Configuration: 1 SSD Intel 530 - OS and Journal. 5 OSD HDD 600G: certified DELL - WD/HITACHI/SEAGATE. Size replication=2. Iops ~ 4k no VM. 15 янв. 2014 г. 15:47 пользователь "Alexandre DERUMIER" написал: > > Hello List, > > I'm going to bu

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Stefan Priebe
Am 15.01.2014 15:34, schrieb Sebastien Han: Hum the Crucial m500 is pretty slow. The biggest one doesn’t even reach 300MB/s. Intel DC S3700 100G showed around 200MB/sec for us. where did you get this values from? I've some 960GB and they all have > 450Mb/s write speed. Also in tests like here

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Sebastien Han
Hum the Crucial m500 is pretty slow. The biggest one doesn’t even reach 300MB/s. Intel DC S3700 100G showed around 200MB/sec for us. Actually, I don’t know the price difference between the crucial and the intel but the intel looks more suitable for me. Especially after Mark’s comment. Séba

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Mark Nelson
On 01/15/2014 08:03 AM, Robert van Leeuwen wrote: Power-Loss Protection: In the rare event that power fails while the drive is operating, power-loss protection helps ensure that data isn’t corrupted. Seems that not all power protected SSDs are created equal: http://lkcl.net/reports/ssd_analysi

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Michael J. Kidd
It's also good to note that the m500 has built in RAIN protection (basically, diagonal parity at the nand level). Should be very good for journal consistency. Sent from my mobile device. Please excuse brevity and typographical errors. On Jan 15, 2014 9:07 AM, "Stefan Priebe" wrote: > Am 15.01

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Stefan Priebe
Am 15.01.2014 15:03, schrieb Robert van Leeuwen: Power-Loss Protection: In the rare event that power fails while the drive is operating, power-loss protection helps ensure that data isn’t corrupted. Seems that not all power protected SSDs are created equal: http://lkcl.net/reports/ssd_analysis

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Alexandre DERUMIER
shed to be a JBOD >>controller. Thanks Robert ! I didn't that supermicro have 2utwin with 2x12disk, seem really interesting ! - Mail original - De: "Robert van Leeuwen" À: "Alexandre DERUMIER" , ceph-users@lists.ceph.com Envoyé: Mercredi 15 Jan

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Robert van Leeuwen
> Power-Loss Protection: In the rare event that power fails while the > drive is operating, power-loss protection helps ensure that data isn’t > corrupted. Seems that not all power protected SSDs are created equal: http://lkcl.net/reports/ssd_analysis.html The m500 is not tested but the m4 is.

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Alexandre DERUMIER
5 Janvier 2014 14:52:18 Objet: Re: [ceph-users] servers advise (dell r515 or supermicro ) Am 15.01.2014 14:33, schrieb Cedric Lemarchand: > > Le 15/01/2014 14:15, Stefan Priebe a écrit : >> >> THe DC S3700 isn't good as squential but the 520 or 525 series has the &g

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Stefan Priebe
Am 15.01.2014 14:33, schrieb Cedric Lemarchand: Le 15/01/2014 14:15, Stefan Priebe a écrit : THe DC S3700 isn't good as squential but the 520 or 525 series has the problem that it doesn't have a capicator. We've used Intel SSDs since the 160 series but for ceph we now go for Crucial m500 (has

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Robert van Leeuwen
> I would like to have this kind of configuration for each node: > > - 2U > - 2,5inch drives > os : 2 disk sas drive > journal : 2 x ssd intel dc s3700 100GB > osd : 10 or 12 x sas Seagate Savvio 10K.6 900GB > Another option could be to use supermicro server, > they have some 2U - 16 disks chass

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Cedric Lemarchand
Le 15/01/2014 14:15, Stefan Priebe a écrit : THe DC S3700 isn't good as squential but the 520 or 525 series has the problem that it doesn't have a capicator. We've used Intel SSDs since the 160 series but for ceph we now go for Crucial m500 (has capicitor). "Curcial m500 has capacitor", coul

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Alexandre DERUMIER
ilities like 525 >>series 120GB. Ok, thanks! - Mail original - De: "Sebastien Han" À: "Alexandre DERUMIER" Cc: "ceph-users" Envoyé: Mercredi 15 Janvier 2014 13:55:39 Objet: Re: [ceph-users] servers advise (dell r515 or supermicro ) Hi Alexa

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Stefan Priebe
Hi Sebastian, Am 15.01.2014 13:55, schrieb Sebastien Han: Hi Alexandre, Are you going with a 10Gb network? It’s not an issue for IOPS but more for the bandwidth. If so read the following: I personally won’t go with a ratio of 1:6 for the journal. I guess 1:5 (or even 1:4) is preferable. SAS

Re: [ceph-users] servers advise (dell r515 or supermicro ....)

2014-01-15 Thread Sebastien Han
Hi Alexandre, Are you going with a 10Gb network? It’s not an issue for IOPS but more for the bandwidth. If so read the following: I personally won’t go with a ratio of 1:6 for the journal. I guess 1:5 (or even 1:4) is preferable. SAS 10K gives you around 140MB/sec for sequential writes. So if y