Re: SSD journal suggestion / rsockets

2012-11-09 Thread Atchley, Scott
On Nov 8, 2012, at 5:00 PM, Joseph Glanville joseph.glanvi...@orionvm.com.au wrote: On 9 November 2012 08:21, Dieter Kasper d.kas...@kabelmail.de wrote: Joseph, I've downloaded and read the presentation from 'Sean Hefty / Intel Corporation' about rsockets, which sounds very promising to

Re: SSD journal suggestion / rsockets

2012-11-09 Thread Joseph Glanville
On 10 November 2012 01:43, Atchley, Scott atchle...@ornl.gov wrote: On Nov 8, 2012, at 5:00 PM, Joseph Glanville joseph.glanvi...@orionvm.com.au wrote: On 9 November 2012 08:21, Dieter Kasper d.kas...@kabelmail.de wrote: Joseph, I've downloaded and read the presentation from 'Sean Hefty /

Re: SSD journal suggestion

2012-11-08 Thread Atchley, Scott
On Nov 8, 2012, at 3:22 AM, Gandalf Corvotempesta gandalf.corvotempe...@gmail.com wrote: 2012/11/8 Mark Nelson mark.nel...@inktank.com: I haven't done much with IPoIB (just RDMA), but my understanding is that it tends to top out at like 15Gb/s. Some others on this mailing list can probably

Re: SSD journal suggestion

2012-11-08 Thread Mark Nelson
On 11/08/2012 07:55 AM, Atchley, Scott wrote: On Nov 8, 2012, at 3:22 AM, Gandalf Corvotempesta gandalf.corvotempe...@gmail.com wrote: 2012/11/8 Mark Nelson mark.nel...@inktank.com: I haven't done much with IPoIB (just RDMA), but my understanding is that it tends to top out at like 15Gb/s.

Re: SSD journal suggestion

2012-11-08 Thread Atchley, Scott
On Nov 8, 2012, at 10:00 AM, Scott Atchley atchle...@ornl.gov wrote: On Nov 8, 2012, at 9:39 AM, Mark Nelson mark.nel...@inktank.com wrote: On 11/08/2012 07:55 AM, Atchley, Scott wrote: On Nov 8, 2012, at 3:22 AM, Gandalf Corvotempesta gandalf.corvotempe...@gmail.com wrote: 2012/11/8

Re: SSD journal suggestion

2012-11-08 Thread Andrey Korolyov
On Thu, Nov 8, 2012 at 7:02 PM, Atchley, Scott atchle...@ornl.gov wrote: On Nov 8, 2012, at 10:00 AM, Scott Atchley atchle...@ornl.gov wrote: On Nov 8, 2012, at 9:39 AM, Mark Nelson mark.nel...@inktank.com wrote: On 11/08/2012 07:55 AM, Atchley, Scott wrote: On Nov 8, 2012, at 3:22 AM,

Re: SSD journal suggestion

2012-11-08 Thread Atchley, Scott
On Nov 8, 2012, at 11:19 AM, Andrey Korolyov and...@xdel.ru wrote: On Thu, Nov 8, 2012 at 7:02 PM, Atchley, Scott atchle...@ornl.gov wrote: On Nov 8, 2012, at 10:00 AM, Scott Atchley atchle...@ornl.gov wrote: On Nov 8, 2012, at 9:39 AM, Mark Nelson mark.nel...@inktank.com wrote: On

Re: SSD journal suggestion

2012-11-08 Thread Joseph Glanville
On 9 November 2012 02:00, Atchley, Scott atchle...@ornl.gov wrote: On Nov 8, 2012, at 9:39 AM, Mark Nelson mark.nel...@inktank.com wrote: On 11/08/2012 07:55 AM, Atchley, Scott wrote: On Nov 8, 2012, at 3:22 AM, Gandalf Corvotempesta gandalf.corvotempe...@gmail.com wrote: 2012/11/8 Mark

Re: SSD journal suggestion / rsockets

2012-11-08 Thread Joseph Glanville
On 9 November 2012 08:21, Dieter Kasper d.kas...@kabelmail.de wrote: Joseph, I've downloaded and read the presentation from 'Sean Hefty / Intel Corporation' about rsockets, which sounds very promising to me. Can you please teach me how to get access to the rsockets source ? Thanks,

Re: SSD journal suggestion / rsockets

2012-11-08 Thread Dieter Kasper
Joseph, I've downloaded and read the presentation from 'Sean Hefty / Intel Corporation' about rsockets, which sounds very promising to me. Can you please teach me how to get access to the rsockets source ? Thanks, -Dieter On Thu, Nov 08, 2012 at 09:12:45PM +0100, Joseph Glanville wrote: On 9

Re: SSD journal suggestion

2012-11-07 Thread Sage Weil
On Wed, 7 Nov 2012, Gandalf Corvotempesta wrote: I'm evaluating some SSD drives as journal. Samsung 840 Pro seems to be the fastest in sequential reads and write. What parameter should I consider for a journal? I think that none of read benchmark are influent because when dumping journal to

Re: SSD journal suggestion

2012-11-07 Thread Mark Nelson
On 11/07/2012 06:28 AM, Gandalf Corvotempesta wrote: 2012/11/7 Sage Weil s...@inktank.com: On Wed, 7 Nov 2012, Gandalf Corvotempesta wrote: I'm evaluating some SSD drives as journal. Samsung 840 Pro seems to be the fastest in sequential reads and write. The 840 Pro seems to reach 485MB/s in

Re: SSD journal suggestion

2012-11-07 Thread Mark Nelson
On 11/07/2012 10:12 AM, Atchley, Scott wrote: On Nov 7, 2012, at 10:01 AM, Mark Nelson mark.nel...@inktank.com wrote: On 11/07/2012 06:28 AM, Gandalf Corvotempesta wrote: 2012/11/7 Sage Weil s...@inktank.com: On Wed, 7 Nov 2012, Gandalf Corvotempesta wrote: I'm evaluating some SSD drives as

Re: SSD journal suggestion

2012-11-07 Thread Atchley, Scott
On Nov 7, 2012, at 10:01 AM, Mark Nelson mark.nel...@inktank.com wrote: On 11/07/2012 06:28 AM, Gandalf Corvotempesta wrote: 2012/11/7 Sage Weil s...@inktank.com: On Wed, 7 Nov 2012, Gandalf Corvotempesta wrote: I'm evaluating some SSD drives as journal. Samsung 840 Pro seems to be the

Re: SSD journal suggestion

2012-11-07 Thread Atchley, Scott
On Nov 7, 2012, at 11:20 AM, Mark Nelson mark.nel...@inktank.com wrote: Right now I'm doing 3 journals per SSD, but topping out at about 1.2-1.4GB/s from the client perspective for the node with 15+ drives and 5 SSDs. It's possible newer versions of the code and tuning may increase that.

Re: SSD journal suggestion

2012-11-07 Thread Mark Nelson
On 11/07/2012 10:35 AM, Atchley, Scott wrote: On Nov 7, 2012, at 11:20 AM, Mark Nelson mark.nel...@inktank.com wrote: Right now I'm doing 3 journals per SSD, but topping out at about 1.2-1.4GB/s from the client perspective for the node with 15+ drives and 5 SSDs. It's possible newer

Re: SSD journal suggestion

2012-11-07 Thread Martin Mailand
Hi, I have 16 SAS disk on a LSI 9266-8i and 4 Intel 520 SSD on a HBA, the node has dual 10G Ethernet. The clients are 4 nodes with dual 10GeB, as test I use rados bench on each client. The aggregated write speed is around 1,6GB/s with single replication. In the first configuration, I had

Re: SSD journal suggestion

2012-11-07 Thread Martin Mailand
Hi, I tested a Arista 7150S-24, a HP5900 and in a few weeks I will get a Mellanox MSX1016. ATM the Arista is may favourite. For the dual 10GeB NICs I tested the Intel X520-DA2 and the Mellanox ConnectX-3. My favourite is the Intel X520-DA2. -martin Am 07.11.2012 22:14, schrieb Gandalf

Re: SSD journal suggestion

2012-11-07 Thread Stefan Priebe
Am 07.11.2012 22:35, schrieb Martin Mailand: Hi, I tested a Arista 7150S-24, a HP5900 and in a few weeks I will get a Mellanox MSX1016. ATM the Arista is may favourite. For the dual 10GeB NICs I tested the Intel X520-DA2 and the Mellanox ConnectX-3. My favourite is the Intel X520-DA2. That's

Re: SSD journal suggestion

2012-11-07 Thread Martin Mailand
Hi Stefan, deep buffers means latency spikes, you should go for fast switching latency. The HP5900 has a latency of 1ms, the Arista and Mellanox of 250ns. And I you should think at the price the HP5900 cost 3 times of the Mellanox. -martin Am 07.11.2012 22:44, schrieb Stefan Priebe: Am

Re: SSD journal suggestion

2012-11-07 Thread Stefan Priebe
Am 07.11.2012 22:55, schrieb Martin Mailand: Hi Stefan, deep buffers means latency spikes, you should go for fast switching latency. The HP5900 has a latency of 1ms, the Arista and Mellanox of 250ns. HP told me they all use the same ships and Arista measures latency while only one port is in

Re: SSD journal suggestion

2012-11-07 Thread Martin Mailand
Hi, I *think* the HP is Broadcom based, the Arista is Fulcrum based, and I don't know which chips Mellanox is using. Our NOC tested both of them, an the Arista was the clear winner, at least in our workload. -martin Am 07.11.2012 22:59, schrieb Stefan Priebe: HP told me they all use the

Re: SSD journal suggestion

2012-11-07 Thread Martin Mailand
good question, probably we do not have enough experience with IPoIB. But it looks good on paper, so it's definitely a try worth. -martin Am 07.11.2012 23:28, schrieb Gandalf Corvotempesta: 2012/11/7 Martin Mailand mar...@tuxadero.com: I tested a Arista 7150S-24, a HP5900 and in a few weeks I

Re: SSD journal suggestion

2012-11-07 Thread Mark Nelson
On 11/07/2012 04:51 PM, Gandalf Corvotempesta wrote: 2012/11/7 Martin Mailand mar...@tuxadero.com: But it looks good on paper, so it's definitely a try worth. is at least 4x times faster than 10gbe and AFAIK should have a lower latency. I'm planning to use infiniband as backend storage