Check these out to: http://www.seagate.com/internal-hard-drives/solid-state-hybrid/1200-ssd/
----- Original Message ----- From: "Christian Balzer" <ch...@gol.com> To: "ceph-users" <ceph-users@lists.ceph.com> Sent: Sunday, January 17, 2016 10:45:56 PM Subject: Re: [ceph-users] Again - state of Ceph NVMe and SSDs Hello, On Sat, 16 Jan 2016 19:06:07 +0100 David wrote: > Hi! > > We’re planning our third ceph cluster and been trying to find how to > maximize IOPS on this one. > > Our needs: > * Pool for MySQL, rbd (mounted as /var/lib/mysql or equivalent on KVM > servers) > * Pool for storage of many small files, rbd (probably dovecot maildir > and dovecot index etc) > I'm running dovecot for several 100k users on 2-node DRBD clusters and for a mail archive server for a few hundred users backed by Ceph/RBD. The later works fine (it's not that busy), but I wouldn't consider replacing the DRBD clusters with Ceph/RBD at this time (higher investment in storage 3x vs 2x and lower performance of course). Depending on your use case you may be just fine of course. > So I’ve been reading up on: > > https://communities.intel.com/community/itpeernetwork/blog/2015/11/20/the-future-ssd-is-here-pcienvme-boosts-ceph-performance > > and ceph-users from october 2015: > > http://www.spinics.net/lists/ceph-users/msg22494.html > > We’re planning something like 5 OSD servers, with: > > * 4x 1.2TB Intel S3510 I'd be wary of that. As in, you're spec'ing the best Intel SSDs money can buy below for journals, but the least write-endurable Intel DC SSDs for OSDs here. Note that write amplification (beyond Ceph and FS journals) is very much a thing, especially with small files. There's a mail about this by me in the ML archives somewhere: http://lists.opennebula.org/pipermail/ceph-users-ceph.com/2014-October/043949.html Unless you're very sure about this being a read-mostly environment I'd go with 3610's at least. > * 8st 4TB HDD > * 2x Intel P3700 Series HHHL PCIe 400GB (one for SSD Pool Journal and > one for HDD pool journal) You may be better off (cost and SPOF wise) with 2 200GB S3700 (not 3710) for the HDD journals, but then again that won't fit into your case, won't it... Given the IOPS limits in Ceph as it is, you're unlikely to see much of difference if you forgo a journal for the SSDs and use shared journals with DC S3610 or 3710 OSD SSDs. Note that as far as pure throughput is concerned (in most operations the least critical factor) your single journal SSD will limit things to the speed of 2 (of your 4) storage SSDs. But then again, your network is probably saturated before that. > * 2x 80GB Intel S3510 raid1 for system > * 256GB RAM Plenty. ^o^ > * 2x 8 core CPU Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz or better > Not sure about Jewel, but SSD OSDs will eat pretty much any and all CPU cycles you can throw at them. This also boils down to the question if having mixed HDD/SSD storage nodes (with the fun of having to set "osd crush update on start = false") is a good idea or not, as opposed to nodes that are optimized for their respective storage hardware (CPU, RAM, network wise). Regards, Christian > This cluster will probably run Hammer LTS unless there are huge > improvements in Infernalis when dealing 4k IOPS. > > The first link above hints at awesome performance. The second one from > the list not so much yet.. > > Is anyone running Hammer or Infernalis with a setup like this? > Is it a sane setup? > Will we become CPU constrained or can we just throw more RAM on it? :D > > Kind Regards, > David Majchrzak -- Christian Balzer Network/Systems Engineer ch...@gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/ _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
