Hi, If we have 12 SATA disks, each 4TB as storage pool. Then how many SSD disks we should have for cache tier usage?
thanks. 2016-05-10 16:40 GMT+08:00 Yoann Moulin <[email protected]>: > Hello, > > I'd like some advices about the setup of a new ceph cluster. Here the use > case : > > RadowGW (S3 and maybe swift for hadoop/spark) will be the main usage. Most > of > the access will be in read only mode. Write access will only be done by the > admin to update the datasets. > > We might use rbd some time to sync data as temp storage (when POSIX is > needed) > but performance will not be an issue here. We might use cephfs in the > futur if > that can replace a filesystem on rdb. > > We gonna start with 16 nodes (up to 24). The configuration of each node is > : > > CPU : 2 x Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz (12c/48t) > Memory : 128GB > OS Storage : 2 x SSD 240GB Intel S3500 DC (raid 1) > Journal or cache Storage : 2 x SSD 400GB Intel S3300 DC (no Raid) > OSD Disk : 10 x HGST ultrastar-7k6000 6TB > Public Network : 1 x 10Gb/s > Private Network : 1 x 10Gb/s > OS : Ubuntu 16.04 > Ceph version : Jewel > > The question is : journal or cache tier (read only) on the SD 400GB Intel > S3300 DC ? > > Each disk is able to write sequentially at 220MB/s. SSDs can write at > ~500MB/s. > if we set 5 journals on each SSDs, SSD will still be the bottleneck (1GB/s > vs > 2GB/s). If we set the journal on OSDs, we can expect a good throughput in > read > on the disk (in case of data not in the cache) and write shouldn't be so > bad > too, even if we have random read on the OSD during the write ? > > SSDs as cache tier seem to be a better usage than only 5 journal on each ? > Is > that correct ? > > We gonna use an EC pool for big files (jerasure 8+2 I think) and a > replicated > pool for small files. > > If I check on http://ceph.com/pgcalc/, in this use case > > replicated pool: pg_num = 8192 for 160 OSDs but 16384 for 240 OSDs > Ec pool : pg_num = 4096 > and pgp_num = pg_num > > Should I set the pg_num to 8192 or 16384 ? what is the impact on the > cluster if > we set the pg_num to 16384 at the beginning ? 16384 is high, isn't it ? > > Thanks for your help > > -- > Yoann Moulin > EPFL IC-IT > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
