13-Sep-15 21:25, Somnath Roy пишет:
<<inline
-----Original Message-----
From: Mike Almateia [mailto:[email protected]]
Sent: Sunday, September 13, 2015 10:39 AM
To: Somnath Roy; ceph-devel
Subject: Re: Question about big EC pool.
13-Sep-15 01:12, Somnath Roy пишет:
12-Sep-15 19:34, Somnath Roy пишет:
I don't think there is any limit from Ceph side..
We are testing with ~768 TB deployment with 4:2 EC on Flash and it is working
well so far..
Thanks & Regards
Somnath
Thanks for answer!
It's very interesting!
What is hardware you use for your the test cluster?
[Somnath] Three 256 TB SanDisk's JBOF (IF100) and 2 heads in front of that ,
so, total of 6 node cluster. FYI, each IF100 can support max 512 TB. Heads are
with 128GB RAM and Xeon 2690 V3 dual socket on each of the server.
What a version of ceph you use?
[Somnath] As of now, it's giant , but, will be moving to Hammer soon..
Good, may be you write a result perfomance/benefits after migrating to
Hammer?
How cluster working in degraded state? Performance degradation is huge?
[Somnath] That's one of the reason we are using Cauchy_good, it's performance
in degraded state is much better. By reducing the recovery traffic (lower
values of recovery settings) , we are able to get significant performance
improvement during degraded state as well...BTW, degradation will depend on how
much data cluster has to recover. In our case, we are seeing ~8% degradation if
say ~64 TB (one ceph node) is failed, but, ~28% if ~128TB (2 node) is down.
This is for 4M reads..
Cool, I see now.
I think that e5-2690 didn't enough for that flash cluster.
[Somnath] In our case and specially for bigger block sizes object use cases,
dual socket E5-2690 should be more than sufficient. We are not able to saturate
that in this case. For smaller block size block use cases we are almost
saturating the cpus with our config though. If you are planning to use EC with
RGW and considering you have object size at least 256K or so, this cpu complex
is good enough IMO.
Ok.
How you have 6 node if as you say "Three 256 TB SanDisk's JBOF (IF100) and 2 heads
in front of that", may be I not realized how IF100 working.
[Somnath] It is 2 ceph nodes connected to each IF100 (you can connect upto 8
servers in front). The IF100 drives are partitioned between 2 head servers. We
used 3 IF100s, so, total 3 * 2 = 6 head nodes or 6 node ceph servers. Hope that
make sense now.
Yes, I understood it now.
Thanks for info!
--
Mike, yes.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
