...there is also a crail blog available, which discusses NVMeF tier design
and performance. See
http://crail.incubator.apache.org/blog/2017/08/crail-nvme-fabrics-v1.html
Thanks,
Bernard.
On 1/26/2018 10:18, Animesh Trivedi wrote:
Hi Dawn - we had some results published at the last year's Spark Summit
conference :
https://www.slideshare.net/databricks/running-apache-spark-on-a-highperformance-cluster-using-rdma-and-nvme-flash-with-patrick-stuedi
see slide 27. We are currently working on testing new setups and workloads.
Cheers,
--
Animesh
On Fri, Jan 26, 2018 at 6:01 AM, Dawn Shepherd <[email protected]>
wrote:
Hi Patrick,
Thank you for you answer. That’s really helpful.
I have another question. Do you have any testing report about the
capability/performance of Crail on a cluster with disaggregated storage? I
didn’t find it on the Crail official site.
On 25 Jan 2018, at 23:40, Patrick Stuedi <[email protected]> wrote:
Hi Dawn,
Not a stupid question at all, a very relevant question. Storage
disaggregation is one of THE key use cases we have built Crail for.
The goal is to enable existing data processing platforms frameworks to
perform efficiently on remote storage (remote as seen from the compute
nodes). This is not straight forward because most frameworks like
hadoop, spark, etc.., have been designed explicitly to take advantage
of local storage. In Crail, due to the low-latency high-throughput
data access, storage disaggregation becomes feasible even for such
frameworks.
In terms of configuration there is nothing in particular that needs to
be done, except that Crail permits you to put the storage nodes remote
to the compute nodes. Both the NVMf and DRAM/RDMA tier are suitable
for this. With regard to NVMf, there are storage enclosure that pack
plenty of flash into a single box and export an NVMf interface, these
boxes can also be used out-of-the box with the Crail NVMf tier.
-Patrick
On Thu, Jan 25, 2018 at 4:05 PM, Dawn Shepherd
<[email protected]> wrote:
How about Crail's ability to support disaggregated storage? I will be
grateful if someone can tell me the detail. Thank you.
