http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/5c23d221/content/blog/2017/11/rdmashuffle.html ---------------------------------------------------------------------- diff --git a/content/blog/2017/11/rdmashuffle.html b/content/blog/2017/11/rdmashuffle.html deleted file mode 100644 index 4482592..0000000 --- a/content/blog/2017/11/rdmashuffle.html +++ /dev/null @@ -1,195 +0,0 @@ -<!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <title>The Apache Crail (Incubating) Project: Spark Shuffle: SparkRDMA vs Crail</title> - <meta name="viewport" content="width=device-width, initial-scale=1.0"> - <link href="http://crail.incubator.apache.org/css/bootstrap.min.css"; rel="stylesheet"> - <link href="http://crail.incubator.apache.org/css/group.css"; rel="stylesheet"> - <link rel="alternate" type="application/atom+xml" title="Atom" - href="http://crail.incubator.apache.org/blog/blog.xml";> - - <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/rdmashuffle-summary.png"; /> - <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/rdmashuffle-summary.png"; /> - </head> - - <body> - <div class="container"> - <div class="header"> - <ul class="nav nav-pills pull-right"> - - - - <li > - <a href="http://crail.incubator.apache.org/";> - Home - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/overview/";> - Overview - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/blog/";> - Blog - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/community/";> - Community - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/documentation/";> - Documentation - </a> - </li> - - </ul> - <a href="http://crail.incubator.apache.org/";> - <img src="http://crail.incubator.apache.org/img/crail_logo.png"; - srcset="http://crail.incubator.apache.org/img/crail_logo.png"; - alt="Crail" id="logo"> - </a> - </div> - - - - <h2>Spark Shuffle: SparkRDMA vs Crail</h2> - - - <p class="meta">17 Nov 2017</p> - -<div class="post"> -<div style="text-align: justify"> -<p> -This blog is comparing the shuffle performance of Crail with SparkRDMA, an alternative RDMA-based shuffle plugin for Spark. -</p> -</div> - -<h3 id="hardware-configuration">Hardware Configuration</h3> - -<p>The specific cluster configuration used for the experiments in this blog:</p> - -<ul> - <li>Cluster - <ul> - <li>8 compute + 1 management node x86_64 cluster</li> - </ul> - </li> - <li>Node configuration - <ul> - <li>CPU: 2 x Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz</li> - <li>DRAM: 96GB DDR3</li> - <li>Network: 1x100Gbit/s Mellanox ConnectX-5</li> - </ul> - </li> - <li>Software - <ul> - <li>Ubuntu 16.04.3 LTS (Xenial Xerus) with Linux kernel version 4.10.0-33-generic</li> - <li><a href="https://github.com/zrlio/crail";>Crail 1.0</a>, commit a45c8382050f471e9342e1c6cf25f9f2001af6b5</li> - <li><a href="">Crail Shuffle plugin</a>, commit 2273b5dd53405cab3389f5c1fc2ee4cd30f02ae6</li> - <li><a href="https://github.com/Mellanox/SparkRDMA";>SparkRDMA</a>, commit d95ce3e370a8e3b5146f4e0ab5e67a19c6f405a5 (latest master on 8th of November 2017)</li> - </ul> - </li> -</ul> - -<h3 id="overview">Overview</h3> -<div style="text-align: justify"> -<p> -Lately there has been an increasing interest in the community to include RDMA networking into data processing frameworks like Spark and Hadoop. One natural spot to integrate RDMA is in the shuffle operation that involves all-to-all network communication pattern. Naturally, due to its performance requirements the shuffle operation is of interest to us as well, and we have developed a Spark plugin for shuffle. In our previous blog posts, we have already shown that the Crail Shuffler achieves great workload-level speedups compared to vanilla Spark. In this blog post, we take a look at another recently proposed design called <a href="https://github.com/Mellanox/SparkRDMA";>SparkRDMA</a> (<a href="https://issues.apache.org/jira/browse/SPARK-22229";>SPARK-22229 JIRA</a>). SparkRDMA proposes to improve the shuffle performance of Spark by performing data transfers over RDMA. For this, the code manages its own off-heap memory which needs to be registered with the NIC for RDMA use. It supports two ways to store shuffle data between the stages: (1) shuffle data is stored in regular files (just like vanilla Spark) but the data transfer is implemented via RDMA, (2) data is stored in memory (allocated and registered for RDMA transfer) and the data transfer is implemented via RDMA. We call it the "last-mile" approach where just the networking operations are replaced by the RDMA operations. -</p> -<p> -In contrast, the Crail shuffler plugin takes a more holistic approach and leverages the high performance of Crail distributed data store to deliver gains. It uses Crail store to efficiently manage I/O resources, storage and networking devices, memory registrations, client sessions, data distribution, etc. Consequently, the shuffle operation becomes as simple as writing and reading files. And recall that Crail store is designed as a fast data bus for the intermediate data. The shuffle operation is just one of many operations that can be accelerated using Crail store. Beyond these operations, the modular architecture of Crail store enables us to seamlessly leverage different storage types (DRAM, NVMe, and more), perform tiering, support disaggregation, share inter-job data, jointly optimize I/O resources for various workloads, etc. These capabilities and performance gains give us confidence in the design choices we made for the Crail project. -</p> -</div> - -<h3 id="performance-comparison">Performance comparison</h3> -<div style="text-align: justify"> -<p>Lets start by quantitatively assessing performance gains from the Crail shuffle plugin and SparkRDMA. As described above, SparkRDMA can be operated in two different modes. Users decide which mode to use by selecting a particular type of shuffle writer (spark.shuffle.rdma.shuffleWriterMethod). The Wrapper shuffle writer writes shuffle data to files between the stages, the Chunked shuffle writer stores shuffle data in memory. We evaluate both writer methods for terasort and SQL equijoin. -</p> -</div> -<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/rdma-shuffle/terasort.svg"; width="550" /></div> -<p><br /></p> -<div style="text-align: justify"> -<p> -First we run <a href="https://github.com/zrlio/crail-spark-terasort";>terasort</a> on our 8+1 machine cluster (see above). We sort 200GB, thus, each node gets 25GB of data (equal distribution). We further did a basic search of the parameter space for each of the systems to find the best possible configuration. In all the experiments we use 8 executors with 12 cores each. Note that in a typical Spark run more CPU cores than assigned are engaged because of garbabge collection, etc. In our test runs assigning 12 cores lead to the best performance. -</p> -<p> -The plot above shows runtimes of the various configuration we run with terasort. SparkRDMA with the Wrapper shuffle writer performance slightly better (3-4%) than vanilla Spark whereas the Chunked shuffle writer shows a 30% overhead. On a quick inspection we found that this overhead stems from memory allocation and registration for the shuffle data that is kept in memory between the stages. Compared to vanilla Spark, Crail's shuffle plugin shows performance improvement of around 235%. -</p> -</div> -<div style="text-align:center"><img src="http://crail.incubator.apache.org/img/blog/rdma-shuffle/sql.svg"; width="550" /></div> -<p><br /></p> - -<div style="text-align: justify"> -<p> -For our second workload we choose the <a href="https://github.com/zrlio/sql-benchmarks";>SQL equijoin</a> with a <a href="https://github.com/zrlio/spark-nullio-fileformat";>special fileformat</a> that allows data to be generated on the fly. By generating data on the fly we eliminate any costs for reading data from storage and focus entirely on the shuffle performance. The shuffle data size is around 148GB. Here the Wrapper shuffle writer is slightly slower than vanilla Spark but instead the Chunked shuffle writer is roughly the same amount faster. The Crail shuffle plugin again delivers a great performance increase over vanilla Spark. -</p> -</div> - -<div style="text-align: justify"> -<p>Please let us know if your have recommendations about how these experiments can be improved.</p> -</div> - -<h3 id="summary">Summary</h3> - -<div style="text-align: justify"> -<p> -These benchmarks validate our belief that a "last-mile" integration cannot deliver the same performance gains as a holistic approach, i.e. one has to look at the whole picture in how to integrate RDMA into Spark applications (and for that matter any framework or application). Only replacing the data transfer alone does not lead to the anticipated performance increase. We learned this the hard way when we intially started working on Crail. -</p> - -</div> - - -</div> - -<!-- - -<div id="disqus_thread"></div> -<script> - -/** -* RECOMMENDED CONFIGURATION VARIABLES: EDIT AND UNCOMMENT THE SECTION BELOW TO INSERT DYNAMIC VALUES FROM YOUR PLATFORM OR CMS. -* LEARN WHY DEFINING THESE VARIABLES IS IMPORTANT: https://disqus.com/admin/universalcode/#configuration-variables*/ -/* -var disqus_config = function () { -this.page.url = PAGE_URL; // Replace PAGE_URL with your page's canonical URL variable -this.page.identifier = PAGE_IDENTIFIER; // Replace PAGE_IDENTIFIER with your page's unique identifier variable -}; -*/ -(function() { // DON'T EDIT BELOW THIS LINE -var d = document, s = d.createElement('script'); -s.src = '//crail-io.disqus.com/embed.js'; -s.setAttribute('data-timestamp', +new Date()); -(d.head || d.body).appendChild(s); -})(); -</script> -<noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript";>comments powered by Disqus.</a></noscript> - ---> - - - <br> - <br> - <div class="footer"> - <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/";>incubation</a> at <a href="https://www.apache.org/";>The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. - </p> - </div> - - </div> <!-- /container --> - - <!-- Support retina images. --> - <script type="text/javascript" - src="http://crail.incubator.apache.org/js/srcset-polyfill.js";></script> - </body> -</html>
http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/5c23d221/content/blog/2018/01/apache.html ---------------------------------------------------------------------- diff --git a/content/blog/2018/01/apache.html b/content/blog/2018/01/apache.html deleted file mode 100644 index bf1bdca..0000000 --- a/content/blog/2018/01/apache.html +++ /dev/null @@ -1,86 +0,0 @@ -<!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <title>The Apache Crail (Incubating) Project: Apache</title> - <meta name="viewport" content="width=device-width, initial-scale=1.0"> - <link href="http://crail.incubator.apache.org/css/bootstrap.min.css"; rel="stylesheet"> - <link href="http://crail.incubator.apache.org/css/group.css"; rel="stylesheet"> - <link rel="alternate" type="application/atom+xml" title="Atom" - href="http://crail.incubator.apache.org/blog/blog.xml";> - - <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/apache-summary.png"; /> - <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/apache-summary.png"; /> - </head> - - <body> - <div class="container"> - <div class="header"> - <ul class="nav nav-pills pull-right"> - - - - <li > - <a href="http://crail.incubator.apache.org/";> - Home - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/overview/";> - Overview - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/blog/";> - Blog - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/community/";> - Community - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/documentation/";> - Documentation - </a> - </li> - - </ul> - <a href="http://crail.incubator.apache.org/";> - <img src="http://crail.incubator.apache.org/img/crail_logo.png"; - srcset="http://crail.incubator.apache.org/img/crail_logo.png"; - alt="Crail" id="logo"> - </a> - </div> - - - - <h2>Apache</h2> - - - <p>Crail is now an Apache Incubator project!</p> - - - <br> - <br> - <div class="footer"> - <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/";>incubation</a> at <a href="https://www.apache.org/";>The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. - </p> - </div> - - </div> <!-- /container --> - - <!-- Support retina images. --> - <script type="text/javascript" - src="http://crail.incubator.apache.org/js/srcset-polyfill.js";></script> - </body> -</html> http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/5c23d221/content/blog/index.html ---------------------------------------------------------------------- diff --git a/content/blog/index.html b/content/blog/index.html deleted file mode 100644 index 732428f..0000000 --- a/content/blog/index.html +++ /dev/null @@ -1,154 +0,0 @@ -<!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <title>The Apache Crail (Incubating) Project: Latest posts</title> - <meta name="viewport" content="width=device-width, initial-scale=1.0"> - <link href="http://crail.incubator.apache.org/css/bootstrap.min.css"; rel="stylesheet"> - <link href="http://crail.incubator.apache.org/css/group.css"; rel="stylesheet"> - <link rel="alternate" type="application/atom+xml" title="Atom" - href="http://crail.incubator.apache.org/blog/blog.xml";> - - <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/blog-summary.png"; /> - <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/blog-summary.png"; /> - </head> - - <body> - <div class="container"> - <div class="header"> - <ul class="nav nav-pills pull-right"> - - - - <li > - <a href="http://crail.incubator.apache.org/";> - Home - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/overview/";> - Overview - </a> - </li> - - - <li class="active"> - <a href="http://crail.incubator.apache.org/blog/";> - Blog - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/community/";> - Community - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/documentation/";> - Documentation - </a> - </li> - - </ul> - <a href="http://crail.incubator.apache.org/";> - <img src="http://crail.incubator.apache.org/img/crail_logo.png"; - srcset="http://crail.incubator.apache.org/img/crail_logo.png"; - alt="Crail" id="logo"> - </a> - </div> - - - - <h2>Latest posts</h2> - - - -<div class="post-preview"> - <a href="/blog/2017/11/crail-metadata.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part III: Metadata - </h2> - - </a> - <p class="post-meta">Posted by Adrian Schuepbach and Patrick Stuedi on November 21, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/11/rdmashuffle.html"> - <h2 class="post-title"> - Spark Shuffle: SparkRDMA vs Crail - </h2> - - </a> - <p class="post-meta">Posted by Jonas Pfefferle, Patrick Stuedi, Animesh Trivedi, Bernard Metzler, Adrian Schuepbach on November 17, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/08/crail-nvme-fabrics-v1.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part II: NVMf - </h2> - - </a> - <p class="post-meta">Posted by Jonas Pfefferle on August 22, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/08/crail-memory.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part I: DRAM - </h2> - - </a> - <p class="post-meta">Posted by Patrick Stuedi on August 18, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/01/sorting.html"> - <h2 class="post-title"> - Sorting on a 100Gbit/s Cluster using Spark/Crail - </h2> - - </a> - <p class="post-meta">Posted by Patrick Stuedi on January 17, 2017</p> -</div> -<hr> - - -<!-- Pager --> - -<ul class="pager"> - - - <li class="next"> - <a href="/blog/page2">Older Posts →</a> - </li> - -</ul> - - - - - <br> - <br> - <div class="footer"> - <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/";>incubation</a> at <a href="https://www.apache.org/";>The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. - </p> - </div> - - </div> <!-- /container --> - - <!-- Support retina images. --> - <script type="text/javascript" - src="http://crail.incubator.apache.org/js/srcset-polyfill.js";></script> - </body> -</html> http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/5c23d221/content/blog/page2/index.html ---------------------------------------------------------------------- diff --git a/content/blog/page2/index.html b/content/blog/page2/index.html deleted file mode 100644 index 9796b04..0000000 --- a/content/blog/page2/index.html +++ /dev/null @@ -1,158 +0,0 @@ -<!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <title>The Apache Crail (Incubating) Project: Latest posts</title> - <meta name="viewport" content="width=device-width, initial-scale=1.0"> - <link href="http://crail.incubator.apache.org/css/bootstrap.min.css"; rel="stylesheet"> - <link href="http://crail.incubator.apache.org/css/group.css"; rel="stylesheet"> - <link rel="alternate" type="application/atom+xml" title="Atom" - href="http://crail.incubator.apache.org/blog/blog.xml";> - - <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/page2-summary.png"; /> - <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/page2-summary.png"; /> - </head> - - <body> - <div class="container"> - <div class="header"> - <ul class="nav nav-pills pull-right"> - - - - <li > - <a href="http://crail.incubator.apache.org/";> - Home - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/overview/";> - Overview - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/blog/";> - Blog - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/community/";> - Community - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/documentation/";> - Documentation - </a> - </li> - - </ul> - <a href="http://crail.incubator.apache.org/";> - <img src="http://crail.incubator.apache.org/img/crail_logo.png"; - srcset="http://crail.incubator.apache.org/img/crail_logo.png"; - alt="Crail" id="logo"> - </a> - </div> - - - - <h2>Latest posts</h2> - - - -<div class="post-preview"> - <a href="/blog/2017/11/crail-metadata.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part III: Metadata - </h2> - - </a> - <p class="post-meta">Posted by Adrian Schuepbach and Patrick Stuedi on November 21, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/11/rdmashuffle.html"> - <h2 class="post-title"> - Spark Shuffle: SparkRDMA vs Crail - </h2> - - </a> - <p class="post-meta">Posted by Jonas Pfefferle, Patrick Stuedi, Animesh Trivedi, Bernard Metzler, Adrian Schuepbach on November 17, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/08/crail-nvme-fabrics-v1.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part II: NVMf - </h2> - - </a> - <p class="post-meta">Posted by Jonas Pfefferle on August 22, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/08/crail-memory.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part I: DRAM - </h2> - - </a> - <p class="post-meta">Posted by Patrick Stuedi on August 18, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/01/sorting.html"> - <h2 class="post-title"> - Sorting on a 100Gbit/s Cluster using Spark/Crail - </h2> - - </a> - <p class="post-meta">Posted by Patrick Stuedi on January 17, 2017</p> -</div> -<hr> - - -<!-- Pager --> - -<ul class="pager"> - - <li class="previous"> - <a href="/blog/">← Newer Posts</a> - </li> - - - <li class="next"> - <a href="/blog/page3">Older Posts →</a> - </li> - -</ul> - - - - - <br> - <br> - <div class="footer"> - <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/";>incubation</a> at <a href="https://www.apache.org/";>The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. - </p> - </div> - - </div> <!-- /container --> - - <!-- Support retina images. --> - <script type="text/javascript" - src="http://crail.incubator.apache.org/js/srcset-polyfill.js";></script> - </body> -</html> http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/5c23d221/content/blog/page3/index.html ---------------------------------------------------------------------- diff --git a/content/blog/page3/index.html b/content/blog/page3/index.html deleted file mode 100644 index dbd2d51..0000000 --- a/content/blog/page3/index.html +++ /dev/null @@ -1,158 +0,0 @@ -<!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <title>The Apache Crail (Incubating) Project: Latest posts</title> - <meta name="viewport" content="width=device-width, initial-scale=1.0"> - <link href="http://crail.incubator.apache.org/css/bootstrap.min.css"; rel="stylesheet"> - <link href="http://crail.incubator.apache.org/css/group.css"; rel="stylesheet"> - <link rel="alternate" type="application/atom+xml" title="Atom" - href="http://crail.incubator.apache.org/blog/blog.xml";> - - <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/page3-summary.png"; /> - <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/page3-summary.png"; /> - </head> - - <body> - <div class="container"> - <div class="header"> - <ul class="nav nav-pills pull-right"> - - - - <li > - <a href="http://crail.incubator.apache.org/";> - Home - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/overview/";> - Overview - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/blog/";> - Blog - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/community/";> - Community - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/documentation/";> - Documentation - </a> - </li> - - </ul> - <a href="http://crail.incubator.apache.org/";> - <img src="http://crail.incubator.apache.org/img/crail_logo.png"; - srcset="http://crail.incubator.apache.org/img/crail_logo.png"; - alt="Crail" id="logo"> - </a> - </div> - - - - <h2>Latest posts</h2> - - - -<div class="post-preview"> - <a href="/blog/2017/11/crail-metadata.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part III: Metadata - </h2> - - </a> - <p class="post-meta">Posted by Adrian Schuepbach and Patrick Stuedi on November 21, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/11/rdmashuffle.html"> - <h2 class="post-title"> - Spark Shuffle: SparkRDMA vs Crail - </h2> - - </a> - <p class="post-meta">Posted by Jonas Pfefferle, Patrick Stuedi, Animesh Trivedi, Bernard Metzler, Adrian Schuepbach on November 17, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/08/crail-nvme-fabrics-v1.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part II: NVMf - </h2> - - </a> - <p class="post-meta">Posted by Jonas Pfefferle on August 22, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/08/crail-memory.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part I: DRAM - </h2> - - </a> - <p class="post-meta">Posted by Patrick Stuedi on August 18, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/01/sorting.html"> - <h2 class="post-title"> - Sorting on a 100Gbit/s Cluster using Spark/Crail - </h2> - - </a> - <p class="post-meta">Posted by Patrick Stuedi on January 17, 2017</p> -</div> -<hr> - - -<!-- Pager --> - -<ul class="pager"> - - <li class="previous"> - <a href="/blog/page2">← Newer Posts</a> - </li> - - - <li class="next"> - <a href="/blog/page4">Older Posts →</a> - </li> - -</ul> - - - - - <br> - <br> - <div class="footer"> - <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/";>incubation</a> at <a href="https://www.apache.org/";>The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. - </p> - </div> - - </div> <!-- /container --> - - <!-- Support retina images. --> - <script type="text/javascript" - src="http://crail.incubator.apache.org/js/srcset-polyfill.js";></script> - </body> -</html> http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/5c23d221/content/blog/page4/index.html ---------------------------------------------------------------------- diff --git a/content/blog/page4/index.html b/content/blog/page4/index.html deleted file mode 100644 index aabfb8b..0000000 --- a/content/blog/page4/index.html +++ /dev/null @@ -1,154 +0,0 @@ -<!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <title>The Apache Crail (Incubating) Project: Latest posts</title> - <meta name="viewport" content="width=device-width, initial-scale=1.0"> - <link href="http://crail.incubator.apache.org/css/bootstrap.min.css"; rel="stylesheet"> - <link href="http://crail.incubator.apache.org/css/group.css"; rel="stylesheet"> - <link rel="alternate" type="application/atom+xml" title="Atom" - href="http://crail.incubator.apache.org/blog/blog.xml";> - - <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/page4-summary.png"; /> - <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/page4-summary.png"; /> - </head> - - <body> - <div class="container"> - <div class="header"> - <ul class="nav nav-pills pull-right"> - - - - <li > - <a href="http://crail.incubator.apache.org/";> - Home - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/overview/";> - Overview - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/blog/";> - Blog - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/community/";> - Community - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/documentation/";> - Documentation - </a> - </li> - - </ul> - <a href="http://crail.incubator.apache.org/";> - <img src="http://crail.incubator.apache.org/img/crail_logo.png"; - srcset="http://crail.incubator.apache.org/img/crail_logo.png"; - alt="Crail" id="logo"> - </a> - </div> - - - - <h2>Latest posts</h2> - - - -<div class="post-preview"> - <a href="/blog/2017/11/crail-metadata.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part III: Metadata - </h2> - - </a> - <p class="post-meta">Posted by Adrian Schuepbach and Patrick Stuedi on November 21, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/11/rdmashuffle.html"> - <h2 class="post-title"> - Spark Shuffle: SparkRDMA vs Crail - </h2> - - </a> - <p class="post-meta">Posted by Jonas Pfefferle, Patrick Stuedi, Animesh Trivedi, Bernard Metzler, Adrian Schuepbach on November 17, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/08/crail-nvme-fabrics-v1.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part II: NVMf - </h2> - - </a> - <p class="post-meta">Posted by Jonas Pfefferle on August 22, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/08/crail-memory.html"> - <h2 class="post-title"> - Crail Storage Performance -- Part I: DRAM - </h2> - - </a> - <p class="post-meta">Posted by Patrick Stuedi on August 18, 2017</p> -</div> -<hr> - -<div class="post-preview"> - <a href="/blog/2017/01/sorting.html"> - <h2 class="post-title"> - Sorting on a 100Gbit/s Cluster using Spark/Crail - </h2> - - </a> - <p class="post-meta">Posted by Patrick Stuedi on January 17, 2017</p> -</div> -<hr> - - -<!-- Pager --> - -<ul class="pager"> - - <li class="previous"> - <a href="/blog/page3">← Newer Posts</a> - </li> - - -</ul> - - - - - <br> - <br> - <div class="footer"> - <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/";>incubation</a> at <a href="https://www.apache.org/";>The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. - </p> - </div> - - </div> <!-- /container --> - - <!-- Support retina images. --> - <script type="text/javascript" - src="http://crail.incubator.apache.org/js/srcset-polyfill.js";></script> - </body> -</html> http://git-wip-us.apache.org/repos/asf/incubator-crail-website/blob/5c23d221/content/community/index.html ---------------------------------------------------------------------- diff --git a/content/community/index.html b/content/community/index.html deleted file mode 100644 index 1a2453f..0000000 --- a/content/community/index.html +++ /dev/null @@ -1,106 +0,0 @@ -<!DOCTYPE html> -<html> - <head> - <meta charset="utf-8"> - <title>The Apache Crail (Incubating) Project: Community</title> - <meta name="viewport" content="width=device-width, initial-scale=1.0"> - <link href="http://crail.incubator.apache.org/css/bootstrap.min.css"; rel="stylesheet"> - <link href="http://crail.incubator.apache.org/css/group.css"; rel="stylesheet"> - <link rel="alternate" type="application/atom+xml" title="Atom" - href="http://crail.incubator.apache.org/blog/blog.xml";> - - <meta property="og:image" content="http://crail.incubator.apache.org/img/blog/preview/community-summary.png"; /> - <meta property="og:image:secure_url" content="http://crail.incubator.apache.org/img/blog/preview/community-summary.png"; /> - </head> - - <body> - <div class="container"> - <div class="header"> - <ul class="nav nav-pills pull-right"> - - - - <li > - <a href="http://crail.incubator.apache.org/";> - Home - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/overview/";> - Overview - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/blog/";> - Blog - </a> - </li> - - - <li class="active"> - <a href="http://crail.incubator.apache.org/community/";> - Community - </a> - </li> - - - <li > - <a href="http://crail.incubator.apache.org/documentation/";> - Documentation - </a> - </li> - - </ul> - <a href="http://crail.incubator.apache.org/";> - <img src="http://crail.incubator.apache.org/img/crail_logo.png"; - srcset="http://crail.incubator.apache.org/img/crail_logo.png"; - alt="Crail" id="logo"> - </a> - </div> - - - - <h2>Community</h2> - - - <p>If you’d like to contribute to Apache Crail, please use the follwing resources.</p> - -<h3 id="soucre-code">Source code</h3> - -The versioned source code history is available from the Apache git repository at <a href="http://git-wip-us.apache.org/repos/asf/incubator-crail.git";>http://git-wip-us.apache.org/repos/asf/incubator-crail.git</a> or from the github mirror <a href="https://github.com/apache/incubator-crail";>https://github.com/apache/incubator-crail</a> - -<h3 id="bug-tracking">Bug tracking</h3> - -<a href="https://issues.apache.org/jira/browse/CRAIL";>https://issues.apache.org/jira/browse/CRAIL</a> - -<h3 id="mailing-list">Mailing list</h3> - -<p>Please join the Crail developer mailing list for discussions and notifications. The list is at: <a href="mailto:d...@crail.incubator.apache.org";>d...@crail.incubator.apache.org</a>.</p> - -<ul> -<li><a href="mailto: dev-subscr...@crail.incubator.apache.org">Subscribe to List</a></li> -<li><a href="mailto: dev-unsubscr...@crail.incubator.apache.org">Unsubscribe from List</a></li> -<li><a href="http://mail-archives.apache.org/mod_mbox/crail-dev/";>Archives</a></li> -</ul> - -<p>To post to the list, it is necessary to subscribe to it.</p> - - - <br> - <br> - <div class="footer"> - <p>Apache Crail is an effort undergoing <a href="https://incubator.apache.org/";>incubation</a> at <a href="https://www.apache.org/";>The Apache Software Foundation (ASF)</a>, sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. - </p> - </div> - - </div> <!-- /container --> - - <!-- Support retina images. --> - <script type="text/javascript" - src="http://crail.incubator.apache.org/js/srcset-polyfill.js";></script> - </body> -</html>
![http://crail.incubator.apache.org/img/blog/preview/rdmashuffle-summary.png"](http://crail.incubator.apache.org/img/blog/preview/rdmashuffle-summary.png"){kind=link}
![http://crail.incubator.apache.org/img/crail_logo.png"](http://crail.incubator.apache.org/img/crail_logo.png"){kind=link}
![http://crail.incubator.apache.org/img/blog/rdma-shuffle/terasort.svg"](http://crail.incubator.apache.org/img/blog/rdma-shuffle/terasort.svg"){kind=link}
![http://crail.incubator.apache.org/img/blog/rdma-shuffle/sql.svg"](http://crail.incubator.apache.org/img/blog/rdma-shuffle/sql.svg"){kind=link}
![http://crail.incubator.apache.org/img/blog/preview/apache-summary.png"](http://crail.incubator.apache.org/img/blog/preview/apache-summary.png"){kind=link}
![http://crail.incubator.apache.org/img/blog/preview/blog-summary.png"](http://crail.incubator.apache.org/img/blog/preview/blog-summary.png"){kind=link}
![http://crail.incubator.apache.org/img/blog/preview/page2-summary.png"](http://crail.incubator.apache.org/img/blog/preview/page2-summary.png"){kind=link}
![http://crail.incubator.apache.org/img/blog/preview/page3-summary.png"](http://crail.incubator.apache.org/img/blog/preview/page3-summary.png"){kind=link}
![http://crail.incubator.apache.org/img/blog/preview/page4-summary.png"](http://crail.incubator.apache.org/img/blog/preview/page4-summary.png"){kind=link}
![http://crail.incubator.apache.org/img/blog/preview/community-summary.png"](http://crail.incubator.apache.org/img/blog/preview/community-summary.png"){kind=link}