Author: dmagda
Date: Thu Feb 8 23:18:20 2018
New Revision: 1823609
URL: http://svn.apache.org/viewvc?rev=1823609&view=rev
Log:
finished clustering and deployment page
Removed:
ignite/site/branches/ignite-7061/features/deploy.html
Modified:
ignite/site/branches/ignite-7061/.htaccess
ignite/site/branches/ignite-7061/arch/clustering.html
Modified: ignite/site/branches/ignite-7061/.htaccess
URL:
http://svn.apache.org/viewvc/ignite/site/branches/ignite-7061/.htaccess?rev=1823609&r1=1823608&r2=1823609&view=diff
==============================================================================
--- ignite/site/branches/ignite-7061/.htaccess (original)
+++ ignite/site/branches/ignite-7061/.htaccess Thu Feb 8 23:18:20 2018
@@ -10,6 +10,7 @@ Redirect 301 collocatedprocessing.html /
Redirect 301 /features/durablememory.html /arch/durablememory.html
Redirect 301 memorycentric.html /arch/memorycentric.html
Redirect 301 /features/persistence.html /arch/persistence.html
+Redirect 301 /features/deploy.html /arch/clustering.html
RewriteEngine On
Modified: ignite/site/branches/ignite-7061/arch/clustering.html
URL:
http://svn.apache.org/viewvc/ignite/site/branches/ignite-7061/arch/clustering.html?rev=1823609&r1=1823608&r2=1823609&view=diff
==============================================================================
--- ignite/site/branches/ignite-7061/arch/clustering.html (original)
+++ ignite/site/branches/ignite-7061/arch/clustering.html Thu Feb 8 23:18:20
2018
@@ -51,171 +51,78 @@ under the License.
<!--#include virtual="/includes/header.html" -->
<main id="main" role="main" class="container">
- <section id="memory-centric" class="page-section">
- <h1 class="first">Collocated Processing</h1>
- <div class="col-sm-12 col-md-12 col-xs-12" style="padding:0 0 20px
0;">
- <div class="col-sm-6 col-md-6 col-xs-12"
style="padding-left:0; padding-right:0">
+ <section id="deployment-options" class="page-section">
+ <h1 class="first">Clustering and Deployment</h1>
+ <div class="col-sm-12 col-md-12 col-xs-12" style="padding-left:0;
padding-right:0;">
+ <div class="col-sm-6 col-md-6 col-xs-12"
style="padding-left:0; padding-right:0;">
<p>
- The disk-centric systems, like RDBMS or NoSQL,
generally utilize the classic client-server
- approach, where the data is brought from the server to
the client side where it gets processed
- and then is usually discarded. This approach does not
scale well as moving the data over the
- network is the most expensive operation in a
distributed system.
+ Apache Ignite cluster is based on the shared-nothing
architecture which implies that all the
+ cluster nodes are equal, independent and
self-sufficient, and there is no single point of failure
+ across the cluster.
</p>
<p>
- A much more scalable approach is
<code>collocated</code> processing that reverses the flow by bringing
- the computations to the servers where the data
actually resides. This approach allows you to
- execute advanced logic or distributed SQL with JOINs
exactly where the data is stored avoiding
- expensive serialization and network trips.
+ Ignite nodes can automatically discover each other
relying on flexible Discovery SPI component.
+ This helps to scale the cluster out or down when
needed and easily.
+ </p>
+ <p>
+ Apache Ignite can run stand-alone, in a cluster,
within Kubernetes and Docker containers, as well
+ as on Apache Mesos and Hadoop Yarn. It can also run on
physical servers as well as within
+ virtual environments or clouds. Technically there are
no limitations on where the cluster
+ can be deployed.
</p>
</div>
- <div class="col-sm-6 col-md-6 col-xs-12"
style="padding-right:0">
- <img class="img-responsive"
src="/images/collocated_processing.png" width="440px" style="float: right;
margin-top: -25px;"/>
+ <div class="col-sm-6 col-md-6 col-xs-12"
style="padding-right:0;">
+ <img class="img-responsive"
src="/images/ignite-deploy.png" width="540px" style="float:right;"/>
</div>
- </div>
+ </div>
- <div class="page-heading">Data Collocation</div>
- <p>
- To start benefiting from the collocated processing, we need to
ensure that the data is properly collocated
- at the first place. If business logic requires to access more
than one entry it can be reasonable to
- collocate dependent entries by storing them on a single
cluster node. This technique is also known as
- affinity collocation of the data.
- </p>
+ <div class="page-heading">Servers, Clients and Protocols</div>
<p>
- In the example below, we have <code>Country</code> and
<code>City</code> tables and want to collocate
- <code>City</code> entries with <code>Country</code> entries
for where a city is located. To achieve this,
- we use the <code>WITH</code> clause and specify
<code>affinityKey=CountryCode</code> as shown below:
+ There are two types of nodes defined in Ignite - client and
server nodes. The nodes server a
+ different purpose.
</p>
- <div class="tab-content">
-
- <div class="tab-pane active" id="sql-tables">
- <pre class="brush:sql">
- CREATE TABLE Country (
- Code CHAR(3),
- Name CHAR(52),
- Continent CHAR(50),
- Region CHAR(26),
- SurfaceArea DECIMAL(10,2),
- Population INT(11),
- Capital INT(11),
- PRIMARY KEY (Code)
- ) WITH "template=partitioned, backups=1";
-
- CREATE TABLE City (
- ID INT(11),
- Name CHAR(35),
- CountryCode CHAR(3),
- District CHAR(20),
- Population INT(11),
- PRIMARY KEY (ID, CountryCode)
- ) WITH "template=partitioned, backups=1,
affinityKey=CountryCode";
- </pre>
- </div>
- </div>
<p>
- By collocating the tables together we can ensure that all the
entries with the same <code>affinityKey</code>
- will be stored on the same cluster node, hence avoiding costly
network trips to fetch data from other
- remote nodes.
+ The server nodes act as containers for data and computations.
Once interconnected, the server nodes will
+ represent a distributed database (or data grid) that stores
the data, participates in queries processing,
+ compute execution, stream processing, etc.
</p>
- <div class="page-heading">SQL and Distributed JOINs</div>
<p>
- Apache Ignite SQL engine will always perform much more
efficiently if a query is run against the
- collocated data. Especially it is crucial for distributed
JOINs supported by Ignite.
+ The client nodes, on their turn, provide an ability to connect
to the distributed database (a group of server nodes)
+ remotely. Ignite native clients allow using the whole set of
Ignite APIs, including SQL, transactions,
+ compute, streaming, services, etc. from the client side.
</p>
<p>
- Taking the example of the two tables created above, let's get
the most inhabited cities across China,
- Russia and the USA joining the data stored in the
<code>Country</code> and <code>City</code> tables, as follows:
+ In addition to the client nodes, you can access the cluster by
means of Ignite binary client protocol,
+ JDBC and ODBC drivers, or REST API.
</p>
- <div class="tab-content">
- <div class="tab-pane active" id="sql-joins-query">
- <pre class="brush:sql">
- SELECT country.name, city.name,
MAX(city.population) as max_pop
- FROM country
- JOIN city ON city.countrycode = country.code
- WHERE country.code IN ('USA','RUS','CHN')
- GROUP BY country.name, city.name
- ORDER BY max_pop DESC;
- </pre>
- </div>
- </div>
+ <div class="page-heading">Public Clouds</div>
<p>
- Since all the cities were collocated with their countries
beforehand, Ignite will execute the query and
- join the data on concrete nodes that store China, Russia and
the USA entries. This approach, again, <i>avoids</i>
- expensive data movement across the network, needed to complete
the query, because all the data will be available
- on the nodes locally.
+ For public cloud environments, Ignite has native integration
with <code class="text-nowrap">Amazon AWS</code>
+ and <code class="text-nowrap">Google Compute Engine</code>.
For all other cloud environments, Ignite
+ integrates with <code class="text-nowrap">Apache
JClouds</code> that has support for most of the
+ <a href="https://jclouds.apache.org/reference/providers/";
target="_blank" class="text-nowrap">existing
+ cloud providers</a>.
</p>
- <div class="page-heading">Distributed Collocated Computations</div>
+ <div class="page-heading">Containers</div>
<p>
- Apache Ignite compute grid and machine learning component
allows to perform computations and execute
- machine learning algorithms in parallel fashion to gain high
performance, low latency, and linear scalability.
- Furthermore, both components highly rely on data collocation
(and collocated processing in general) giving
- the ways to optimize specific tasks or calculations.
+ Ignite can freely run within container environments. Ignite's
integration with <code>Docker</code>
+ allows to automatically build and deploy user code into Ignite
prior to server startup.
</p>
<p>
- For instance, let's assume that a blizzard is approaching New
York. As a telecommunication company,
- you have to send a warning text message to 8 million New
Yorkers.
- With the client-server approach the company has to move all
<nobr>8 million (!)</nobr> records
- from the database to the client text messaging application,
which does not scale.
- </p>
- <p>
- A much more efficient approach would be to send the
text-messaging logic, implemented with compute grid,
- to the cluster node responsible for storing the New York
residents.
- This approach moves only 1 computation instead of 8 million
records across the network, and performs a lot better.
+ Apache Ignite cluster can also be deployed in
<code>Kubernetes</code> that allows automatic deployment and management of
+ containerized applications.
</p>
+ <div class="page-heading">Resource Managers</div>
<p>
- Here is an example of how this logic might look like:
+ Ignite provides native integration for <code
class="text-nowrap">Hadoop Yarn</code>,<code>Kubernetes</code>,
+ and <code class="text-nowrap">Apache Mesos</code> making it
very easy to seamlessly deploy Ignite
+ into specific environments and manage it there.
</p>
- <!--<ul id="compute-example" class="nav nav-tabs">-->
- <!--<li class="active"><a href="#compute-task"
aria-controls="profile" data-toggle="tab">Message Broadcasting Logic</a></li>-->
- <!--</ul>-->
-
- <div class="tab-content">
-
- <div class="tab-pane active" id="compute-task">
- <pre class="brush:java">
-
-Ignite ignite = ...
-
-// NewYork ID.
-long newYorkId = 2;
-
-// Sending the logic to a cluster node that stores NewYork and all its
inhabitants.
-ignite.compute().affinityRun("City", newYorkId, new IgniteRunnable() {
- @IgniteInstanceResource
- Ignite ignite;
-
- @Override
- public void run() {
- // Getting an access to Persons cache.
- IgniteCache<BinaryObject, BinaryObject> people =
ignite.cache("Person").withKeepBinary();
-
-
- ScanQuery<BinaryObject, BinaryObject> query = new ScanQuery
<BinaryObject, BinaryObject>();
-
- try (QueryCursor<Cache.Entry<BinaryObject, BinaryObject>>
cursor = people.query(query)) {
- // Iteration over the local cluster node data using the scan query.
- for (Cache.Entry<BinaryObject, BinaryObject> entry : cursor) {
- BinaryObject personKey = entry.getKey();
-
- // Picking NewYorker's only.
- if (personKey.<Long>field("CITY_ID") == newYorkId) {
- person = entry.getValue();
-
- // Sending the warning message to the person.
- }
- }
- }
- }
-}
-
- </pre>
- </div>
- </div>
-
- <div class="page-heading">More on Collocated Processing</div>
- <table class="formatted" name="More on Ignite Transactions">
+ <div class="page-heading">More on Clustering and Deployment</div>
+ <table class="formatted" name="Deployment Options Features">
<thead>
<tr>
<th width="35%" class="left">Feature</th>
@@ -224,59 +131,127 @@ ignite.compute().affinityRun("City", new
</thead>
<tbody>
<tr>
- <td class="features-left">Affinity Collocation</td>
+ <td class="left">Clients and Servers</td>
<td>
<p>
- If business logic requires to access more than one
entry it can be reasonable to
- collocate dependent entries by storing them on a
single cluster node:
+ There are two types of nodes defined in Ignite -
client and server nodes. The nodes server a
+ different purpose:
</p>
<div class="page-links">
- <a
href="https://apacheignite.readme.io/docs/affinity-collocation";
target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
+ <a
href="https://apacheignite.readme.io/docs/clients-vs-servers";
target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
- <td class="features-left">Collocated Computations</td>
+ <td class="left">Cross-platform Support</td>
<td>
<p>
- It is also possible to route computations to the
nodes where the data is stored:
+ Applications can access the cluster from numerous
programming languages as well as JDBC/ODBC
+ drivers, and additional protocols:
</p>
<div class="page-links">
- <a
href="https://apacheignite.readme.io/docs/collocate-compute-and-data";
target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
+ <a href="/features/multilanguage.html"
target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
- <td class="features-left">Compute Grid</td>
+ <td class="left">Zero Deployment</td>
<td>
<p>
- Distributed computations are performed in parallel
fashion to gain high performance, low latency, and linear scalability:
+ Ignite nodes automatically become aware of custom
classes, eliminating the need to explicitly deploy them:
</p>
<div class="page-links">
- <a
href="https://apacheignite.readme.io/docs/compute-grid"; target="docs">Docs for
this feature <i class="fa fa-angle-double-right"></i></a>
+ <a
href="https://apacheignite.readme.io/docs/zero-deployment"; target="docs">Docs
for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
- <td class="features-left">Distributed JOINs</td>
+ <td class="left">Dynamic Schema Changes</td>
<td>
<p>
- Ignite supports collocated and non-collocated
distributed SQL joins:
+ Ignite stores objects in binary format eliminating
the need to deploy classes on server nodes:
</p>
<div class="page-links">
- <a
href="https://apacheignite-sql.readme.io/docs/distributed-joins";
target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
+ <a
href="https://apacheignite.readme.io/docs/binary-marshaller"; target="docs">Docs
for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
- <td class="features-left">Machine Learning</td>
+ <td class="left">Stand-Alone Clusters</td>
<td>
<p>
- Ignite machine learning component allows users to
run ML/DL training and inference directly
- on the data stored in an Ignite cluster and
provides ML and DL algorithms:
+ Ignite nodes can automatically discover each
other. This helps to scale the cluster
+ when needed, without having to restart the whole
cluster - simply start up new additional
+ nodes and they will automatically join the
existing work loads:
</p>
- <div class="page-links">
- <a
href="https://apacheignite.readme.io/docs/machine-learning"; target="docs">Docs
for this feature <i class="fa fa-angle-double-right"></i></a>
+ <div class="page-links">
+ <a
href="http://apacheignite.readme.io/docs/cluster"; target="docs">Docs for this
Feature <i class="fa fa-angle-double-right"></i></a>
+ </div>
+ </td>
+ </tr>
+ <tr>
+ <td class="left">Docker Container</td>
+ <td>
+ <p>
+ <code>Docker</code> allows to package Ignite
deployment with all the dependencies into a
+ standard container. Docker automates downloading
the Ignite release, deploying users'
+ code into Ignite, and configuring nodes. It also
automatically starts up the fully
+ configured Ignite node:
+ </p>
+ <div class="page-links">
+ <a
href="http://apacheignite.readme.io/docs/docker-deployment"; target="docs">Docs
for this Feature <i class="fa fa-angle-double-right"></i></a>
+ </div>
+ </td>
+ </tr>
+ <tr>
+ <td class="left">Kubernetes</td>
+ <td>
+ <p>
+ Apache Ignite cluster can be easily deployed in
and maintained by <code>Kubernetes</code> which is an
+ open-source system for automating deployment,
scaling, and management of containerized applications:
+ </p>
+ <div class="page-links">
+ <a
href="https://apacheignite.readme.io/docs/kubernetes-deployment";
target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
+ </div>
+ </td>
+ </tr>
+ <tr>
+ <td class="left">Public Clouds</td>
+ <td>
+ <p>
+ For public cloud environments, Ignite has native
integration with <code class="text-nowrap">Amazon AWS</code>
+ and <code class="text-nowrap">Google Compute
Engine</code>. For all other cloud environments, Ignite
+ integrates with <code class="text-nowrap">Apache
JClouds</code> which has support for most of the
+ <a
href="https://jclouds.apache.org/reference/providers/"; target="_blank"
class="text-nowrap">existing cloud providers</a>.
+ </p>
+ <div class="page-links" style="line-height: 1.3;">
+ <a
href="https://apacheignite-mix.readme.io/docs/amazon-aws"; target="docs">Amazon
AWS Integration <i class="fa fa-angle-double-right"></i></a><br/>
+ <a
href="https://apacheignite-mix.readme.io/docs/google-compute-engine";
target="docs">Google Cloud Integration <i class="fa
fa-angle-double-right"></i></a><br/>
+ <a
href="https://apacheignite-mix.readme.io/docs/jclouds"; target="docs">Apache
JClouds Integration <i class="fa fa-angle-double-right"></i></a>
+ </div>
+ </td>
+ </tr>
+ <tr>
+ <td class="left">Apache Mesos</td>
+ <td>
+ <p>
+ Ignite provides native integration for
<code>Apache Mesos</code> making it very easy
+ to deploy into Mesos data centers, such as, for
example, Hadoop and Spark environments:
+ </p>
+ <div class="page-links">
+ <a
href="http://apacheignite.readme.io/docs/mesos-deployment"; target="docs">Docs
for this Feature <i class="fa fa-angle-double-right"></i></a>
+ </div>
+ </td>
+ </tr>
+ <tr>
+ <td class="left">Hadoop Yarn</td>
+ <td>
+ <p>
+ Ignite provides native integration for
<code>Hadoop Yarn</code> making it very easy
+ to deploy into Hadoop and Spark environments:
+ </p>
+ <div class="page-links">
+ <a
href="http://apacheignite.readme.io/docs/yarn-deployment"; target="docs">Docs
for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
