This is an automated email from the ASF dual-hosted git repository.
dmagda pushed a commit to branch IGNITE-7595
in repository https://gitbox.apache.org/repos/asf/ignite.git
The following commit(s) were added to refs/heads/IGNITE-7595 by this push:
new 0a5c414 Ignite docs: ported some missed pages from readme
0a5c414 is described below
commit 0a5c414cd80fb50e864f9e6909207352abc3d15b
Author: Denis Magda <dma...@gridgain.com>
AuthorDate: Mon Oct 5 14:27:19 2020 -0700
Ignite docs: ported some missed pages from readme
---
docs/_data/toc.yaml | 8 +
docs/_docs/data-structures/id-generator.adoc | 62 ++++++++
docs/_docs/distributed-locks.adoc | 45 ++++++
docs/_docs/messaging.adoc | 92 ++++++++++++
.../yardstick-benchmarking.adoc | 162 +++++++++++++++++++++
5 files changed, 369 insertions(+)
diff --git a/docs/_data/toc.yaml b/docs/_data/toc.yaml
index d675358..d33963b 100644
--- a/docs/_data/toc.yaml
+++ b/docs/_data/toc.yaml
@@ -336,6 +336,8 @@
url: key-value-api/continuous-queries
- title: Using Ignite Services
url: services/services
+- title: Using Ignite Messaging
+ url: messaging
- title: Distributed Data Structures
items:
- title: Queue and Set
@@ -348,6 +350,10 @@
url: data-structures/atomic-sequence
- title: Semaphore
url: data-structures/semaphore
+ - title: ID Generator
+ url: data-structures/id-generator
+- title: Distributed Locks
+ url: distributed-locks
- title: REST API
url: restapi
- title: .NET Specific
@@ -535,3 +541,5 @@
url: /perf-and-troubleshooting/troubleshooting
- title: Handling Exceptions
url: /perf-and-troubleshooting/handling-exceptions
+ - title: Benchmarking With Yardstick
+ url: /perf-and-troubleshooting/yardstick-benchmarking
diff --git a/docs/_docs/data-structures/id-generator.adoc
b/docs/_docs/data-structures/id-generator.adoc
new file mode 100644
index 0000000..0ad395b
--- /dev/null
+++ b/docs/_docs/data-structures/id-generator.adoc
@@ -0,0 +1,62 @@
+= Distributed ID Generator
+
+== Overview
+
+The distributed atomic sequence provided by the `IgniteCacheAtomicSequence`
interface is similar to the distributed atomic long,
+but its value can only go up. It also supports reserving a range of values to
avoid costly network trips or cluster updates
+every time a sequence must provide a next value. That is, when you perform
`incrementAndGet()` (or any other atomic operation)
+on an atomic sequence, the data structure reserves ahead a range of values,
which are guaranteed to be unique across the
+cluster for this sequence instance.
+
+As a result, the atomic sequence is a suitable and efficient data structure
for the implementation of a
+distributed ID generator. For instance, such a generator can be used to
produce unique primary keys across the whole cluster.
+
+Here is an example of how atomic sequence can be created:
+
+[tabs]
+--
+tab:Java[]
+[source, java]
+----
+Ignite ignite = Ignition.ignite();
+
+IgniteAtomicSequence seq = ignite.atomicSequence(
+ "seqName", // Sequence name.
+ 0, // Initial value for sequence.
+ true // Create if it does not exist.
+);
+----
+--
+
+Below is a simple usage example:
+
+[tabs]
+--
+tab:Java[]
+[source, java]
+----
+Ignite ignite = Ignition.ignite();
+
+// Initialize atomic sequence.
+final IgniteAtomicSequence seq = ignite.atomicSequence("seqName", 0, true);
+
+// Increment atomic sequence.
+for (int i = 0; i < 20; i++) {
+ long currentValue = seq.get();
+ long newValue = seq.incrementAndGet();
+
+ ...
+}
+----
+--
+
+== Sequence Reserve Size
+
+The key parameter of `IgniteAtomicSequence` is `atomicSequenceReserveSize`
which is the number of sequence values reserved
+per node. When a node tries to obtain an instance of `IgniteAtomicSequence`, a
number of sequence values will be reserved
+for that node and consequent increments of sequence will happen locally
without communication with other nodes, until
+the next reservation has to be made.
+
+The default value for `atomicSequenceReserveSize` is `1000`. This default
setting can be changed by modifying the
+`atomicSequenceReserveSize` property of `AtomicConfiguration`.
+
diff --git a/docs/_docs/distributed-locks.adoc
b/docs/_docs/distributed-locks.adoc
new file mode 100644
index 0000000..3198340
--- /dev/null
+++ b/docs/_docs/distributed-locks.adoc
@@ -0,0 +1,45 @@
+= Distributed Locks
+
+== Overview
+
+Ignite transactions acquire distributed locks implicitly. However, there are
certain use cases when you might need to
+acquire the locks explicitly. The `lock()` method of the `IgniteCache` API
returns an instance of `java.util.concurrent.locks.Lock`
+that lets you define explicit distributed locks for any given key. Locks can
also be acquired on a collection of objects using the
+`IgniteCache.lockAll()` method.
+
+[tabs]
+--
+tab:Java[]
+[source, java]
+----
+IgniteCache<String, Integer> cache = ignite.cache("myCache");
+
+// Create a lock for the given key
+Lock lock = cache.lock("keyLock");
+try {
+ // Acquire the lock
+ lock.lock();
+
+ cache.put("Hello", 11);
+ cache.put("World", 22);
+}
+finally {
+ // Release the lock
+ lock.unlock();
+}
+----
+--
+
+[NOTE]
+====
+[discrete]
+=== Atomicity Mode
+In Ignite, locks are supported only for the `TRANSACTIONAL` atomicity mode,
which can be set via the
+`CacheConfiguration.atomicityMode` parameter.
+====
+
+== Locks and Transactions
+
+Explicit locks are not transactional and cannot not be used from within
transactions (exception will be thrown).
+If you do need explicit locking within transactions, then you should use the
`TransactionConcurrency.PESSIMISTIC` concurrency
+control for transactions which will acquire explicit locks for relevant
cluster data requests.
diff --git a/docs/_docs/messaging.adoc b/docs/_docs/messaging.adoc
new file mode 100644
index 0000000..4647501
--- /dev/null
+++ b/docs/_docs/messaging.adoc
@@ -0,0 +1,92 @@
+= Topic-Based Messaging With Apache Ignite
+
+== Overview
+
+Ignite distributed messaging enables topic-based cluster-wide communication
between all nodes. Messages via a specified
+message topic can be distributed to all or sub-group of nodes that have
subscribed to that topic.
+
+Ignite messaging is based on the publish-subscribe paradigm where publishers
and subscribers are tethered together with
+a common topic. When one of the nodes sends a message `A` for topic `T`, it is
published on all nodes that have subscribed to `T`.
+
+[NOTE]
+====
+[discrete]
+Any new node joining the cluster automatically gets subscribed to all the
topics that other nodes in the cluster
+(or link:distributed-computing/cluster-groups[cluster group]) are subscribed
to.
+====
+
+== IgniteMessaging
+
+Distributed messaging functionality in Ignite is available via the
`IgniteMessaging` interface. You can get an instance
+of `IgniteMessaging` like so:
+
+[tabs]
+--
+tab:Java[]
+[source, java]
+----
+Ignite ignite = Ignition.ignite();
+
+// Messaging instance over this cluster.
+IgniteMessaging msg = ignite.message();
+
+// Messaging instance over given cluster group (in this case, remote nodes).
+IgniteMessaging rmtMsg = ignite.message(ignite.cluster().forRemotes());
+----
+--
+
+== Publish Messages
+
+Send methods help sending/publishing messages with a specified message topic
to all nodes. Messages can be sent
+in _ordered_ or _unordered_ manner.
+
+=== Ordered Messages
+
+The `sendOrdered(...)` method can be used if you want to receive messages in
the order they were sent. The timeout parameter
+is passed to specify how long a message will stay in the queue to wait for
messages that are supposed to be sent before
+this message. If the timeout expires, then all the messages that have not yet
arrived for a given topic on that node will be ignored.
+
+=== Unordered Messages
+
+The `send(...)` methods do not guarantee message ordering. This means that,
when you sequentially send message `A` and
+message `B`, you are not guaranteed that the target node first receives `A`
and then `B`.
+
+== Subscribe for Messages
+
+Listen methods help to listen/subscribe for messages. When these methods are
called, a listener with specified message
+topic is registered on all (or sub-group of ) nodes to listen for new
messages. With listen methods, a predicate is
+passed that returns a boolean value which tells the listener to continue or
stop listening for new messages.
+
+=== Local Listen
+
+The `localListen(...)` method registers a message listener with specified
topic only on the local node and listens for
+messages from any node in the _given_ cluster group.
+
+=== Remote Listen
+
+The `remoteListen(...)` method registers message listeners with specified
topic on all nodes in the _given_ cluster group
+and listens for messages from any node in _this_ cluster group.
+
+== Example
+
+[tabs]
+--
+tab:Java[]
+[source, java]
+----
+Ignite ignite = Ignition.ignite();
+
+IgniteMessaging rmtMsg = ignite.message(ignite.cluster().forRemotes());
+
+// Add listener for ordered messages on all remote nodes.
+rmtMsg.remoteListen("MyOrderedTopic", (nodeId, msg) -> {
+ System.out.println("Received ordered message [msg=" + msg + ", from=" +
nodeId + ']');
+
+ return true; // Return true to continue listening.
+});
+
+// Send ordered messages to remote nodes.
+for (int i = 0; i < 10; i++)
+ rmtMsg.sendOrdered("MyOrderedTopic", Integer.toString(i),0);
+----
+--
diff --git a/docs/_docs/perf-and-troubleshooting/yardstick-benchmarking.adoc
b/docs/_docs/perf-and-troubleshooting/yardstick-benchmarking.adoc
new file mode 100644
index 0000000..65954f7
--- /dev/null
+++ b/docs/_docs/perf-and-troubleshooting/yardstick-benchmarking.adoc
@@ -0,0 +1,162 @@
+= Benchmarking Ignite With Yardstick Framework
+
+== Yardstick Ignite Benchmarks
+
+Apache Ignite benchmarks are written on top of the Yardstick Framework,
allowing you to measure the performance of
+various Apache Ignite components and modules. The documentation below
describes how to execute and configure pre-assembled
+benchmarks. If you need to add new benchmarks
+or build existing one, then refer to the instructions from Ignite's
`DEVNOTES.txt` file in the source directory.
+
+Visit the https://github.com/gridgain/yardstick[Yardstick Repository,
window=_blank] for more details on the resulting graphs generation
+and how the framework works.
+
+== Running Ignite Benchmarks Locally
+
+The simplest way to start with benchmarking is to use one of the executable
scripts available under the `benchmarks/bin` directory:
+
+[tabs]
+--
+tab:Shell[]
+[source, shell]
+----
+./bin/benchmark-run-all.sh config/benchmark-sample.properties
+----
+--
+
+The command above will benchmark the cache `put` operations for a distributed
atomic cache. The results of the benchmark
+will be added to an auto-generated `output/results-{DATE-TIME}+\` directory.
+
+If the `./bin/benchmark-run-all.sh` command is executed as-is without any
parameters and modifications in the configuration
+file, then all the available benchmarks will be executed on a local machine
using the `config/benchmark.properties`
+configuration. In case of any issues, refer to the logs that are added to an
auto-generated `output/logs-{DATE-TIME}` directory.
+
+For more information about available benchmarks and configuration parameters,
refer to the
+<<existing-benchmarks,Existing Benchmarks>> and
<<properties-and-command-line-arguments,Properties And Command Line Arguments>>
+sections below.
+
+== Running Ignite Benchmarks Remotely
+
+To benchmark Apache Ignite across several remote hosts:
+
+. Go to `config/ignite-remote-config.xml` and replace
`<value>127.0.0.1:47500..47509</value>` with actual IPs of all the remote
+hosts. Refer to the documentation section below if you prefer to use another
kind of IP finder: link:clustering/clustering[Cluster Configuration]
+. Go to `config/benchmark-remote-sample.properties` and replace `localhost`
with actual IPs of the remote hosts in the following places:
+`SERVERS=localhost,localhost`
+`DRIVERS=localhost,localhost`
+ where the `DRIVER` is a host (usually an Ignite client node) that executes
benchmarking logic. `SERVERS` are Ignite nodes
+that are benchmarked. Replace the `localhost` occurrences in the same places
in the `config/benchmark-remote.properties`
+file if you plan to execute a full set of benchmarks available.
+. Upload Ignite Yardstick Benchmarks to one of your `DRIVERS` host in its own
working directory.
+. Log in on the remote host that will be the `DRIVER`, and execute the
following command:
++
+[tabs]
+--
+tab:Shell[]
+[source, shell]
+----
+./bin/benchmark-run-all.sh config/benchmark-remote-sample.properties
+----
+--
+
+By default, all the necessary files will be automatically uploaded from the
host in which you run the command above to
+every other host to the same path. If you prefer to do it manually set the
`AUTO_COPY` variable in property file to `false`.
+
+The command above will benchmark the cache put operation for a distributed
atomic cache. The results of the benchmark will
+be added to an auto-generated `output/results-{DATE-TIME}` directory.
+
+If you want to execute all the available benchmarks across the remote hosts,
then execute the following command on the `DRIVER` side:
+[tabs]
+--
+tab:Shell[]
+[source, shell]
+----
+./bin/benchmark-run-all.sh config/benchmark-remote.properties
+----
+--
+
+== Existing Benchmarks
+
+The following benchmarks are provided by default:
+
+. `GetBenchmark` - benchmarks atomic distributed cache get operation.
+. `PutBenchmark` - benchmarks atomic distributed cache put operation.
+. `PutGetBenchmark` - benchmarks atomic distributed cache put and get
operations together.
+. `PutTxBenchmark` - benchmarks transactional distributed cache put operation.
+. `PutGetTxBenchmark` - benchmarks transactional distributed cache put and get
operations together.
+. `SqlQueryBenchmark` - benchmarks distributed SQL query over cached data.
+. `SqlQueryJoinBenchmark` - benchmarks distributed SQL query with a Join over
cached data.
+. `SqlQueryPutBenchmark` - benchmarks distributed SQL query with simultaneous
cache updates.
+. `AffinityCallBenchmark` - benchmarks affinity call operation.
+. `ApplyBenchmark` - benchmarks apply operation.
+. `BroadcastBenchmark` - benchmarks broadcast operations.
+. `ExecuteBenchmark` - benchmarks execute operations.
+. `RunBenchmark` - benchmarks running task operations.
+. `PutGetOffHeapBenchmark` - benchmarks atomic distributed cache put and get
operations together off-heap.
+. `PutGetOffHeapValuesBenchmark` - benchmarks atomic distributed cache put
value operations off-heap.
+. `PutOffHeapBenchmark` - benchmarks atomic distributed cache put operations
off-heap.
+. `PutOffHeapValuesBenchmark` - benchmarks atomic distributed cache get value
operations off-heap.
+. `PutTxOffHeapBenchmark` - benchmarks transactional distributed cache put
operation off-heap.
+. `PutTxOffHeapValuesBenchmark` - benchmarks transactional distributed cache
put value operation off-heap.
+. `SqlQueryOffHeapBenchmark` -benchmarks distributed SQL query over cached
data off-heap.
+. `SqlQueryJoinOffHeapBenchmark` - benchmarks distributed SQL query with a
Join over cached data off-heap.
+. `SqlQueryPutOffHeapBenchmark` - benchmarks distributed SQL query with
simultaneous cache updates off-heap.
+. `PutAllBenchmark` - benchmarks atomic distributed cache batch put operation.
+. `PutAllTxBenchmark` - benchmarks transactional distributed cache batch put
operation.
+
+== Properties And Command Line Arguments
+
+Note that this section only describes configuration parameters specific to
Ignite benchmarks, and not for Yardstick framework.
+To run Ignite benchmarks and generate graphs, you will need to run them using
the Yardstick framework scripts in the `bin` folder.
+
+Refer to the
https://github.com/gridgain/yardstick/blob/master/README.md[Yardstick
Documentation, window=_blank] for common Yardstick
+properties and command line arguments for running Yardstick scripts.
+
+The following Ignite benchmark properties can be defined in the benchmark
configuration:
+
+* `-b <num>` or `--backups <num>` - Number of backups for every key.
+* `-cfg <path>` or `--Config <path>` - Path to Ignite configuration file.
+* `-cs` or `--cacheStore` - Enable or disable cache store readThrough,
writeThrough.
+* `-cl` or `--client` - Client flag. Use this flag if you running more than
one `DRIVER`, otherwise additional drivers would behave like a `servers`.
+* `-nc` or `--nearCache` - Near cache flag.
+* `-nn <num>` or `--nodeNumber <num>` - Number of nodes (automatically set in
`benchmark.properties`); used to wait for the specified number of nodes to
start.
+* `-sm <mode>` or `-syncMode <mode>` - Synchronization mode (defined in
CacheWriteSynchronizationMode`).
+* `-r <num>` or `--range` - Range of keys that are randomly generated for
cache operations.
+* `-rd or --restartdelay` - Restart delay in seconds.
+* `-rs or --restartsleep` - Restart sleep in seconds.
+* `-rth <host>` or `--restHost <host>` - REST TCP host.
+* `-rtp <num>` or `--restPort <num>` - REST TCP port, indicates that a Ignite
node is ready to process Ignite Clients.
+* `-ss` or `--syncSend` - Flag indicating whether synchronous send is used in
`TcpCommunicationSpi`.
+* `-txc <value>` or `--txConcurrency <value>` - Cache transaction concurrency
control, either `OPTIMISTIC` or `PESSIMISTIC` (defined in `CacheTxConcurrency`).
+* `-txi <value>` or `--txIsolation <value>` - Cache transaction isolation
(defined in `CacheTxIsolation`).
+* `-wb` or `--writeBehind` - Enable or disable writeBehind for cache store.
+
+For example, if you want to run 2 `IgniteNode` servers on localhost with the
`PutBenchmark` benchmark, number of
+backups set to `1`, and synchronization mode set to `PRIMARY_SYNC`, then the
following configuration should be specified
+in the `benchmark.properties` file:
+[tabs]
+--
+tab:Shell[]
+[source, shell]
+----
+SERVER_HOSTS=localhost,localhost
+...
+
+# Note that -dn and -sn, which stand for data node and server node,
+# are native Yardstick parameters and are documented in
+# Yardstick framework.
+CONFIGS="-b 1 -sm PRIMARY_SYNC -dn PutBenchmark`IgniteNode"
+----
+--
+
+== Building From Sources
+
+Run `mvn clean package -Pyardstick -pl modules/yardstick -am -DskipTests` in
the Apache Ignite root directory.
+
+This command will compile the project and also unpack the scripts from
`yardstick-resources.zip` file to `modules/yardstick/target/assembly/bin`
directory.
+
+Artifacts can be found in the `modules/yardstick/target/assembly` directory.
+
+== Custom Ignite Benchmarks
+
+All benchmarks extend the `AbstractBenchmark` class. A new benchmark should
also extend this abstract class and
+implement the `test` method (this is the method that actually tests
performance).
