Author: bobby
Date: Thu Mar 17 02:56:15 2016
New Revision: 1735362
URL: http://svn.apache.org/viewvc?rev=1735362&view=rev
Log:
Some more cleanup of some of the release specific pages
Added:
storm/branches/bobby-versioned-site/releases/0.10.0/Daemon-Fault-Tolerance.md
- copied, changed from r1735360,
storm/branches/bobby-versioned-site/releases/0.10.0/Fault-tolerance.md
storm/branches/bobby-versioned-site/releases/0.10.0/images/ack_tree.png
(with props)
storm/branches/bobby-versioned-site/releases/0.10.0/images/tuple-dag.png
(with props)
storm/branches/bobby-versioned-site/releases/0.10.0/images/tuple_tree.png
(with props)
Removed:
storm/branches/bobby-versioned-site/releases/0.10.0/Acking-framework-implementation.md
storm/branches/bobby-versioned-site/releases/0.10.0/Fault-tolerance.md
Modified:
storm/branches/bobby-versioned-site/releases/0.10.0/Guaranteeing-message-processing.md
storm/branches/bobby-versioned-site/releases/0.10.0/Implementation-docs.md
storm/branches/bobby-versioned-site/releases/0.10.0/index.md
Copied:
storm/branches/bobby-versioned-site/releases/0.10.0/Daemon-Fault-Tolerance.md
(from r1735360,
storm/branches/bobby-versioned-site/releases/0.10.0/Fault-tolerance.md)
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.10.0/Daemon-Fault-Tolerance.md?p2=storm/branches/bobby-versioned-site/releases/0.10.0/Daemon-Fault-Tolerance.md&p1=storm/branches/bobby-versioned-site/releases/0.10.0/Fault-tolerance.md&r1=1735360&r2=1735362&rev=1735362&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.10.0/Fault-tolerance.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.10.0/Daemon-Fault-Tolerance.md
Thu Mar 17 02:56:15 2016
@@ -1,14 +1,14 @@
---
-title: Fault Tolerance
+title: Daemon Fault Tolerance
layout: documentation
documentation: true
version: v0.10.0
---
-This page explains the design details of Storm that make it a fault-tolerant
system.
+Storm has several different daemon processes. Nimbus that schedules workers,
supervisors that launch and kill workers, the log viewer that gives access to
logs, and the UI that shows the status of a cluster.
## What happens when a worker dies?
-When a worker dies, the supervisor will restart it. If it continuously fails
on startup and is unable to heartbeat to Nimbus, Nimbus will reassign the
worker to another machine.
+When a worker dies, the supervisor will restart it. If it continuously fails
on startup and is unable to heartbeat to Nimbus, Nimbus will reschedule the
worker.
## What happens when a node dies?
Modified:
storm/branches/bobby-versioned-site/releases/0.10.0/Guaranteeing-message-processing.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.10.0/Guaranteeing-message-processing.md?rev=1735362&r1=1735361&r2=1735362&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.10.0/Guaranteeing-message-processing.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.10.0/Guaranteeing-message-processing.md
Thu Mar 17 02:56:15 2016
@@ -4,7 +4,8 @@ layout: documentation
documentation: true
version: v0.10.0
---
-Storm guarantees that each message coming off a spout will be fully processed.
This page describes how Storm accomplishes this guarantee and what you have to
do as a user to benefit from Storm's reliability capabilities.
+Storm offers several different levels of guaranteed message processing,
includeing best effort, at least once, and exactly once through
[Trident](Trident-tutorial.html).
+This page describes how Storm can guarantee at least once processing.
### What does it mean for a message to be "fully processed"?
@@ -132,12 +133,11 @@ In contrast, bolts that do aggregations
### How do I make my applications work correctly given that tuples can be
replayed?
-As always in software design, the answer is "it depends." Storm 0.7.0
introduced the "transactional topologies" feature, which enables you to get
fully fault-tolerant exactly-once messaging semantics for most computations.
Read more about transactional topologies [here](Transactional-topologies.html).
-
+As always in software design, the answer is "it depends." If you really want
exactly once semantics use the [Trident](Trident-tutorial.html) API. In some
cases, like with a lot of analytics, dropping data is OK so disabling the fault
tolerance by setting the number of acker bolts to 0
[Config.TOPOLOGY_ACKERS](javadocs/backtype/storm/Config.html#TOPOLOGY_ACKERS).
But in some cases you want to be sure that everything was processed at least
once and nothing was dropped. This is especially useful if all operations are
idenpotent or if deduping can happen aferwards.
### How does Storm implement reliability in an efficient way?
-A Storm topology has a set of special "acker" tasks that track the DAG of
tuples for every spout tuple. When an acker sees that a DAG is complete, it
sends a message to the spout task that created the spout tuple to ack the
message. You can set the number of acker tasks for a topology in the topology
configuration using
[Config.TOPOLOGY_ACKERS](javadocs/backtype/storm/Config.html#TOPOLOGY_ACKERS).
Storm defaults TOPOLOGY_ACKERS to one task -- you will need to increase this
number for topologies processing large amounts of messages.
+A Storm topology has a set of special "acker" tasks that track the DAG of
tuples for every spout tuple. When an acker sees that a DAG is complete, it
sends a message to the spout task that created the spout tuple to ack the
message. You can set the number of acker tasks for a topology in the topology
configuration using
[Config.TOPOLOGY_ACKERS](javadocs/backtype/storm/Config.html#TOPOLOGY_ACKERS).
Storm defaults TOPOLOGY_ACKERS to one task per worker.
The best way to understand Storm's reliability implementation is to look at
the lifecycle of tuples and tuple DAGs. When a tuple is created in a topology,
whether in a spout or a bolt, it is given a random 64 bit id. These ids are
used by ackers to track the tuple DAG for every spout tuple.
Modified:
storm/branches/bobby-versioned-site/releases/0.10.0/Implementation-docs.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.10.0/Implementation-docs.md?rev=1735362&r1=1735361&r2=1735362&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.10.0/Implementation-docs.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.10.0/Implementation-docs.md
Thu Mar 17 02:56:15 2016
@@ -9,7 +9,6 @@ This section of the wiki is dedicated to
- [Structure of the codebase](Structure-of-the-codebase.html)
- [Lifecycle of a topology](Lifecycle-of-a-topology.html)
- [Message passing implementation](Message-passing-implementation.html)
-- [Acking framework implementation](Acking-framework-implementation.html)
- [Metrics](Metrics.html)
- How transactional topologies work
- subtopology for TransactionalSpout
Added: storm/branches/bobby-versioned-site/releases/0.10.0/images/ack_tree.png
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.10.0/images/ack_tree.png?rev=1735362&view=auto
==============================================================================
Binary file - no diff available.
Propchange:
storm/branches/bobby-versioned-site/releases/0.10.0/images/ack_tree.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: storm/branches/bobby-versioned-site/releases/0.10.0/images/tuple-dag.png
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.10.0/images/tuple-dag.png?rev=1735362&view=auto
==============================================================================
Binary file - no diff available.
Propchange:
storm/branches/bobby-versioned-site/releases/0.10.0/images/tuple-dag.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: storm/branches/bobby-versioned-site/releases/0.10.0/images/tuple_tree.png
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.10.0/images/tuple_tree.png?rev=1735362&view=auto
==============================================================================
Binary file - no diff available.
Propchange:
storm/branches/bobby-versioned-site/releases/0.10.0/images/tuple_tree.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Modified: storm/branches/bobby-versioned-site/releases/0.10.0/index.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.10.0/index.md?rev=1735362&r1=1735361&r2=1735362&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.10.0/index.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.10.0/index.md Thu Mar 17
02:56:15 2016
@@ -10,7 +10,7 @@ version: v0.10.0
* [Concepts](Concepts.html)
* [Configuration](Configuration.html)
* [Guaranteeing message processing](Guaranteeing-message-processing.html)
-* [Fault-tolerance](Fault-tolerance.html)
+* [Daemon Fault Tolerance](Daemon-Fault-Tolerance.html)
* [Command line client](Command-line-client.html)
* [REST API](STORM-UI-REST-API.html)
* [Understanding the parallelism of a Storm
topology](Understanding-the-parallelism-of-a-Storm-topology.html)
