Author: bobby
Date: Thu Mar 17 20:19:17 2016
New Revision: 1735492
URL: http://svn.apache.org/viewvc?rev=1735492&view=rev
Log:
Updated 0.9.6 docs
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Acking-framework-implementation.md
storm/branches/bobby-versioned-site/releases/0.9.6/Clojure-DSL.md
storm/branches/bobby-versioned-site/releases/0.9.6/Command-line-client.md
storm/branches/bobby-versioned-site/releases/0.9.6/Common-patterns.md
storm/branches/bobby-versioned-site/releases/0.9.6/Concepts.md
storm/branches/bobby-versioned-site/releases/0.9.6/Configuration.md
storm/branches/bobby-versioned-site/releases/0.9.6/Contributing-to-Storm.md
storm/branches/bobby-versioned-site/releases/0.9.6/Creating-a-new-Storm-project.md
storm/branches/bobby-versioned-site/releases/0.9.6/DSLs-and-multilang-adapters.md
storm/branches/bobby-versioned-site/releases/0.9.6/Defining-a-non-jvm-language-dsl-for-storm.md
storm/branches/bobby-versioned-site/releases/0.9.6/Distributed-RPC.md
storm/branches/bobby-versioned-site/releases/0.9.6/Documentation.md
storm/branches/bobby-versioned-site/releases/0.9.6/FAQ.md
storm/branches/bobby-versioned-site/releases/0.9.6/Fault-tolerance.md
storm/branches/bobby-versioned-site/releases/0.9.6/Guaranteeing-message-processing.md
storm/branches/bobby-versioned-site/releases/0.9.6/Hooks.md
storm/branches/bobby-versioned-site/releases/0.9.6/Implementation-docs.md
storm/branches/bobby-versioned-site/releases/0.9.6/Installing-native-dependencies.md
storm/branches/bobby-versioned-site/releases/0.9.6/Kestrel-and-Storm.md
storm/branches/bobby-versioned-site/releases/0.9.6/Lifecycle-of-a-topology.md
storm/branches/bobby-versioned-site/releases/0.9.6/Local-mode.md
storm/branches/bobby-versioned-site/releases/0.9.6/Maven.md
storm/branches/bobby-versioned-site/releases/0.9.6/Message-passing-implementation.md
storm/branches/bobby-versioned-site/releases/0.9.6/Metrics.md
storm/branches/bobby-versioned-site/releases/0.9.6/Multilang-protocol.md
storm/branches/bobby-versioned-site/releases/0.9.6/Powered-By.md
storm/branches/bobby-versioned-site/releases/0.9.6/Project-ideas.md
storm/branches/bobby-versioned-site/releases/0.9.6/Rationale.md
storm/branches/bobby-versioned-site/releases/0.9.6/Running-topologies-on-a-production-cluster.md
storm/branches/bobby-versioned-site/releases/0.9.6/SECURITY.md
storm/branches/bobby-versioned-site/releases/0.9.6/STORM-UI-REST-API.md
storm/branches/bobby-versioned-site/releases/0.9.6/Serialization-(prior-to-0.6.0).md
storm/branches/bobby-versioned-site/releases/0.9.6/Serialization.md
storm/branches/bobby-versioned-site/releases/0.9.6/Serializers.md
storm/branches/bobby-versioned-site/releases/0.9.6/Setting-up-a-Storm-cluster.md
storm/branches/bobby-versioned-site/releases/0.9.6/Setting-up-development-environment.md
storm/branches/bobby-versioned-site/releases/0.9.6/Spout-implementations.md
storm/branches/bobby-versioned-site/releases/0.9.6/Storm-multi-language-protocol-(versions-0.7.0-and-below).md
storm/branches/bobby-versioned-site/releases/0.9.6/Structure-of-the-codebase.md
storm/branches/bobby-versioned-site/releases/0.9.6/Support-for-non-java-languages.md
storm/branches/bobby-versioned-site/releases/0.9.6/Transactional-topologies.md
storm/branches/bobby-versioned-site/releases/0.9.6/Trident-API-Overview.md
storm/branches/bobby-versioned-site/releases/0.9.6/Trident-spouts.md
storm/branches/bobby-versioned-site/releases/0.9.6/Trident-state.md
storm/branches/bobby-versioned-site/releases/0.9.6/Trident-tutorial.md
storm/branches/bobby-versioned-site/releases/0.9.6/Troubleshooting.md
storm/branches/bobby-versioned-site/releases/0.9.6/Tutorial.md
storm/branches/bobby-versioned-site/releases/0.9.6/Understanding-the-parallelism-of-a-Storm-topology.md
storm/branches/bobby-versioned-site/releases/0.9.6/Using-non-JVM-languages-with-Storm.md
storm/branches/bobby-versioned-site/releases/0.9.6/index.md
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Acking-framework-implementation.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Acking-framework-implementation.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Acking-framework-implementation.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Acking-framework-implementation.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
[Storm's
acker](https://github.com/apache/incubator-storm/blob/46c3ba7/storm-core/src/clj/backtype/storm/daemon/acker.clj#L28)
tracks completion of each tupletree with a checksum hash: each time a tuple is
sent, its value is XORed into the checksum, and each time a tuple is acked its
value is XORed in again. If all tuples have been successfully acked, the
checksum will be zero (the odds that the checksum will be zero otherwise are
vanishingly small).
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Clojure-DSL.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Clojure-DSL.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Clojure-DSL.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Clojure-DSL.md Thu Mar
17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
Storm comes with a Clojure DSL for defining spouts, bolts, and topologies. The
Clojure DSL has access to everything the Java API exposes, so if you're a
Clojure user you can code Storm topologies without touching Java at all. The
Clojure DSL is defined in the source in the
[backtype.storm.clojure](https://github.com/apache/incubator-storm/blob/0.5.3/src/clj/backtype/storm/clojure.clj)
namespace.
@@ -36,11 +37,11 @@ The maps of spout and bolt specs are map
#### spout-spec
-`spout-spec` takes as arguments the spout implementation (an object that
implements [IRichSpout](/apidocs/backtype/storm/topology/IRichSpout.html)) and
optional keyword arguments. The only option that exists currently is the `:p`
option, which specifies the parallelism for the spout. If you omit `:p`, the
spout will execute as a single task.
+`spout-spec` takes as arguments the spout implementation (an object that
implements [IRichSpout](javadocs/backtype/storm/topology/IRichSpout.html)) and
optional keyword arguments. The only option that exists currently is the `:p`
option, which specifies the parallelism for the spout. If you omit `:p`, the
spout will execute as a single task.
#### bolt-spec
-`bolt-spec` takes as arguments the input declaration for the bolt, the bolt
implementation (an object that implements
[IRichBolt](/apidocs/backtype/storm/topology/IRichBolt.html)), and optional
keyword arguments.
+`bolt-spec` takes as arguments the input declaration for the bolt, the bolt
implementation (an object that implements
[IRichBolt](javadocs/backtype/storm/topology/IRichBolt.html)), and optional
keyword arguments.
The input declaration is a map from stream ids to stream groupings. A stream
id can have one of two forms:
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Command-line-client.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Command-line-client.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Command-line-client.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Command-line-client.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This page describes all the commands that are possible with the "storm"
command line client. To learn how to set up your "storm" client to talk to a
remote cluster, follow the instructions in [Setting up development
environment](Setting-up-a-development-environment.html).
@@ -23,7 +24,7 @@ These commands are:
Syntax: `storm jar topology-jar-path class ...`
-Runs the main method of `class` with the specified arguments. The storm jars
and configs in `~/.storm` are put on the classpath. The process is configured
so that [StormSubmitter](/apidocs/backtype/storm/StormSubmitter.html) will
upload the jar at `topology-jar-path` when the topology is submitted.
+Runs the main method of `class` with the specified arguments. The storm jars
and configs in `~/.storm` are put on the classpath. The process is configured
so that [StormSubmitter](javadocs/backtype/storm/StormSubmitter.html) will
upload the jar at `topology-jar-path` when the topology is submitted.
### kill
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Common-patterns.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Common-patterns.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Common-patterns.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Common-patterns.md Thu
Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This page lists a variety of common patterns in Storm topologies.
@@ -37,7 +38,7 @@ If you want reliability in your data pro
If the bolt emits tuples, then you may want to use multi-anchoring to ensure
reliability. It all depends on the specific application. See [Guaranteeing
message processing](Guaranteeing-message-processing.html) for more details on
how reliability works.
### BasicBolt
-Many bolts follow a similar pattern of reading an input tuple, emitting zero
or more tuples based on that input tuple, and then acking that input tuple
immediately at the end of the execute method. Bolts that match this pattern are
things like functions and filters. This is such a common pattern that Storm
exposes an interface called
[IBasicBolt](/apidocs/backtype/storm/topology/IBasicBolt.html) that automates
this pattern for you. See [Guaranteeing message
processing](Guaranteeing-message-processing.html) for more information.
+Many bolts follow a similar pattern of reading an input tuple, emitting zero
or more tuples based on that input tuple, and then acking that input tuple
immediately at the end of the execute method. Bolts that match this pattern are
things like functions and filters. This is such a common pattern that Storm
exposes an interface called
[IBasicBolt](javadocs/backtype/storm/topology/IBasicBolt.html) that automates
this pattern for you. See [Guaranteeing message
processing](Guaranteeing-message-processing.html) for more information.
### In-memory caching + fields grouping combo
@@ -73,11 +74,11 @@ This pattern works because of the fields
### TimeCacheMap for efficiently keeping a cache of things that have been
recently updated
-You sometimes want to keep a cache in memory of items that have been recently
"active" and have items that have been inactive for some time be automatically
expires. [TimeCacheMap](/apidocs/backtype/storm/utils/TimeCacheMap.html) is an
efficient data structure for doing this and provides hooks so you can insert
callbacks whenever an item is expired.
+You sometimes want to keep a cache in memory of items that have been recently
"active" and have items that have been inactive for some time be automatically
expires. [TimeCacheMap](javadocs/backtype/storm/utils/TimeCacheMap.html) is an
efficient data structure for doing this and provides hooks so you can insert
callbacks whenever an item is expired.
### CoordinatedBolt and KeyedFairBolt for Distributed RPC
-When building distributed RPC applications on top of Storm, there are two
common patterns that are usually needed. These are encapsulated by
[CoordinatedBolt](/apidocs/backtype/storm/task/CoordinatedBolt.html) and
[KeyedFairBolt](/apidocs/backtype/storm/task/KeyedFairBolt.html) which are part
of the "standard library" that ships with the Storm codebase.
+When building distributed RPC applications on top of Storm, there are two
common patterns that are usually needed. These are encapsulated by
[CoordinatedBolt](javadocs/backtype/storm/task/CoordinatedBolt.html) and
[KeyedFairBolt](javadocs/backtype/storm/task/KeyedFairBolt.html) which are part
of the "standard library" that ships with the Storm codebase.
`CoordinatedBolt` wraps the bolt containing your logic and figures out when
your bolt has received all the tuples for any given request. It makes heavy use
of direct streams to do this.
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Concepts.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Concepts.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Concepts.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Concepts.md Thu Mar 17
20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This page lists the main concepts of Storm and links to resources where you
can find more information. The concepts discussed are:
@@ -19,7 +20,7 @@ The logic for a realtime application is
**Resources:**
-* [TopologyBuilder](/apidocs/backtype/storm/topology/TopologyBuilder.html):
use this class to construct topologies in Java
+* [TopologyBuilder](javadocs/backtype/storm/topology/TopologyBuilder.html):
use this class to construct topologies in Java
* [Running topologies on a production
cluster](Running-topologies-on-a-production-cluster.html)
* [Local mode](Local-mode.html): Read this to learn how to develop and test
topologies in local mode.
@@ -27,30 +28,30 @@ The logic for a realtime application is
The stream is the core abstraction in Storm. A stream is an unbounded sequence
of tuples that is processed and created in parallel in a distributed fashion.
Streams are defined with a schema that names the fields in the stream's tuples.
By default, tuples can contain integers, longs, shorts, bytes, strings,
doubles, floats, booleans, and byte arrays. You can also define your own
serializers so that custom types can be used natively within tuples.
-Every stream is given an id when declared. Since single-stream spouts and
bolts are so common,
[OutputFieldsDeclarer](/apidocs/backtype/storm/topology/OutputFieldsDeclarer.html)
has convenience methods for declaring a single stream without specifying an
id. In this case, the stream is given the default id of "default".
+Every stream is given an id when declared. Since single-stream spouts and
bolts are so common,
[OutputFieldsDeclarer](javadocs/backtype/storm/topology/OutputFieldsDeclarer.html)
has convenience methods for declaring a single stream without specifying an
id. In this case, the stream is given the default id of "default".
**Resources:**
-* [Tuple](/apidocs/backtype/storm/tuple/Tuple.html): streams are composed of
tuples
-*
[OutputFieldsDeclarer](/apidocs/backtype/storm/topology/OutputFieldsDeclarer.html):
used to declare streams and their schemas
+* [Tuple](javadocs/backtype/storm/tuple/Tuple.html): streams are composed of
tuples
+*
[OutputFieldsDeclarer](javadocs/backtype/storm/topology/OutputFieldsDeclarer.html):
used to declare streams and their schemas
* [Serialization](Serialization.html): Information about Storm's dynamic
typing of tuples and declaring custom serializations
-* [ISerialization](/apidocs/backtype/storm/serialization/ISerialization.html):
custom serializers must implement this interface
-*
[CONFIG.TOPOLOGY_SERIALIZATIONS](/apidocs/backtype/storm/Config.html#TOPOLOGY_SERIALIZATIONS):
custom serializers can be registered using this configuration
+* [ISerialization](javadocs/backtype/storm/serialization/ISerialization.html):
custom serializers must implement this interface
+*
[CONFIG.TOPOLOGY_SERIALIZATIONS](javadocs/backtype/storm/Config.html#TOPOLOGY_SERIALIZATIONS):
custom serializers can be registered using this configuration
### Spouts
A spout is a source of streams in a topology. Generally spouts will read
tuples from an external source and emit them into the topology (e.g. a Kestrel
queue or the Twitter API). Spouts can either be __reliable__ or __unreliable__.
A reliable spout is capable of replaying a tuple if it failed to be processed
by Storm, whereas an unreliable spout forgets about the tuple as soon as it is
emitted.
-Spouts can emit more than one stream. To do so, declare multiple streams using
the `declareStream` method of
[OutputFieldsDeclarer](/apidocs/backtype/storm/topology/OutputFieldsDeclarer.html)
and specify the stream to emit to when using the `emit` method on
[SpoutOutputCollector](/apidocs/backtype/storm/spout/SpoutOutputCollector.html).
+Spouts can emit more than one stream. To do so, declare multiple streams using
the `declareStream` method of
[OutputFieldsDeclarer](javadocs/backtype/storm/topology/OutputFieldsDeclarer.html)
and specify the stream to emit to when using the `emit` method on
[SpoutOutputCollector](javadocs/backtype/storm/spout/SpoutOutputCollector.html).
The main method on spouts is `nextTuple`. `nextTuple` either emits a new tuple
into the topology or simply returns if there are no new tuples to emit. It is
imperative that `nextTuple` does not block for any spout implementation,
because Storm calls all the spout methods on the same thread.
-The other main methods on spouts are `ack` and `fail`. These are called when
Storm detects that a tuple emitted from the spout either successfully completed
through the topology or failed to be completed. `ack` and `fail` are only
called for reliable spouts. See [the
Javadoc](/apidocs/backtype/storm/spout/ISpout.html) for more information.
+The other main methods on spouts are `ack` and `fail`. These are called when
Storm detects that a tuple emitted from the spout either successfully completed
through the topology or failed to be completed. `ack` and `fail` are only
called for reliable spouts. See [the
Javadoc](javadocs/backtype/storm/spout/ISpout.html) for more information.
**Resources:**
-* [IRichSpout](/apidocs/backtype/storm/topology/IRichSpout.html): this is the
interface that spouts must implement.
+* [IRichSpout](javadocs/backtype/storm/topology/IRichSpout.html): this is the
interface that spouts must implement.
* [Guaranteeing message processing](Guaranteeing-message-processing.html)
### Bolts
@@ -59,52 +60,52 @@ All processing in topologies is done in
Bolts can do simple stream transformations. Doing complex stream
transformations often requires multiple steps and thus multiple bolts. For
example, transforming a stream of tweets into a stream of trending images
requires at least two steps: a bolt to do a rolling count of retweets for each
image, and one or more bolts to stream out the top X images (you can do this
particular stream transformation in a more scalable way with three bolts than
with two).
-Bolts can emit more than one stream. To do so, declare multiple streams using
the `declareStream` method of
[OutputFieldsDeclarer](/apidocs/backtype/storm/topology/OutputFieldsDeclarer.html)
and specify the stream to emit to when using the `emit` method on
[OutputCollector](/apidocs/backtype/storm/task/OutputCollector.html).
+Bolts can emit more than one stream. To do so, declare multiple streams using
the `declareStream` method of
[OutputFieldsDeclarer](javadocs/backtype/storm/topology/OutputFieldsDeclarer.html)
and specify the stream to emit to when using the `emit` method on
[OutputCollector](javadocs/backtype/storm/task/OutputCollector.html).
-When you declare a bolt's input streams, you always subscribe to specific
streams of another component. If you want to subscribe to all the streams of
another component, you have to subscribe to each one individually.
[InputDeclarer](/apidocs/backtype/storm/topology/InputDeclarer.html) has
syntactic sugar for subscribing to streams declared on the default stream id.
Saying `declarer.shuffleGrouping("1")` subscribes to the default stream on
component "1" and is equivalent to `declarer.shuffleGrouping("1",
DEFAULT_STREAM_ID)`.
+When you declare a bolt's input streams, you always subscribe to specific
streams of another component. If you want to subscribe to all the streams of
another component, you have to subscribe to each one individually.
[InputDeclarer](javadocs/backtype/storm/topology/InputDeclarer.html) has
syntactic sugar for subscribing to streams declared on the default stream id.
Saying `declarer.shuffleGrouping("1")` subscribes to the default stream on
component "1" and is equivalent to `declarer.shuffleGrouping("1",
DEFAULT_STREAM_ID)`.
-The main method in bolts is the `execute` method which takes in as input a new
tuple. Bolts emit new tuples using the
[OutputCollector](/apidocs/backtype/storm/task/OutputCollector.html) object.
Bolts must call the `ack` method on the `OutputCollector` for every tuple they
process so that Storm knows when tuples are completed (and can eventually
determine that its safe to ack the original spout tuples). For the common case
of processing an input tuple, emitting 0 or more tuples based on that tuple,
and then acking the input tuple, Storm provides an
[IBasicBolt](/apidocs/backtype/storm/topology/IBasicBolt.html) interface which
does the acking automatically.
+The main method in bolts is the `execute` method which takes in as input a new
tuple. Bolts emit new tuples using the
[OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) object.
Bolts must call the `ack` method on the `OutputCollector` for every tuple they
process so that Storm knows when tuples are completed (and can eventually
determine that its safe to ack the original spout tuples). For the common case
of processing an input tuple, emitting 0 or more tuples based on that tuple,
and then acking the input tuple, Storm provides an
[IBasicBolt](javadocs/backtype/storm/topology/IBasicBolt.html) interface which
does the acking automatically.
-Its perfectly fine to launch new threads in bolts that do processing
asynchronously.
[OutputCollector](/apidocs/backtype/storm/task/OutputCollector.html) is
thread-safe and can be called at any time.
+Its perfectly fine to launch new threads in bolts that do processing
asynchronously.
[OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) is
thread-safe and can be called at any time.
**Resources:**
-* [IRichBolt](/apidocs/backtype/storm/topology/IRichBolt.html): this is
general interface for bolts.
-* [IBasicBolt](/apidocs/backtype/storm/topology/IBasicBolt.html): this is a
convenience interface for defining bolts that do filtering or simple functions.
-* [OutputCollector](/apidocs/backtype/storm/task/OutputCollector.html): bolts
emit tuples to their output streams using an instance of this class
+* [IRichBolt](javadocs/backtype/storm/topology/IRichBolt.html): this is
general interface for bolts.
+* [IBasicBolt](javadocs/backtype/storm/topology/IBasicBolt.html): this is a
convenience interface for defining bolts that do filtering or simple functions.
+* [OutputCollector](javadocs/backtype/storm/task/OutputCollector.html): bolts
emit tuples to their output streams using an instance of this class
* [Guaranteeing message processing](Guaranteeing-message-processing.html)
### Stream groupings
Part of defining a topology is specifying for each bolt which streams it
should receive as input. A stream grouping defines how that stream should be
partitioned among the bolt's tasks.
-There are seven built-in stream groupings in Storm, and you can implement a
custom stream grouping by implementing the
[CustomStreamGrouping](/apidocs/backtype/storm/grouping/CustomStreamGrouping.html)
interface:
+There are seven built-in stream groupings in Storm, and you can implement a
custom stream grouping by implementing the
[CustomStreamGrouping](javadocs/backtype/storm/grouping/CustomStreamGrouping.html)
interface:
1. **Shuffle grouping**: Tuples are randomly distributed across the bolt's
tasks in a way such that each bolt is guaranteed to get an equal number of
tuples.
2. **Fields grouping**: The stream is partitioned by the fields specified in
the grouping. For example, if the stream is grouped by the "user-id" field,
tuples with the same "user-id" will always go to the same task, but tuples with
different "user-id"'s may go to different tasks.
3. **All grouping**: The stream is replicated across all the bolt's tasks. Use
this grouping with care.
4. **Global grouping**: The entire stream goes to a single one of the bolt's
tasks. Specifically, it goes to the task with the lowest id.
5. **None grouping**: This grouping specifies that you don't care how the
stream is grouped. Currently, none groupings are equivalent to shuffle
groupings. Eventually though, Storm will push down bolts with none groupings to
execute in the same thread as the bolt or spout they subscribe from (when
possible).
-6. **Direct grouping**: This is a special kind of grouping. A stream grouped
this way means that the __producer__ of the tuple decides which task of the
consumer will receive this tuple. Direct groupings can only be declared on
streams that have been declared as direct streams. Tuples emitted to a direct
stream must be emitted using one of the
[emitDirect](/apidocs/backtype/storm/task/OutputCollector.html#emitDirect(int,
int, java.util.List) methods. A bolt can get the task ids of its consumers by
either using the provided
[TopologyContext](/apidocs/backtype/storm/task/TopologyContext.html) or by
keeping track of the output of the `emit` method in
[OutputCollector](/apidocs/backtype/storm/task/OutputCollector.html) (which
returns the task ids that the tuple was sent to).
+6. **Direct grouping**: This is a special kind of grouping. A stream grouped
this way means that the __producer__ of the tuple decides which task of the
consumer will receive this tuple. Direct groupings can only be declared on
streams that have been declared as direct streams. Tuples emitted to a direct
stream must be emitted using one of the
[emitDirect](javadocs/backtype/storm/task/OutputCollector.html#emitDirect(int,
int, java.util.List) methods. A bolt can get the task ids of its consumers by
either using the provided
[TopologyContext](javadocs/backtype/storm/task/TopologyContext.html) or by
keeping track of the output of the `emit` method in
[OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) (which
returns the task ids that the tuple was sent to).
7. **Local or shuffle grouping**: If the target bolt has one or more tasks in
the same worker process, tuples will be shuffled to just those in-process
tasks. Otherwise, this acts like a normal shuffle grouping.
**Resources:**
-* [TopologyBuilder](/apidocs/backtype/storm/topology/TopologyBuilder.html):
use this class to define topologies
-* [InputDeclarer](/apidocs/backtype/storm/topology/InputDeclarer.html): this
object is returned whenever `setBolt` is called on `TopologyBuilder` and is
used for declaring a bolt's input streams and how those streams should be
grouped
-* [CoordinatedBolt](/apidocs/backtype/storm/task/CoordinatedBolt.html): this
bolt is useful for distributed RPC topologies and makes heavy use of direct
streams and direct groupings
+* [TopologyBuilder](javadocs/backtype/storm/topology/TopologyBuilder.html):
use this class to define topologies
+* [InputDeclarer](javadocs/backtype/storm/topology/InputDeclarer.html): this
object is returned whenever `setBolt` is called on `TopologyBuilder` and is
used for declaring a bolt's input streams and how those streams should be
grouped
+* [CoordinatedBolt](javadocs/backtype/storm/task/CoordinatedBolt.html): this
bolt is useful for distributed RPC topologies and makes heavy use of direct
streams and direct groupings
### Reliability
Storm guarantees that every spout tuple will be fully processed by the
topology. It does this by tracking the tree of tuples triggered by every spout
tuple and determining when that tree of tuples has been successfully completed.
Every topology has a "message timeout" associated with it. If Storm fails to
detect that a spout tuple has been completed within that timeout, then it fails
the tuple and replays it later.
-To take advantage of Storm's reliability capabilities, you must tell Storm
when new edges in a tuple tree are being created and tell Storm whenever you've
finished processing an individual tuple. These are done using the
[OutputCollector](/apidocs/backtype/storm/task/OutputCollector.html) object
that bolts use to emit tuples. Anchoring is done in the `emit` method, and you
declare that you're finished with a tuple using the `ack` method.
+To take advantage of Storm's reliability capabilities, you must tell Storm
when new edges in a tuple tree are being created and tell Storm whenever you've
finished processing an individual tuple. These are done using the
[OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) object
that bolts use to emit tuples. Anchoring is done in the `emit` method, and you
declare that you're finished with a tuple using the `ack` method.
This is all explained in much more detail in [Guaranteeing message
processing](Guaranteeing-message-processing.html).
### Tasks
-Each spout or bolt executes as many tasks across the cluster. Each task
corresponds to one thread of execution, and stream groupings define how to send
tuples from one set of tasks to another set of tasks. You set the parallelism
for each spout or bolt in the `setSpout` and `setBolt` methods of
[TopologyBuilder](/apidocs/backtype/storm/topology/TopologyBuilder.html).
+Each spout or bolt executes as many tasks across the cluster. Each task
corresponds to one thread of execution, and stream groupings define how to send
tuples from one set of tasks to another set of tasks. You set the parallelism
for each spout or bolt in the `setSpout` and `setBolt` methods of
[TopologyBuilder](javadocs/backtype/storm/topology/TopologyBuilder.html).
### Workers
@@ -112,4 +113,4 @@ Topologies execute across one or more wo
**Resources:**
-*
[Config.TOPOLOGY_WORKERS](/apidocs/backtype/storm/Config.html#TOPOLOGY_WORKERS):
this config sets the number of workers to allocate for executing the topology
\ No newline at end of file
+*
[Config.TOPOLOGY_WORKERS](javadocs/backtype/storm/Config.html#TOPOLOGY_WORKERS):
this config sets the number of workers to allocate for executing the topology
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Configuration.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Configuration.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Configuration.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Configuration.md Thu Mar
17 20:19:17 2016
@@ -1,9 +1,10 @@
---
layout: documentation
+version: v0.9.6
---
Storm has a variety of configurations for tweaking the behavior of nimbus,
supervisors, and running topologies. Some configurations are system
configurations and cannot be modified on a topology by topology basis, whereas
other configurations can be modified per topology.
-Every configuration has a default value defined in
[defaults.yaml](https://github.com/apache/incubator-storm/blob/master/conf/defaults.yaml)
in the Storm codebase. You can override these configurations by defining a
storm.yaml in the classpath of Nimbus and the supervisors. Finally, you can
define a topology-specific configuration that you submit along with your
topology when using
[StormSubmitter](/apidocs/backtype/storm/StormSubmitter.html). However, the
topology-specific configuration can only override configs prefixed with
"TOPOLOGY".
+Every configuration has a default value defined in
[defaults.yaml](https://github.com/apache/incubator-storm/blob/master/conf/defaults.yaml)
in the Storm codebase. You can override these configurations by defining a
storm.yaml in the classpath of Nimbus and the supervisors. Finally, you can
define a topology-specific configuration that you submit along with your
topology when using
[StormSubmitter](javadocs/backtype/storm/StormSubmitter.html). However, the
topology-specific configuration can only override configs prefixed with
"TOPOLOGY".
Storm 0.7.0 and onwards lets you override configuration on a
per-bolt/per-spout basis. The only configurations that can be overriden this
way are:
@@ -22,8 +23,8 @@ The preference order for configuration v
**Resources:**
-* [Config](/apidocs/backtype/storm/Config.html): a listing of all
configurations as well as a helper class for creating topology specific
configurations
+* [Config](javadocs/backtype/storm/Config.html): a listing of all
configurations as well as a helper class for creating topology specific
configurations
*
[defaults.yaml](https://github.com/apache/incubator-storm/blob/master/conf/defaults.yaml):
the default values for all configurations
* [Setting up a Storm cluster](Setting-up-a-Storm-cluster.html): explains how
to create and configure a Storm cluster
* [Running topologies on a production
cluster](Running-topologies-on-a-production-cluster.html): lists useful
configurations when running topologies on a cluster
-* [Local mode](Local-mode.html): lists useful configurations when using local
mode
\ No newline at end of file
+* [Local mode](Local-mode.html): lists useful configurations when using local
mode
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Contributing-to-Storm.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Contributing-to-Storm.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Contributing-to-Storm.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Contributing-to-Storm.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
### Getting started with contributing
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Creating-a-new-Storm-project.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Creating-a-new-Storm-project.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Creating-a-new-Storm-project.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Creating-a-new-Storm-project.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This page outlines how to set up a Storm project for development. The steps
are:
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/DSLs-and-multilang-adapters.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/DSLs-and-multilang-adapters.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/DSLs-and-multilang-adapters.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/DSLs-and-multilang-adapters.md
Thu Mar 17 20:19:17 2016
@@ -1,9 +1,10 @@
---
layout: documentation
+version: v0.9.6
---
* [Scala DSL](https://github.com/velvia/ScalaStorm)
* [JRuby DSL](https://github.com/colinsurprenant/redstorm)
* [Clojure DSL](Clojure-DSL.html)
* [Storm/Esper integration](https://github.com/tomdz/storm-esper): Streaming
SQL on top of Storm
* [io-storm](https://github.com/gphat/io-storm): Perl multilang adapter
-* [storm-php](https://github.com/lazyshot/storm-php): PHP multilang adapter
\ No newline at end of file
+* [storm-php](https://github.com/lazyshot/storm-php): PHP multilang adapter
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Defining-a-non-jvm-language-dsl-for-storm.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Defining-a-non-jvm-language-dsl-for-storm.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Defining-a-non-jvm-language-dsl-for-storm.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Defining-a-non-jvm-language-dsl-for-storm.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
The right place to start to learn how to make a non-JVM DSL for Storm is
[storm-core/src/storm.thrift](https://github.com/apache/incubator-storm/blob/master/storm-core/src/storm.thrift).
Since Storm topologies are just Thrift structures, and Nimbus is a Thrift
daemon, you can create and submit topologies in any language.
@@ -33,4 +34,4 @@ Then you can connect to Nimbus using the
void submitTopology(1: string name, 2: string uploadedJarLocation, 3: string
jsonConf, 4: StormTopology topology) throws (1: AlreadyAliveException e, 2:
InvalidTopologyException ite);
```
-Finally, one of the key things to do in a non-JVM DSL is make it easy to
define the entire topology in one file (the bolts, spouts, and the definition
of the topology).
\ No newline at end of file
+Finally, one of the key things to do in a non-JVM DSL is make it easy to
define the entire topology in one file (the bolts, spouts, and the definition
of the topology).
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Distributed-RPC.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Distributed-RPC.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Distributed-RPC.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Distributed-RPC.md Thu
Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
The idea behind distributed RPC (DRPC) is to parallelize the computation of
really intense functions on the fly using Storm. The Storm topology takes in as
input a stream of function arguments, and it emits an output stream of the
results for each of those function calls.
@@ -22,7 +23,7 @@ A client sends the DRPC server the name
### LinearDRPCTopologyBuilder
-Storm comes with a topology builder called
[LinearDRPCTopologyBuilder](/apidocs/backtype/storm/drpc/LinearDRPCTopologyBuilder.html)
that automates almost all the steps involved for doing DRPC. These include:
+Storm comes with a topology builder called
[LinearDRPCTopologyBuilder](javadocs/backtype/storm/drpc/LinearDRPCTopologyBuilder.html)
that automates almost all the steps involved for doing DRPC. These include:
1. Setting up the spout
2. Returning the results to the DRPC server
@@ -194,4 +195,4 @@ The rest of the topology should be self-
### Advanced
* KeyedFairBolt for weaving the processing of multiple requests at the same
time
-* How to use `CoordinatedBolt` directly
\ No newline at end of file
+* How to use `CoordinatedBolt` directly
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Documentation.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Documentation.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Documentation.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Documentation.md Thu Mar
17 20:19:17 2016
@@ -1,9 +1,10 @@
---
layout: documentation
+version: v0.9.6
---
### Basics of Storm
-* [Javadoc](http://nathanmarz.github.com/storm)
+* [Javadoc](javadocs/index.html)
* [Concepts](Concepts.html)
* [Configuration](Configuration.html)
* [Guaranteeing message processing](Guaranteeing-message-processing.html)
@@ -47,4 +48,4 @@ Trident is an alternative interface to S
* [Defining a non-JVM language DSL for
Storm](Defining-a-non-jvm-language-dsl-for-storm.html)
* [Multilang protocol](Multilang-protocol.html) (how to provide support for
another language)
-* [Implementation docs](Implementation-docs.html)
\ No newline at end of file
+* [Implementation docs](Implementation-docs.html)
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/FAQ.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/FAQ.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/FAQ.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/FAQ.md Thu Mar 17
20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
## Best Practices
@@ -118,4 +119,4 @@ You cannot know that all events are coll
* Set a time limit using domain knowledge
* Introduce a _punctuation_: a record known to come after all records in the
given time bucket. Trident uses this scheme to know when a batch is complete.
If you for instance receive records from a set of sensors, each in order for
that sensor, then once all sensors have sent you a 3:02:xx or later timestamp
lets you know you can commit.
* When possible, make your process incremental: each value that comes in makes
the answer more an more true. A Trident ReducerAggregator is an operator that
takes a prior result and a set of new records and returns a new result. This
lets the result be cached and serialized to a datastore; if a server drops off
line for a day and then comes back with a full day's worth of data in a rush,
the old results will be calmly retrieved and updated.
-* Lambda architecture: Record all events into an archival store (S3, HBase,
HDFS) on receipt. in the fast layer, once the time window is clear, process the
bucket to get an actionable answer, and ignore everything older than the time
window. Periodically run a global aggregation to calculate a "correct" answer.
\ No newline at end of file
+* Lambda architecture: Record all events into an archival store (S3, HBase,
HDFS) on receipt. in the fast layer, once the time window is clear, process the
bucket to get an actionable answer, and ignore everything older than the time
window. Periodically run a global aggregation to calculate a "correct" answer.
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Fault-tolerance.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Fault-tolerance.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Fault-tolerance.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Fault-tolerance.md Thu
Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This page explains the design details of Storm that make it a fault-tolerant
system.
@@ -25,4 +26,4 @@ So the answer is that Nimbus is "sort of
## How does Storm guarantee data processing?
-Storm provides mechanisms to guarantee data processing even if nodes die or
messages are lost. See [Guaranteeing message
processing](Guaranteeing-message-processing.html) for the details.
\ No newline at end of file
+Storm provides mechanisms to guarantee data processing even if nodes die or
messages are lost. See [Guaranteeing message
processing](Guaranteeing-message-processing.html) for the details.
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Guaranteeing-message-processing.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Guaranteeing-message-processing.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Guaranteeing-message-processing.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Guaranteeing-message-processing.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
Storm guarantees that each message coming off a spout will be fully processed.
This page describes how Storm accomplishes this guarantee and what you have to
do as a user to benefit from Storm's reliability capabilities.
@@ -23,11 +24,11 @@ This topology reads sentences off of a K
-Storm considers a tuple coming off a spout "fully processed" when the tuple
tree has been exhausted and every message in the tree has been processed. A
tuple is considered failed when its tree of messages fails to be fully
processed within a specified timeout. This timeout can be configured on a
topology-specific basis using the
[Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS](/apidocs/backtype/storm/Config.html#TOPOLOGY_MESSAGE_TIMEOUT_SECS)
configuration and defaults to 30 seconds.
+Storm considers a tuple coming off a spout "fully processed" when the tuple
tree has been exhausted and every message in the tree has been processed. A
tuple is considered failed when its tree of messages fails to be fully
processed within a specified timeout. This timeout can be configured on a
topology-specific basis using the
[Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS](javadocs/backtype/storm/Config.html#TOPOLOGY_MESSAGE_TIMEOUT_SECS)
configuration and defaults to 30 seconds.
### What happens if a message is fully processed or fails to be fully
processed?
-To understand this question, let's take a look at the lifecycle of a tuple
coming off of a spout. For reference, here is the interface that spouts
implement (see the [Javadoc](/apidocs/backtype/storm/spout/ISpout.html) for
more information):
+To understand this question, let's take a look at the lifecycle of a tuple
coming off of a spout. For reference, here is the interface that spouts
implement (see the [Javadoc](javadocs/backtype/storm/spout/ISpout.html) for
more information):
```java
public interface ISpout extends Serializable {
@@ -134,7 +135,7 @@ As always in software design, the answer
### How does Storm implement reliability in an efficient way?
-A Storm topology has a set of special "acker" tasks that track the DAG of
tuples for every spout tuple. When an acker sees that a DAG is complete, it
sends a message to the spout task that created the spout tuple to ack the
message. You can set the number of acker tasks for a topology in the topology
configuration using
[Config.TOPOLOGY_ACKERS](/apidocs/backtype/storm/Config.html#TOPOLOGY_ACKERS).
Storm defaults TOPOLOGY_ACKERS to one task -- you will need to increase this
number for topologies processing large amounts of messages.
+A Storm topology has a set of special "acker" tasks that track the DAG of
tuples for every spout tuple. When an acker sees that a DAG is complete, it
sends a message to the spout task that created the spout tuple to ack the
message. You can set the number of acker tasks for a topology in the topology
configuration using
[Config.TOPOLOGY_ACKERS](javadocs/backtype/storm/Config.html#TOPOLOGY_ACKERS).
Storm defaults TOPOLOGY_ACKERS to one task -- you will need to increase this
number for topologies processing large amounts of messages.
The best way to understand Storm's reliability implementation is to look at
the lifecycle of tuples and tuple DAGs. When a tuple is created in a topology,
whether in a spout or a bolt, it is given a random 64 bit id. These ids are
used by ackers to track the tuple DAG for every spout tuple.
@@ -176,4 +177,4 @@ There are three ways to remove reliabili
The second way is to remove reliability on a message by message basis. You can
turn off tracking for an individual spout tuple by omitting a message id in the
`SpoutOutputCollector.emit` method.
-Finally, if you don't care if a particular subset of the tuples downstream in
the topology fail to be processed, you can emit them as unanchored tuples.
Since they're not anchored to any spout tuples, they won't cause any spout
tuples to fail if they aren't acked.
\ No newline at end of file
+Finally, if you don't care if a particular subset of the tuples downstream in
the topology fail to be processed, you can emit them as unanchored tuples.
Since they're not anchored to any spout tuples, they won't cause any spout
tuples to fail if they aren't acked.
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Hooks.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Hooks.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Hooks.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Hooks.md Thu Mar 17
20:19:17 2016
@@ -1,7 +1,8 @@
---
layout: documentation
+version: v0.9.6
---
-Storm provides hooks with which you can insert custom code to run on any
number of events within Storm. You create a hook by extending the
[BaseTaskHook](/apidocs/backtype/storm/hooks/BaseTaskHook.html) class and
overriding the appropriate method for the event you want to catch. There are
two ways to register your hook:
+Storm provides hooks with which you can insert custom code to run on any
number of events within Storm. You create a hook by extending the
[BaseTaskHook](javadocs/backtype/storm/hooks/BaseTaskHook.html) class and
overriding the appropriate method for the event you want to catch. There are
two ways to register your hook:
-1. In the open method of your spout or prepare method of your bolt using the
[TopologyContext#addTaskHook](/apidocs/backtype/storm/task/TopologyContext.html)
method.
-2. Through the Storm configuration using the
["topology.auto.task.hooks"](/apidocs/backtype/storm/Config.html#TOPOLOGY_AUTO_TASK_HOOKS)
config. These hooks are automatically registered in every spout or bolt, and
are useful for doing things like integrating with a custom monitoring system.
\ No newline at end of file
+1. In the open method of your spout or prepare method of your bolt using the
[TopologyContext#addTaskHook](javadocs/backtype/storm/task/TopologyContext.html)
method.
+2. Through the Storm configuration using the
["topology.auto.task.hooks"](javadocs/backtype/storm/Config.html#TOPOLOGY_AUTO_TASK_HOOKS)
config. These hooks are automatically registered in every spout or bolt, and
are useful for doing things like integrating with a custom monitoring system.
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Implementation-docs.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Implementation-docs.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Implementation-docs.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Implementation-docs.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This section of the wiki is dedicated to explaining how Storm is implemented.
You should have a good grasp of how to use Storm before reading these sections.
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Installing-native-dependencies.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Installing-native-dependencies.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Installing-native-dependencies.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Installing-native-dependencies.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
The native dependencies are only needed on actual Storm clusters. When running
Storm in local mode, Storm uses a pure Java messaging system so that you don't
need to install native dependencies on your development machine.
@@ -35,4 +36,4 @@ To get the JZMQ build to work, you may n
3. Upgrade autoconf on your machine
4. Follow the instructions in [this blog
post](http://blog.pmorelli.com/getting-zeromq-and-jzmq-running-on-mac-os-x)
-If you run into any errors when running `./configure`, [this
thread](http://stackoverflow.com/questions/3522248/how-do-i-compile-jzmq-for-zeromq-on-osx)
may provide a solution.
\ No newline at end of file
+If you run into any errors when running `./configure`, [this
thread](http://stackoverflow.com/questions/3522248/how-do-i-compile-jzmq-for-zeromq-on-osx)
may provide a solution.
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Kestrel-and-Storm.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Kestrel-and-Storm.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Kestrel-and-Storm.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Kestrel-and-Storm.md Thu
Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This page explains how to use to Storm to consume items from a Kestrel cluster.
@@ -195,4 +196,4 @@ Than, wait about 5 seconds in order to a
Now execute the program to add items to the queue and launch the Storm
topology. The order in which you launch the programs is of no importance.
-If you run the topology with TOPOLOGY_DEBUG you should see tuples being
emitted in the topology.
\ No newline at end of file
+If you run the topology with TOPOLOGY_DEBUG you should see tuples being
emitted in the topology.
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Lifecycle-of-a-topology.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Lifecycle-of-a-topology.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Lifecycle-of-a-topology.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Lifecycle-of-a-topology.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
(**NOTE**: this page is based on the 0.7.1 code; many things have changed
since then, including a split between tasks and executors, and a reorganization
of the code under `storm-core/src` rather than `src/`.)
@@ -77,4 +78,4 @@ First a couple of important notes about
- This causes the topology to be deactivated for the wait time before its
actually shut down. This gives the topology a chance to finish processing what
it's currently processing before shutting down the workers
- Changing the status during the kill transition ensures that the kill
protocol is fault-tolerant to Nimbus crashing. On startup, if the status of the
topology is "killed", Nimbus schedules the remove event to run "wait time
seconds" in the future
[code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L111)
- Removing a topology cleans out the assignment and static information from ZK
[code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L116)
-- A separate cleanup thread runs the `do-cleanup` function which will clean up
the heartbeat dir and the jars/configs stored locally.
[code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L577)
\ No newline at end of file
+- A separate cleanup thread runs the `do-cleanup` function which will clean up
the heartbeat dir and the jars/configs stored locally.
[code](https://github.com/apache/incubator-storm/blob/0.7.1/src/clj/backtype/storm/daemon/nimbus.clj#L577)
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Local-mode.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Local-mode.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Local-mode.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Local-mode.md Thu Mar 17
20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
Local mode simulates a Storm cluster in process and is useful for developing
and testing topologies. Running topologies in local mode is similar to running
topologies [on a cluster](Running-topologies-on-a-production-cluster.html).
@@ -11,7 +12,7 @@ import backtype.storm.LocalCluster;
LocalCluster cluster = new LocalCluster();
```
-You can then submit topologies using the `submitTopology` method on the
`LocalCluster` object. Just like the corresponding method on
[StormSubmitter](/apidocs/backtype/storm/StormSubmitter.html), `submitTopology`
takes a name, a topology configuration, and the topology object. You can then
kill a topology using the `killTopology` method which takes the topology name
as an argument.
+You can then submit topologies using the `submitTopology` method on the
`LocalCluster` object. Just like the corresponding method on
[StormSubmitter](javadocs/backtype/storm/StormSubmitter.html), `submitTopology`
takes a name, a topology configuration, and the topology object. You can then
kill a topology using the `killTopology` method which takes the topology name
as an argument.
To shutdown a local cluster, simple call:
@@ -21,7 +22,7 @@ cluster.shutdown();
### Common configurations for local mode
-You can see a full list of configurations
[here](/apidocs/backtype/storm/Config.html).
+You can see a full list of configurations
[here](javadocs/backtype/storm/Config.html).
1. **Config.TOPOLOGY_MAX_TASK_PARALLELISM**: This config puts a ceiling on the
number of threads spawned for a single component. Oftentimes production
topologies have a lot of parallelism (hundreds of threads) which places
unreasonable load when trying to test the topology in local mode. This config
lets you easy control that parallelism.
-2. **Config.TOPOLOGY_DEBUG**: When this is set to true, Storm will log a
message every time a tuple is emitted from any spout or bolt. This is extremely
useful for debugging.
\ No newline at end of file
+2. **Config.TOPOLOGY_DEBUG**: When this is set to true, Storm will log a
message every time a tuple is emitted from any spout or bolt. This is extremely
useful for debugging.
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Maven.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Maven.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Maven.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Maven.md Thu Mar 17
20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
To develop topologies, you'll need the Storm jars on your classpath. You
should either include the unpacked jars in the classpath for your project or
use Maven to include Storm as a development dependency. Storm is hosted on
Clojars (a Maven repository). To include Storm in your project as a development
dependency, add the following to your pom.xml:
@@ -53,4 +54,4 @@ Build javadocs with
### Building a Storm Release
-Use the file `bin/build_release.sh` to make a zipfile like the ones you would
download (and like what the bin files require in order to run daemons).
\ No newline at end of file
+Use the file `bin/build_release.sh` to make a zipfile like the ones you would
download (and like what the bin files require in order to run daemons).
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Message-passing-implementation.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Message-passing-implementation.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Message-passing-implementation.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Message-passing-implementation.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
(Note: this walkthrough is out of date as of 0.8.0. 0.8.0 revamped the message
passing infrastructure to be based on the Disruptor)
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Metrics.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Metrics.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Metrics.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Metrics.md Thu Mar 17
20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
Storm exposes a metrics interface to report summary statistics across the full
topology.
It's used internally to track the numbers you see in the Nimbus UI console:
counts of executes and acks; average process latency per bolt; worker heap
usage; and so forth.
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Multilang-protocol.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Multilang-protocol.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Multilang-protocol.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Multilang-protocol.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This page explains the multilang protocol as of Storm 0.7.1. Versions prior to
0.7.1 used a somewhat different protocol, documented
[here](Storm-multi-language-protocol-(versions-0.7.0-and-below\).html).
@@ -218,4 +219,4 @@ A "log" will log a message in the worker
```
* Note that, as of version 0.7.1, there is no longer any need for a
- shell bolt to 'sync'.
\ No newline at end of file
+ shell bolt to 'sync'.
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Powered-By.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Powered-By.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Powered-By.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Powered-By.md Thu Mar 17
20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
Want to be added to this page? Send an email
[here](mailto:[email protected]).
@@ -1025,4 +1026,4 @@ We are using Storm to track internet thr
</tr>
-</table>
\ No newline at end of file
+</table>
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Project-ideas.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Project-ideas.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Project-ideas.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Project-ideas.md Thu Mar
17 20:19:17 2016
@@ -1,6 +1,7 @@
---
layout: documentation
+version: v0.9.6
---
* **DSLs for non-JVM languages:** These DSL's should be all-inclusive and not
require any Java for the creation of topologies, spouts, or bolts. Since
topologies are [Thrift](http://thrift.apache.org/) structs, Nimbus is a Thrift
service, and bolts can be written in any language, this is possible.
* **Online machine learning algorithms:** Something like
[Mahout](http://mahout.apache.org/) but for online algorithms
- * **Suite of performance benchmarks:** These benchmarks should test Storm's
performance on CPU and IO intensive workloads. There should be benchmarks for
different classes of applications, such as stream processing (where throughput
is the priority) and distributed RPC (where latency is the priority).
\ No newline at end of file
+ * **Suite of performance benchmarks:** These benchmarks should test Storm's
performance on CPU and IO intensive workloads. There should be benchmarks for
different classes of applications, such as stream processing (where throughput
is the priority) and distributed RPC (where latency is the priority).
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Rationale.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Rationale.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Rationale.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Rationale.md Thu Mar 17
20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
The past decade has seen a revolution in data processing. MapReduce, Hadoop,
and related technologies have made it possible to store and process data at
scales previously unthinkable. Unfortunately, these data processing
technologies are not realtime systems, nor are they meant to be. There's no
hack that will turn Hadoop into a realtime system; realtime data processing has
a fundamentally different set of requirements than batch processing.
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Running-topologies-on-a-production-cluster.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Running-topologies-on-a-production-cluster.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Running-topologies-on-a-production-cluster.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Running-topologies-on-a-production-cluster.md
Thu Mar 17 20:19:17 2016
@@ -1,11 +1,12 @@
---
layout: documentation
+version: v0.9.6
---
Running topologies on a production cluster is similar to running in [Local
mode](Local-mode.html). Here are the steps:
-1) Define the topology (Use
[TopologyBuilder](/apidocs/backtype/storm/topology/TopologyBuilder.html) if
defining using Java)
+1) Define the topology (Use
[TopologyBuilder](javadocs/backtype/storm/topology/TopologyBuilder.html) if
defining using Java)
-2) Use [StormSubmitter](/apidocs/backtype/storm/StormSubmitter.html) to submit
the topology to the cluster. `StormSubmitter` takes as input the name of the
topology, a configuration for the topology, and the topology itself. For
example:
+2) Use [StormSubmitter](javadocs/backtype/storm/StormSubmitter.html) to submit
the topology to the cluster. `StormSubmitter` takes as input the name of the
topology, a configuration for the topology, and the topology itself. For
example:
```java
Config conf = new Config();
@@ -45,7 +46,7 @@ You can find out how to configure your `
### Common configurations
-There are a variety of configurations you can set per topology. A list of all
the configurations you can set can be found
[here](/apidocs/backtype/storm/Config.html). The ones prefixed with "TOPOLOGY"
can be overridden on a topology-specific basis (the other ones are cluster
configurations and cannot be overridden). Here are some common ones that are
set for a topology:
+There are a variety of configurations you can set per topology. A list of all
the configurations you can set can be found
[here](javadocs/backtype/storm/Config.html). The ones prefixed with "TOPOLOGY"
can be overridden on a topology-specific basis (the other ones are cluster
configurations and cannot be overridden). Here are some common ones that are
set for a topology:
1. **Config.TOPOLOGY_WORKERS**: This sets the number of worker processes to
use to execute the topology. For example, if you set this to 25, there will be
25 Java processes across the cluster executing all the tasks. If you had a
combined 150 parallelism across all components in the topology, each worker
process will have 6 tasks running within it as threads.
2. **Config.TOPOLOGY_ACKERS**: This sets the number of tasks that will track
tuple trees and detect when a spout tuple has been fully processed. Ackers are
an integral part of Storm's reliability model and you can read more about them
on [Guaranteeing message processing](Guaranteeing-message-processing.html).
@@ -72,4 +73,4 @@ To update a running topology, the only o
The best place to monitor a topology is using the Storm UI. The Storm UI
provides information about errors happening in tasks and fine-grained stats on
the throughput and latency performance of each component of each running
topology.
-You can also look at the worker logs on the cluster machines.
\ No newline at end of file
+You can also look at the worker logs on the cluster machines.
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/SECURITY.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/SECURITY.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/SECURITY.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/SECURITY.md Thu Mar 17
20:19:17 2016
@@ -2,6 +2,7 @@
title: Running Apache Storm Securely
layout: documentation
documentation: true
+version: v0.9.6
---
# Running Apache Storm Securely
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/STORM-UI-REST-API.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/STORM-UI-REST-API.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/STORM-UI-REST-API.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/STORM-UI-REST-API.md Thu
Mar 17 20:19:17 2016
@@ -2,6 +2,7 @@
title: Storm UI REST API
layout: documentation
documentation: true
+version: v0.9.6
---
# Storm UI REST API
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Serialization-(prior-to-0.6.0).md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Serialization-%28prior-to-0.6.0%29.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Serialization-(prior-to-0.6.0).md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Serialization-(prior-to-0.6.0).md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
Tuples can be comprised of objects of any types. Since Storm is a distributed
system, it needs to know how to serialize and deserialize objects when they're
passed between tasks. By default Storm can serialize ints, shorts, longs,
floats, doubles, bools, bytes, strings, and byte arrays, but if you want to use
another type in your tuples, you'll need to implement a custom serializer.
@@ -19,7 +20,7 @@ Let's dive into Storm's API for defining
#### Creating a serializer
-Custom serializers implement the
[ISerialization](/apidocs/backtype/storm/serialization/ISerialization.html)
interface. Implementations specify how to serialize and deserialize types into
a binary format.
+Custom serializers implement the
[ISerialization](javadocs/backtype/storm/serialization/ISerialization.html)
interface. Implementations specify how to serialize and deserialize types into
a binary format.
The interface looks like this:
@@ -45,6 +46,6 @@ Once you create a serializer, you need t
Serializer registrations are done through the Config.TOPOLOGY_SERIALIZATIONS
config and is simply a list of serialization class names.
-Storm provides helpers for registering serializers in a topology config. The
[Config](/apidocs/backtype/storm/Config.html) class has a method called
`addSerialization` that takes in a serializer class to add to the config.
+Storm provides helpers for registering serializers in a topology config. The
[Config](javadocs/backtype/storm/Config.html) class has a method called
`addSerialization` that takes in a serializer class to add to the config.
-There's an advanced config called Config.TOPOLOGY_SKIP_MISSING_SERIALIZATIONS.
If you set this to true, Storm will ignore any serializations that are
registered but do not have their code available on the classpath. Otherwise,
Storm will throw errors when it can't find a serialization. This is useful if
you run many topologies on a cluster that each have different serializations,
but you want to declare all the serializations across all topologies in the
`storm.yaml` files.
\ No newline at end of file
+There's an advanced config called Config.TOPOLOGY_SKIP_MISSING_SERIALIZATIONS.
If you set this to true, Storm will ignore any serializations that are
registered but do not have their code available on the classpath. Otherwise,
Storm will throw errors when it can't find a serialization. This is useful if
you run many topologies on a cluster that each have different serializations,
but you want to declare all the serializations across all topologies in the
`storm.yaml` files.
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Serialization.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Serialization.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Serialization.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Serialization.md Thu Mar
17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This page is about how the serialization system in Storm works for versions
0.6.0 and onwards. Storm used a different serialization system prior to 0.6.0
which is documented on [Serialization (prior to
0.6.0)](Serialization-\(prior-to-0.6.0\).html).
@@ -39,7 +40,7 @@ topology.kryo.register:
`com.mycompany.CustomType1` and `com.mycompany.CustomType3` will use the
`FieldsSerializer`, whereas `com.mycompany.CustomType2` will use
`com.mycompany.serializer.CustomType2Serializer` for serialization.
-Storm provides helpers for registering serializers in a topology config. The
[Config](/apidocs/backtype/storm/Config.html) class has a method called
`registerSerialization` that takes in a registration to add to the config.
+Storm provides helpers for registering serializers in a topology config. The
[Config](javadocs/backtype/storm/Config.html) class has a method called
`registerSerialization` that takes in a registration to add to the config.
There's an advanced config called
`Config.TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS`. If you set this to true,
Storm will ignore any serializations that are registered but do not have their
code available on the classpath. Otherwise, Storm will throw errors when it
can't find a serialization. This is useful if you run many topologies on a
cluster that each have different serializations, but you want to declare all
the serializations across all topologies in the `storm.yaml` files.
@@ -57,4 +58,4 @@ Storm 0.7.0 lets you set component-speci
When a topology is submitted, a single set of serializations is chosen to be
used by all components in the topology for sending messages. This is done by
merging the component-specific serializer registrations with the regular set of
serialization registrations. If two components define serializers for the same
class, one of the serializers is chosen arbitrarily.
-To force a serializer for a particular class if there's a conflict between two
component-specific registrations, just define the serializer you want to use in
the topology-specific configuration. The topology-specific configuration has
precedence over component-specific configurations for serialization
registrations.
\ No newline at end of file
+To force a serializer for a particular class if there's a conflict between two
component-specific registrations, just define the serializer you want to use in
the topology-specific configuration. The topology-specific configuration has
precedence over component-specific configurations for serialization
registrations.
Modified: storm/branches/bobby-versioned-site/releases/0.9.6/Serializers.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Serializers.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Serializers.md (original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Serializers.md Thu Mar
17 20:19:17 2016
@@ -1,4 +1,5 @@
---
layout: documentation
+version: v0.9.6
---
-* [storm-json](https://github.com/rapportive-oss/storm-json): Simple JSON
serializer for Storm
\ No newline at end of file
+* [storm-json](https://github.com/rapportive-oss/storm-json): Simple JSON
serializer for Storm
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Setting-up-a-Storm-cluster.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Setting-up-a-Storm-cluster.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Setting-up-a-Storm-cluster.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Setting-up-a-Storm-cluster.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This page outlines the steps for getting a Storm cluster up and running. If
you're on AWS, you should check out the
[storm-deploy](https://github.com/nathanmarz/storm-deploy/wiki) project.
[storm-deploy](https://github.com/nathanmarz/storm-deploy/wiki) completely
automates the provisioning, configuration, and installation of Storm clusters
on EC2. It also sets up Ganglia for you so you can monitor CPU, disk, and
network usage.
@@ -80,4 +81,4 @@ The last step is to launch all the Storm
2. **Supervisor**: Run the command "bin/storm supervisor" under supervision on
each worker machine. The supervisor daemon is responsible for starting and
stopping worker processes on that machine.
3. **UI**: Run the Storm UI (a site you can access from the browser that gives
diagnostics on the cluster and topologies) by running the command "bin/storm
ui" under supervision. The UI can be accessed by navigating your web browser to
http://{nimbus host}:8080.
-As you can see, running the daemons is very straightforward. The daemons will
log to the logs/ directory in wherever you extracted the Storm release.
\ No newline at end of file
+As you can see, running the daemons is very straightforward. The daemons will
log to the logs/ directory in wherever you extracted the Storm release.
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Setting-up-development-environment.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Setting-up-development-environment.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Setting-up-development-environment.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Setting-up-development-environment.md
Thu Mar 17 20:19:17 2016
@@ -1,9 +1,10 @@
---
layout: documentation
+version: v0.9.6
---
This page outlines what you need to do to get a Storm development environment
set up. In summary, the steps are:
-1. Download a [Storm release](..//downloads.html) , unpack it, and put the
unpacked `bin/` directory on your PATH
+1. Download a [Storm release](/releases.html) , unpack it, and put the
unpacked `bin/` directory on your PATH
2. To be able to start and stop topologies on a remote cluster, put the
cluster information in `~/.storm/storm.yaml`
More detail on each of these steps is below.
@@ -18,7 +19,7 @@ Let's quickly go over the relationship b
### Installing a Storm release locally
-If you want to be able to submit topologies to a remote cluster from your
machine, you should install a Storm release locally. Installing a Storm release
will give you the `storm` client that you can use to interact with remote
clusters. To install Storm locally, download a release [from
here](https://github.com/apache/incubator-storm/downloads) and unzip it
somewhere on your computer. Then add the unpacked `bin/` directory onto your
`PATH` and make sure the `bin/storm` script is executable.
+If you want to be able to submit topologies to a remote cluster from your
machine, you should install a Storm release locally. Installing a Storm release
will give you the `storm` client that you can use to interact with remote
clusters. To install Storm locally, download a release [from
here](/releases.html) and unzip it somewhere on your computer. Then add the
unpacked `bin/` directory onto your `PATH` and make sure the `bin/storm` script
is executable.
Installing a Storm release locally is only for interacting with remote
clusters. For developing and testing topologies in local mode, it is
recommended that you use Maven to include Storm as a dev dependency for your
project. You can read more about using Maven for this purpose on
[Maven](Maven.html).
@@ -36,4 +37,4 @@ Alternatively, if you use the [storm-dep
lein run :deploy --attach --name mystormcluster
```
-More information is on the storm-deploy
[wiki](https://github.com/nathanmarz/storm-deploy/wiki)
\ No newline at end of file
+More information is on the storm-deploy
[wiki](https://github.com/nathanmarz/storm-deploy/wiki)
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Spout-implementations.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Spout-implementations.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
--- storm/branches/bobby-versioned-site/releases/0.9.6/Spout-implementations.md
(original)
+++ storm/branches/bobby-versioned-site/releases/0.9.6/Spout-implementations.md
Thu Mar 17 20:19:17 2016
@@ -1,8 +1,9 @@
---
layout: documentation
+version: v0.9.6
---
* [storm-kestrel](https://github.com/nathanmarz/storm-kestrel): Adapter to use
Kestrel as a spout
* [storm-amqp-spout](https://github.com/rapportive-oss/storm-amqp-spout):
Adapter to use AMQP source as a spout
* [storm-jms](https://github.com/ptgoetz/storm-jms): Adapter to use a JMS
source as a spout
* [storm-redis-pubsub](https://github.com/sorenmacbeth/storm-redis-pubsub): A
spout that subscribes to a Redis pubsub stream
-*
[storm-beanstalkd-spout](https://github.com/haitaoyao/storm-beanstalkd-spout):
A spout that subscribes to a beanstalkd queue
\ No newline at end of file
+*
[storm-beanstalkd-spout](https://github.com/haitaoyao/storm-beanstalkd-spout):
A spout that subscribes to a beanstalkd queue
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Storm-multi-language-protocol-(versions-0.7.0-and-below).md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Storm-multi-language-protocol-%28versions-0.7.0-and-below%29.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Storm-multi-language-protocol-(versions-0.7.0-and-below).md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Storm-multi-language-protocol-(versions-0.7.0-and-below).md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
This page explains the multilang protocol for versions 0.7.0 and below. The
protocol changed in version 0.7.1.
@@ -119,4 +120,4 @@ A "log" will log a message in the worker
Note: This command is not JSON encoded, it is sent as a simple string.
-This lets the parent bolt know that the script has finished processing and is
ready for another tuple.
\ No newline at end of file
+This lets the parent bolt know that the script has finished processing and is
ready for another tuple.
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Structure-of-the-codebase.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Structure-of-the-codebase.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Structure-of-the-codebase.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Structure-of-the-codebase.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
There are three distinct layers to Storm's codebase.
@@ -40,16 +41,16 @@ Note that the structure spouts also have
The interfaces for Storm are generally specified as Java interfaces. The main
interfaces are:
-1. [IRichBolt](/apidocs/backtype/storm/topology/IRichBolt.html)
-2. [IRichSpout](/apidocs/backtype/storm/topology/IRichSpout.html)
-3. [TopologyBuilder](/apidocs/backtype/storm/topology/TopologyBuilder.html)
+1. [IRichBolt](javadocs/backtype/storm/topology/IRichBolt.html)
+2. [IRichSpout](javadocs/backtype/storm/topology/IRichSpout.html)
+3. [TopologyBuilder](javadocs/backtype/storm/topology/TopologyBuilder.html)
The strategy for the majority of the interfaces is to:
1. Specify the interface using a Java interface
2. Provide a base class that provides default implementations when appropriate
-You can see this strategy at work with the
[BaseRichSpout](/apidocs/backtype/storm/topology/base/BaseRichSpout.html)
class.
+You can see this strategy at work with the
[BaseRichSpout](javadocs/backtype/storm/topology/base/BaseRichSpout.html)
class.
Spouts and bolts are serialized into the Thrift definition of the topology as
described above.
@@ -137,4 +138,4 @@ Here's a summary of the purpose of the m
[backtype.storm.util](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/util.clj):
Contains generic utility functions used throughout the code base.
-[backtype.storm.zookeeper](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/zookeeper.clj):
Clojure wrapper around the Zookeeper API and implements some "high-level"
stuff like "mkdirs" and "delete-recursive".
\ No newline at end of file
+[backtype.storm.zookeeper](https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/zookeeper.clj):
Clojure wrapper around the Zookeeper API and implements some "high-level"
stuff like "mkdirs" and "delete-recursive".
Modified:
storm/branches/bobby-versioned-site/releases/0.9.6/Support-for-non-java-languages.md
URL:
http://svn.apache.org/viewvc/storm/branches/bobby-versioned-site/releases/0.9.6/Support-for-non-java-languages.md?rev=1735492&r1=1735491&r2=1735492&view=diff
==============================================================================
---
storm/branches/bobby-versioned-site/releases/0.9.6/Support-for-non-java-languages.md
(original)
+++
storm/branches/bobby-versioned-site/releases/0.9.6/Support-for-non-java-languages.md
Thu Mar 17 20:19:17 2016
@@ -1,5 +1,6 @@
---
layout: documentation
+version: v0.9.6
---
* [Scala DSL](https://github.com/velvia/ScalaStorm)
* [JRuby DSL](https://github.com/colinsurprenant/storm-jruby)
