[
https://issues.apache.org/jira/browse/STORM-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533574#comment-14533574
]
ASF GitHub Bot commented on STORM-561:
--------------------------------------
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/546#discussion_r29905214
--- Diff: external/flux/flux-examples/README.md ---
@@ -0,0 +1,68 @@
+# Flux Examples
+A collection of examples illustrating various capabilities.
+
+## Building From Source and Running
+
+Checkout the projects source and perform a top level Maven build (i.e.
from the `flux` directory):
+
+```bash
+git clone https://github.com/apache/storm.git
+cd storm
+mvn install -DskipTests=true
+```
+
+This will create a shaded (i.e. "fat" or "uber") jar in the
`external/flux/flux-examples/target` directory that can run/deployed with
+the `storm` command:
+
+```bash
+cd flux-examples
+storm jar ./target/flux-examples-*-SNAPSHOT.jar org.apache.storm.flux.Flux
--local ./src/main/resources/simple_wordcount.yaml
+```
+
+The example YAML files are also packaged in the examples jar, so they can
also be referenced with Flux's `--resource`
+command line switch:
+
+```bash
+storm jar ./target/flux-examples-*.jar org.apache.storm.flux.Flux --local
--resource /simple_wordcount.yaml
+```
+
+## Available Examples
+
+### [simple_wordcount.yaml](src/main/resources/simple_wordcount.yaml)
+
+This is a very basic wordcount example using Java spouts and bolts. It
simply logs the running count of each word
+received.
+
+### [multilang.yaml](src/main/resources/multilang.yaml)
+
+Another wordcount example that uses a spout written in JavaScript
(node.js), a bolt written in Python, and two bolts
+written in java.
+
+### [kafka_spout.yaml](src/main/resources/kafka_spout.yaml)
+This example illustrates how to configure Storm's `storm-kafka` spout
using Flux YAML DSL `components`, `references`,
+and `constructor arguments` constructs.
+
+### [simple_hdfs.yaml](src/main/resources/simple_hdfs.yaml)
+
+This example demonstrates using Flux to setup a storm-hdfs bolt to write
to an HDFS cluster. It also demonstrates Flux's
+variable substitution/filtering feature.
+
+To run the `simple_hdfs.yaml` example, copy the `hdfs_bolt.properties`
file to a convenient location and change, at
+least, the property `hdfs.url` to point to a HDFS cluster. Then you can
run the example something like:
+
+```bash
+storm jar ./target/flux-examples-*.jar org.apache.storm.flux.Flux --local
./src/main/resources/simple_hdfs.yaml --filter my_hdfs_bolt.properties
+```
+
+### [simple_hbase.yaml](src/main/resources/simple_hbase.yaml)
+
+This example illustrates how to use Flux to setup a storm-hbase bolt to
write to HBase.
+
+In order to use this example, you will need to edit the `src/main
resrouces/hbase-site.xml` file to reflect your HBase
--- End diff --
You may want to change how to run as same as simple_hdfs.yaml since you
prepared hbase_bolt.properties and simple_hbase.yaml is ready to reflect.
> Add ability to create topologies dynamically
> --------------------------------------------
>
> Key: STORM-561
> URL: https://issues.apache.org/jira/browse/STORM-561
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Nathan Leung
> Assignee: P. Taylor Goetz
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> It would be nice if a storm topology could be built dynamically, instead of
> requiring a recompile to change parameters (e.g. number of workers, number of
> tasks, layout, etc).
> I would propose the following data structures for building core storm
> topologies. I haven't done a design for trident yet but the intention would
> be to add trident support when core storm support is complete (or in parallel
> if there are other people working on it):
> {code}
> // fields value and arguments are mutually exclusive
> class Argument {
> String argumentType; // Class used to lookup arguments in
> method/constructor
> String implementationType; // Class used to create this argument
> String value; // String used to construct this argument
> List<Argument> arguments; // arguments used to build this argument
> }
> class Dependency {
> String upstreamComponent; // name of upstream component
> String grouping;
> List<Argument> arguments; // arguments for the grouping
> }
> class StormSpout {
> String name;
> String klazz; // Class of this spout
> List <Argument> arguments;
> int numTasks;
> int numExecutors;
> }
> class StormBolt {
> String name;
> String klazz; // Class of this bolt
> List <Argument> arguments;
> int numTasks;
> int numExecutors;
> List<Dependency> dependencies;
> }
> class StormTopologyRepresentation {
> String name;
> List<StormSpout> spouts;
> List<StormBolt> bolts;
> Map config;
> int numWorkers;
> }
> {code}
> Topology creation will be built on top of the data structures above. The
> benefits:
> * Dependency free. Code to unmarshal from json, xml, etc, can be kept in
> extensions, or as examples, and users can write a different unmarshaller if
> they want to use a different text representation.
> * support arbitrary spout and bolts types
> * support of all groupings, streams, via reflections
> * ability to specify configuration map via config file
> * reification of spout / bolt / dependency arguments
> ** recursive argument reification for complex objects
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
