Add tutorial
Project: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/commit/283d0fc4 Tree: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/tree/283d0fc4 Diff: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/diff/283d0fc4 Branch: refs/heads/master Commit: 283d0fc46896a64dafd31879173ac1f81f32a8df Parents: 749300a Author: Steve Blackmon @steveblackmon <[email protected]> Authored: Mon Apr 25 11:04:59 2016 -0500 Committer: Steve Blackmon @steveblackmon <[email protected]> Committed: Mon Apr 25 11:04:59 2016 -0500 ---------------------------------------------------------------------- src/site/markdown/tutorial/build.md | 81 ++++++++++++++++++++++++++ src/site/markdown/tutorial/database.md | 44 ++++++++++++++ src/site/markdown/tutorial/docker.md | 18 ++++++ src/site/markdown/tutorial/environment.md | 28 +++++++++ src/site/site.xml | 37 +++++++----- src/site/site_en.xml | 81 ++++++++++++++------------ 6 files changed, 235 insertions(+), 54 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/markdown/tutorial/build.md ---------------------------------------------------------------------- diff --git a/src/site/markdown/tutorial/build.md b/src/site/markdown/tutorial/build.md new file mode 100644 index 0000000..1608b28 --- /dev/null +++ b/src/site/markdown/tutorial/build.md @@ -0,0 +1,81 @@ +## Build streams and streams-examples from source + +This tutorial assumes you are using linux or Mac OS X. + +### Setup Tools + +You'll need the following tools installed in your command line: + +* Git +* Java SDK +* Maven +* Docker + +#### Git + + `git -version` + +| Possible result | Explanation | +|-----------------|-------------| +| bash: git: No such file or directory | You need to install git | +| git version < 2.7 | You should upgrade git for security reasons | +| git version > 2.7 | You are all good | + +#### Maven and Java SDK + +Run from your command line: + + `mvn -version` + +| Possible result | Explanation | +|-----------------|-------------| +| -bash: mvn: command not found | You need to install maven | +| Error: JAVA_HOME is not defined correctly. | You need to install JDK | +| Apache Maven >= 3.2.5+\nJava Version >= 1.7.0u72) | You're all good | +| Apache Maven >= 3.2.5+\nJava Version >= 1.8.0u25) | You're all good | +| Apache Maven < 3.2.5 | You need a newer version of maven | +| Java Version < 1.7.0u72 | You need a newer version of maven | +| Java Version < 1.8.0u25 | You need a newer JDK | + +#### Docker + +Run from your command line: + + `docker version` + +| Possible result | Explanation | +|-----------------|-------------| +| bash: docker: No such file or directory | You need to install docker | +| Client: Version: < 1.0.0 | You need a newer version of docker | +| Server: Version: < 1.0.0 | You need a newer version of docker | +| Client: Version: > 1.0.0\nServer: Version: > 1.0.0 | You are all good | + +See [streams-project-index.html](http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html "streams-project/index.html") for more information. + +### Download Sources + +Run from your command line: + + `git clone https://github.com/apache/incubator-streams` + `git clone https://github.com/apache/incubator-streams-examples` + +### Build Projects + +Run from your command line: + + `export MAVEN_OPTS="-Xmx2G"` + `cd incubator-streams` + `mvn clean install -Dmaven.test.skip.exec=true` + +| Possible result | Explanation | +|-----------------|-------------| +| BUILD SUCCESSFUL | You are all good | +| BUILD FAILED | Check yourself | + + `cd ../incubator-streams-examples` + `mvn clean package` + +| Possible result | Explanation | +|-----------------|-------------| +| BUILD SUCCESSFUL | You are all good | +| BUILD FAILED | Check yourself | \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/markdown/tutorial/database.md ---------------------------------------------------------------------- diff --git a/src/site/markdown/tutorial/database.md b/src/site/markdown/tutorial/database.md new file mode 100644 index 0000000..b546826 --- /dev/null +++ b/src/site/markdown/tutorial/database.md @@ -0,0 +1,44 @@ +## Set up databases to store and analyze streams content + +This tutorial assumes you are using linux or Mac OS X. + +### Confirm Docker is healthy + +We'll assume you've got docker up and running. + +Run from your command line: + + `docker ps` + +If you see a (possibly empty) list of running containers, you are good. + +### Run Elasticsearch + +Elasticsearch is a great database for storing content from your streams. + + `docker run -d --name elasticsearch elasticsearch` + +### Add Elasticsearch container details to your configuration + + `cd $STREAMS` + `export DOCKERHOST=$(docker-machine ip)` + +Put the following into elasticsearch.conf + + include "reference.conf" + elasticsearch { + hosts = [ + ${DOCKERHOST} + ] + protocol = "tcp" + port = 9300 + index = "streams" + indexes = [ + "streams" + ] + types = [ + "page" + "post" + ] + } + http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/markdown/tutorial/docker.md ---------------------------------------------------------------------- diff --git a/src/site/markdown/tutorial/docker.md b/src/site/markdown/tutorial/docker.md new file mode 100644 index 0000000..fa5ffec --- /dev/null +++ b/src/site/markdown/tutorial/docker.md @@ -0,0 +1,18 @@ +## Set up a local environment to run streams + +First of all, you do not run 'Streams' as software. Rather, you run 'streams' which use Streams components and libraries under the cover. + +Streams components can be embedded in a variety of data processing frameworks based on the problem and performance requirements at hand. + +A great way to get started collecting and indexing data is with streams-runtime-local and Docker. + +### Confirm Docker is healthy + +We'll assume you've got docker up and running. + +Run from your command line: + + `docker ps` + +If you see a (possibly empty) list of running containers, you are good. + http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/markdown/tutorial/environment.md ---------------------------------------------------------------------- diff --git a/src/site/markdown/tutorial/environment.md b/src/site/markdown/tutorial/environment.md new file mode 100644 index 0000000..b86dd7e --- /dev/null +++ b/src/site/markdown/tutorial/environment.md @@ -0,0 +1,28 @@ +### Create a folder to hold streams configuration files + +Pick a place on your file system that's easy to remember and reference with an absolution path. + +Create a new empty folder there and export the path to a variables. + + `mkdir ~/streams` + `export STREAMS=$(cd ~streams; pwd)` + +### Create a configuration file to hold secrets + + `cd $STREAMS` + `touch reference.conf` + +To get started, put the following into reference.conf + + twitter { + oauth { + consumerKey = "" + consumerSecret = "" + accessToken = "" + accessTokenSecret = "" + } + } + +Visit developer.twitter.com to obtain the above fields and put them your reference.conf + + http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/site.xml ---------------------------------------------------------------------- diff --git a/src/site/site.xml b/src/site/site.xml index e06e9f2..4087ae7 100644 --- a/src/site/site.xml +++ b/src/site/site.xml @@ -75,26 +75,31 @@ <item name="streams-examples" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/"; /> </menu> <menu name="Getting Started"> - <item name="Learn more about Activity Streams" href="http://activitystrea.ms"; /> - <item name="Check out streams-project web site" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/"; /> - <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-pojo/index.html"; /> - <item name="Set up a local environment to run streams" /> - <item name="Set up a local database to store streams data" /> - <item name="Build and run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html"; /> - <item name="Browse and search the twitter data you collected" /> - <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/streams-provider-twitter/index.html"; /> - <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/index.html"; /> - <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-components/index.html"; /> - <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/index.html"; /> + <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-pojo/index.html"; /> + <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/streams-provider-twitter/index.html"; /> + <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html"; /> + <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-components/index.html"; /> + <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html"; /> + <item name="Browse streams providers, processors, and persisters" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html"; /> </menu> - <menu name="Foundation"> - <item name="Foundation Info" href="http://www.apache.org/"; /> - <item name="Sponsorship" href="http://www.apache.org/foundation/sponsorship.html"; /> - <item name="Thanks" href="http://www.apache.org/foundation/thanks.html"; /> + <menu name="Tutorial"> + <item name="Build streams and streams-examples from source" href="tutorial/build.html" /> + <item name="Create a local folder to configure streams" href="tutorial/environment.html"/> + <item name="Set up a local docker environment" href="tutorial/docker.html" /> + <item name="Run a local database to store activity documents" href="tutorial/database.html" /> + <item name="Run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html"; /> + <item name="Browse and search the twitter data you collected" href="tutorial/browse.html" /> </menu> <menu name="Other Resources"> - <item name="Streams Proposal Wiki Page" href="http://wiki.apache.org/incubator/StreamsProposal"; /> + <item name="ActivityStreams Homepage" href="http://activitystrea.ms"; /> <item name="ActivityStreams Specifications" href="http://activitystrea.ms/specs/"; /> + <item name="Streams Blog" href="http://blogs.apache.org/streams/"; /> + <item name="Streams Proposal Wiki Page" href="http://wiki.apache.org/incubator/StreamsProposal"; /> + </menu> + <menu name="Foundation"> + <item name="Foundation Info" href="http://www.apache.org/"; /> + <item name="Sponsorship" href="http://www.apache.org/foundation/sponsorship.html"; /> + <item name="Thanks" href="http://www.apache.org/foundation/thanks.html"; /> </menu> </body> </project> http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/283d0fc4/src/site/site_en.xml ---------------------------------------------------------------------- diff --git a/src/site/site_en.xml b/src/site/site_en.xml index e06e9f2..effd07e 100644 --- a/src/site/site_en.xml +++ b/src/site/site_en.xml @@ -25,11 +25,11 @@ <version>1.4</version> </skin> <custom> - <fluidoSkin> - <topBarEnabled>true</topBarEnabled> - <navBarStyle>navbar-inverse</navBarStyle> - <sideBarEnabled>false</sideBarEnabled> - </fluidoSkin> + <fluidoSkin> + <topBarEnabled>true</topBarEnabled> + <navBarStyle>navbar-inverse</navBarStyle> + <sideBarEnabled>false</sideBarEnabled> + </fluidoSkin> </custom> <bannerLeft> <name>Apache Streams</name> @@ -45,8 +45,8 @@ </bannerRight> <body> <breadcrumbs> - <item name="Incubator" href="http://incubator.apache.org/"/> - <item name="Streams" href="http://streams.incubator.apache.org/"/> + <item name="Incubator" href="http://incubator.apache.org/"/> + <item name="Streams" href="http://streams.incubator.apache.org/"/> </breadcrumbs> <menu ref="parent" inherit="top"/> <menu name="Overview"> @@ -57,44 +57,49 @@ <item name="Frequently Asked Questions" href="faq.html" /> </menu> <menu name="Details"> - <item name="Project License" href="license.html" /> - <item name="Mailing Lists" href="mail-lists.html" /> - <item name="Project Team" href="team-list.html" /> - <item name="Continuous Integration" href="integration.html"/> - <item name="Issue Tracking" href="issue-tracking.html" /> - <item name="Source Repositories" href="source-repository.html" /> - <item name="Dependency Info" href="dependency-info.html" /> - <item name="Distribution Management" href="distribution-management.html" /> - <item name="Plugin Management" href="plugin-management.html" /> - <item name="Release Setup" href="release-setup.html" /> - <item name="Release Process" href="release.html" /> + <item name="Project License" href="license.html" /> + <item name="Mailing Lists" href="mail-lists.html" /> + <item name="Project Team" href="team-list.html" /> + <item name="Continuous Integration" href="integration.html"/> + <item name="Issue Tracking" href="issue-tracking.html" /> + <item name="Source Repositories" href="source-repository.html" /> + <item name="Dependency Info" href="dependency-info.html" /> + <item name="Distribution Management" href="distribution-management.html" /> + <item name="Plugin Management" href="plugin-management.html" /> + <item name="Release Setup" href="release-setup.html" /> + <item name="Release Process" href="release.html" /> </menu> <menu name="Projects"> - <item name="streams-master" /> - <item name="streams-project" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project"; /> - <item name="streams-examples" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/"; /> + <item name="streams-master" /> + <item name="streams-project" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project"; /> + <item name="streams-examples" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/"; /> </menu> <menu name="Getting Started"> - <item name="Learn more about Activity Streams" href="http://activitystrea.ms"; /> - <item name="Check out streams-project web site" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/"; /> - <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-pojo/index.html"; /> - <item name="Set up a local environment to run streams" /> - <item name="Set up a local database to store streams data" /> - <item name="Build and run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html"; /> - <item name="Browse and search the twitter data you collected" /> - <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/streams-provider-twitter/index.html"; /> - <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/index.html"; /> - <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-components/index.html"; /> - <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/0.2-incubating/streams-project/streams-contrib/index.html"; /> + <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-pojo/index.html"; /> + <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/streams-provider-twitter/index.html"; /> + <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html"; /> + <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-components/index.html"; /> + <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html"; /> + <item name="Browse streams providers, processors, and persisters" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html"; /> </menu> - <menu name="Foundation"> - <item name="Foundation Info" href="http://www.apache.org/"; /> - <item name="Sponsorship" href="http://www.apache.org/foundation/sponsorship.html"; /> - <item name="Thanks" href="http://www.apache.org/foundation/thanks.html"; /> + <menu name="Tutorial"> + <item name="Build streams and streams-examples from source" href="tutorial/build.html" /> + <item name="Create a local folder to configure streams" href="tutorial/environment.html"/> + <item name="Set up a local docker environment" href="tutorial/docker.html" /> + <item name="Run a local database to store activity documents" href="tutorial/database.html" /> + <item name="Run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html"; /> + <item name="Browse and search the twitter data you collected" href="tutorial/browse.html" /> </menu> <menu name="Other Resources"> - <item name="Streams Proposal Wiki Page" href="http://wiki.apache.org/incubator/StreamsProposal"; /> - <item name="ActivityStreams Specifications" href="http://activitystrea.ms/specs/"; /> + <item name="ActivityStreams Homepage" href="http://activitystrea.ms"; /> + <item name="ActivityStreams Specifications" href="http://activitystrea.ms/specs/"; /> + <item name="Streams Blog" href="http://blogs.apache.org/streams/"; /> + <item name="Streams Proposal Wiki Page" href="http://wiki.apache.org/incubator/StreamsProposal"; /> + </menu> + <menu name="Foundation"> + <item name="Foundation Info" href="http://www.apache.org/"; /> + <item name="Sponsorship" href="http://www.apache.org/foundation/sponsorship.html"; /> + <item name="Thanks" href="http://www.apache.org/foundation/thanks.html"; /> </menu> </body> </project>
