Repository: incubator-beam-site Updated Branches: refs/heads/asf-site cb975b411 -> 1c9f85626
Update main landing page and Beam Overview page Update get-started/beam-overview.md and src/index.md with v2 main page content Add Apache Apex & other additions to Overview & landing page. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/236b3b6a Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/tree/236b3b6a Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/diff/236b3b6a Branch: refs/heads/asf-site Commit: 236b3b6ada6688b0b2b015b502929f707154b461 Parents: cb975b4 Author: Hadar Hod <[email protected]> Authored: Mon Nov 21 11:18:20 2016 -0800 Committer: Davor Bonaci <[email protected]> Committed: Wed Nov 23 17:10:46 2016 -0800 ---------------------------------------------------------------------- src/get-started/beam-overview.md | 13 +++++- src/get-started/index.md | 28 ++++++------ src/index.md | 80 +++++------------------------------ 3 files changed, 36 insertions(+), 85 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/236b3b6a/src/get-started/beam-overview.md ---------------------------------------------------------------------- diff --git a/src/get-started/beam-overview.md b/src/get-started/beam-overview.md index 1b43ca8..6756a50 100644 --- a/src/get-started/beam-overview.md +++ b/src/get-started/beam-overview.md @@ -6,9 +6,10 @@ redirect_from: - /use/beam-overview/ - /docs/use/beam-overview/ --- + # Apache Beam Overview -Apache Beam is an open source, unified programming model that you can use to create a data processing **pipeline**. You start by building a program that defines the pipeline using one of the open source Beam SDKs. The pipeline is then executed by one of Beam's supported **distributed processing back-ends**, which include [Apache Flink](http://flink.apache.org), [Apache Spark](http://spark.apache.org), and [Google Cloud Dataflow](https://cloud.google.com/dataflow). +Apache Beam is an open source, unified programming model that you can use to create a data processing **pipeline**. You start by building a program that defines the pipeline using one of the open source Beam SDKs. The pipeline is then executed by one of Beam's supported **distributed processing back-ends**, which include [Apache Apex](http://apex.apache.org), [Apache Flink](http://flink.apache.org), [Apache Spark](http://spark.apache.org), and [Google Cloud Dataflow](https://cloud.google.com/dataflow). Beam is particularly useful for [Embarrassingly Parallel](http://en.wikipedia.org/wiki/Embarassingly_parallel) data processing tasks, in which the problem can be decomposed into many smaller bundles of data that can be processed independently and in parallel. You can also use Beam for Extract, Transform, and Load (ETL) tasks and pure data integration. These tasks are useful for moving data between different storage media and data sources, transforming data into a more desirable format, or loading data onto a new system. @@ -49,7 +50,7 @@ Beam currently supports Runners that work with the following distributed process <th>Status</th> </tr> <tr> - <td>Google Cloud Dataflow</td> + <td>Apache Apex</td> <td>In Development</td> </tr> <tr> @@ -60,6 +61,14 @@ Beam currently supports Runners that work with the following distributed process <td>Apache Spark</td> <td>In Development</td> </tr> +<tr> + <td>Google Cloud Dataflow</td> + <td>In Development</td> +</tr> </table> **Note:** You can always execute your pipeline locally for testing and debugging purposes. + +## Getting Started with Apache Beam + +Get started using Beam for your data processing tasks by following the [Quickstart]({{ site.baseurl }}/get-started/quickstart) and the [WordCount Examples Walkthrough]({{ site.baseurl }}/get-started/wordcount-example). http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/236b3b6a/src/get-started/index.md ---------------------------------------------------------------------- diff --git a/src/get-started/index.md b/src/get-started/index.md index 4ab55af..69831ed 100644 --- a/src/get-started/index.md +++ b/src/get-started/index.md @@ -7,28 +7,28 @@ redirect_from: - /docs/use/ --- -# Using Apache Beam +# Get Started with Apache Beam Learn to use Beam to create data processing pipelines that run on supported processing back-ends: -#### [Beam Overview]({{ site.baseurl}}/use/beam-overview) +#### [Beam Overview]({{ site.baseurl}}/get-started/beam-overview) + Learn about the Beam model, the currently available Beam SDKs and Runners, and Beam's native I/O connectors. -#### [Quickstart]({{ site.baseurl }}/use/quickstart/) -Learn how to set up a Beam project and run a simple example Beam pipeline on your local machine. +#### [Quickstart]({{ site.baseurl }}/get-started/quickstart/) -#### [Release Information]({{ site.baseurl}}/use/releases) -Find information on the latest Beam releases, including versioning and release notes. +Learn how to set up a Beam project and run a simple example Beam pipeline on your local machine. #### Example Walkthroughs + See detailed walkthroughs of complete Beam pipelines. -* [WordCount]({{ site.baseurl }}/use/wordcount-example): Simple example pipelines that demonstrate basic Beam programming, including debugging and testing -* [Mobile Gaming]({{ site.baseurl }}/use/mobile-gaming-example): A series of more advanced pipelines that demonstrate use cases in the mobile gaming domain +* [WordCount]({{ site.baseurl }}/get-started/wordcount-example): Simple example pipelines that demonstrate basic Beam programming, including debugging and testing +* [Mobile Gaming]({{ site.baseurl }}/get-started/mobile-gaming-example): A series of more advanced pipelines that demonstrate use cases in the mobile gaming domain + +#### [Downloads and Releases]({{ site.baseurl}}/get-started/downloads) + +Find download links and information on the latest Beam releases, including versioning and release notes. -#### Support -Find resources to help you use Beam. +#### [Support]({{ site.baseurl }}/get-started/support) -* [Mailing Lists]({{ site.baseurl }}/use/mailing-lists/) -* [Issue Tracking]({{ site.baseurl }}/use/issue-tracking/) -* [Beam on Stack Overflow](http://stackoverflow.com/questions/tagged/apache-beam) -* [Slack Channel](http://apachebeam.slack.com) +Find resources, such as mailing lists and issue tracking, to help you use Beam. Ask questions and discuss topics via [Stack Overflow](http://stackoverflow.com/questions/tagged/apache-beam) or on Beam's [Slack Channel](http://apachebeam.slack.com). http://git-wip-us.apache.org/repos/asf/incubator-beam-site/blob/236b3b6a/src/index.md ---------------------------------------------------------------------- diff --git a/src/index.md b/src/index.md index e16d1bf..e5d47bd 100644 --- a/src/index.md +++ b/src/index.md @@ -1,85 +1,28 @@ --- layout: default --- -<div class="alert alert-info alert-dismissible" role="alert"> -<span class="glyphicon glyphicon-flag" aria-hidden="true"></span> -<button type="button" class="close" data-dismiss="alert" aria-label="Close"><span aria-hidden="true">×</span></button> -The Apache Beam project is in the process of bootstrapping. This includes the website -- so please file issues you find in <a href="{{ site.baseurl }}/use/issue-tracking/">Jira</a>. Thanks! -</div> # Apache Beam (incubating) -Apache Beam is an open source, unified programming model that you can use to create a data processing **pipeline**. You start by building a program that defines the pipeline using one of the open source Beam SDKs. The pipeline is then executed by one of Beam's supported **distributed processing back-ends**, which include [Apache Apex](http://apex.apache.org), [Apache Flink](http://flink.apache.org), [Apache Spark](http://spark.apache.org), and [Google Cloud Dataflow](https://cloud.google.com/dataflow). - -Beam is particularly useful for [Embarrassingly Parallel](http://en.wikipedia.org/wiki/Embarassingly_parallel) data processing tasks, in which the problem can be decomposed into many smaller bundles of data that can be processed independently and in parallel. You can also use Beam for Extract, Transform, and Load (ETL) tasks and pure data integration. These tasks are useful for moving data between different storage media and data sources, transforming data into a more desirable format, or loading data onto a new system. - -## Apache Beam SDKs - -The Beam SDKs provide a unified programming model that can represent and transform data sets of any size, whether the input is a finite data set from a batch data source, or an infinite data set from a streaming data source. The Beam SDKs use the same classes to represent both bounded and unbounded data, and the same transforms to operate on that data. You use the Beam SDK of your choice to build a program that defines your data processing pipeline. - -Beam currently supports the following language-specific SDKs: +#### Apache Beam provides an advanced unified programming model, allowing you to implement batch and streaming data processing jobs that can run on any execution engine. -<table class="table table-condensed"> -<tr> - <th>Language</th> - <th>SDK Status</th> -</tr> -<tr> - <td>Java</td> - <td>Active Development</td> -</tr> -<tr> - <td>Python</td> - <td>Coming Soon</td> -</tr> -<tr> - <td>Other</td> - <td>TBD</td> -</tr> -</table> +Apache Beam is: +* **UNIFIED** - Use a single programming model for both batch and streaming use cases. +* **PORTABLE** - Execute pipelines on multiple execution environments, including Apache Apex, Apache Flink, Apache Spark, and Google Cloud Dataflow. +* **EXTENSIBLE** - Write and share new SDKs, IO connectors, and transformation libraries. -## Apache Beam Pipeline Runners +## Get Started -The Beam Pipeline Runners translate the data processing pipeline you define with your Beam program into the API compatible with the distributed processing back-end of your choice. When you run your Beam program, you'll need to specify the appropriate runner for the back-end where you want to execute your pipeline. +To use Beam for your data processing tasks, start by reading the [Beam Overview]({{ site.baseurl }}/get-started/beam-overview) and performing the steps in the [Quickstart]({{ site.baseurl }}/get-started/quickstart). Then dive into the [Documentation]({{ site.baseurl }}/documentation/) section for in-depth concepts and reference materials for the Beam model, SDKs, and runners. -Beam currently supports Runners that work with the following distributed processing back-ends: +## Contribute -<table class="table table-condensed"> -<tr> - <th>Runner</th> - <th>Status</th> -</tr> -<tr> - <td>Google Cloud Dataflow</td> - <td>In Development</td> -</tr> -<tr> - <td>Apache Flink</td> - <td>In Development</td> -</tr> -<tr> - <td>Apache Spark</td> - <td>In Development</td> -</tr> -<tr> - <td>Apache Apex</td> - <td>In Development</td> -</tr> -</table> - -**Note:** You can always execute your pipeline locally for testing and debugging purposes. - -## Getting Started with Apache Beam - -Interested in working with Apache Beam? Great! Here's how to get started: - -* If you are interested in using Beam for your data processing tasks, use the [Get Started]({{ site.baseurl }}/get-started/) section for an overview, quickstart, and examples. Then dive into the [Documentation]({{ site.baseurl }}/documentation/) section to get up to speed on the details. -* If you're interested in contributing to the Beam SDKs or Runners, start with the [Contribute]({{ site.baseurl }}/contribute/) section. +Beam is an [Apache Software Foundation](http://www.apache.org) project, available under the Apache v2 license. Beam is an open source community and contributions are greatly appreciated! If you'd like to contribute, please see the [Contribute]({{ site.baseurl }}/contribute/) section. <hr> <div class="row"> <div class="col-md-6"> - <h3>Blog</h3> + <h2>Blog</h2> <div class="list-group"> {% for post in site.posts %} <a class="list-group-item" href="{{ post.url | prepend: site.baseurl }}">{{ post.date | date: "%b %-d, %Y" }} - {{ post.title }}</a> @@ -87,7 +30,7 @@ Interested in working with Apache Beam? Great! Here's how to get started: </div> </div> <div class="col-md-6"> - <h3>Twitter</h3> + <h2>Twitter</h2> <a class="twitter-timeline" href="https://twitter.com/ApacheBeam"; data-widget-id="697809684422533120">Tweets by @ApacheBeam</a> <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+"://platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script> </div> @@ -99,5 +42,4 @@ Interested in working with Apache Beam? Great! Here's how to get started: Apache Beam is an effort undergoing incubation at [The Apache Software Foundation (ASF)](http://www.apache.org) sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. - Apache Beam (incubating) is available under [Apache License, version 2.0](http://www.apache.org/licenses/LICENSE-2.0).
