pcoet commented on a change in pull request #17136: URL: https://github.com/apache/beam/pull/17136#discussion_r835533947
########## File path: website/www/site/content/en/contribute/_index.md ########## @@ -24,366 +24,217 @@ See the License for the specific language governing permissions and limitations under the License. --> -# Contribution guide +# Contribute to Apache Beam + +Apache Beam is an open source project developed and maintained by a friendly community of users, contributors, committers, and project management committee (PMC) members. + +Everyone is welcome to join and contribute to the project in multiple ways, not only with code contributions. Contributions like asking questions, reporting bugs, proposing new features, improving documentation, adding to the website, organizing events, or writing blog posts among other contributions, are welcome and recognized. + +### What’s In It for You? + - Become a part of the world’s largest, most welcoming open source community + - Drive innovation for next-gen distributed data processing + - Make an impact by bringing advanced technology solutions into reality + - Empower your brand recognition and thought leadership in the data world + +### Ways You Can Contribute + +Here’s a list of things you can do to get started contributing: + +<table class="table table-bordered"> + <tr> + <th>Contribution</th> + <th>How to do it</th> + </tr> + <tr> + <td>Participate and share ideas</td> + <td class="list-text-size-md"> + <ol> + <li>Start by <a href="https://beam.apache.org/contribute/#accounts-and-permissions";>getting an Apache Beam Jira account</a>.</li> + <li>Subscribe to our <a href="https://beam.apache.org/community/contact-us/";>mailing lists</a> (<a href="mailto:[email protected]";>[email protected]</a> and <a href="mailto:[email protected]";>[email protected]</a>).</li> + <li>Stay in the loop of our <a href="https://issues.apache.org/jira/projects/BEAM/issues";>Jira product backlog</a>, participate in discussions to brainstorm solutions, and help the community solve their mailing list questions.</li> + </ol> + </td> + </tr> + <tr> + <td>File a bug</td> + <td class="list-text-size-md"> + <ol class="no-bullet"> + <li>Create a Jira ticket and mark it as a bug</li> + <br><span>OR</span><br><br> + <li>Send an email to <a href="mailto:[email protected]";>[email protected]</a> with subject: [Bug] <bug summary> and include as much detail as possible.</li> + </ol> + </td> + </tr> + <tr> + <td>Propose a feature</td> + <td class="list-text-size-md"> + <ol class="no-bullet"> + <li>Create a Jira ticket and mark it as a feature request</li> + <br><span>OR</span><br><br> + <li>Send an email to <a href="mailto:[email protected]";>[email protected]</a> with subject: [Feature Proposal] <your idea>.</li> + </ol> + </td> + </tr> + <tr> + <td>Improve documentation</td> + <td> + Click the edit icon in our documentation and it’ll take you to our <a href="https://github.com/apache/beam/";>GitHub</a>, where you can automatically make the change you want and send a Pull Request. + </td> + </tr> + <tr> + <td>Improve the website</td> + <td class="list-text-size-md"> + <ul> + <li>Follow the website <a href="https://beam.apache.org/contribute/get-started-contributing/";>Contribution Guide</a> to make changes yourself.</li> + <li>Check our <a href="https://issues.apache.org/jira/browse/BEAM-13229?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20website%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC";>Website Backlog</a> in Jira to find rooms for improvement areas, or propose one.</li> + </ul> + </td> + </tr> + <tr> + <td>Write a blogpost</td> + <td> + Tell a story about your custom use cases or share your recommendations and best practices. + </td> + </tr> + <tr> + <td>Share your success story</td> + <td> + Visit our <a href="https://beam.apache.org/case-studies/";>Case Studies</a> page and share your story or add your company’s logo. + </td> + </tr> + <tr> + <td>Produce a social media publication</td> + <td> + Share the use cases that Beam helps empower or the data processing performance you achieved. Spread the word about the upcoming Beam Summits and conferences. + </td> + </tr> + <tr> + <td>Organize an event</td> + <td> + Send an email to <a href="mailto:[email protected]";>[email protected]</a> to share your idea about the Beam event you want to organize. + </td> + </tr> + <tr> + <td>Build a new feature like a new IO connector or PTransform</td> + <td class="list-text-size-md"> + <ul> + <li>Create and contribute a custom <a href="https://beam.apache.org/documentation/programming-guide/#pipeline-io";>I/O connector</a> for a specific data storage system you want to connect to, if it is not on <a href="https://beam.apache.org/documentation/io/built-in/";>the I/O list yet</a>.</li> + <li>Look at our project backlog to get some inspiration on what to build.</li> + <li>Follow our <a href="https://beam.apache.org/contribute/get-started-contributing/";>Code Contribution Guide</a>.</li> + </ul> + </td> + </tr> + <tr> + <td>Share a code sample or template</td> + <td class="list-text-size-md"> + <ol> + <li>Read our <a href="https://beam.apache.org/contribute/get-started-contributing/";>Code Contribution Guide</a>.</li> + <li>Create your code sample or template. </li> + <li><a href="https://beam.apache.org/contribute/get-started-contributing/#make-your-change";>Make a Pull Request</a> to add it to the Beam example folder.</li> + </ol> + </td> + </tr> + <tr> + <td>Test a release candidate</td> + <td> + Anybody can <a href="https://beam.apache.org/contribute/release-guide/";>propose a release</a> via the [email protected] mailing list. Try Apache Beam releases in your projects, vote for release candidates, inform the community about the results and any issues found via <a href="mailto:[email protected]";>[email protected]</a>. Learn more about how to validate a Beam release <a href="https://beam.apache.org/blog/validate-beam-release/";>here</a>. + </td> + </tr> + <tr> + <td>Other ideas?</td> + <td> + If you want to contribute in another way and you’re not sure how to get started, send a proposal to <a href="mailto:[email protected]";>[email protected]</a>. + </td> + </tr> +</table> + + +## Become a Committer + +If you like contributing to Apache Beam, you are active in the Apache Beam mailing lists, Slack channel, or you attend/organize events, then you are a candidate to become a project committer. + +Committers are community members that have write access to the project’s repositories (i.e., they can modify the code, documentation, and website by themselves and also assist with code reviews and accept other contributions). + +Even though there is no strict protocol to become a committer, the most important thing we look for is that you display the traits of an Apache Beam committer and that you have been consistently contributing to the project and community. + +Candidates for new committers are suggested by current committers or PMC members, and voted upon by the PMC team. So if you would like to become a committer, engage with the community and keep contributing to Apache Beam, you might also want to talk to other committers and ask for their advice and guidance. Feel free to email any committer directly and ask for their advice! Review comment: "Apache Beam, you might" -> "Apache Beam. You might" ########## File path: website/www/site/content/en/contribute/get-started-contributing.md ########## @@ -0,0 +1,383 @@ +--- +title: "Beam Contribution Guide" +type: "contribute" +layout: "arrow_template" +--- + +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Contribution guide + +<a class="arrow-list-header" data-toggle="collapse" href="#collapseOverview" role="button" aria-expanded="false" aria-controls="collapseOverview"> + {{< figure src="/images/arrow-icon_list.svg">}} + +## Overview + +</a> + +<div class="collapse dont-collapse-sm" id="collapseOverview"> + +There are lots of opportunities to contribute. You can for example: + +- ask or answer questions on [[email protected]](https://beam.apache.org/community/contact-us/) or + [stackoverflow](https://stackoverflow.com/questions/tagged/apache-beam) +- review proposed design ideas on [[email protected]](https://beam.apache.org/community/contact-us/) +- improve the documentation +- file [bug reports](https://issues.apache.org/jira/projects/BEAM/issues) +- test releases +- review [changes](https://github.com/apache/beam/pulls) +- write new examples +- improve your favorite language SDK (Java, Python, Go, etc) +- improve specific runners (Apache Flink, Apache Spark, Google + Cloud Dataflow, etc) +- improve or add IO connectors +- add new transform libraries (statistics, ML, image processing, etc) +- work on the core programming model (what is a Beam pipeline and how does it + run?) +- improve the developer experience (for example, Windows guides) +- add answers to the [contribution FAQ]( + https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ) +- organize local meetups of users or contributors to Apache Beam + +</div> + +<a class="arrow-list-header" data-toggle="collapse" href="#collapseContributing" role="button" aria-expanded="false" aria-controls="collapseContributing"> + {{< figure src="/images/arrow-icon_list.svg">}} + +## Contributing code + + </a> + +<div class="collapse dont-collapse-sm" id="collapseContributing"> + +Below is a tutorial for contributing code to Beam, covering our tools and typical process in +detail. + +### Prerequisites + +- A GitHub account. +- A Linux, macOS, or Microsoft Windows development environment +- Java JDK 8 installed +- [Docker](https://www.docker.com/) installed for some tasks including building worker containers and testing this website + changes locally. +- For SDK Development: + - [Go](https://golang.org) 1.12 or later installed for Go SDK development + - Python 3.x interpreters. You will need Python interpreters for all Python versions supported by Beam. + Interpreters should be installed and available in shell via `python3.x` commands. For more information, see: + Python installation tips in [Developer Wiki](https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-InstallingPythoninterpreters). +- For large contributions, a signed [Individual Contributor License + Agreement](https://www.apache.org/licenses/icla.pdf) (ICLA) to the Apache + Software Foundation (ASF). + + +### Configuration options +You have two options for configuring your development environment: +- Local: + - Manually installing the prerequisites listed above. + - Using the automated script for Linux and macOS. +- Container: using a Docker image. + +#### Local: Debian-based Distribution +##### Manual steps +To install these in a Debian-based distribution: +1. Execute: + +``` +sudo apt-get install \ + openjdk-8-jdk \ + python-setuptools \ + python-pip \ + virtualenv \ + tox \ + docker-ce +``` + +2. On some systems, like Ubuntu 20.04, install these: + +``` +pip3 install grpcio-tools mypy-protobuf +``` + +3. If you you develop in GO: + 1. Install [Go](https://golang.org/doc/install). + 1. Check BEAM repo is in: `$GOPATH/src/github.com/apache/` + 1. At the end, it should look like this: `$GOPATH/src/github.com/apache/beam` +4. Once Go is installed, install goavro: + +``` +$ export GOPATH=`pwd`/sdks/go/examples/.gogradle/project_gopath +$ go get github.com/linkedin/goavro +``` + +**Important**: gLinux users should configure their machines for sudoless Docker. + +##### Automated script for Linux and macOS + +You can install these in a Debian-based distribution for Linux or macOs using the [local-env-setup.sh](https://github.com/apache/beam/blob/master/local-env-setup.sh) script, which is part of the Beam repo. It contains: + +* pip3 packages +* go packages +* goavro +* JDK 8 +* Python +* Docker + +To install: + +1. Execute: +``` +./local-env-setup.sh +``` + +#### Container: Docker-based + +Alternatively, you can use the Docker based local development environment to wrap your clone of the Beam repo +into a container meeting the requirements above. + +You can start this container using the [start-build-env.sh](https://github.com/apache/beam/blob/master/start-build-env.sh) +script which is part of the Beam repo: + +1. Execute: +``` +./start-build-env.sh +``` + +### Connect With the Beam community + +1. Consider subscribing to the [dev@ mailing list](/community/contact-us/), especially + if you plan to make more than one change or the change will be large. All decisions happen on the + public dev list. +1. (Optionally) Join the [#beam channel of the ASF slack](/community/contact-us/). +1. Create an account on [Beam issue tracker (JIRA)](https://issues.apache.org/jira/projects/BEAM/issues) + (anyone can do this). + +### Share your intent + +1. Find or create an issue in the [Beam issue tracker (JIRA)](https://issues.apache.org/jira/projects/BEAM/issues). + Tracking your work in an issue will avoid duplicated or conflicting work, and provide + a place for notes. Later, your pull request will be linked to the issue as well. +1. If you want to get involved but don't have a project in mind, check our list of open starter tasks, + [https://s.apache.org/beam-starter-tasks](https://s.apache.org/beam-starter-tasks). +1. Assign the issue to yourself. To get the permission to do so, email + the [dev@ mailing list](/community/contact-us) + to introduce yourself and to be added as a contributor in the Beam issue tracker including your + ASF Jira Username. For example [this welcome email](https://lists.apache.org/thread.html/e6018c2aaf7dc7895091434295e5b0fafe192b975e3e3761fcf0cda7@%3Cdev.beam.apache.org%3E). +1. If your change is large or it is your first change, it is a good idea to + [discuss it on the dev@ mailing list](/community/contact-us/). +1. For large changes create a design doc + ([template](https://s.apache.org/beam-design-doc-template), + [examples](https://s.apache.org/beam-design-docs)) and email it to the [dev@ mailing list](/community/contact-us). + +### Development Setup {#development-setup} + +1. Check [Git workflow tips](https://cwiki.apache.org/confluence/display/BEAM/Git+Tips) if you need help with git forking, cloning, branching, committing, pull requests, and squashing commits. +1. Clone the git repository. You can download it anywhere you like. + + $ mkdir -p ~/go/src/github.com/apache + $ cd ~/go/src/github.com/apache + $ git clone https://github.com/apache/beam + $ cd beam + + - For Go development: + We recommend putting it in your [`$GOPATH`](https://golang.org/doc/gopath_code#GOPATH) (`$HOME/go` by default on Unix systems). + 1. Clone the repo, and update your branch as normal + $ git clone https://github.com/apache/beam.git + $ cd beam + $ git remote add <GitHub_user> [email protected]:<GitHub_user>/beam.git + $ git fetch --all + 1. Get or Update all the Go SDK dependencies + $ go get -u ./... + +1. Check the environment was set up correctly. + - **Option 1**: validate the Go, Java, and Python environments: + + **Important**: Make sure you have activated Python development. +``` +./gradlew :checkSetup +``` + - **Option 2**: Run independent checks: + - For **Go development**: + 1. Execute: +``` +export GOLANG_PROTOBUF_REGISTRATION_CONFLICT=ignore +./gradlew :sdks:go:examples:wordCount +``` + - For **Python development**: Review comment: Markdown not rendering... ########## File path: website/www/site/content/en/contribute/_index.md ########## @@ -24,366 +24,217 @@ See the License for the specific language governing permissions and limitations under the License. --> -# Contribution guide +# Contribute to Apache Beam + +Apache Beam is an open source project developed and maintained by a friendly community of users, contributors, committers, and project management committee (PMC) members. + +Everyone is welcome to join and contribute to the project in multiple ways, not only with code contributions. Contributions like asking questions, reporting bugs, proposing new features, improving documentation, adding to the website, organizing events, or writing blog posts among other contributions, are welcome and recognized. + +### What’s In It for You? + - Become a part of the world’s largest, most welcoming open source community + - Drive innovation for next-gen distributed data processing + - Make an impact by bringing advanced technology solutions into reality + - Empower your brand recognition and thought leadership in the data world + +### Ways You Can Contribute + +Here’s a list of things you can do to get started contributing: + +<table class="table table-bordered"> + <tr> + <th>Contribution</th> + <th>How to do it</th> + </tr> + <tr> + <td>Participate and share ideas</td> + <td class="list-text-size-md"> + <ol> + <li>Start by <a href="https://beam.apache.org/contribute/#accounts-and-permissions";>getting an Apache Beam Jira account</a>.</li> + <li>Subscribe to our <a href="https://beam.apache.org/community/contact-us/";>mailing lists</a> (<a href="mailto:[email protected]";>[email protected]</a> and <a href="mailto:[email protected]";>[email protected]</a>).</li> + <li>Stay in the loop of our <a href="https://issues.apache.org/jira/projects/BEAM/issues";>Jira product backlog</a>, participate in discussions to brainstorm solutions, and help the community solve their mailing list questions.</li> + </ol> + </td> + </tr> + <tr> + <td>File a bug</td> + <td class="list-text-size-md"> + <ol class="no-bullet"> + <li>Create a Jira ticket and mark it as a bug</li> + <br><span>OR</span><br><br> + <li>Send an email to <a href="mailto:[email protected]";>[email protected]</a> with subject: [Bug] <bug summary> and include as much detail as possible.</li> + </ol> + </td> + </tr> + <tr> + <td>Propose a feature</td> + <td class="list-text-size-md"> + <ol class="no-bullet"> + <li>Create a Jira ticket and mark it as a feature request</li> + <br><span>OR</span><br><br> + <li>Send an email to <a href="mailto:[email protected]";>[email protected]</a> with subject: [Feature Proposal] <your idea>.</li> + </ol> + </td> + </tr> + <tr> + <td>Improve documentation</td> + <td> + Click the edit icon in our documentation and it’ll take you to our <a href="https://github.com/apache/beam/";>GitHub</a>, where you can automatically make the change you want and send a Pull Request. + </td> + </tr> + <tr> + <td>Improve the website</td> + <td class="list-text-size-md"> + <ul> + <li>Follow the website <a href="https://beam.apache.org/contribute/get-started-contributing/";>Contribution Guide</a> to make changes yourself.</li> + <li>Check our <a href="https://issues.apache.org/jira/browse/BEAM-13229?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20website%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC";>Website Backlog</a> in Jira to find rooms for improvement areas, or propose one.</li> Review comment: "find rooms for improvement areas" -> this doesn't quite make sense to me (?) ########## File path: website/www/site/content/en/contribute/get-started-contributing.md ########## @@ -0,0 +1,383 @@ +--- +title: "Beam Contribution Guide" +type: "contribute" +layout: "arrow_template" +--- + +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Contribution guide + +<a class="arrow-list-header" data-toggle="collapse" href="#collapseOverview" role="button" aria-expanded="false" aria-controls="collapseOverview"> + {{< figure src="/images/arrow-icon_list.svg">}} + +## Overview + +</a> + +<div class="collapse dont-collapse-sm" id="collapseOverview"> + +There are lots of opportunities to contribute. You can for example: + +- ask or answer questions on [[email protected]](https://beam.apache.org/community/contact-us/) or + [stackoverflow](https://stackoverflow.com/questions/tagged/apache-beam) +- review proposed design ideas on [[email protected]](https://beam.apache.org/community/contact-us/) +- improve the documentation +- file [bug reports](https://issues.apache.org/jira/projects/BEAM/issues) +- test releases +- review [changes](https://github.com/apache/beam/pulls) +- write new examples +- improve your favorite language SDK (Java, Python, Go, etc) +- improve specific runners (Apache Flink, Apache Spark, Google + Cloud Dataflow, etc) +- improve or add IO connectors +- add new transform libraries (statistics, ML, image processing, etc) +- work on the core programming model (what is a Beam pipeline and how does it + run?) +- improve the developer experience (for example, Windows guides) +- add answers to the [contribution FAQ]( + https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ) +- organize local meetups of users or contributors to Apache Beam + +</div> + +<a class="arrow-list-header" data-toggle="collapse" href="#collapseContributing" role="button" aria-expanded="false" aria-controls="collapseContributing"> + {{< figure src="/images/arrow-icon_list.svg">}} + +## Contributing code + + </a> + +<div class="collapse dont-collapse-sm" id="collapseContributing"> + +Below is a tutorial for contributing code to Beam, covering our tools and typical process in +detail. + +### Prerequisites + +- A GitHub account. +- A Linux, macOS, or Microsoft Windows development environment +- Java JDK 8 installed +- [Docker](https://www.docker.com/) installed for some tasks including building worker containers and testing this website + changes locally. +- For SDK Development: + - [Go](https://golang.org) 1.12 or later installed for Go SDK development + - Python 3.x interpreters. You will need Python interpreters for all Python versions supported by Beam. + Interpreters should be installed and available in shell via `python3.x` commands. For more information, see: + Python installation tips in [Developer Wiki](https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-InstallingPythoninterpreters). +- For large contributions, a signed [Individual Contributor License + Agreement](https://www.apache.org/licenses/icla.pdf) (ICLA) to the Apache + Software Foundation (ASF). + + +### Configuration options +You have two options for configuring your development environment: +- Local: + - Manually installing the prerequisites listed above. + - Using the automated script for Linux and macOS. +- Container: using a Docker image. + +#### Local: Debian-based Distribution +##### Manual steps +To install these in a Debian-based distribution: +1. Execute: + +``` +sudo apt-get install \ + openjdk-8-jdk \ + python-setuptools \ + python-pip \ + virtualenv \ + tox \ + docker-ce +``` + +2. On some systems, like Ubuntu 20.04, install these: + +``` +pip3 install grpcio-tools mypy-protobuf +``` + +3. If you you develop in GO: + 1. Install [Go](https://golang.org/doc/install). + 1. Check BEAM repo is in: `$GOPATH/src/github.com/apache/` + 1. At the end, it should look like this: `$GOPATH/src/github.com/apache/beam` +4. Once Go is installed, install goavro: + +``` +$ export GOPATH=`pwd`/sdks/go/examples/.gogradle/project_gopath +$ go get github.com/linkedin/goavro +``` + +**Important**: gLinux users should configure their machines for sudoless Docker. + +##### Automated script for Linux and macOS + +You can install these in a Debian-based distribution for Linux or macOs using the [local-env-setup.sh](https://github.com/apache/beam/blob/master/local-env-setup.sh) script, which is part of the Beam repo. It contains: + +* pip3 packages +* go packages +* goavro +* JDK 8 +* Python +* Docker + +To install: + +1. Execute: +``` +./local-env-setup.sh +``` + +#### Container: Docker-based + +Alternatively, you can use the Docker based local development environment to wrap your clone of the Beam repo +into a container meeting the requirements above. + +You can start this container using the [start-build-env.sh](https://github.com/apache/beam/blob/master/start-build-env.sh) +script which is part of the Beam repo: + +1. Execute: +``` +./start-build-env.sh +``` + +### Connect With the Beam community + +1. Consider subscribing to the [dev@ mailing list](/community/contact-us/), especially + if you plan to make more than one change or the change will be large. All decisions happen on the + public dev list. +1. (Optionally) Join the [#beam channel of the ASF slack](/community/contact-us/). +1. Create an account on [Beam issue tracker (JIRA)](https://issues.apache.org/jira/projects/BEAM/issues) + (anyone can do this). + +### Share your intent + +1. Find or create an issue in the [Beam issue tracker (JIRA)](https://issues.apache.org/jira/projects/BEAM/issues). + Tracking your work in an issue will avoid duplicated or conflicting work, and provide + a place for notes. Later, your pull request will be linked to the issue as well. +1. If you want to get involved but don't have a project in mind, check our list of open starter tasks, + [https://s.apache.org/beam-starter-tasks](https://s.apache.org/beam-starter-tasks). +1. Assign the issue to yourself. To get the permission to do so, email + the [dev@ mailing list](/community/contact-us) + to introduce yourself and to be added as a contributor in the Beam issue tracker including your + ASF Jira Username. For example [this welcome email](https://lists.apache.org/thread.html/e6018c2aaf7dc7895091434295e5b0fafe192b975e3e3761fcf0cda7@%3Cdev.beam.apache.org%3E). +1. If your change is large or it is your first change, it is a good idea to + [discuss it on the dev@ mailing list](/community/contact-us/). +1. For large changes create a design doc + ([template](https://s.apache.org/beam-design-doc-template), + [examples](https://s.apache.org/beam-design-docs)) and email it to the [dev@ mailing list](/community/contact-us). + +### Development Setup {#development-setup} + +1. Check [Git workflow tips](https://cwiki.apache.org/confluence/display/BEAM/Git+Tips) if you need help with git forking, cloning, branching, committing, pull requests, and squashing commits. +1. Clone the git repository. You can download it anywhere you like. + + $ mkdir -p ~/go/src/github.com/apache + $ cd ~/go/src/github.com/apache + $ git clone https://github.com/apache/beam + $ cd beam + + - For Go development: + We recommend putting it in your [`$GOPATH`](https://golang.org/doc/gopath_code#GOPATH) (`$HOME/go` by default on Unix systems). + 1. Clone the repo, and update your branch as normal + $ git clone https://github.com/apache/beam.git + $ cd beam + $ git remote add <GitHub_user> [email protected]:<GitHub_user>/beam.git + $ git fetch --all + 1. Get or Update all the Go SDK dependencies + $ go get -u ./... + +1. Check the environment was set up correctly. + - **Option 1**: validate the Go, Java, and Python environments: Review comment: The Markdown is not rendering correctly in any of these steps. ########## File path: website/www/site/content/en/contribute/_index.md ########## @@ -24,366 +24,217 @@ See the License for the specific language governing permissions and limitations under the License. --> -# Contribution guide +# Contribute to Apache Beam + +Apache Beam is an open source project developed and maintained by a friendly community of users, contributors, committers, and project management committee (PMC) members. + +Everyone is welcome to join and contribute to the project in multiple ways, not only with code contributions. Contributions like asking questions, reporting bugs, proposing new features, improving documentation, adding to the website, organizing events, or writing blog posts among other contributions, are welcome and recognized. + +### What’s In It for You? + - Become a part of the world’s largest, most welcoming open source community + - Drive innovation for next-gen distributed data processing + - Make an impact by bringing advanced technology solutions into reality + - Empower your brand recognition and thought leadership in the data world + +### Ways You Can Contribute + +Here’s a list of things you can do to get started contributing: + +<table class="table table-bordered"> + <tr> + <th>Contribution</th> + <th>How to do it</th> + </tr> + <tr> + <td>Participate and share ideas</td> + <td class="list-text-size-md"> + <ol> + <li>Start by <a href="https://beam.apache.org/contribute/#accounts-and-permissions";>getting an Apache Beam Jira account</a>.</li> + <li>Subscribe to our <a href="https://beam.apache.org/community/contact-us/";>mailing lists</a> (<a href="mailto:[email protected]";>[email protected]</a> and <a href="mailto:[email protected]";>[email protected]</a>).</li> + <li>Stay in the loop of our <a href="https://issues.apache.org/jira/projects/BEAM/issues";>Jira product backlog</a>, participate in discussions to brainstorm solutions, and help the community solve their mailing list questions.</li> + </ol> + </td> + </tr> + <tr> + <td>File a bug</td> + <td class="list-text-size-md"> + <ol class="no-bullet"> + <li>Create a Jira ticket and mark it as a bug</li> + <br><span>OR</span><br><br> + <li>Send an email to <a href="mailto:[email protected]";>[email protected]</a> with subject: [Bug] <bug summary> and include as much detail as possible.</li> + </ol> + </td> + </tr> + <tr> + <td>Propose a feature</td> + <td class="list-text-size-md"> + <ol class="no-bullet"> + <li>Create a Jira ticket and mark it as a feature request</li> + <br><span>OR</span><br><br> + <li>Send an email to <a href="mailto:[email protected]";>[email protected]</a> with subject: [Feature Proposal] <your idea>.</li> + </ol> + </td> + </tr> + <tr> + <td>Improve documentation</td> + <td> + Click the edit icon in our documentation and it’ll take you to our <a href="https://github.com/apache/beam/";>GitHub</a>, where you can automatically make the change you want and send a Pull Request. + </td> + </tr> + <tr> + <td>Improve the website</td> + <td class="list-text-size-md"> + <ul> + <li>Follow the website <a href="https://beam.apache.org/contribute/get-started-contributing/";>Contribution Guide</a> to make changes yourself.</li> + <li>Check our <a href="https://issues.apache.org/jira/browse/BEAM-13229?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20website%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC";>Website Backlog</a> in Jira to find rooms for improvement areas, or propose one.</li> + </ul> + </td> + </tr> + <tr> + <td>Write a blogpost</td> + <td> + Tell a story about your custom use cases or share your recommendations and best practices. + </td> + </tr> + <tr> + <td>Share your success story</td> + <td> + Visit our <a href="https://beam.apache.org/case-studies/";>Case Studies</a> page and share your story or add your company’s logo. + </td> + </tr> + <tr> + <td>Produce a social media publication</td> + <td> + Share the use cases that Beam helps empower or the data processing performance you achieved. Spread the word about the upcoming Beam Summits and conferences. + </td> + </tr> + <tr> + <td>Organize an event</td> + <td> + Send an email to <a href="mailto:[email protected]";>[email protected]</a> to share your idea about the Beam event you want to organize. + </td> + </tr> + <tr> + <td>Build a new feature like a new IO connector or PTransform</td> + <td class="list-text-size-md"> + <ul> + <li>Create and contribute a custom <a href="https://beam.apache.org/documentation/programming-guide/#pipeline-io";>I/O connector</a> for a specific data storage system you want to connect to, if it is not on <a href="https://beam.apache.org/documentation/io/built-in/";>the I/O list yet</a>.</li> + <li>Look at our project backlog to get some inspiration on what to build.</li> + <li>Follow our <a href="https://beam.apache.org/contribute/get-started-contributing/";>Code Contribution Guide</a>.</li> + </ul> + </td> + </tr> + <tr> + <td>Share a code sample or template</td> + <td class="list-text-size-md"> + <ol> + <li>Read our <a href="https://beam.apache.org/contribute/get-started-contributing/";>Code Contribution Guide</a>.</li> + <li>Create your code sample or template. </li> + <li><a href="https://beam.apache.org/contribute/get-started-contributing/#make-your-change";>Make a Pull Request</a> to add it to the Beam example folder.</li> + </ol> + </td> + </tr> + <tr> + <td>Test a release candidate</td> + <td> + Anybody can <a href="https://beam.apache.org/contribute/release-guide/";>propose a release</a> via the [email protected] mailing list. Try Apache Beam releases in your projects, vote for release candidates, inform the community about the results and any issues found via <a href="mailto:[email protected]";>[email protected]</a>. Learn more about how to validate a Beam release <a href="https://beam.apache.org/blog/validate-beam-release/";>here</a>. + </td> + </tr> + <tr> + <td>Other ideas?</td> + <td> + If you want to contribute in another way and you’re not sure how to get started, send a proposal to <a href="mailto:[email protected]";>[email protected]</a>. + </td> + </tr> +</table> + + +## Become a Committer + +If you like contributing to Apache Beam, you are active in the Apache Beam mailing lists, Slack channel, or you attend/organize events, then you are a candidate to become a project committer. + +Committers are community members that have write access to the project’s repositories (i.e., they can modify the code, documentation, and website by themselves and also assist with code reviews and accept other contributions). + +Even though there is no strict protocol to become a committer, the most important thing we look for is that you display the traits of an Apache Beam committer and that you have been consistently contributing to the project and community. + +Candidates for new committers are suggested by current committers or PMC members, and voted upon by the PMC team. So if you would like to become a committer, engage with the community and keep contributing to Apache Beam, you might also want to talk to other committers and ask for their advice and guidance. Feel free to email any committer directly and ask for their advice! + +## The Traits of an Apache Beam Committer + +{{< contributor/row_of_traits committer_traits >}} + +## Become a Project Management Committee (PMC) Member + +The PMC is the official managing body of the project as per the Apache Software Foundation governance model: “The Apache Way”. PMC members vote to set Apache Beam’s community and technical direction and are directly responsible for overseeing releases and the healthy development of the community. + +PMC members are also responsible for ensuring their project follows certain core requirements set by the board or other corporate officers of the ASF, like following Legal, Branding, and Infrastructure related requirements, along with ensuring their community operates within the [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct#code-of-conduct). + +Since the PMC members are responsible for establishing the Apache Beam vision and leading our community to work in harmony to achieve it, we look for committed individuals who can perform the responsibilities of the PMC, as well as lead our community to fulfill its vision. This is why we strive to build a diverse PMC that complements skills and expertise sharing the same sets of values. + +Ideally, we find candidates among active community members that have shown initiative to shape the direction of Beam and have shown willingness to learn the official processes, such as how to create or verify for releases, how to ensure trademark compliance for events, among others. We also expect PMC candidates to be level-headed, constructive, supportive, and willing to “disagree and commit” at times. Review comment: "verify for releases" -> "verify releases" ########## File path: website/www/site/content/en/contribute/get-started-contributing.md ########## @@ -0,0 +1,383 @@ +--- +title: "Beam Contribution Guide" +type: "contribute" +layout: "arrow_template" +--- + +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Contribution guide + +<a class="arrow-list-header" data-toggle="collapse" href="#collapseOverview" role="button" aria-expanded="false" aria-controls="collapseOverview"> + {{< figure src="/images/arrow-icon_list.svg">}} + +## Overview + +</a> + +<div class="collapse dont-collapse-sm" id="collapseOverview"> + +There are lots of opportunities to contribute. You can for example: + +- ask or answer questions on [[email protected]](https://beam.apache.org/community/contact-us/) or + [stackoverflow](https://stackoverflow.com/questions/tagged/apache-beam) +- review proposed design ideas on [[email protected]](https://beam.apache.org/community/contact-us/) +- improve the documentation +- file [bug reports](https://issues.apache.org/jira/projects/BEAM/issues) +- test releases +- review [changes](https://github.com/apache/beam/pulls) +- write new examples +- improve your favorite language SDK (Java, Python, Go, etc) +- improve specific runners (Apache Flink, Apache Spark, Google + Cloud Dataflow, etc) +- improve or add IO connectors +- add new transform libraries (statistics, ML, image processing, etc) +- work on the core programming model (what is a Beam pipeline and how does it + run?) +- improve the developer experience (for example, Windows guides) +- add answers to the [contribution FAQ]( + https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ) +- organize local meetups of users or contributors to Apache Beam + +</div> + +<a class="arrow-list-header" data-toggle="collapse" href="#collapseContributing" role="button" aria-expanded="false" aria-controls="collapseContributing"> + {{< figure src="/images/arrow-icon_list.svg">}} + +## Contributing code + + </a> + +<div class="collapse dont-collapse-sm" id="collapseContributing"> + +Below is a tutorial for contributing code to Beam, covering our tools and typical process in +detail. + +### Prerequisites + +- A GitHub account. +- A Linux, macOS, or Microsoft Windows development environment +- Java JDK 8 installed +- [Docker](https://www.docker.com/) installed for some tasks including building worker containers and testing this website + changes locally. +- For SDK Development: + - [Go](https://golang.org) 1.12 or later installed for Go SDK development + - Python 3.x interpreters. You will need Python interpreters for all Python versions supported by Beam. + Interpreters should be installed and available in shell via `python3.x` commands. For more information, see: + Python installation tips in [Developer Wiki](https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-InstallingPythoninterpreters). +- For large contributions, a signed [Individual Contributor License + Agreement](https://www.apache.org/licenses/icla.pdf) (ICLA) to the Apache + Software Foundation (ASF). + + +### Configuration options +You have two options for configuring your development environment: +- Local: + - Manually installing the prerequisites listed above. + - Using the automated script for Linux and macOS. +- Container: using a Docker image. + +#### Local: Debian-based Distribution +##### Manual steps +To install these in a Debian-based distribution: +1. Execute: + +``` +sudo apt-get install \ + openjdk-8-jdk \ + python-setuptools \ + python-pip \ + virtualenv \ + tox \ + docker-ce +``` + +2. On some systems, like Ubuntu 20.04, install these: + +``` +pip3 install grpcio-tools mypy-protobuf +``` + +3. If you you develop in GO: + 1. Install [Go](https://golang.org/doc/install). + 1. Check BEAM repo is in: `$GOPATH/src/github.com/apache/` + 1. At the end, it should look like this: `$GOPATH/src/github.com/apache/beam` +4. Once Go is installed, install goavro: + +``` +$ export GOPATH=`pwd`/sdks/go/examples/.gogradle/project_gopath +$ go get github.com/linkedin/goavro +``` + +**Important**: gLinux users should configure their machines for sudoless Docker. + +##### Automated script for Linux and macOS + +You can install these in a Debian-based distribution for Linux or macOs using the [local-env-setup.sh](https://github.com/apache/beam/blob/master/local-env-setup.sh) script, which is part of the Beam repo. It contains: + +* pip3 packages +* go packages +* goavro +* JDK 8 +* Python +* Docker + +To install: + +1. Execute: +``` +./local-env-setup.sh +``` + +#### Container: Docker-based + +Alternatively, you can use the Docker based local development environment to wrap your clone of the Beam repo +into a container meeting the requirements above. + +You can start this container using the [start-build-env.sh](https://github.com/apache/beam/blob/master/start-build-env.sh) +script which is part of the Beam repo: + +1. Execute: +``` +./start-build-env.sh +``` + +### Connect With the Beam community + +1. Consider subscribing to the [dev@ mailing list](/community/contact-us/), especially + if you plan to make more than one change or the change will be large. All decisions happen on the + public dev list. +1. (Optionally) Join the [#beam channel of the ASF slack](/community/contact-us/). +1. Create an account on [Beam issue tracker (JIRA)](https://issues.apache.org/jira/projects/BEAM/issues) + (anyone can do this). + +### Share your intent + +1. Find or create an issue in the [Beam issue tracker (JIRA)](https://issues.apache.org/jira/projects/BEAM/issues). + Tracking your work in an issue will avoid duplicated or conflicting work, and provide + a place for notes. Later, your pull request will be linked to the issue as well. +1. If you want to get involved but don't have a project in mind, check our list of open starter tasks, + [https://s.apache.org/beam-starter-tasks](https://s.apache.org/beam-starter-tasks). +1. Assign the issue to yourself. To get the permission to do so, email + the [dev@ mailing list](/community/contact-us) + to introduce yourself and to be added as a contributor in the Beam issue tracker including your + ASF Jira Username. For example [this welcome email](https://lists.apache.org/thread.html/e6018c2aaf7dc7895091434295e5b0fafe192b975e3e3761fcf0cda7@%3Cdev.beam.apache.org%3E). +1. If your change is large or it is your first change, it is a good idea to + [discuss it on the dev@ mailing list](/community/contact-us/). +1. For large changes create a design doc + ([template](https://s.apache.org/beam-design-doc-template), + [examples](https://s.apache.org/beam-design-docs)) and email it to the [dev@ mailing list](/community/contact-us). + +### Development Setup {#development-setup} + +1. Check [Git workflow tips](https://cwiki.apache.org/confluence/display/BEAM/Git+Tips) if you need help with git forking, cloning, branching, committing, pull requests, and squashing commits. +1. Clone the git repository. You can download it anywhere you like. + + $ mkdir -p ~/go/src/github.com/apache + $ cd ~/go/src/github.com/apache + $ git clone https://github.com/apache/beam + $ cd beam + + - For Go development: + We recommend putting it in your [`$GOPATH`](https://golang.org/doc/gopath_code#GOPATH) (`$HOME/go` by default on Unix systems). + 1. Clone the repo, and update your branch as normal + $ git clone https://github.com/apache/beam.git + $ cd beam + $ git remote add <GitHub_user> [email protected]:<GitHub_user>/beam.git + $ git fetch --all + 1. Get or Update all the Go SDK dependencies + $ go get -u ./... + +1. Check the environment was set up correctly. + - **Option 1**: validate the Go, Java, and Python environments: + + **Important**: Make sure you have activated Python development. +``` +./gradlew :checkSetup +``` + - **Option 2**: Run independent checks: + - For **Go development**: + 1. Execute: +``` +export GOLANG_PROTOBUF_REGISTRATION_CONFLICT=ignore +./gradlew :sdks:go:examples:wordCount +``` + - For **Python development**: + 1. Execute: +``` +./gradlew :sdks:python:wordCount +``` + - For **Java development**: Review comment: Please review the markdown in all of these steps. It's being treated as code, so e.g. `**foo**` is not rendering. ########## File path: website/www/site/content/en/contribute/get-started-contributing.md ########## @@ -0,0 +1,383 @@ +--- +title: "Beam Contribution Guide" +type: "contribute" +layout: "arrow_template" +--- + +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Contribution guide + +<a class="arrow-list-header" data-toggle="collapse" href="#collapseOverview" role="button" aria-expanded="false" aria-controls="collapseOverview"> + {{< figure src="/images/arrow-icon_list.svg">}} + +## Overview + +</a> + +<div class="collapse dont-collapse-sm" id="collapseOverview"> + +There are lots of opportunities to contribute. You can for example: + +- ask or answer questions on [[email protected]](https://beam.apache.org/community/contact-us/) or + [stackoverflow](https://stackoverflow.com/questions/tagged/apache-beam) +- review proposed design ideas on [[email protected]](https://beam.apache.org/community/contact-us/) +- improve the documentation +- file [bug reports](https://issues.apache.org/jira/projects/BEAM/issues) +- test releases +- review [changes](https://github.com/apache/beam/pulls) +- write new examples +- improve your favorite language SDK (Java, Python, Go, etc) +- improve specific runners (Apache Flink, Apache Spark, Google + Cloud Dataflow, etc) +- improve or add IO connectors +- add new transform libraries (statistics, ML, image processing, etc) +- work on the core programming model (what is a Beam pipeline and how does it + run?) +- improve the developer experience (for example, Windows guides) +- add answers to the [contribution FAQ]( + https://cwiki.apache.org/confluence/display/BEAM/Contributor+FAQ) +- organize local meetups of users or contributors to Apache Beam + +</div> + +<a class="arrow-list-header" data-toggle="collapse" href="#collapseContributing" role="button" aria-expanded="false" aria-controls="collapseContributing"> + {{< figure src="/images/arrow-icon_list.svg">}} + +## Contributing code + + </a> + +<div class="collapse dont-collapse-sm" id="collapseContributing"> + +Below is a tutorial for contributing code to Beam, covering our tools and typical process in +detail. + +### Prerequisites + +- A GitHub account. +- A Linux, macOS, or Microsoft Windows development environment +- Java JDK 8 installed +- [Docker](https://www.docker.com/) installed for some tasks including building worker containers and testing this website + changes locally. +- For SDK Development: + - [Go](https://golang.org) 1.12 or later installed for Go SDK development + - Python 3.x interpreters. You will need Python interpreters for all Python versions supported by Beam. + Interpreters should be installed and available in shell via `python3.x` commands. For more information, see: + Python installation tips in [Developer Wiki](https://cwiki.apache.org/confluence/display/BEAM/Python+Tips#PythonTips-InstallingPythoninterpreters). +- For large contributions, a signed [Individual Contributor License + Agreement](https://www.apache.org/licenses/icla.pdf) (ICLA) to the Apache + Software Foundation (ASF). + + +### Configuration options +You have two options for configuring your development environment: +- Local: + - Manually installing the prerequisites listed above. + - Using the automated script for Linux and macOS. +- Container: using a Docker image. + +#### Local: Debian-based Distribution +##### Manual steps +To install these in a Debian-based distribution: +1. Execute: + +``` +sudo apt-get install \ + openjdk-8-jdk \ + python-setuptools \ + python-pip \ + virtualenv \ + tox \ + docker-ce +``` + +2. On some systems, like Ubuntu 20.04, install these: + +``` +pip3 install grpcio-tools mypy-protobuf +``` + +3. If you you develop in GO: + 1. Install [Go](https://golang.org/doc/install). + 1. Check BEAM repo is in: `$GOPATH/src/github.com/apache/` + 1. At the end, it should look like this: `$GOPATH/src/github.com/apache/beam` +4. Once Go is installed, install goavro: + +``` +$ export GOPATH=`pwd`/sdks/go/examples/.gogradle/project_gopath +$ go get github.com/linkedin/goavro +``` + +**Important**: gLinux users should configure their machines for sudoless Docker. + +##### Automated script for Linux and macOS + +You can install these in a Debian-based distribution for Linux or macOs using the [local-env-setup.sh](https://github.com/apache/beam/blob/master/local-env-setup.sh) script, which is part of the Beam repo. It contains: + +* pip3 packages +* go packages +* goavro +* JDK 8 +* Python +* Docker + +To install: + +1. Execute: +``` +./local-env-setup.sh +``` + +#### Container: Docker-based + +Alternatively, you can use the Docker based local development environment to wrap your clone of the Beam repo +into a container meeting the requirements above. + +You can start this container using the [start-build-env.sh](https://github.com/apache/beam/blob/master/start-build-env.sh) +script which is part of the Beam repo: + +1. Execute: +``` +./start-build-env.sh +``` + +### Connect With the Beam community + +1. Consider subscribing to the [dev@ mailing list](/community/contact-us/), especially + if you plan to make more than one change or the change will be large. All decisions happen on the + public dev list. +1. (Optionally) Join the [#beam channel of the ASF slack](/community/contact-us/). +1. Create an account on [Beam issue tracker (JIRA)](https://issues.apache.org/jira/projects/BEAM/issues) + (anyone can do this). + +### Share your intent + +1. Find or create an issue in the [Beam issue tracker (JIRA)](https://issues.apache.org/jira/projects/BEAM/issues). + Tracking your work in an issue will avoid duplicated or conflicting work, and provide + a place for notes. Later, your pull request will be linked to the issue as well. +1. If you want to get involved but don't have a project in mind, check our list of open starter tasks, + [https://s.apache.org/beam-starter-tasks](https://s.apache.org/beam-starter-tasks). +1. Assign the issue to yourself. To get the permission to do so, email + the [dev@ mailing list](/community/contact-us) + to introduce yourself and to be added as a contributor in the Beam issue tracker including your + ASF Jira Username. For example [this welcome email](https://lists.apache.org/thread.html/e6018c2aaf7dc7895091434295e5b0fafe192b975e3e3761fcf0cda7@%3Cdev.beam.apache.org%3E). +1. If your change is large or it is your first change, it is a good idea to + [discuss it on the dev@ mailing list](/community/contact-us/). +1. For large changes create a design doc + ([template](https://s.apache.org/beam-design-doc-template), + [examples](https://s.apache.org/beam-design-docs)) and email it to the [dev@ mailing list](/community/contact-us). + +### Development Setup {#development-setup} + +1. Check [Git workflow tips](https://cwiki.apache.org/confluence/display/BEAM/Git+Tips) if you need help with git forking, cloning, branching, committing, pull requests, and squashing commits. +1. Clone the git repository. You can download it anywhere you like. + + $ mkdir -p ~/go/src/github.com/apache + $ cd ~/go/src/github.com/apache Review comment: Can you confirm that these commands are right? I don't understand why you'd create a directory at `~/go/src/github.com/apache` before cloning the repo. Unless this is something specific for Go...? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
