[
https://issues.apache.org/jira/browse/BEAM-4501?focusedWorklogId=151866&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151866
]
ASF GitHub Bot logged work on BEAM-4501:
----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Oct/18 21:06
Start Date: 05/Oct/18 21:06
Worklog Time Spent: 10m
Work Description: swegner closed pull request #6533: [BEAM-4501] Update
website contribution and release docs
URL: https://github.com/apache/beam/pull/6533
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/website/README.md b/website/README.md
index 0fe188268bb..acf79f91c4a 100644
--- a/website/README.md
+++ b/website/README.md
@@ -1,16 +1,24 @@
-# Apache Beam website
-
-_**The website source code is currently being migrated from
-https://github.com/apache/beam-site, but is not yet ready. Please
-continue to contribute changes at
-[apache/beam-site](https://github.com/apache/beam-site) according to
-the [website contribution
guide](https://beam.apache.org/contribute/#contributing-to-the-website).
-You can track migration progress via
-[[BEAM-4493]](https://issues.apache.org/jira/browse/BEAM-4493).**_
-
-This is the website for [Apache Beam](https://beam.apache.org/), hosted at:
-
- https://beam.apache.org/
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+These are the main sources of the website for Apache Beam, hosted at
+https://beam.apache.org/.
## About this site
@@ -18,55 +26,32 @@ The Beam website is built using
[Jekyll](http://jekyllrb.com/). Additionally,
for additional formatting capabilities, this website uses
[Twitter Bootstrap](http://getbootstrap.com/).
-### Repository Structure
-
-This repository contains:
-
-1. `src/`: the source of the site, including markdown files containing the
bulk of the content
-1. `content/`: html generated from the markdown (which is what is actually
hosted on the website)
-
-## Development Workflow
-
-### Setup
-
-You need Ruby version >= 2.2.0 to build the project.
-
-Install [Ruby Gems](https://rubygems.org/pages/download), a package management
framework for Ruby.
+Documentation generated from source code, such as Javadoc and Pydoc, is stored
+separately on the [beam-site
+repository](https://github.com/apache/beam-site/tree/release-docs).
-Install [Bundler](http://bundler.io/v1.3/rationale.html), which we use to
specify dependencies and ensure
-a consistent environment for building the website, even across multiple
developers on different machines:
+## Active development
- $ gem install bundler
+Website development requires Docker installed if you wish to preview changes
and
+run website tests.
-Use Bundler to download the versions of each dependency specified in the
website's `Gemfile.lock`,
-including [Jekyll](https://jekyllrb.com/):
+The following command is used to build and serve the website locally.
- $ bundle install --deployment
+ $ ./gradlew -PpublishJekyllPort :beam-website:serveWebsite
-This will install a number of gems in a local `./vendor` directory.
+Any changes made locally will trigger a rebuild of the website.
-### Active development
+Websites tests may be run using this command:
-Launch Jekyll via Bundler in order to guarantee that the appropriate versions
of the dependencies are used:
+ $ ./gradlew :beam-website:testWebsite
- $ bundle exec jekyll serve
+## Website push
-Jekyll will start a webserver on port `4000`. As you make changes to the
-content, Jekyll will rebuild it automatically. This is helpful if you want to
see
-how your changes will render in realtime.
-
-In addition, check for dead links and the like by running the tests via:
-
- $ bundle exec rake test
-
-Running Jekyll may cause the `content/` directory to be generated.
-Merging autogenerated content can get tricky, so regenerating content is
-responsibility of the committer doing the final merge of your PR, and
-your PR should not contain changes to that directory.
-
-Before sending the PR for review, please run:
-
- $ git checkout -- content
+After a PR is merged, a background Jenkins job will automatically generate and
+push [website
+content](https://github.com/apache/beam/tree/asf-site/website/generated-content)
+to the asf-site branch. This content is later picked up and pushed to
+https://beam.apache.org/.
## Additional Information
@@ -87,19 +72,3 @@ and then run `bundle update`, which will regenerate the
complete `Gemfile.lock`.
Make sure that the updated `Gemfile.lock` is included in your pull request.
For more information,
see the Bundler [documentation](http://bundler.io/v1.3/rationale.html).
-## Apache License
-
-Except as otherwise noted this software is licensed under the
-[Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.html)
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
diff --git a/website/build.gradle b/website/build.gradle
index eb52de7dae8..715b02a234b 100644
--- a/website/build.gradle
+++ b/website/build.gradle
@@ -65,8 +65,12 @@ task createDockerContainer(type: Exec) {
ext.containerId = {
return standardOutput.toString().trim()
}
+ def extraOptions = ''
+ if (project.hasProperty('publishJekyllPort')) {
+ extraOptions = '-p 127.0.0.1:4000:4000'
+ }
commandLine '/bin/bash', '-c',
- "docker create -v $project.rootDir:$dockerWorkDir -u \$(id -u):\$(id -g)
$dockerImageTag"
+ "docker create -v $project.rootDir:$dockerWorkDir -u \$(id -u):\$(id -g)
$extraOptions $dockerImageTag"
}
task startDockerContainer(type: Exec) {
@@ -115,6 +119,24 @@ task buildWebsite(type: Exec) {
}
build.dependsOn buildWebsite
+task serveWebsite(type: Exec) {
+ dependsOn startDockerContainer, setupBuildDir
+ finalizedBy stopAndRemoveDockerContainer
+ inputs.files 'Gemfile.lock', '_config.yml'
+ inputs.dir 'src'
+ outputs.dir "$buildDir/.sass-cache"
+ outputs.dir buildContentDir
+ commandLine 'docker', 'exec',
+ "${->startDockerContainer.containerId()}", '/bin/bash', '-c',
+ """cd $dockerWorkDir/build/website && \
+ bundle exec jekyll serve \
+ --config $dockerWorkDir/website/_config.yml \
+ --incremental \
+ --source $dockerWorkDir/website/src \
+ --host 0.0.0.0
+ """
+}
+
task testWebsite(type: Exec) {
dependsOn startDockerContainer, buildWebsite
finalizedBy stopAndRemoveDockerContainer
diff --git a/website/src/contribute/index.md b/website/src/contribute/index.md
index 9390e65a95e..c061f388964 100644
--- a/website/src/contribute/index.md
+++ b/website/src/contribute/index.md
@@ -273,12 +273,10 @@ a read-only collaborator to apache/beam by opening an
INFRA ticket. -->
## Contributing to the website
-The Beam website is in the [Beam Site GitHub
-mirror](https://github.com/apache/beam-site) repository in the `asf-site`
-branch (_not_ `master`). The
-[README](https://github.com/apache/beam-site/blob/asf-site/README.md) there
-explains how to modify different parts of the site. The GitHub workflow is the
-same - make your change and open a pull request.
+The Beam website is in the `/website` directory of the repo. The
+[README](https://github.com/apache/beam/blob/master/website) there explains how
+to modify different parts of the site. The GitHub workflow is the same - make
+your change and open a pull request.
Issues are tracked in the
[website](https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20component%20%3D%20website)
diff --git a/website/src/contribute/release-guide.md
b/website/src/contribute/release-guide.md
index e4dd601635f..d3072c7ca31 100644
--- a/website/src/contribute/release-guide.md
+++ b/website/src/contribute/release-guide.md
@@ -167,8 +167,24 @@ please submit your GPG public key into [MIT PGP Public Key
Server](http://pgp.mi
#### Website development setup
-Prepare to update the Beam website by following the [website development
-instructions]({{ site.baseurl }}/contribute/website-contributions/).
+Updating the Beam website requires submitting PRs to both the main
`apache/beam`
+repo and the `apache/beam-site` repo. The first contains reference manuals
+generated from SDK code, while the second updates the current release version
+number.
+
+You should already have setup a local clone of `apache/beam`. Setting up a
clone
+of `apache/beam-site` is similar:
+
+ $ git clone -b release-docs https://github.com/apache/beam-site.git
+ $ cd beam-site
+ $ git remote add <GitHub_user> g...@github.com:<GitHub_user>/beam-site.git
+ $ git fetch --all
+ $ git checkout -b <my-branch> origin/release-docs
+
+Further instructions on website development on `apache/beam` is
+[here](https://github.com/apache/beam/blob/master/website). Background
+information about how the website is updated can be found in [Beam-Site
+Automation Reliability](https://s.apache.org/beam-site-automation).
#### Register to PyPI
@@ -449,15 +465,15 @@ For this step, we recommend you using automation script
to create a RC, but you
When prompted for a description, enter “Apache Beam, version X, release
candidate Y”.
1. Stage source release into dist.apache.org dev
[repo](https://dist.apache.org/repos/dist/dev/beam/).
1. Stage,sign and hash python binaries into dist.apache.ord dev repo python
dir
- 1. Create a PR to update beam-site, changes includes:
+ 1. Create a PR to update beam and beam-site, changes includes:
* Copy python doc into beam-site
* Copy java doc into beam-site
- * Update release version into
[_config.yml](https://github.com/apache/beam-site/blob/asf-site/_config.yml).
+ * Update release version into
[_config.yml](https://github.com/apache/beam/blob/master/website/_config.yml).
* Tasks you need to do manually
- 1. Add new release into src/get-started/downloads.md
- 1. Update last release download links in src/get-started/downloads.md
- 1. Update the Pydoc link on this page to point to the new version (in
src/documentation/sdks/pydoc/current.md.
+ 1. Add new release into `website/src/get-started/downloads.md`.
+ 1. Update last release download links in
`website/src/get-started/downloads.md`.
+ 1. Update `website/src/.htaccess` to redirect to the new version.
### Run all steps manually
@@ -547,7 +563,6 @@ Make sure you have ```tox``` installed:
```
pip install tox
-
```
Create the Python SDK documentation using sphinx by running a helper script.
```
@@ -555,33 +570,41 @@ cd sdks/python && tox -e docs
```
By default the Pydoc is generated in `sdks/python/target/docs/_build`. Let
`${PYDOC_ROOT}` be the absolute path to `_build`.
-#### Propose a pull request for website updates
-
-The final step of building the candidate is to propose a website pull request.
+#### Propose pull requests for website updates
-Start by updating `release_latest` version flag in the top-level
`_config.yml`, and list the new release in the [Apache Beam Downloads]({{
site.baseurl }}/get-started/downloads/), linking to the source code download
and the Release Notes in JIRA.
+Beam publishes API reference manuals for each release on the website. For Java
+and Python SDKs, that’s Javadoc and PyDoc, respectively. The final step of
+building the candidate is to propose website pull requests that update these
+manuals.
-Beam publishes API reference manual for each release on the website. For Java
SDK, that’s Javadoc.
+Merge the pull requests only after finalizing the release. To avoid invalid
+redirects for the 'current' version, merge these PRs in the order listed. Once
+the PR is merged, the new contents will get picked up automatically and served
+to the Beam website, usually within an hour.
-One of the artifacts created in the release contains the Javadoc for the
-website. To update the website, you must unpack this jar file from the release
-candidate into the source tree of the website.
+**PR 1: apache/beam-site**
-Add the new Javadoc to [SDK API Reference
page](https://beam.apache.org/releases/javadoc/) page, as follows:
+This pull request is against the `apache/beam-site` repo, on the `release-docs`
+branch.
-* Unpack the Maven artifact `org.apache.beam:beam-sdks-java-javadoc` into some
temporary location. Call this `${JAVADOC_TMP}`.
-* Copy the generated Javadoc into the website repository: `cp -r
${JAVADOC_TMP} src/documentation/sdks/javadoc/${RELEASE}`.
-* Set up the necessary git commands to account for the new and deleted files
from the javadoc.
-* Update the Javadoc link on this page to point to the new version (in
`src/documentation/sdks/javadoc/current.md`).
+* Add the new Javadoc to [SDK API Reference
page](https://beam.apache.org/releases/javadoc/) page, as follows:
+ * Unpack the Maven artifact `org.apache.beam:beam-sdks-java-javadoc` into
some temporary location. Call this `${JAVADOC_TMP}`.
+ * Copy the generated Javadoc into the website repository: `cp -r
${JAVADOC_TMP} javadoc/${RELEASE}`.
+* Add the new Pydoc to [SDK API Reference
page](https://beam.apache.org/releases/pydoc/) page, as follows:
+ * Copy the generated Pydoc into the website repository: `cp -r ${PYDOC_ROOT}
pydoc/${RELEASE}`.
+ * Remove `.doctrees` directory.
+* Stage files using: `git add --all javadoc/ pydoc/`.
-##### Create Pydoc
-Add the new Pydoc to [SDK API Reference
page](https://beam.apache.org/releases/pydoc/) page, as follows:
+**PR 2: apache/beam**
-* Copy the generated Pydoc into the website repository: `cp -r ${PYDOC_ROOT}
src/documentation/sdks/pydoc/${RELEASE}`.
-* Remove `.doctrees` directory.
-* Update the Pydoc link on this page to point to the new version (in
`src/documentation/sdks/pydoc/current.md`).
+This pull request is against the `apache/beam` repo, on the `master` branch.
-Finally, propose a pull request with these changes. (Don’t merge before
finalizing the release.)
+* Update the `release_latest` version flag in `/website/_config.yml`, and list
+ the new release in `/website/src/get-started/downloads.md`, linking to the
+ source code download and the Release Notes in JIRA.
+* Update the `RedirectMatch` rule in
+
[/website/src/.htaccess](https://github.com/apache/beam/blob/master/website/src/.htaccess)
+ to point to the new release. See file history for examples.
#### Build and stage python wheels
diff --git a/website/src/contribute/website-contributions.md
b/website/src/contribute/website-contributions.md
deleted file mode 100644
index 7ec24586331..00000000000
--- a/website/src/contribute/website-contributions.md
+++ /dev/null
@@ -1,84 +0,0 @@
----
-layout: section
-title: "Beam Website Contributions"
-permalink: /contribute/website-contributions/
-section_menu: section-menu/contribute.html
----
-<!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-
-# Website Contributions
-
-Contributions to the website require the same steps as contributions to the
-main repository, so start with the instructions for [Contributing
-Code]({{ site.baseurl }}/contribute/contribution-guide).
-
-The Beam website is in the [Beam Site GitHub
-mirror](https://github.com/apache/beam-site) repository in the `asf-site`
-branch (_not_ `master`).
-
-Issues are tracked in the
-[website](https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20component%20%3D%20website)
-component in JIRA.
-
-## One-time Setup
-
-The [README file](https://github.com/apache/beam-site/blob/asf-site/README.md)
-in the website repository has more information on how to set up the required
-dependencies for your development environment.
-
-The general guidelines for cloning a repository can be adjusted to use the
-`asf-site` branch of `beam-site`:
-
- $ git clone -b asf-site https://github.com/apache/beam-site.git
- $ cd beam-site
- $ git remote add <GitHub_user>
g...@github.com:<GitHub_user>/beam-site.git
- $ git fetch --all
- $ git checkout -b <my-branch> origin/asf-site
-
-## Working on your change
-
-While you are working on your pull request, you can test and develop live by
-running the following command in the root folder of the website:
-
- $ bundle exec jekyll serve --incremental
-
-Jekyll will start a webserver on port 4000. As you make changes to the content,
-Jekyll will rebuild it automatically.
-
-In addition, you can run the tests to valid your links using:
-
- $ bundle exec rake test
-
-Both of these commands will cause the `content/` directory to be generated.
-Merging autogenerated content can get tricky, so please leave this directory
-out of your commits and pull request by doing:
-
- $ git checkout -- content
-
-When you are ready, submit a pull request using the [Beam Site GitHub
-mirror](https://github.com/apache/beam-site), including the JIRA issue as
-usual.
-
-During review, committers will patch in your PR, generate the static
-`content/`, and review the changes.
-
-## Committing website changes (committers only)
-
-We have a bot that runs tests and merges changes to the website.
-When the pull request is fully ready to be merged, a committer
-can comment
-
- @asfgit merge
-
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 151866)
Time Spent: 5h 50m (was: 5h 40m)
> Update contribution guide for new website contribution process
> --------------------------------------------------------------
>
> Key: BEAM-4501
> URL: https://issues.apache.org/jira/browse/BEAM-4501
> Project: Beam
> Issue Type: Sub-task
> Components: website
> Reporter: Scott Wegner
> Assignee: Udi Meiri
> Priority: Major
> Labels: beam-site-automation-reliability
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
