This is an automated email from the ASF dual-hosted git repository.
kou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow-site.git
The following commit(s) were added to refs/heads/master by this push:
new 1e94b29 ARROW-4473: [Website] Support test site deployment (take 2)
(#9)
1e94b29 is described below
commit 1e94b294e78214446708bf6975923a46fab4e139
Author: Neal Richardson <neal.p.richard...@gmail.com>
AuthorDate: Wed Aug 14 19:43:34 2019 -0700
ARROW-4473: [Website] Support test site deployment (take 2) (#9)
* Relative URL fixes
* Add build script
* One more relative URL
* Update README to reflect new location and instructions for gh-pages setup
* README formatting
* Wrap commit actions in status --porcelain check
* Feedback from @kou on the build script
* chmod img/arrow.png
---
.gitignore | 1 +
.travis.yml | 4 ++
README.md | 74 ++++++++++++++-------
_config.yml | 1 +
_includes/blog_header.html | 2 +-
...fast-python-serialization-with-ray-and-arrow.md | 12 ++--
_posts/2018-03-22-go-code-donation.md | 2 +-
_posts/2019-01-25-r-spark-improvements.md | 2 +-
build-and-deploy.sh | 50 ++++++++++++++
img/arrow.png | Bin
index.html | 4 +-
11 files changed, 118 insertions(+), 34 deletions(-)
diff --git a/.gitignore b/.gitignore
index 3a88173..fbfc4b7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,3 +7,4 @@ asf-site
build/
.bundle/
ruby/
+.DS_Store
diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 0000000..998c0bd
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,4 @@
+language: ruby
+cache: bundler
+
+script: ./build-and-deploy.sh
diff --git a/README.md b/README.md
index 8dddbf0..7980799 100644
--- a/README.md
+++ b/README.md
@@ -21,14 +21,19 @@
## Overview
-Site content is maintained in the main Arrow repository, mostly in markdown
-format. [Jekyll](https://jekyllrb.com/) is used to generate HTML files that can
-then be committed to the [arrow-site](https://github.com/apache/arrow-site)
-repository.
+[Jekyll](https://jekyllrb.com/) is used to generate HTML files from the
+markdown + templates in this repository. The built version of the site is kept
+on the `asf-site` branch, which gets deployed to https://arrow.apache.org.
## Adding Content
-To add a blog post, create a new markdown file in the `_posts` directory,
following the model of existing posts. In the front matter, you should specify
an "author". This should be your Apache ID if you have one, or it can just be
your name. To add additional metadata about yourself (GitHub ID, website), add
yourself to `_data/contributors.yml`. This object is keyed by `apacheId`, so
use that as the `author` in your post. (It doesn't matter if the ID actually
exists in the ASF; all metad [...]
+To add a blog post, create a new markdown file in the `_posts` directory,
+following the model of existing posts. In the front matter, you should specify
+an "author". This should be your Apache ID if you have one, or it can just be
+your name. To add additional metadata about yourself (GitHub ID, website), add
+yourself to `_data/contributors.yml`. This object is keyed by `apacheId`, so
+use that as the `author` in your post. (It doesn't matter if the ID actually
+exists in the ASF; all metadata is local to this project.)
## Prerequisites
@@ -47,6 +52,7 @@ such cases the following configuration option may help:
bundle config build.nokogiri --use-system-libraries
```
+
`nokogiri` depends on the `libxml2` and `libxslt1` libraries, which can be
installed on Debian-like systems with
@@ -54,27 +60,48 @@ installed on Debian-like systems with
apt-get install libxml2-dev libxslt1-dev
```
-If you are planning to publish the website, you must clone the arrow-site git
-repository. Run this command from the `site` directory so that `asf-site` is a
-subdirectory of `site`.
-
-```shell
-git clone --branch=asf-site https://github.com/apache/arrow-site.git asf-site
-```
-
## Previewing the site
-From the `site` directory, run the following to generate HTML files and run the
-web site locally.
+Run the following to generate HTML files and run the web site locally.
```
bundle exec jekyll serve
```
+If you're working on a fork of `apache/arrow-site`, you can get a development
+version of the site built off of your `master` branch published using GitHub
+Pages and Travis-CI. There are a couple of quick steps to enable this:
+
+1. Create a `gh-pages` branch on your fork, based on the `asf-site` branch
+(`git checkout asf-site && git checkout -b gh-pages && git push origin
gh-pages`)
+2. In the settings for your fork (https://github.com/$YOU/arrow-site/settings),
+turn on GitHub Pages and set it to the gh-pages branch
+3. Go to https://travis-ci.org/account/repositories and enable Travis builds on
+your fork
+4. Go to https://github.com/settings/tokens and create a GitHub personal access
+token with `public_repo` scope
+5. In the settings in Travis for your fork
+(https://travis-ci.org/$YOU/arrow-site/settings), add an environment variable
+called GITHUB_PAT, using the token you just created. To keep the token value
+secret, **do not toggle on "Display value in build log"** (i.e. the default is
+secret).
+
+After doing this, commits to the master branch of your fork will be
+automatically built and published to https://$YOU.github.io/arrow-site/. This
+can help Arrow committers preview your changes more easily before accepting
+patches.
+
## Publishing
-After following the above instructions, run the following commands from the
-`site` directory:
+Publishing the site happens automatically on commits to the `master` branch on
+`apache/arrow-site`. Alternatively, to build and publish locally, clone the
+`arrow-site` repository again as a subdirectory:
+
+```shell
+git clone --branch=asf-site https://github.com/apache/arrow-site.git asf-site
+```
+
+To build and copy the contents of the built site to that second clone,
```shell
JEKYLL_ENV=production bundle exec jekyll build
@@ -91,12 +118,13 @@ git push
### Updating Code Documentation
-To update the documentation, run the script `./dev/gen_apidocs.sh`. This script
-will run the code documentation tools in a fixed environment.
+To update the documentation, you can run the script `./dev/gen_apidocs.sh` in
+the `apache/arrow` repository. This script will run the code documentation
+tools in a fixed environment.
#### C (GLib)
-First, build Apache Arrow C++ and Apache Arrow GLib.
+First, build Apache Arrow C++ and Apache Arrow GLib. This assumes that you
have checkouts your forks of `arrow` and `arrow-site` alongside each other in
your file system.
```
mkdir -p ../cpp/build
@@ -110,7 +138,7 @@ cd ../../c_glib
--with-arrow-cpp-build-type=debug \
--enable-gtk-doc
LD_LIBRARY_PATH=$PWD/../cpp/build/debug make GTK_DOC_V_XREF=": "
-rsync -r doc/reference/html/ ../site/asf-site/docs/c_glib/
+rsync -r doc/reference/html/ ../../arrow-site/asf-site/docs/c_glib/
```
#### Javascript
@@ -118,7 +146,7 @@ rsync -r doc/reference/html/ ../site/asf-site/docs/c_glib/
```
cd ../js
npm run doc
-rsync -r doc/ ../site/asf-site/docs/js
+rsync -r doc/ ../../arrow-site/asf-site/docs/js
```
-Then add/commit/push from the site/asf-site git checkout.
+Then add/commit/push from the `asf-site/` git checkout.
diff --git a/_config.yml b/_config.yml
index ae420e4..57c75c2 100644
--- a/_config.yml
+++ b/_config.yml
@@ -35,6 +35,7 @@ exclude:
- scripts
- README.md
- vendor
+ - build-and-deploy.sh
collections:
docs:
diff --git a/_includes/blog_header.html b/_includes/blog_header.html
index a5a1bac..d12fd5f 100644
--- a/_includes/blog_header.html
+++ b/_includes/blog_header.html
@@ -16,7 +16,7 @@ limitations under the License.
{% endcomment %}
<h1>
{{ post.title }}
- <a href="{{ post.url }}" class="permalink" title="Permalink">∞</a>
+ <a href="{{ site.baseurl }}{{ post.url }}" class="permalink"
title="Permalink">∞</a>
</h1>
{% capture discard %}
diff --git a/_posts/2017-10-15-fast-python-serialization-with-ray-and-arrow.md
b/_posts/2017-10-15-fast-python-serialization-with-ray-and-arrow.md
index f5f3cca..e320bff 100644
--- a/_posts/2017-10-15-fast-python-serialization-with-ray-and-arrow.md
+++ b/_posts/2017-10-15-fast-python-serialization-with-ray-and-arrow.md
@@ -106,8 +106,8 @@ mistake. This is a consequence of the support for zero-copy
reads (the savings
largely come from the lack of memory movement).
<div align="center">
-<img src="{{ site.base-url
}}/assets/fast_python_serialization_with_ray_and_arrow/speedups0.png"
width="365" height="255">
-<img src="{{ site.base-url
}}/assets/fast_python_serialization_with_ray_and_arrow/speedups1.png"
width="365" height="255">
+<img src="{{ site.baseurl
}}/assets/fast_python_serialization_with_ray_and_arrow/speedups0.png"
width="365" height="255">
+<img src="{{ site.baseurl
}}/assets/fast_python_serialization_with_ray_and_arrow/speedups1.png"
width="365" height="255">
</div>
Note that the biggest wins are with deserialization. The speedups here are
@@ -123,8 +123,8 @@ deserialization the bottleneck.
cannot take advantage of shared memory, the results are comparable to pickle.
<div align="center">
-<img src="{{ site.base-url
}}/assets/fast_python_serialization_with_ray_and_arrow/speedups2.png"
width="365" height="255">
-<img src="{{ site.base-url
}}/assets/fast_python_serialization_with_ray_and_arrow/speedups3.png"
width="365" height="255">
+<img src="{{ site.baseurl
}}/assets/fast_python_serialization_with_ray_and_arrow/speedups2.png"
width="365" height="255">
+<img src="{{ site.baseurl
}}/assets/fast_python_serialization_with_ray_and_arrow/speedups3.png"
width="365" height="255">
</div>
These are just a few examples of interesting Python objects. The most important
@@ -194,7 +194,7 @@ and can be efficiently shared and accessed using shared
memory.
Note that the actual object would be laid out in memory as shown below.
<div align="center">
-<img src="{{ site.base-url
}}/assets/fast_python_serialization_with_ray_and_arrow/python_object.png"
width="600">
+<img src="{{ site.baseurl
}}/assets/fast_python_serialization_with_ray_and_arrow/python_object.png"
width="600">
</div>
<div><i>The layout of a Python object in the heap. Each box is allocated in a
different memory region, and arrows between boxes represent pointers.</i></div>
@@ -203,7 +203,7 @@ different memory region, and arrows between boxes represent
pointers.</i></div>
The Arrow serialized representation would be as follows.
<div align="center">
-<img src="{{ site.base-url
}}/assets/fast_python_serialization_with_ray_and_arrow/arrow_object.png"
width="400">
+<img src="{{ site.baseurl
}}/assets/fast_python_serialization_with_ray_and_arrow/arrow_object.png"
width="400">
</div>
<div><i>The memory layout of the Arrow-serialized object.</i></div>
<br />
diff --git a/_posts/2018-03-22-go-code-donation.md
b/_posts/2018-03-22-go-code-donation.md
index 41ddace..baa1ff9 100644
--- a/_posts/2018-03-22-go-code-donation.md
+++ b/_posts/2018-03-22-go-code-donation.md
@@ -52,7 +52,7 @@ instruction sets. We are looking forward to working more
closely with the Go
community on solving in-memory analytics and data interoperability problems.
<div align="center">
-<img src="{{ site.base-url }}/img/native_go_implementation.png"
+<img src="{{ site.baseurl }}/img/native_go_implementation.png"
alt="Apache Arrow implementations and bindings"
width="60%" class="img-responsive">
</div>
diff --git a/_posts/2019-01-25-r-spark-improvements.md
b/_posts/2019-01-25-r-spark-improvements.md
index a06d7aa..4b8fd2d 100644
--- a/_posts/2019-01-25-r-spark-improvements.md
+++ b/_posts/2019-01-25-r-spark-improvements.md
@@ -176,7 +176,7 @@ Unit: seconds
```
<div align="center">
-<img src="{{ site.base-url }}/img/arrow-r-spark-transforming.png"
+<img src="{{ site.baseurl }}/img/arrow-r-spark-transforming.png"
alt="Transforming data with R in Spark with and without Arrow"
width="60%" class="img-responsive">
</div>
diff --git a/build-and-deploy.sh b/build-and-deploy.sh
new file mode 100755
index 0000000..a03931b
--- /dev/null
+++ b/build-and-deploy.sh
@@ -0,0 +1,50 @@
+#!/bin/bash
+set -ev
+
+if [ "${TRAVIS_BRANCH}" = "master" ] && [ "${TRAVIS_PULL_REQUEST}" = "false"
]; then
+
+ if [ -z "${GITHUB_PAT}" ]; then
+ # Don't build because we can't publish
+ echo "To publish the site, you must set a GITHUB_PAT at"
+ echo "https://travis-ci.org/${TRAVIS_REPO_SLUG}/settings";
+ exit 1
+ fi
+
+ # Set git config so that the author of the deployed site commit is the same
+ # as the author of the commit we're building
+ AUTHOR_EMAIL=$(git log -1 --pretty=format:%ae)
+ AUTHOR_NAME=$(git log -1 --pretty=format:%an)
+ git config --global user.email "${AUTHOR_EMAIL}"
+ git config --global user.name "${AUTHOR_NAME}"
+
+ if [ "${TRAVIS_REPO_SLUG}" = "apache/arrow-site" ]; then
+ # Production
+ TARGET_BRANCH=asf-site
+ BASE_URL=
+ else
+ # On a fork, so we'll deploy to GitHub Pages
+ TARGET_BRANCH=gh-pages
+ # You could supply an alternate BASE_URL, but that's not necessary
+ # because we can infer it based on GitHub Pages conventions
+ if [ -z "${BASE_URL}" ]; then
+ BASE_URL=$(echo $TRAVIS_REPO_SLUG | sed -e 's@.*/@/@')
+ fi
+ fi
+
+ # Build
+ JEKYLL_ENV=production bundle exec jekyll build --baseurl="${BASE_URL}"
+
+ # Publish
+ git clone -b ${TARGET_BRANCH}
https://${GITHUB_PAT}@github.com/$TRAVIS_REPO_SLUG.git OUTPUT
+ rsync -a --delete --exclude '/.git/' --exclude '/docs/' build/ OUTPUT/
+ cd OUTPUT
+
+ if [ "$(git status --porcelain)" != "" ]; then
+ # There are changes to the built site
+ git add .
+ git commit -m "Updating built site (build ${TRAVIS_BUILD_NUMBER})"
+ git push origin ${TARGET_BRANCH}
+ else
+ echo "No changes to the built site"
+ fi
+fi
diff --git a/img/arrow.png b/img/arrow.png
old mode 100755
new mode 100644
diff --git a/index.html b/index.html
index 4aab88e..3c863a0 100644
--- a/index.html
+++ b/index.html
@@ -54,7 +54,7 @@ layout: default
<h2>Advantages of a Common Data Layer</h2>
<div class="row">
<div class="col-md-6">
- <img src="img/copy.png" alt="common data layer" class="img-fluid mx-auto"
/>
+ <img src="{{ site.baseurl }}/img/copy.png" alt="common data layer"
class="img-fluid mx-auto" />
<ul>
<li>Each system has its own internal memory format</li>
<li>70-80% computation wasted on serialization and deserialization</li>
@@ -62,7 +62,7 @@ layout: default
</ul>
</div>
<div class="col-md-6">
- <img src="img/shared.png" alt="common data layer" class="img-fluid
mx-auto" />
+ <img src="{{ site.baseurl }}/img/shared.png" alt="common data layer"
class="img-fluid mx-auto" />
<ul>
<li>All systems utilize the same memory format</li>
<li>No overhead for cross-system communication</li>
