[
https://issues.apache.org/jira/browse/CASSANDRA-16066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17326637#comment-17326637
]
Anthony Grasso edited comment on CASSANDRA-16066 at 6/26/21, 3:09 AM:
----------------------------------------------------------------------
A near complete implementation of the new website tooling can be found on my
cassandra-website [fork|https://github.com/ossarga/cassandra-website] on branch
[anthony/CASSANDRA-16066|https://github.com/ossarga/cassandra-website/tree/anthony/CASSANDRA-16066].
Items left to do:
* Complete repository README updates
* Update CI commands and triggers
If you look through the changes on the branch above you will notice the website
repository code is separated into two main parts. These parts are represented
by the directories at the root level of the project. Specifically the structure
of the repository is:
{code:java}
ROOT
- site-content
- site-ui
{code}
h2. Site UI
The 'site-ui' directory contains only the UI styling files that determines the
look and feel of the site. A ui-bundle.zip file containing the styling
information will be generated from the contents of this directory. Generation
of the ui-bundle.zip will be done using {{gulp}} launched inside a Docker
container.
h2. Site Content
The 'site-content' directory contains all the raw page information e.g. where
to download, developer guidelines, how to commit patches, etc. The live website
HTML is generated from the contents of this directory. Generation of the HTML
content is done by {{antora}} launched inside a Docker container. As part of
the website HTML generation, the _ui-bundle.zip_ file, and the Cassandra
documentation location are passed to {{antora}}. It uses the ui-bundle.zip to
style the website. The Cassandra documentation location will be used to gather
and generate documentation for each Cassandra version.
h1. Developer Examples
The development cycle is very similar to the existing website development
cycle. As such, to test changes before committing, it is a requirement that you
build the website locally. Building the Apache Cassandra website takes a number
of steps. To make things easier we have provided a suite of tools to build the
full website in a few simple commands and have it ready to commit via git.
h2. Tooling components
The tooling is made up of the following components
* Run script: {{./run.sh}} - Provides a single command line interface that
generates the docker commands to run the website and UI docker containers.
Using the containers, it can build the Cassandra website UI components,
generate the Cassandra versioned documentation, and generate the website HTML.
* Website Docker container: {{Dockerfile}} and {{docker-entrypoint.sh}} -
Contains the libraries necessary to generate the Cassandra versioned
documentation, and generate the website HTML using Antora.
* Antora Site YAML script: {{bin/site_yaml_generator.py}} - Used by the
Website Docker container to create the YAML file that defines the sources for
the website content, optionally the cassandra versioned documentation, and the
UI styling bundle to apply.
* Website UI Docker container: {{site-ui/Dockerfile}} and
{{site-ui/docker-entrypoint.sh}} - Contains the libraries necessary to generate
the UI bundle ZIP file the is applied by Antora to style the website and
documentation.
h2. Building Prerequisites
To build and run the Docker container you will need {{Docker}} version 2.0.0.0
or greater. If you need a copy of the site code you will need {{git}} as well.
h2. Building the Website
If you need a copy of the site code run this command:
{code:java}
$ git clone https://github.com/apache/cassandra-website.git
$ cd ./cassandra-website
{code}
To build the website only, run the following command from within the
{{./cassandra-website}} directory (assuming you used the above clone command).
{code:java}
$ ./run.sh website build
{code}
*Tip:* In order to prevent root-owned modified files in your repository, the
container user, {{build}}, is set up with a default {{UID=1000:GID=1000}},
which is usually the first user configured on a linux machine. If your local
user is different you should set up the container user with your local host
user's UID:GID, replace the above with:
{code:java}
$ ./run.sh -v UID_ARG=$(id -u) -v GID_ARG=$(id -g) website container
$ ./run.sh website build
{code}
This will build the website content using your local copy of the
cassandra-website, and the current checked-out branch. Use this command if you
want to make a change to a top-level webpage without building the docs for any
versions of cassandra.
Once building has completed, the HTML content will be in the
{{./site-content/build/html/}} directory ready to be reviewed and committed.
h2. Build the Website when Developing
The website tooling is very flexible and allows for a wide range of development
scenarios.
h3. Build the website from a different branch
You can tell the website builder to use a different branch to the one you are
on. This can be done using the following command.
{code:java}
$ ./run.sh -b cassandra-website:my_branch website build
{code}
This will build the website content using your local copy of the
cassandra-website, and the branch named {{my_branch}}.
h3. Build the website using a local clone of the repository
You can tell the website builder to use a different clone or fork of the
repository.
To build using another local copy of the cassandra-website run the following
command.
{code:java}
$ ./run.sh -u cassandra-website:/local/path/to/cassandra-website website build
{code}
This will build the website using the contents of the local repository located
in */local/path/to/cassandra-website*
h3. Build the website using a remote clone of the repository
To build using a remote copy of the cassandra-website run the following command.
{code:java}
$ ./run.sh -u
cassandra-website:https://github.com/my_orgranisation/cassandra-website.git
website build
{code}
This will build the website using the contents of the remote repository located
at *[https://github.com/my_orgranisation/cassandra-website.git*]
*Tip:* The {{trunk}} branch is used by default in both cases above. You can
specify a different branch using the {{-b}} option as per the example in the
*Build a different branch* section.
h2. Previewing the Website
An offline preview mode exists if you want to view the website as you make
changes to the content. Preview mode can be launched using the following
command.
{code:java}
$ ./run.sh website preview
{code}
The site can be viewed on [http://localhost:5151|http://localhost:5151/].
The {{preview}} command uses the same code path as the {{build}} command. It
will build the website content using your local copy of the cassandra-website,
and the current checked-out branch. Additionally, it will then start a
webserver to serve the HTML, and start a process that monitors the content
files in your local copy of the repository. If a change is made to a content
file, the website HTML will automatically be regenerated.
Press {{Ctrl+C}} to stop the preview server and end the continuous build.
*Tip:* You may need to refresh your browser when the auto rendering of the site
is complete.
All options that are available in the {{build}} command can be used by the
{{preview}} command. Hence, the options used in the previous examples are
applicable to the {{preview}} command as well.
h1. Advanced Developer Examples
The cassandra-website tooling can also be used to perform a number of other
operations:
* Generate the Apache Cassandra documentation.
* Update the website styling and behaviour.
h2. Generating the Cassandra Documentation
The Docker container that generates the version documentation has a local copy
of the Apache Cassandra repository. The document generation process commits the
generated AsciiDoc to the local repository. This is done so {{antora}} can
generate website HTML as well as the HTML for the various versions of the
documentation using the committed AsciiDoc in each branch.
However, you can use your own local copy or fork of the Cassandra repository as
the source for the versioned documentation. This can be done by supplying a
local path to the {{run.sh}} script. In this case if you are rendering multiple
versions of the documentation, the tooling in the Docker container will commit
the generated AsciiDoc files for each of the branches you specified to your
repository.
To avoid the tooling committing to your local copy of the Cassandra repository,
you can specify a remote URL to the repository. In this case the tooling in the
Docker container will clone the repository inside the container. The generated
AsciiDoc files will be committed to the repository inside the container.
h3. Generate docs using a local clone of the repository
To generate the latest version of the Cassandra docs using a local copy or fork
of the Cassandra repository, run the following command.
{code:java}
$ ./run.sh -u cassandra:/local/path/to/cassandra/repository -b cassandra:trunk
website docs
{code}
This will build the documentation using the contents of the {{trunk}} branch in
the local repository located in */local/path/to/cassandra-website*
The output of this command will be AsciiDoc ({{.adoc}}) files that {{antora}}
can render into HTML. Note that {{antora}} is never executed when only the
Cassandra versioned documentation is generated.
h3. Generate docs using a remote clone of the repository
To generate the latest version of the Cassandra docs using a remote copy of the
Cassandra repository, run the following command.
{code:java}
$ ./run.sh -u cassandra:https://github.com/my_orgranisation/cassandra.git -b
cassandra:trunk website docs
{code}
This will build the documentation using the contents of the {{trunk}} branch in
the remote repository fork located at
*[https://github.com/my_orgranisation/cassandra.git*].
h3. Generate docs for multiple Cassandra versions
To generate multiple versions of the Cassandra documentation using a local copy
or fork of the Cassandra repository, run the following command.
{code:java}
$ ./run.sh -u cassandra:/local/path/to/cassandra/repository -b
cassandra:trunk,cassandra-3.11,my_branch website docs
{code}
In the above command, multiple branches separated by a comma ({{,}}) can be
specified in the {{-b}} option.
was (Author: anthony grasso):
A near complete implementation of the new website tooling can be found on my
cassandra-website [fork|https://github.com/ossarga/cassandra-website] on branch
[anthony/CASSANDRA-16066_rebased|https://github.com/ossarga/cassandra-website/commits/anthony/CASSANDRA-16066_rebased].
Items left to do:
* Complete repository README updates
* Update CI commands and triggers
If you look through the changes on the branch above you will notice the website
repository code is separated into two main parts. These parts are represented
by the directories at the root level of the project. Specifically the structure
of the repository is:
{code}
ROOT
- site-content
- site-ui
{code}
h2. Site UI
The 'site-ui' directory contains only the UI styling files that determines the
look and feel of the site. A ui-bundle.zip file containing the styling
information will be generated from the contents of this directory. Generation
of the ui-bundle.zip will be done using {{gulp}} launched inside a Docker
container.
h2. Site Content
The 'site-content' directory contains all the raw page information e.g. where
to download, developer guidelines, how to commit patches, etc. The live website
HTML is generated from the contents of this directory. Generation of the HTML
content is done by {{antora}} launched inside a Docker container. As part of
the website HTML generation, the _ui-bundle.zip_ file, and the Cassandra
documentation location are passed to {{antora}}. It uses the ui-bundle.zip to
style the website. The Cassandra documentation location will be used to gather
and generate documentation for each Cassandra version.
h1. Developer Examples
The development cycle is very similar to the existing website development
cycle. As such, to test changes before committing, it is a requirement that you
build the website locally. Building the Apache Cassandra website takes a number
of steps. To make things easier we have provided a suite of tools to build the
full website in a few simple commands and have it ready to commit via git.
h2. Tooling components
The tooling is made up of the following components
* Run script: {{./run.sh}} - Provides a single command line interface that
generates the docker commands to run the website and UI docker containers.
Using the containers, it can build the Cassandra website UI components,
generate the Cassandra versioned documentation, and generate the website HTML.
* Website Docker container: {{Dockerfile}} and {{docker-entrypoint.sh}} -
Contains the libraries necessary to generate the Cassandra versioned
documentation, and generate the website HTML using Antora.
* Antora Site YAML script: {{bin/site_yaml_generator.py}} - Used by the Website
Docker container to create the YAML file that defines the sources for the
website content, optionally the cassandra versioned documentation, and the UI
styling bundle to apply.
* Website UI Docker container: {{site-ui/Dockerfile}} and
{{site-ui/docker-entrypoint.sh}} - Contains the libraries necessary to generate
the UI bundle ZIP file the is applied by Antora to style the website and
documentation.
h2. Building Prerequisites
To build and run the Docker container you will need {{Docker}} version 2.0.0.0
or greater. If you need a copy of the site code you will need {{git}} as well.
h2. Building the Website
If you need a copy of the site code run this command:
{code}
$ git clone https://github.com/apache/cassandra-website.git
$ cd ./cassandra-website
{code}
To build the website only, run the following command from within the
{{./cassandra-website}} directory (assuming you used the above clone command).
{code}
$ ./run.sh website build
{code}
*Tip:* In order to prevent root-owned modified files in your repository, the
container user, {{build}}, is set up with a default {{UID=1000:GID=1000}},
which is usually the first user configured on a linux machine. If your local
user is different you should set up the container user with your local host
user's UID:GID, replace the above with:
{code}
$ ./run.sh -v UID_ARG=$(id -u) -v GID_ARG=$(id -g) website container
$ ./run.sh website build
{code}
This will build the website content using your local copy of the
cassandra-website, and the current checked-out branch. Use this command if you
want to make a change to a top-level webpage without building the docs for any
versions of cassandra.
Once building has completed, the HTML content will be in the
{{./site-content/build/html/}} directory ready to be reviewed and committed.
h2. Build the Website when Developing
The website tooling is very flexible and allows for a wide range of development
scenarios.
h3. Build the website from a different branch
You can tell the website builder to use a different branch to the one you are
on. This can be done using the following command.
{code}
$ ./run.sh -b cassandra-website:my_branch website build
{code}
This will build the website content using your local copy of the
cassandra-website, and the branch named {{my_branch}}.
h3. Build the website using a local clone of the repository
You can tell the website builder to use a different clone or fork of the
repository.
To build using another local copy of the cassandra-website run the following
command.
{code}
$ ./run.sh -u cassandra-website:/local/path/to/cassandra-website website build
{code}
This will build the website using the contents of the local repository located
in */local/path/to/cassandra-website*
h3. Build the website using a remote clone of the repository
To build using a remote copy of the cassandra-website run the following command.
{code}
$ ./run.sh -u
cassandra-website:https://github.com/my_orgranisation/cassandra-website.git
website build
{code}
This will build the website using the contents of the remote repository located
at *https://github.com/my_orgranisation/cassandra-website.git*
*Tip:* The {{trunk}} branch is used by default in both cases above. You can
specify a different branch using the {{-b}} option as per the example in the
*Build a different branch* section.
h2. Previewing the Website
An offline preview mode exists if you want to view the website as you make
changes to the content. Preview mode can be launched using the following
command.
{code}
$ ./run.sh website preview
{code}
The site can be viewed on [http://localhost:5151|http://localhost:5151].
The {{preview}} command uses the same code path as the {{build}} command. It
will build the website content using your local copy of the cassandra-website,
and the current checked-out branch. Additionally, it will then start a
webserver to serve the HTML, and start a process that monitors the content
files in your local copy of the repository. If a change is made to a content
file, the website HTML will automatically be regenerated.
Press {{Ctrl+C}} to stop the preview server and end the continuous build.
*Tip:* You may need to refresh your browser when the auto rendering of the site
is complete.
All options that are available in the {{build}} command can be used by the
{{preview}} command. Hence, the options used in the previous examples are
applicable to the {{preview}} command as well.
h1. Advanced Developer Examples
The cassandra-website tooling can also be used to perform a number of other
operations:
* Generate the Apache Cassandra documentation.
* Update the website styling and behaviour.
h2. Generating the Cassandra Documentation
The Docker container that generates the version documentation has a local copy
of the Apache Cassandra repository. The document generation process commits the
generated AsciiDoc to the local repository. This is done so {{antora}} can
generate website HTML as well as the HTML for the various versions of the
documentation using the committed AsciiDoc in each branch.
However, you can use your own local copy or fork of the Cassandra repository as
the source for the versioned documentation. This can be done by supplying a
local path to the {{run.sh}} script. In this case if you are rendering multiple
versions of the documentation, the tooling in the Docker container will commit
the generated AsciiDoc files for each of the branches you specified to your
repository.
To avoid the tooling committing to your local copy of the Cassandra repository,
you can specify a remote URL to the repository. In this case the tooling in the
Docker container will clone the repository inside the container. The generated
AsciiDoc files will be committed to the repository inside the container.
h3. Generate docs using a local clone of the repository
To generate the latest version of the Cassandra docs using a local copy or fork
of the Cassandra repository, run the following command.
{code}
$ ./run.sh -u cassandra:/local/path/to/cassandra/repository -b cassandra:trunk
website docs
{code}
This will build the documentation using the contents of the {{trunk}} branch in
the local repository located in */local/path/to/cassandra-website*
The output of this command will be AsciiDoc ({{.adoc}}) files that {{antora}}
can render into HTML. Note that {{antora}} is never executed when only the
Cassandra versioned documentation is generated.
h3. Generate docs using a remote clone of the repository
To generate the latest version of the Cassandra docs using a remote copy of the
Cassandra repository, run the following command.
{code}
$ ./run.sh -u cassandra:https://github.com/my_orgranisation/cassandra.git -b
cassandra:trunk website docs
{code}
This will build the documentation using the contents of the {{trunk}} branch in
the remote repository fork located at
*https://github.com/my_orgranisation/cassandra.git*.
h3. Generate docs for multiple Cassandra versions
To generate multiple versions of the Cassandra documentation using a local copy
or fork of the Cassandra repository, run the following command.
{code}
$ ./run.sh -u cassandra:/local/path/to/cassandra/repository -b
cassandra:trunk,cassandra-3.11,my_branch website docs
{code}
In the above command, multiple branches separated by a comma ({{,}}) can be
specified in the {{-b}} option.
> Create tooling and update repository layout to render new website
> -----------------------------------------------------------------
>
> Key: CASSANDRA-16066
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16066
> Project: Cassandra
> Issue Type: Task
> Components: Documentation/Website
> Reporter: Anthony Grasso
> Assignee: Anthony Grasso
> Priority: Normal
> Attachments: image-2020-09-05-13-24-13-606.png,
> image-2020-09-06-07-17-14-705.png
>
>
> *We want to modernise the way the website is built*
> Rework the cassandra-website repository to generate a UI bundle containing
> resources that Antora will use when generating the Cassandra documents and
> website.
> *The existing method is starting to become dated*
> The documentation and website templates are currently in markdown format.
> Sphinx is used to generate the Cassandra documentation and Jekyll generates
> the website content. One of the major issues with the existing website
> tooling is that the live preview server (render site as it is being updated)
> is really slow. There is a preview server that is really fast, however it is
> unable to detect changes to the content and render automatically.
> *We are migrating the docs to be rendered with Antora*
> The work in CASSANDRA-16029 is converting the document templates to AsciiDoc
> format. Sphinx is being replaced by Antora to generate the documentation
> content. This change has two advantages:
> * More flexibility if the Apache Cassandra documentation look and feel needs
> to be updated or redesigned.
> * More modern look and feel to the documentation.
> *We can use Antora to generate the website as well*
> Antora could also be used to generate the Cassandra website content. As
> suggested on the [mailing
> list|https://www.mail-archive.com/[email protected]/msg15577.html]
> this would require the existing markdown templates to be converted to
> AsciiDoc as well.
> *Antora needs a UI bundle to style content*
> For Antora to generate the document content and potentially the website
> content it requires a UI bundle (ui-bundle.zip). The UI bundle contains the
> HTML templates (layouts, partials, and helpers), CSS, JavaScript, fonts, and
> (site-wide) images. As such, it provides both the visual theme and user
> interactions for the documentation. Effectively the UI bundle is the
> templates and styling that are applied to the documentation and website
> content.
> *The [cassandra-website|https://github.com/apache/cassandra-website]
> repository can be used to generate the UI bundle*
> All the resources associated with templating and styling the documentation
> and website can be placed in the
> [cassandra-website|https://github.com/apache/cassandra-website] repository.
> In this case the repository would serve two purposes;
> * Generation of the UI bundle resources.
> * Serve the production website content.
> *The [cassandra|https://github.com/apache/cassandra] repository would contain
> the documentation material, while the rest of the non-versioned pages would
> contain that material*
> * All other material that is used to generate documentation would live in
> the [cassandra|https://github.com/apache/cassandra] repository. In this case
> Antora would run on the
> [cassandra/doc|https://github.com/apache/cassandra/doc] directory and pull in
> a UI bundle released on the GitHub
> [cassandra-website|https://github.com/apache/cassandra-website] repository.
> The content generated by Antora using the site.yml file located in this repo
> can be used to preview the docs for review.
> * All other non-versioned material, such as the blogs, development
> instructions, and third-party page would live in the GitHub
> [cassandra-website|https://github.com/apache/cassandra-website] repository.
> Antora will generate the full website using the site.yml located in this
> repo, using both as content sources the material located in both the
> [cassandra|https://github.com/apache/cassandra] and
> [cassandra-website|https://github.com/apache/cassandra-website] repositories.
> *Design, content, and layout would remain the same*
> This proposal means the roles of each repository will change. In addition,
> the workflow to publish material will change as well. However, the website
> design, content, and layout will remain the same.
> As an added bonus the tooling would allow for a live preview server that is
> fast and responsive. However, the time to build and generate the {{nodetool}}
> and Cassandra YAML AssciDoc files will still take in the order of minutes to
> complete.
> *The [cassandra-website|https://github.com/apache/cassandra-website]
> repository would need to be modified*
> The following changes would need to be made to the
> [cassandra-website|https://github.com/apache/cassandra-website] repository:
> ||File/Directory||Action||Reason||
> |[content/|https://github.com/apache/cassandra-website/content]|keep|Production
> site content served from this directory|
> |[src/_data/|https://github.com/apache/cassandra-website/src/_data]|delete|_site.yml_
> and _antora.yml_ include this info|
> |[src/_includes/|https://github.com/apache/cassandra-website/src/_includes]|delete|Replace
> with UI bundle components|
> |[src/_layout/|https://github.com/apache/cassandra-website/src/_layout]|delete|Replace
> with UI bundle components|
> |[src/_plugins/|https://github.com/apache/cassandra-website/src/_plugins]|delete|Replace
> with UI bundle components|
> |[src/_posts/|https://github.com/apache/cassandra-website/src/_posts]|move|Convert
> to AsciiDoc format and place in
> [cassandra|https://github.com/apache/cassandra] repository|
> |[src/_sass/|https://github.com/apache/cassandra-website/src/_sass]|delete|Replace
> with UI bundle components|
> |[src/_templates/|https://github.com/apache/cassandra-website/src/_templates]|move|Convert
> to AsciiDoc format (template format for blog posts) and place in
> [cassandra|https://github.com/apache/cassandra] repository|
> |[src/blog/|https://github.com/apache/cassandra-website/src/blog]|delete|Replace
> with _index.adoc_ that will be the initial page for blogs|
> |[src/css/|https://github.com/apache/cassandra-website/src/css]|delete|Replace
> with UI bundle components|
> |[src/doc/|https://github.com/apache/cassandra-website/src/doc]|delete|Content
> already generated with Antora in
> [cassandra|https://github.com/apache/cassandra] repository|
> |[src/icons/|https://github.com/apache/cassandra-website/src/icons]|keep|Probably
> needed for site generation but might be moved to a different folder|
> |[src/img/|https://github.com/apache/cassandra-website/src/img]|keep|Probably
> needed for site generation but might be moved to a different folder|
> |[src/js/|https://github.com/apache/cassandra-website/src/js/]|delete|Replace
> with UI bundle components|
> |[src/Gemfile|https://github.com/apache/cassandra-website/src/Gemfile]|delete|Replace
> with node modules|
> |[src/Gemfile.lock|https://github.com/apache/cassandra-website/src/Gemfile.lock]|delete|Replace
> with node modules|
> |[src/Makefile|https://github.com/apache/cassandra-website/src/Makefile]|delete|Replace
> build mechanism with Gulp and Docker|
> |[src/README|https://github.com/apache/cassandra-website/src/README]|delete|Place
> information and instructions in top level README|
> |[src/_config.yml|https://github.com/apache/cassandra-website/src/_config.yml]|delete|Replaced
> by _site.yml_|
> |[src/apachecon_cfp.md|https://github.com/apache/cassandra-website/src/apachecon_cfp.md]|delete??|This
> file is out of date|
> |[src/community.md|https://github.com/apache/cassandra-website/src/community.md]|move|Convert
> to AsciiDoc and place in [cassandra|https://github.com/apache/cassandra]
> repository|
> |[src/download.md|https://github.com/apache/cassandra-website/src/download.md]|move|Convert
> to AsciiDoc and place in [cassandra|https://github.com/apache/cassandra]
> repository|
> |[src/index.html|https://github.com/apache/cassandra-website/src/index.html]|keep|Might
> need to convert to AsciiDoc and place in
> [cassandra|https://github.com/apache/cassandra] repository|
> |[src/robot.txt|https://github.com/apache/cassandra-website/src/robot.txt]|keep|Needed
> by site|
> |[src/third-party.md|https://github.com/apache/cassandra-website/src/third-party.md]|move|Convert
> to AsciiDoc and place in [cassandra|https://github.com/apache/cassandra]
> repository|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
