errose28 commented on code in PR #6916:
URL: https://github.com/apache/ozone/pull/6916#discussion_r1742281020
##########
.github/workflows/ci.yml:
##########
@@ -166,6 +166,99 @@ jobs:
path: |
~/.m2/repository/org/apache/ozone
retention-days: 1
+ build-config-doc:
+ needs:
+ - build
+ runs-on: ubuntu-20.04
+ steps:
+ - name: Checkout project
+ uses: actions/checkout@v4
+ - name: Set up Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.x'
+ - name: Download the source artifact
+ uses: actions/download-artifact@v4
+ with:
+ name: ozone-bin
+ path: ozone-bin
+ - name: Extract the source tarball
+ run: |
+ mkdir -p ozone-bin/extracted
+ tar -xzf ozone-bin/ozone-*-SNAPSHOT.tar.gz -C ozone-bin/extracted
+ - name: Calculate hash of the existing Configurations.md
Review Comment:
Since the config doc needs to get generated anyways I think we can simplify
all of these check steps down to just two:
1. Generate md file.
2. Using GH actions conditionals and
[hash](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/evaluate-expressions-in-workflows-and-actions#hashfiles),
set a flag/variable the job if the hashes don't match.
- This way we do not need the intermediate hash txt files.
Unfortunately it looks like we will still need the "hashes differ" check in
each of the following steps due to GH actions limitations.
##########
.github/workflows/ci.yml:
##########
@@ -166,6 +166,99 @@ jobs:
path: |
~/.m2/repository/org/apache/ozone
retention-days: 1
+ build-config-doc:
+ needs:
+ - build
+ runs-on: ubuntu-20.04
+ steps:
+ - name: Checkout project
+ uses: actions/checkout@v4
+ - name: Set up Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.x'
+ - name: Download the source artifact
+ uses: actions/download-artifact@v4
+ with:
+ name: ozone-bin
+ path: ozone-bin
+ - name: Extract the source tarball
+ run: |
+ mkdir -p ozone-bin/extracted
+ tar -xzf ozone-bin/ozone-*-SNAPSHOT.tar.gz -C ozone-bin/extracted
Review Comment:
The workflow should succeed for commits that represent released versions as
well. These will not have SNAPSHOT in their name.
##########
.github/workflows/ci.yml:
##########
@@ -166,6 +166,99 @@ jobs:
path: |
~/.m2/repository/org/apache/ozone
retention-days: 1
+ build-config-doc:
+ needs:
+ - build
+ runs-on: ubuntu-20.04
+ steps:
+ - name: Checkout project
+ uses: actions/checkout@v4
+ - name: Set up Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.x'
+ - name: Download the source artifact
+ uses: actions/download-artifact@v4
+ with:
+ name: ozone-bin
+ path: ozone-bin
+ - name: Extract the source tarball
+ run: |
+ mkdir -p ozone-bin/extracted
+ tar -xzf ozone-bin/ozone-*-SNAPSHOT.tar.gz -C ozone-bin/extracted
+ - name: Calculate hash of the existing Configurations.md
+ id: existing-hash
+ run: |
+ if [ -f hadoop-hdds/docs/content/tools/Configurations.md ]; then
+ sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > existing_hash.txt
+ else
+ echo "No existing file found." > existing_hash.txt
+ fi
+ - name: Run the Python script to convert XML properties into Markdown
+ run: python3 dev-support/ci/xml_to_md.py ozone-bin/extracted >
hadoop-hdds/docs/content/tools/Configurations.md
+ - name: Calculate hash of the new Configurations.md
+ id: new-hash
+ run: sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > new_hash.txt
+ - name: Compare hashes and decide to commit
+ id: hash-check
+ run: |
+ EXISTING_HASH=$(cat existing_hash.txt)
+ NEW_HASH=$(cat new_hash.txt)
+ if [ "$EXISTING_HASH" != "$NEW_HASH" ]; then
+ echo "Configurations.md has changed, proceeding to commit and
create PRs."
+ echo "hashes_differ=true" >> $GITHUB_ENV
+ else
+ echo "Configurations.md is unchanged, skipping commit and PR
creation."
+ echo "hashes_differ=false" >> $GITHUB_ENV
+ fi
+ - name: Commit and push changes to apache/ozone
+ if: env.hashes_differ == 'true'
+ run: |
+ git config --global user.name 'Github Actions'
+ git config --global user.email '[email protected]'
+ git add hadoop-hdds/docs/content/tools/Configurations.md
+ git commit -m "[Auto] Update Configurations.md from $GITHUB_SHA"
+ git push origin HEAD:master
+ - name: Create Pull Request in apache/ozone
+ if: env.hashes_differ == 'true'
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ gh pr create --title "[Auto] Update Configurations.md" --body "This
is an automated pull request."
Review Comment:
Github provides metadata about the event that triggered the action. See
[context](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/accessing-contextual-information-about-workflow-runs#github-context).
This will be helpful to have in the PR description. For example:
```
This is an automated Pull Request to update the website documentation of
Ozone's default configuration. It was triggered by the workflow {{
github.workflow }} run on {{ github.ref_name }}. Please delete the
$SOURCE_BRANCH branch from the repo when the PR is merged or closed.
```
##########
.github/workflows/ci.yml:
##########
@@ -166,6 +166,99 @@ jobs:
path: |
~/.m2/repository/org/apache/ozone
retention-days: 1
+ build-config-doc:
+ needs:
+ - build
+ runs-on: ubuntu-20.04
+ steps:
+ - name: Checkout project
+ uses: actions/checkout@v4
+ - name: Set up Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.x'
+ - name: Download the source artifact
+ uses: actions/download-artifact@v4
+ with:
+ name: ozone-bin
+ path: ozone-bin
+ - name: Extract the source tarball
+ run: |
+ mkdir -p ozone-bin/extracted
+ tar -xzf ozone-bin/ozone-*-SNAPSHOT.tar.gz -C ozone-bin/extracted
+ - name: Calculate hash of the existing Configurations.md
+ id: existing-hash
+ run: |
+ if [ -f hadoop-hdds/docs/content/tools/Configurations.md ]; then
+ sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > existing_hash.txt
+ else
+ echo "No existing file found." > existing_hash.txt
+ fi
+ - name: Run the Python script to convert XML properties into Markdown
+ run: python3 dev-support/ci/xml_to_md.py ozone-bin/extracted >
hadoop-hdds/docs/content/tools/Configurations.md
+ - name: Calculate hash of the new Configurations.md
+ id: new-hash
+ run: sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > new_hash.txt
+ - name: Compare hashes and decide to commit
+ id: hash-check
+ run: |
+ EXISTING_HASH=$(cat existing_hash.txt)
+ NEW_HASH=$(cat new_hash.txt)
+ if [ "$EXISTING_HASH" != "$NEW_HASH" ]; then
+ echo "Configurations.md has changed, proceeding to commit and
create PRs."
+ echo "hashes_differ=true" >> $GITHUB_ENV
+ else
+ echo "Configurations.md is unchanged, skipping commit and PR
creation."
+ echo "hashes_differ=false" >> $GITHUB_ENV
+ fi
+ - name: Commit and push changes to apache/ozone
+ if: env.hashes_differ == 'true'
+ run: |
+ git config --global user.name 'Github Actions'
+ git config --global user.email '[email protected]'
+ git add hadoop-hdds/docs/content/tools/Configurations.md
+ git commit -m "[Auto] Update Configurations.md from $GITHUB_SHA"
+ git push origin HEAD:master
+ - name: Create Pull Request in apache/ozone
+ if: env.hashes_differ == 'true'
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ gh pr create --title "[Auto] Update Configurations.md" --body "This
is an automated pull request."
+ - name: Checkout ozone-site repository
+ if: env.hashes_differ == 'true'
+ uses: actions/checkout@v4
+ with:
+ repository: apache/ozone-site
+ ref: 'HDDS-9225-website-v2'
+ path: ozone-site
+ - name: Copy MD file to ozone-site repository
+ if: env.hashes_differ == 'true'
+ run: |
+ TARGET_DIR=$(find ozone-site/docs -type d -name
'*-administrator-guide' -exec find {} -type d -name '*-configuration' -print
-quit \;)
+ echo "Target directory: $TARGET_DIR"
+ if [ -z "$TARGET_DIR" ]; then
+ echo "Target directory not found."
+ exit 1
+ fi
+ cp hadoop-hdds/docs/content/tools/Configurations.md
"$TARGET_DIR/configurations.md"
+ - name: Commit and push changes to apache/ozone-site
+ if: env.hashes_differ == 'true'
+ run: |
+ cd ozone-site
+ git config --global user.name 'Github Actions'
+ git config --global user.email '[email protected]'
+ ADD_FILE=$(find docs -type d -name '*-administrator-guide' -exec
find {} -type d -name '*-configuration' -print -quit \;)
+ git add "$ADD_FILE/configurations.md"
+ git commit -m "[Auto] Update configurations.md page from ozone
$GITHUB_SHA"
+ git push origin HEAD:HDDS-9225-website-v2
+ - name: Create Pull Request in apache/ozone-site
+ if: env.hashes_differ == 'true'
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ cd ozone-site
+ gh pr create --title "[Auto] Update configurations.md page" --body
"This is an automated pull request from the ozone repository."
Review Comment:
Similar comment about PR description from before applies here as well.
##########
.github/workflows/ci.yml:
##########
@@ -166,6 +166,99 @@ jobs:
path: |
~/.m2/repository/org/apache/ozone
retention-days: 1
+ build-config-doc:
+ needs:
+ - build
+ runs-on: ubuntu-20.04
+ steps:
+ - name: Checkout project
+ uses: actions/checkout@v4
+ - name: Set up Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.x'
+ - name: Download the source artifact
+ uses: actions/download-artifact@v4
+ with:
+ name: ozone-bin
+ path: ozone-bin
+ - name: Extract the source tarball
+ run: |
+ mkdir -p ozone-bin/extracted
+ tar -xzf ozone-bin/ozone-*-SNAPSHOT.tar.gz -C ozone-bin/extracted
+ - name: Calculate hash of the existing Configurations.md
+ id: existing-hash
+ run: |
+ if [ -f hadoop-hdds/docs/content/tools/Configurations.md ]; then
+ sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > existing_hash.txt
+ else
+ echo "No existing file found." > existing_hash.txt
+ fi
+ - name: Run the Python script to convert XML properties into Markdown
+ run: python3 dev-support/ci/xml_to_md.py ozone-bin/extracted >
hadoop-hdds/docs/content/tools/Configurations.md
+ - name: Calculate hash of the new Configurations.md
+ id: new-hash
+ run: sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > new_hash.txt
+ - name: Compare hashes and decide to commit
+ id: hash-check
+ run: |
+ EXISTING_HASH=$(cat existing_hash.txt)
+ NEW_HASH=$(cat new_hash.txt)
+ if [ "$EXISTING_HASH" != "$NEW_HASH" ]; then
+ echo "Configurations.md has changed, proceeding to commit and
create PRs."
+ echo "hashes_differ=true" >> $GITHUB_ENV
+ else
+ echo "Configurations.md is unchanged, skipping commit and PR
creation."
+ echo "hashes_differ=false" >> $GITHUB_ENV
+ fi
+ - name: Commit and push changes to apache/ozone
+ if: env.hashes_differ == 'true'
+ run: |
+ git config --global user.name 'Github Actions'
+ git config --global user.email '[email protected]'
+ git add hadoop-hdds/docs/content/tools/Configurations.md
+ git commit -m "[Auto] Update Configurations.md from $GITHUB_SHA"
+ git push origin HEAD:master
+ - name: Create Pull Request in apache/ozone
+ if: env.hashes_differ == 'true'
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ gh pr create --title "[Auto] Update Configurations.md" --body "This
is an automated pull request."
+ - name: Checkout ozone-site repository
+ if: env.hashes_differ == 'true'
+ uses: actions/checkout@v4
+ with:
+ repository: apache/ozone-site
+ ref: 'HDDS-9225-website-v2'
+ path: ozone-site
+ - name: Copy MD file to ozone-site repository
+ if: env.hashes_differ == 'true'
+ run: |
+ TARGET_DIR=$(find ozone-site/docs -type d -name
'*-administrator-guide' -exec find {} -type d -name '*-configuration' -print
-quit \;)
+ echo "Target directory: $TARGET_DIR"
+ if [ -z "$TARGET_DIR" ]; then
+ echo "Target directory not found."
+ exit 1
+ fi
+ cp hadoop-hdds/docs/content/tools/Configurations.md
"$TARGET_DIR/configurations.md"
+ - name: Commit and push changes to apache/ozone-site
+ if: env.hashes_differ == 'true'
+ run: |
+ cd ozone-site
+ git config --global user.name 'Github Actions'
+ git config --global user.email '[email protected]'
+ ADD_FILE=$(find docs -type d -name '*-administrator-guide' -exec
find {} -type d -name '*-configuration' -print -quit \;)
Review Comment:
We should save the absolute path to file as a variable in the previous step
instead o f recomputing it.
##########
.github/workflows/ci.yml:
##########
@@ -166,6 +166,99 @@ jobs:
path: |
~/.m2/repository/org/apache/ozone
retention-days: 1
+ build-config-doc:
Review Comment:
We need conditionals so this only runs on pushes to the master branch in the
main Ozone repo:
```
if: ${{ github.repository == 'apache/ozone' && github.event_name == 'push'
&& github.ref_name == 'master' }}
```
##########
.github/workflows/ci.yml:
##########
@@ -166,6 +166,99 @@ jobs:
path: |
~/.m2/repository/org/apache/ozone
retention-days: 1
+ build-config-doc:
+ needs:
+ - build
+ runs-on: ubuntu-20.04
+ steps:
+ - name: Checkout project
+ uses: actions/checkout@v4
+ - name: Set up Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.x'
+ - name: Download the source artifact
+ uses: actions/download-artifact@v4
+ with:
+ name: ozone-bin
+ path: ozone-bin
+ - name: Extract the source tarball
+ run: |
+ mkdir -p ozone-bin/extracted
+ tar -xzf ozone-bin/ozone-*-SNAPSHOT.tar.gz -C ozone-bin/extracted
+ - name: Calculate hash of the existing Configurations.md
+ id: existing-hash
+ run: |
+ if [ -f hadoop-hdds/docs/content/tools/Configurations.md ]; then
+ sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > existing_hash.txt
+ else
+ echo "No existing file found." > existing_hash.txt
+ fi
+ - name: Run the Python script to convert XML properties into Markdown
+ run: python3 dev-support/ci/xml_to_md.py ozone-bin/extracted >
hadoop-hdds/docs/content/tools/Configurations.md
+ - name: Calculate hash of the new Configurations.md
+ id: new-hash
+ run: sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > new_hash.txt
+ - name: Compare hashes and decide to commit
+ id: hash-check
+ run: |
+ EXISTING_HASH=$(cat existing_hash.txt)
+ NEW_HASH=$(cat new_hash.txt)
+ if [ "$EXISTING_HASH" != "$NEW_HASH" ]; then
+ echo "Configurations.md has changed, proceeding to commit and
create PRs."
+ echo "hashes_differ=true" >> $GITHUB_ENV
+ else
+ echo "Configurations.md is unchanged, skipping commit and PR
creation."
+ echo "hashes_differ=false" >> $GITHUB_ENV
+ fi
+ - name: Commit and push changes to apache/ozone
+ if: env.hashes_differ == 'true'
+ run: |
+ git config --global user.name 'Github Actions'
+ git config --global user.email '[email protected]'
+ git add hadoop-hdds/docs/content/tools/Configurations.md
+ git commit -m "[Auto] Update Configurations.md from $GITHUB_SHA"
+ git push origin HEAD:master
+ - name: Create Pull Request in apache/ozone
+ if: env.hashes_differ == 'true'
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ gh pr create --title "[Auto] Update Configurations.md" --body "This
is an automated pull request."
+ - name: Checkout ozone-site repository
+ if: env.hashes_differ == 'true'
+ uses: actions/checkout@v4
+ with:
+ repository: apache/ozone-site
+ ref: 'HDDS-9225-website-v2'
+ path: ozone-site
+ - name: Copy MD file to ozone-site repository
+ if: env.hashes_differ == 'true'
+ run: |
+ TARGET_DIR=$(find ozone-site/docs -type d -name
'*-administrator-guide' -exec find {} -type d -name '*-configuration' -print
-quit \;)
+ echo "Target directory: $TARGET_DIR"
+ if [ -z "$TARGET_DIR" ]; then
+ echo "Target directory not found."
+ exit 1
+ fi
+ cp hadoop-hdds/docs/content/tools/Configurations.md
"$TARGET_DIR/configurations.md"
+ - name: Commit and push changes to apache/ozone-site
+ if: env.hashes_differ == 'true'
+ run: |
+ cd ozone-site
+ git config --global user.name 'Github Actions'
+ git config --global user.email '[email protected]'
+ ADD_FILE=$(find docs -type d -name '*-administrator-guide' -exec
find {} -type d -name '*-configuration' -print -quit \;)
+ git add "$ADD_FILE/configurations.md"
+ git commit -m "[Auto] Update configurations.md page from ozone
$GITHUB_SHA"
+ git push origin HEAD:HDDS-9225-website-v2
Review Comment:
Similar comment from before about source and target branches applies here as
well.
##########
.github/workflows/ci.yml:
##########
@@ -166,6 +166,99 @@ jobs:
path: |
~/.m2/repository/org/apache/ozone
retention-days: 1
+ build-config-doc:
+ needs:
+ - build
+ runs-on: ubuntu-20.04
+ steps:
+ - name: Checkout project
+ uses: actions/checkout@v4
+ - name: Set up Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.x'
+ - name: Download the source artifact
+ uses: actions/download-artifact@v4
+ with:
+ name: ozone-bin
+ path: ozone-bin
+ - name: Extract the source tarball
+ run: |
+ mkdir -p ozone-bin/extracted
+ tar -xzf ozone-bin/ozone-*-SNAPSHOT.tar.gz -C ozone-bin/extracted
+ - name: Calculate hash of the existing Configurations.md
+ id: existing-hash
+ run: |
+ if [ -f hadoop-hdds/docs/content/tools/Configurations.md ]; then
+ sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > existing_hash.txt
+ else
+ echo "No existing file found." > existing_hash.txt
+ fi
+ - name: Run the Python script to convert XML properties into Markdown
+ run: python3 dev-support/ci/xml_to_md.py ozone-bin/extracted >
hadoop-hdds/docs/content/tools/Configurations.md
+ - name: Calculate hash of the new Configurations.md
+ id: new-hash
+ run: sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > new_hash.txt
+ - name: Compare hashes and decide to commit
+ id: hash-check
+ run: |
+ EXISTING_HASH=$(cat existing_hash.txt)
+ NEW_HASH=$(cat new_hash.txt)
+ if [ "$EXISTING_HASH" != "$NEW_HASH" ]; then
+ echo "Configurations.md has changed, proceeding to commit and
create PRs."
+ echo "hashes_differ=true" >> $GITHUB_ENV
+ else
+ echo "Configurations.md is unchanged, skipping commit and PR
creation."
+ echo "hashes_differ=false" >> $GITHUB_ENV
+ fi
+ - name: Commit and push changes to apache/ozone
+ if: env.hashes_differ == 'true'
+ run: |
+ git config --global user.name 'Github Actions'
+ git config --global user.email '[email protected]'
+ git add hadoop-hdds/docs/content/tools/Configurations.md
+ git commit -m "[Auto] Update Configurations.md from $GITHUB_SHA"
+ git push origin HEAD:master
+ - name: Create Pull Request in apache/ozone
+ if: env.hashes_differ == 'true'
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ gh pr create --title "[Auto] Update Configurations.md" --body "This
is an automated pull request."
+ - name: Checkout ozone-site repository
+ if: env.hashes_differ == 'true'
+ uses: actions/checkout@v4
+ with:
+ repository: apache/ozone-site
+ ref: 'HDDS-9225-website-v2'
+ path: ozone-site
+ - name: Copy MD file to ozone-site repository
+ if: env.hashes_differ == 'true'
+ run: |
+ TARGET_DIR=$(find ozone-site/docs -type d -name
'*-administrator-guide' -exec find {} -type d -name '*-configuration' -print
-quit \;)
Review Comment:
I don't think we need the recursive search provided by find, and it might
actually be more predictable to leave it out. I think we can simplify this:
```suggestion
TARGET_DIR=$(ls docs/*-administrator-guide/*-configuration)
```
##########
.github/workflows/ci.yml:
##########
@@ -166,6 +166,99 @@ jobs:
path: |
~/.m2/repository/org/apache/ozone
retention-days: 1
+ build-config-doc:
+ needs:
+ - build
+ runs-on: ubuntu-20.04
+ steps:
+ - name: Checkout project
+ uses: actions/checkout@v4
+ - name: Set up Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.x'
+ - name: Download the source artifact
+ uses: actions/download-artifact@v4
+ with:
+ name: ozone-bin
+ path: ozone-bin
+ - name: Extract the source tarball
+ run: |
+ mkdir -p ozone-bin/extracted
+ tar -xzf ozone-bin/ozone-*-SNAPSHOT.tar.gz -C ozone-bin/extracted
+ - name: Calculate hash of the existing Configurations.md
+ id: existing-hash
+ run: |
+ if [ -f hadoop-hdds/docs/content/tools/Configurations.md ]; then
+ sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > existing_hash.txt
+ else
+ echo "No existing file found." > existing_hash.txt
+ fi
+ - name: Run the Python script to convert XML properties into Markdown
+ run: python3 dev-support/ci/xml_to_md.py ozone-bin/extracted >
hadoop-hdds/docs/content/tools/Configurations.md
+ - name: Calculate hash of the new Configurations.md
+ id: new-hash
+ run: sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > new_hash.txt
+ - name: Compare hashes and decide to commit
+ id: hash-check
+ run: |
+ EXISTING_HASH=$(cat existing_hash.txt)
+ NEW_HASH=$(cat new_hash.txt)
+ if [ "$EXISTING_HASH" != "$NEW_HASH" ]; then
+ echo "Configurations.md has changed, proceeding to commit and
create PRs."
+ echo "hashes_differ=true" >> $GITHUB_ENV
+ else
+ echo "Configurations.md is unchanged, skipping commit and PR
creation."
+ echo "hashes_differ=false" >> $GITHUB_ENV
+ fi
+ - name: Commit and push changes to apache/ozone
+ if: env.hashes_differ == 'true'
+ run: |
+ git config --global user.name 'Github Actions'
+ git config --global user.email '[email protected]'
+ git add hadoop-hdds/docs/content/tools/Configurations.md
+ git commit -m "[Auto] Update Configurations.md from $GITHUB_SHA"
+ git push origin HEAD:master
Review Comment:
I think this is adding the commit directly to master. This should actually
go to a source branch based off the latest master with a unique name. In the
following PR step, we should explicitly specify the source and target branches
for the PR. I think we will have to manually delete the branch when the PR is
merged. I'm not sure how to do something like dependabot where the branch gets
automatically deleted when the PR is merged.
##########
.github/workflows/ci.yml:
##########
@@ -166,6 +166,99 @@ jobs:
path: |
~/.m2/repository/org/apache/ozone
retention-days: 1
+ build-config-doc:
+ needs:
+ - build
+ runs-on: ubuntu-20.04
+ steps:
+ - name: Checkout project
+ uses: actions/checkout@v4
+ - name: Set up Python
+ uses: actions/setup-python@v4
+ with:
+ python-version: '3.x'
+ - name: Download the source artifact
+ uses: actions/download-artifact@v4
+ with:
+ name: ozone-bin
+ path: ozone-bin
+ - name: Extract the source tarball
+ run: |
+ mkdir -p ozone-bin/extracted
+ tar -xzf ozone-bin/ozone-*-SNAPSHOT.tar.gz -C ozone-bin/extracted
+ - name: Calculate hash of the existing Configurations.md
+ id: existing-hash
+ run: |
+ if [ -f hadoop-hdds/docs/content/tools/Configurations.md ]; then
+ sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > existing_hash.txt
+ else
+ echo "No existing file found." > existing_hash.txt
+ fi
+ - name: Run the Python script to convert XML properties into Markdown
+ run: python3 dev-support/ci/xml_to_md.py ozone-bin/extracted >
hadoop-hdds/docs/content/tools/Configurations.md
+ - name: Calculate hash of the new Configurations.md
+ id: new-hash
+ run: sha256sum hadoop-hdds/docs/content/tools/Configurations.md | awk
'{ print $1 }' > new_hash.txt
+ - name: Compare hashes and decide to commit
+ id: hash-check
+ run: |
+ EXISTING_HASH=$(cat existing_hash.txt)
+ NEW_HASH=$(cat new_hash.txt)
+ if [ "$EXISTING_HASH" != "$NEW_HASH" ]; then
+ echo "Configurations.md has changed, proceeding to commit and
create PRs."
+ echo "hashes_differ=true" >> $GITHUB_ENV
+ else
+ echo "Configurations.md is unchanged, skipping commit and PR
creation."
+ echo "hashes_differ=false" >> $GITHUB_ENV
+ fi
+ - name: Commit and push changes to apache/ozone
+ if: env.hashes_differ == 'true'
+ run: |
+ git config --global user.name 'Github Actions'
+ git config --global user.email '[email protected]'
+ git add hadoop-hdds/docs/content/tools/Configurations.md
+ git commit -m "[Auto] Update Configurations.md from $GITHUB_SHA"
+ git push origin HEAD:master
+ - name: Create Pull Request in apache/ozone
+ if: env.hashes_differ == 'true'
+ env:
+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ gh pr create --title "[Auto] Update Configurations.md" --body "This
is an automated pull request."
+ - name: Checkout ozone-site repository
+ if: env.hashes_differ == 'true'
+ uses: actions/checkout@v4
+ with:
+ repository: apache/ozone-site
+ ref: 'HDDS-9225-website-v2'
+ path: ozone-site
+ - name: Copy MD file to ozone-site repository
+ if: env.hashes_differ == 'true'
+ run: |
+ TARGET_DIR=$(find ozone-site/docs -type d -name
'*-administrator-guide' -exec find {} -type d -name '*-configuration' -print
-quit \;)
+ echo "Target directory: $TARGET_DIR"
+ if [ -z "$TARGET_DIR" ]; then
+ echo "Target directory not found."
+ exit 1
+ fi
+ cp hadoop-hdds/docs/content/tools/Configurations.md
"$TARGET_DIR/configurations.md"
Review Comment:
This page is currently called "Configuration Key Appendix" in the new
website. We should name the file `<number>-appendix.md`. This will give it the
URL `<domain>/docs/administrator-guide/configuration/appendix`. The current set
of changes would produce a more confusing URL:
`<domain>/docs/administrator-guide/configuration/configuration`.
The number at the beginning of the name is required by the new website's CI
to define an absolute ordering of pages in the sdebar. We probably want this to
be the last page in the section always. We could list all the current numbers
and add 1 to make it the next, but there is actually no requirement that the
numbers are consecutive, they work more like weights. To keep this simple I
would suggest naming this file `99-appendix.md` so it is always last assuming
there are less than 99 immediate children of the `configuration` directory.
This should always be true otherwise the website organization would be a mess,
but the PR gives us a change to check this assumption if needed. Let's add a
comment to the script here explaining this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
