Anastasia Filippova created HDDS-15999:
------------------------------------------
Summary: Add GitHub Action to trigger Algolia DocSearch crawler
reindex on demand and on a daily schedule
Key: HDDS-15999
URL: https://issues.apache.org/jira/browse/HDDS-15999
Project: Apache Ozone
Issue Type: Task
Reporter: Anastasia Filippova
h3. Background
The Apache Ozone website (https://ozone.apache.org) uses Algolia DocSearch (App
ID in docusaurus.config.js, index {{Apache Ozone website}}). The Algolia
crawler refreshes the search index on a long fixed interval (currently ~30
days), so documentation updates on the live site are slow to appear in search.
h3. Goal
Add automation in apache/ozone-site to call the Algolia Crawler REST API and
start a reindex when appropriate, without manual dashboard-only workflows.
h3. Proposed solution
* New workflow {{.github/workflows/algolia-reindex.yml}} with:
** {{workflow_dispatch}} with optional {{force}} input (skip the recent master
activity gate).
** Daily schedule (e.g. 12:00 UTC) that reindexes only if {{master}} had
commits in the last 24 hours.
* Shell script {{.github/scripts/algolia_reindex.sh}} that POSTs to the Algolia
Crawler reindex API using Crawler Basic auth (not the public search-only API
key from the site config).
* Operator notes for required GitHub Actions repository secrets
({{.github/resource/algolia_reindex_secrets.md}}).
h3. Notes
* The crawler indexes the published site, not git; there can be a delay after
master publish via asf-site / gitpubsub. A follow-up could chain reindex after
successful publish with a delay if search still lags.
* Workflow must pass existing repo lint (yamllint indentation for schedule list
items matches other workflows).
h3. Post-merge (ASF repo admins)
Configure repository secrets on apache/ozone-site:
* ALGOLIA_CRAWLER_USER_ID
* ALGOLIA_CRAWLER_API_KEY
* ALGOLIA_CRAWLER_ID
(Sources documented in Algolia Crawler dashboard.)
h3. Acceptance criteria
* Manual workflow run with force=true returns HTTP 2xx and starts a crawl
visible in the Algolia Crawler dashboard.
* Scheduled run skips reindex when master has no commits in the last 24 hours
(green job, clear log message).
* Scheduled run triggers reindex when master had commits in the last 24 hours.
* No crawler or admin API credentials committed to the repository.
* CI lint passes for new workflow YAML.
h3. Test plan
# Merge PR to apache/ozone-site; add the three repository secrets.
# Actions → algolia-reindex → Run workflow on master with force enabled.
# Confirm crawl task in Algolia dashboard and updated search results after
crawl completes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
