gopidesupavan opened a new pull request, #47814: URL: https://github.com/apache/airflow/pull/47814
### Why Currently, the Airflow website is served from the GitHub branch from airflow-site repo. As package versions increase, we have observed occasional failures in building the website and pushing documentation to GitHub Pages branches. Additionally, the growing size of the documentation has made managing the publishing process increasingly difficult. To address these challenges, we have discussed publishing the documentation to S3 and serving it from there. ### What Adding Breeze Commands and GitHub Workflow to Publish Documentation to S3 These steps will be handled by the release manager. Current Documentation Publishing Process: Build documentation Publish documentation to airflow-site (a locally checked-out repository of airflow-site) Add back references And pushing changes to airflow-site. We are introducing an additional step: publishing documentation to S3 (publish-docs-to-s3). Once the release manager finalizes a release and new version tags are available in the Airflow repository, a new GitHub workflow—"Publish Docs to S3"—has been created. This process is manual and must be triggered by the release manager. Workflow Requirements: ref: The tag to check out exclude-docs: Optional; packages to exclude from documentation publishing docs-list-as-string: Optional; list of packages to build documentation for Default Workflow Steps: Build CI images Check out the specified ref tag Build documentation Create a temporary airflow-site directory for initial document publishing Add back references Publish documentation to S3 If a matching version already exists in the destination, it will not be copied. To override existing versions, use the --overwrite flag in the release command. An example command sync stable version docs to s3. ``` breeze release-management publish-docs-to-s3 --source-dir-path /tmp/docs-archive --destination-location s3://airflow-docs/docs --stable-versions ``` An example command sync all docs to S3 (Useful in when scenarios like bulk upload, if we fix anything in on historical versions this command will be useful to sync all ) ``` breeze release-management publish-docs-to-s3 --source-dir-path /tmp/airflow-site/docs-archive --destination-location s3://airflow-docs/docs --publish-all-docs ``` <img width="1728" alt="image" src="https://github.com/user-attachments/assets/4f613bd8-978a-4f97-85c2-ede8aaa45b4f"; /> Next steps: New workflows will be created in airflow-site-archive repo and this job is nightly or manually triggered job. it will sync s3 docs to airflow-site-archive repo so that we will have upto date docs in github repo. Discussion thread: https://apache-airflow.slack.com/archives/CJ1LVREHX/p1737287674851989?thread_ts=1737113729.883969&cid=CJ1LVREHX <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <!-- Thank you for contributing! Please make sure that your code changes are covered with tests. And in case of new features or big changes remember to adjust the documentation. Feel free to ping committers for the review! In case of an existing issue, reference it using one of the following: closes: #ISSUE related: #ISSUE How to write a good git commit message: http://chris.beams.io/posts/git-commit/ --> <!-- Please keep an empty line above the dashes. --> --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
