potiuk commented on PR #50375: URL: https://github.com/apache/airflow/pull/50375#issuecomment-2866074784
Yeah. I think we can work out the workflows for this step-by-step with @kaxil and @ephraimbuddy and others who would be the release managers. Right after we improve and finalize the docs for regular workflows, we can work out how to do the "staging" ones. For now Kaxil for example publishes Airflow docs with surge.sh - which is fine but it would be better if we can do the same publishing for RC as for the regular releases. For example when we are relasing the images we have the same workflow -> just "Release -> type version-> workflow will appropriately build the images with or without -`-pre` option and automatically using the rc versions of the providers (and handles all the complex cases) and with or without updating the "katest" image link (for example when we release an RC image we never update "latest" aliases in DockerHub. Here, ti's a bit different story, because we have to first work out how we are publishing the staging site as well. Not only the documentation for airflow and providers. And we do not have it automated yet, but we can - and we planned to have a dedicated branch "staging" that you can push in "airflow-site" and then the site will automatically be published at "staging.airflow.apache.org" - that will allow us to iterate on "site" itself. Then I imagine when you are releasing RC candidates, you could choose "staging" from the list (that might be even a bit simpler than now - just additional "staging" checkbox that might even be selected by default) or have a separate "Staging Publishing' workflow - and then the RC docs would be published at "staging.airflow.apache.org" as well. Also we could always select staging by default when tag contains pre-release version (rc/alpha etc.) and do not give freedom to the RM to select staging. One thing to note there is a question of RC links in pypi - currently the links are nicely redirecting [Documentation](https://airflow.apache.org/docs/apache-airflow-providers-common-messaging/1.0.1rc1) for example - and ... it nicely redirects to 1.0.1 (I thiink it 's a side effect of the index.html fixing that @gopidesupavan implemented. Probably we could do the same with future links that are for versions we have not released yet - but we could redirect them to staging - so for example: https://airflow.apache.org/docs/apache-airflow-providers-common-messaging/1.0.2rc1/ (when 1.0.2 is not yet released and docs not published) would redirect to https://staging.airflow.apache.org/docs/apache-airflow-providers-common-messaging/1.0.2/ -> and use the documentation from there. In this case the workflows would look like this: 1) When you want to check a new "site" modification -> push to "staging" branch in "airflow-site" and it will be visible under "staging.airflow.apache.org". Later merging it to "main" branch will publish them to "airflow.apache.org" - so far, so good. 2) When you want to make a release candidate for providers, airlfow , docker-stack, all-providers -> you run the same workflow as for final publishing (choose rc* tag to use) -> they will also land at staging.airlfow.apache.org, links in PyPi packages will point to "apache.airflow.org/ .... x.y.zrc2" - but since documentation will not be published in "airflow.apache.org" - that documentation will get riderected to "staging.airflow.apache.org/.....x.y.z/" which will be already published by release manager. 3) once we publish the final release, the RM will re-run the docs push (but will select "no staging" option) and the docs will land in "airflow.apache.org" -> and the links from RC candidates in PyPI will no longer redirect to staging, they will point to the final release (of airflow or providers). Does it sound like a good plan - @eladkal @kaxil @ephraimbuddy ? What would be the preference for workflows of publishing airlfow/provider docs ? * two separate workflow (PROD/STAGING push?) * one workflow with checkbox? (what should be the default) * or maybe use the TAG and take away the freedom of choosing staging or not If we get answer to those questions - we can proceed with implementing staging the chosen way :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
