[ 
https://issues.apache.org/jira/browse/SPARK-61458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ruifeng Zheng updated SPARK-61458:
----------------------------------
    Description: 
CI builds documentation, but the daily GitHub Pages workflow independently 
rebuilds the same site.

Publish every upstream documentation build as a branch-specific 
docs-preview-<branch>-<sha>-<attempt> artifact retained for one day. Name the 
archive site-<sha>-<attempt>.tar.bz2. Use the built commit hash and workflow 
run attempt so retries can upload without overwriting existing artifacts. Pages 
selects the newest artifact by its creation date, independently of commit-hash 
order.

In the upstream master documentation job, set the Spark version, short version, 
search version filter, and PySpark version to In-Progress before the complete 
Jekyll build. Export RELEASE_VERSION and build the preview with SKIP_RDOC=1. 
Keep the initial build without API docs, perform only one complete build, and 
retain change-based API documentation skips only for forks. Other upstream 
branches retain their source versions.

Have Pages select the unexpired master preview artifact with the newest 
creation date across all result pages, download that exact artifact by ID, and 
deploy it without rebuilding or modifying its HTML. Keep the daily schedule and 
manual dispatch. Fail if no artifact is available, leaving the existing 
deployment in place. A successful documentation build can be reused even when 
unrelated jobs in the same CI run fail.


  was:
CI builds documentation, but the daily GitHub Pages workflow independently 
rebuilds the same site.

Publish every upstream documentation build as a branch-specific 
docs-preview-<branch>-<timestamp> artifact retained for one day. Include a UTC 
timestamp in both the artifact name and archive filename, and do not overwrite 
existing artifacts.

In the upstream master documentation job, set the Spark version, short version, 
search version filter, and PySpark version to In-Progress before the complete 
Jekyll build. Export RELEASE_VERSION and build the preview with SKIP_RDOC=1. 
Keep the initial build without API docs, perform only one complete build, and 
retain change-based API documentation skips only for forks. Other upstream 
branches retain their source versions.

Have Pages select the unexpired master preview artifact with the newest 
creation date across all result pages, download that exact artifact by ID, and 
deploy it without rebuilding or modifying its HTML. Keep the daily schedule and 
manual dispatch. Fail if no artifact is available, leaving the existing 
deployment in place. A successful documentation build can be reused even when 
unrelated jobs in the same CI run fail.



> Reuse post-merge documentation artifacts for GitHub Pages
> ---------------------------------------------------------
>
>                 Key: SPARK-61458
>                 URL: https://issues.apache.org/jira/browse/SPARK-61458
>             Project: Spark
>          Issue Type: Improvement
>          Components: Project Infra
>    Affects Versions: 5.0.0
>            Reporter: Ruifeng Zheng
>            Priority: Major
>              Labels: pull-request-available
>
> CI builds documentation, but the daily GitHub Pages workflow independently 
> rebuilds the same site.
> Publish every upstream documentation build as a branch-specific 
> docs-preview-<branch>-<sha>-<attempt> artifact retained for one day. Name the 
> archive site-<sha>-<attempt>.tar.bz2. Use the built commit hash and workflow 
> run attempt so retries can upload without overwriting existing artifacts. 
> Pages selects the newest artifact by its creation date, independently of 
> commit-hash order.
> In the upstream master documentation job, set the Spark version, short 
> version, search version filter, and PySpark version to In-Progress before the 
> complete Jekyll build. Export RELEASE_VERSION and build the preview with 
> SKIP_RDOC=1. Keep the initial build without API docs, perform only one 
> complete build, and retain change-based API documentation skips only for 
> forks. Other upstream branches retain their source versions.
> Have Pages select the unexpired master preview artifact with the newest 
> creation date across all result pages, download that exact artifact by ID, 
> and deploy it without rebuilding or modifying its HTML. Keep the daily 
> schedule and manual dispatch. Fail if no artifact is available, leaving the 
> existing deployment in place. A successful documentation build can be reused 
> even when unrelated jobs in the same CI run fail.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to