nchammas commented on PR #57015: URL: https://github.com/apache/spark/pull/57015#issuecomment-4927277153
> I'm not sure how it will affect Spark's GA budget My goal is to migrate some checks out of the massive `build_and_test.yml` and into smaller workflows like `docs.yml`, so the impact should be neutral. > can we integrate this to our current building so it doesn't trigger another doc build? I tried for several days to do this unsuccessfully. `build_and_test.yml` implements a non-standard checkout pattern that totally breaks GitHub's problem matchers. https://github.com/apache/spark/blob/ef58ab3bd90d07efebd4a2e05007b8a36c5d2bee/.github/workflows/build_and_test.yml#L493-L506 For problem matchers (and likely other productivity features of GitHub) to work correctly, they need the local checkout to have a reference to the same commit as what's on the PR. Because we checkout `master` and merge the PR into it, we create a new, synthetic commit that doesn't exist anywhere else. We do this so that we know PRs are always tested against the latest `master`. That way tests don't pass on the PR but later fail post-merge because `master` changed. But we can address this concern in some other ways: 1. Conservative approach with weaker guarantees than we currently have: Checkout the fork "normally" as I am doing in `docs.yml`. If the common root of the PR branch vs. target branch is stale (e.g. older than ~1 week), fail fast. 2. Radical approach with the same guarantees as we currently have: Adopt a [merge queue](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue) and overhaul `dev/merge_spark_pr.py` to use it. If we are not willing to consider either approach, then I think it will be impossible to get inline PR annotations working. Over time, we may also find there are other GitHub features that are difficult or impossible to use because of our non-standard checkout pattern. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
