This is an automated email from the ASF dual-hosted git repository.
Yicong-Huang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 7782448cd2f8 [SPARK-58929][INFRA] Add a dry-run mode to
merge_spark_pr.py
7782448cd2f8 is described below
commit 7782448cd2f8cfd74f8ae0677930304154506047
Author: Yicong Huang <[email protected]>
AuthorDate: Tue Aug 25 18:00:44 2026 +0000
[SPARK-58929][INFRA] Add a dry-run mode to merge_spark_pr.py
### What changes were proposed in this pull request?
Adds a dry-run mode to `dev/merge_spark_pr.py` that runs every read-only
step for real but suppresses every effect that leaves the local machine,
logging a `DRY-RUN: would <action> ...` line instead.
Every production mutation goes through an injectable client with a
`Production` and a `DryRun` subclass, and `main()` constructs the `DryRun`
variants when `--dry-run` is set, so the dry-run behavior lives entirely in
those subclasses:
- `Git` / `DryRunGit`: git I/O. `push()` -- the only command that mutates
`PUSH_REMOTE_NAME` -- is the one method `DryRunGit` overrides;
fetch/checkout/merge/commit/cherry-pick/rev-parse still run for real on the
throwaway `PR_TOOL_*` branches that `clean_up` removes, so conflicts and the
computed merge hash stay realistic.
- `GitHub` / `DryRunGitHub`: `close_pr()` and `comment()`. Reads go through
the module-level `get_json()`.
- `Jira` / `DryRunJira`: component and fixVersion updates, assignee, the
contributor-role grant, and the resolve transition. Reads stay on the
module-level `asf_jira` client.
Read-only steps still run so the flow is faithful: fetching the PR, the
JIRA issue lookup and `project_versions`, and JIRA/GitHub token validation
(`initialize_jira()` and `get_json()` are not routed through the dry-run
clients), so a dry run exercises the same credentials a real merge needs.
Interactive prompts continue to work.
Dry-run is enabled with a `--dry-run` / `-n` flag (the CLI now uses
`argparse`) or the `DRY_RUN` environment variable, which seeds the flag's
default -- consistent with the existing `SKIP_VERSION_CHECK` env toggle in this
script.
### Why are the changes needed?
Before this change there was no way to rehearse the full merge flow without
producing real side effects. The existing affordances cover only part of it:
doctests exercise the pure logic; `SKIP_VERSION_CHECK` lets a committer iterate
against a real PR; and `PR_REMOTE_NAME` / `PUSH_REMOTE_NAME` can point the `git
push` at a scratch fork. But `GITHUB_API_BASE` and `JIRA_API_BASE` are
hardcoded to apache, so the PR close/comment and JIRA resolve paths cannot be
redirected and always hit t [...]
### Does this PR introduce _any_ user-facing change?
No. This is a committer-facing dev tool; the new mode is opt-in and the
default behavior is unchanged.
### How was this patch tested?
- Doctests: `python -m doctest dev/merge_spark_pr.py` passes.
- Manual smoke test of the dry-run clients: `DryRunGit.push` logs and does
not push while `DryRunGit.run` still executes read-only git;
`DryRunGitHub.close_pr`/`comment` log;
`DryRunJira.update_components`/`add_fix_versions`/`resolve_issue`/`assign`/`grant_contributor`
all log and make no API call. With dry-run off, the default `Production`
clients are used and behavior is unchanged.
- `argparse` parsing verified for the optional PR-number positional and the
`--dry-run`/`-n` flag, including the `DRY_RUN` env default.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #58198 from Yicong-Huang/add-dry-run-merge-script.
Authored-by: Yicong Huang <[email protected]>
Signed-off-by: Yicong-Huang <[email protected]>
---
dev/merge_spark_pr.py | 369 +++++++++++++++++++++++++++++++++++---------------
1 file changed, 263 insertions(+), 106 deletions(-)
diff --git a/dev/merge_spark_pr.py b/dev/merge_spark_pr.py
index e7cb861c073b..366181a4b380 100755
--- a/dev/merge_spark_pr.py
+++ b/dev/merge_spark_pr.py
@@ -37,6 +37,7 @@
# have added remotes corresponding to both (i) the github apache Spark
# mirror and (ii) the apache git repo.
+import argparse
import json
import os
import re
@@ -82,6 +83,16 @@ JIRA_CONNECT_TIMEOUT =
float(os.environ.get("JIRA_CONNECT_TIMEOUT", "3.05"))
# exceeding your IP's unauthenticated request rate limit. You can create an
OAuth key at
# https://github.com/settings/tokens. This script only requires the
"public_repo" scope.
GITHUB_OAUTH_KEY = os.environ.get("GITHUB_OAUTH_KEY")
+# Setting the DRY_RUN env var to any non-empty value makes the --dry-run/-n
flag default to on
+# (see build_arg_parser); the flag is the primary interface and can also turn
it on explicitly.
+# Consistent with the SKIP_VERSION_CHECK env toggle above. In a dry run every
read-only step and
+# local git op still runs -- fetching the PR, JIRA lookup, project_versions,
JIRA/GitHub token
+# validation, and the local squash-merge and cherry-picks on the throwaway
PR_TOOL_* branches so
+# conflicts and the computed merge hash stay realistic -- but every outbound
effect is routed
+# through a DryRun* client (see Git/GitHub/Jira below) that logs a "DRY-RUN:
would ..." line
+# instead of executing it: the git push to PUSH_REMOTE_NAME, the GitHub PR
close/comment, and all
+# JIRA writes (component and fixVersion updates, assignment, and the resolve
transition).
+DRY_RUN_ENV = bool(os.environ.get("DRY_RUN"))
GITHUB_BASE = "https://github.com/apache/spark/pull";
@@ -483,36 +494,63 @@ def find_merge_commit(pr_num, pr_events):
return None, None
-def close_pr(pr_num):
- url = "%s/pulls/%s" % (GITHUB_API_BASE, pr_num)
- data = json.dumps({"state": "closed"}).encode("utf-8")
- request = Request(url, data=data, method="PATCH")
- request.add_header("Content-Type", "application/json")
- request.add_header("Accept", "application/vnd.github+json")
- if GITHUB_OAUTH_KEY:
- request.add_header("Authorization", "token %s" % GITHUB_OAUTH_KEY)
- try:
- return json.load(urlopen(request))
- except HTTPError as e:
- print_error("Failed to close PR #%s: HTTP %s %s" % (pr_num, e.code,
e.reason))
- return None
+class GitHub:
+ """GitHub REST writes used by the merge script -- the single seam for PR
mutations.
+
+ Reads go through the module-level get_json(); only the two mutations
(closing the PR and
+ posting a comment) live here, so DryRunGitHub can override them without
touching any read
+ path. main() constructs GitHub() normally and DryRunGitHub() for a dry run.
+ """
+ def close_pr(self, pr_num):
+ url = "%s/pulls/%s" % (GITHUB_API_BASE, pr_num)
+ data = json.dumps({"state": "closed"}).encode("utf-8")
+ request = Request(url, data=data, method="PATCH")
+ request.add_header("Content-Type", "application/json")
+ request.add_header("Accept", "application/vnd.github+json")
+ if GITHUB_OAUTH_KEY:
+ request.add_header("Authorization", "token %s" % GITHUB_OAUTH_KEY)
+ try:
+ return json.load(urlopen(request))
+ except HTTPError as e:
+ print_error("Failed to close PR #%s: HTTP %s %s" % (pr_num,
e.code, e.reason))
+ return None
-def comment_pr(pr_num, body):
- url = "%s/issues/%s/comments" % (GITHUB_API_BASE, pr_num)
- data = json.dumps({"body": body}).encode("utf-8")
- request = Request(url, data=data, method="POST")
- request.add_header("Content-Type", "application/json")
- request.add_header("Accept", "application/vnd.github+json")
- if GITHUB_OAUTH_KEY:
+ def comment(self, pr_num, body):
+ # Posting a comment is a write and needs auth; without a token, skip
rather than 401.
+ if not GITHUB_OAUTH_KEY:
+ print_error("GITHUB_OAUTH_KEY is not set; skipping the comment.")
+ return None
+ url = "%s/issues/%s/comments" % (GITHUB_API_BASE, pr_num)
+ data = json.dumps({"body": body}).encode("utf-8")
+ request = Request(url, data=data, method="POST")
+ request.add_header("Content-Type", "application/json")
+ request.add_header("Accept", "application/vnd.github+json")
request.add_header("Authorization", "token %s" % GITHUB_OAUTH_KEY)
- try:
- return json.load(urlopen(request))
- except HTTPError as e:
- print_error("Failed to comment on PR #%s: HTTP %s %s" % (pr_num,
e.code, e.reason))
+ try:
+ return json.load(urlopen(request))
+ except HTTPError as e:
+ print_error("Failed to comment on PR #%s: HTTP %s %s" % (pr_num,
e.code, e.reason))
+ return None
+
+
+class DryRunGitHub(GitHub):
+ """Logs the intended PR close/comment instead of calling the GitHub API."""
+
+ def close_pr(self, pr_num):
+ print("DRY-RUN: would close PR #%s via the GitHub API." % pr_num)
+ return None
+
+ def comment(self, pr_num, body):
+ print("DRY-RUN: would post the following comment on PR #%s:\n%s" %
(pr_num, body))
return None
+# Set in main(): GitHub() normally, DryRunGitHub() for a dry run. Module-level
default so the
+# top-level except handler and any early failure path have a usable client.
+github = GitHub()
+
+
def post_merge_comment(pr_num, merged_commits):
"""Post a comment on the PR recording every branch the change landed on
and a
link to the resulting commit, so the merge is traceable from the PR page.
@@ -533,10 +571,7 @@ def post_merge_comment(pr_num, merged_commits):
"\n%s\n\n%s\n%s"
% (bold("Posting merge comment on PR #%s:" % pr_num), bold(summary),
attribution)
)
- if not GITHUB_OAUTH_KEY:
- print_error("GITHUB_OAUTH_KEY is not set; skipping the merge comment.")
- return
- comment_pr(pr_num, body)
+ github.comment(pr_num, body)
def fail(msg):
@@ -545,12 +580,41 @@ def fail(msg):
sys.exit(-1)
-def run_cmd(cmd):
- print(cmd)
- if isinstance(cmd, list):
- return subprocess.check_output(cmd).decode("utf-8")
- else:
- return subprocess.check_output(cmd.split(" ")).decode("utf-8")
+class Git:
+ """Runs the merge-flow git commands for the merge script.
+
+ This is the seam for the merge/backport flow, not every git invocation:
+ check_script_up_to_date() shells out to git merge-base directly. ``run``
executes read-only
+ and local commands (fetch, checkout, merge, commit, cherry-pick,
rev-parse, config, branch
+ bookkeeping); those are safe even in a dry run because the merge and
cherry-picks happen on
+ the throwaway PR_TOOL_* branches that clean_up removes. ``push`` is the
only command that
+ mutates the shared repo at PUSH_REMOTE_NAME, so it is the one method
DryRunGit overrides;
+ everything else it inherits and runs for real, keeping conflict detection
and the computed
+ merge hash realistic. main() constructs Git() normally and DryRunGit() for
a dry run.
+ """
+
+ def run(self, cmd):
+ print(cmd)
+ if isinstance(cmd, list):
+ return subprocess.check_output(cmd).decode("utf-8")
+ else:
+ return subprocess.check_output(cmd.split(" ")).decode("utf-8")
+
+ def push(self, remote, local_ref, remote_ref):
+ return self.run("git push %s %s:%s" % (remote, local_ref, remote_ref))
+
+
+class DryRunGit(Git):
+ """Logs the intended push instead of updating the remote; all other git
runs for real."""
+
+ def push(self, remote, local_ref, remote_ref):
+ print("DRY-RUN: would push %s to %s:%s" % (local_ref, remote,
remote_ref))
+ return ""
+
+
+# Set in main(): Git() normally, DryRunGit() for a dry run. Module-level
default so clean_up and
+# the top-level except handler have a usable client even if main() fails early.
+git = Git()
class SkipCherryPick(Exception):
@@ -567,7 +631,7 @@ def continue_maybe(prompt, cherry=False):
if get_input(f"{prompt} (y/N): ", ["y", "n", ""]) != "y":
if cherry:
try:
- run_cmd("git cherry-pick --abort")
+ git.run("git cherry-pick --abort")
except Exception:
print_error("Unable to abort and get back to the state before
cherry-pick")
print("Skipping this cherry-pick; the merge continues.")
@@ -579,26 +643,26 @@ def continue_maybe(prompt, cherry=False):
def clean_up():
if "original_head" in globals():
print("Restoring head pointer to %s" % original_head)
- run_cmd("git checkout %s" % original_head)
+ git.run("git checkout %s" % original_head)
- branches = run_cmd("git branch").replace(" ", "").split("\n")
+ branches = git.run("git branch").replace(" ", "").split("\n")
for branch in list(filter(lambda x: x.startswith(BRANCH_PREFIX),
branches)):
print("Deleting local branch %s" % branch)
- run_cmd("git branch -D %s" % branch)
+ git.run("git branch -D %s" % branch)
# merge the requested PR and return the merge hash
def merge_pr(pr_num, target_ref, title, body, pr_repo_desc, pr_author,
co_authors):
pr_branch_name = "%s_MERGE_PR_%s" % (BRANCH_PREFIX, pr_num)
target_branch_name = "%s_MERGE_PR_%s_%s" % (BRANCH_PREFIX, pr_num,
target_ref.upper())
- run_cmd("git fetch %s pull/%s/head:%s" % (PR_REMOTE_NAME, pr_num,
pr_branch_name))
- run_cmd("git fetch %s %s:%s" % (PUSH_REMOTE_NAME, target_ref,
target_branch_name))
- run_cmd("git checkout %s" % target_branch_name)
+ git.run("git fetch %s pull/%s/head:%s" % (PR_REMOTE_NAME, pr_num,
pr_branch_name))
+ git.run("git fetch %s %s:%s" % (PUSH_REMOTE_NAME, target_ref,
target_branch_name))
+ git.run("git checkout %s" % target_branch_name)
had_conflicts = False
try:
- run_cmd(["git", "merge", pr_branch_name, "--squash"])
+ git.run(["git", "merge", pr_branch_name, "--squash"])
except Exception as e:
msg = "Error merging: %s\nWould you like to manually fix-up this
merge?" % e
continue_maybe(msg)
@@ -621,8 +685,8 @@ def merge_pr(pr_num, target_ref, title, body, pr_repo_desc,
pr_author, co_author
# to people every time someone creates a public fork of Spark.
merge_message_flags += ["-m", body.replace("@", "")]
- committer_name = run_cmd("git config --get user.name").strip()
- committer_email = run_cmd("git config --get user.email").strip()
+ committer_name = git.run("git config --get user.name").strip()
+ committer_email = git.run("git config --get user.email").strip()
if had_conflicts:
message = "This patch had conflicts when merged, resolved
by\nCommitter: %s <%s>" % (
@@ -642,19 +706,19 @@ def merge_pr(pr_num, target_ref, title, body,
pr_repo_desc, pr_author, co_author
merge_message_flags += ["-m", authors]
- run_cmd(["git", "commit", '--author="%s"' % primary_author] +
merge_message_flags)
+ git.run(["git", "commit", '--author="%s"' % primary_author] +
merge_message_flags)
continue_maybe(
"Merge complete (local ref %s). Push to %s?" % (target_branch_name,
PUSH_REMOTE_NAME)
)
try:
- run_cmd("git push %s %s:%s" % (PUSH_REMOTE_NAME, target_branch_name,
target_ref))
+ git.push(PUSH_REMOTE_NAME, target_branch_name, target_ref)
except Exception as e:
clean_up()
print_error("Exception while pushing: %s" % e)
- merge_hash = run_cmd("git rev-parse %s" % target_branch_name).strip()
+ merge_hash = git.run("git rev-parse %s" % target_branch_name).strip()
clean_up()
print("Pull request #%s merged!" % pr_num)
print("Merge hash: %s" % merge_hash)
@@ -670,11 +734,11 @@ def _do_cherry_pick(pr_num, merge_hash, pick_ref):
"""
pick_branch_name = "%s_PICK_PR_%s_%s" % (BRANCH_PREFIX, pr_num,
pick_ref.upper())
- run_cmd("git fetch %s %s:%s" % (PUSH_REMOTE_NAME, pick_ref,
pick_branch_name))
- run_cmd("git checkout %s" % pick_branch_name)
+ git.run("git fetch %s %s:%s" % (PUSH_REMOTE_NAME, pick_ref,
pick_branch_name))
+ git.run("git checkout %s" % pick_branch_name)
try:
- run_cmd("git cherry-pick -sx %s" % merge_hash)
+ git.run("git cherry-pick -sx %s" % merge_hash)
except Exception as e:
msg = "Error cherry-picking: %s\nWould you like to manually fix-up
this merge?" % e
continue_maybe(msg, True)
@@ -686,11 +750,11 @@ def _do_cherry_pick(pr_num, merge_hash, pick_ref):
)
try:
- run_cmd("git push %s %s:%s" % (PUSH_REMOTE_NAME, pick_branch_name,
pick_ref))
+ git.push(PUSH_REMOTE_NAME, pick_branch_name, pick_ref)
except Exception as e:
fail("Exception while pushing: %s" % e)
- pick_hash = run_cmd("git rev-parse %s" % pick_branch_name).strip()
+ pick_hash = git.run("git rev-parse %s" % pick_branch_name).strip()
clean_up()
print("Pull request #%s picked into %s!" % (pr_num, pick_ref))
@@ -709,7 +773,7 @@ def branches_with_merge_footer(pr_num, branch_names):
"""
try:
landed = _branches_with_merge_footer(
- pr_num, PUSH_REMOTE_NAME, lambda args: run_cmd(["git"] + args)
+ pr_num, PUSH_REMOTE_NAME, lambda args: git.run(["git"] + args)
)
except Exception as e:
print_error("Could not scan for existing backports of #%s (%s)." %
(pr_num, e))
@@ -1112,11 +1176,7 @@ def reconcile_jira_components(issue, title_components):
# Append the PR title's components, keeping the existing ones first.
new_names = list(dict.fromkeys(current + title_jira_components))
- try:
- issue.update(fields={"components": [{"name": n} for n in new_names]})
- print("Updated JIRA %s components to: %s" % (issue.key, ",
".join(new_names)))
- except Exception as e:
- print_error("Failed to update components on JIRA %s: %s" % (issue.key,
e))
+ jira_ops.update_components(issue, new_names)
def get_jira_issue(prompt, default_jira_id=""):
@@ -1241,37 +1301,17 @@ def resolve_jira_issue(
if not jira_fix_versions:
print("No new fix versions selected for JIRA issue %s; no update
needed." % issue.key)
return
- issue.update(
- fields={"fixVersions": [v.raw for v in existing_fix_versions] +
jira_fix_versions}
- )
- try:
- print_jira_issue_summary(asf_jira.issue(issue.key))
- except Exception:
- print("Unable to fetch JIRA issue %s after updating fix versions"
% issue.key)
- print(
- "Successfully updated %s with additional fixVersions=%s!"
- % (issue.key, [v["name"] for v in jira_fix_versions])
- )
+ jira_ops.add_fix_versions(issue, existing_fix_versions,
jira_fix_versions)
return
resolve = list(filter(lambda a: a["name"] == "Resolve Issue",
asf_jira.transitions(issue.key)))[
0
]
resolution = list(filter(lambda r: r.raw["name"] == "Fixed",
asf_jira.resolutions()))[0]
- asf_jira.transition_issue(
- issue.key,
- resolve["id"],
- fixVersions=jira_fix_versions,
- comment=comment,
- resolution={"id": resolution.raw["id"]},
+ jira_ops.resolve_issue(
+ issue, resolve["id"], jira_fix_versions, comment,
resolution.raw["id"], fix_versions
)
- try:
- print_jira_issue_summary(asf_jira.issue(issue.key))
- except Exception:
- print("Unable to fetch JIRA issue %s after resolving" % issue.key)
- print("Successfully resolved %s with fixVersions=%s!" % (issue.key,
fix_versions))
-
def choose_jira_assignee(issue):
"""
@@ -1306,7 +1346,7 @@ def choose_jira_assignee(issue):
# assume it's a user id, and try to assign (might fail, we
just prompt again)
assignee = asf_jira.user(raw_assignee)
try:
- assign_issue(issue.key, assignee.name)
+ jira_ops.assign(issue.key, assignee.name)
except Exception as e:
if (
e.__class__.__name__ == "JIRAError"
@@ -1317,8 +1357,8 @@ def choose_jira_assignee(issue):
"User '%s' cannot be assigned, add to contributors
role and try again?"
% assignee.name
)
- grant_contributor_role(assignee.name)
- assign_issue(issue.key, assignee.name)
+ jira_ops.grant_contributor(assignee.name)
+ jira_ops.assign(issue.key, assignee.name)
else:
raise e
return assignee
@@ -1329,23 +1369,101 @@ def choose_jira_assignee(issue):
print("Error assigning JIRA, try again (or leave blank and fix
manually)")
-def grant_contributor_role(user: str):
- role = asf_jira.project_role("SPARK", 10010)
- role.add_user(user)
- print("Successfully added user '%s' to contributors role" % user)
-
+class Jira:
+ """ASF JIRA writes used by the merge script -- the single seam for JIRA
mutations.
-def assign_issue(issue: int, assignee: str) -> bool:
- """
- Assign an issue to a user, which is a shorthand for
jira.client.JIRA.assign_issue.
- The original one has an issue that it will search users again and only
choose the assignee
- from 20 candidates. If it's unmatched, it picks the head blindly. In our
case, the assignee
- is already resolved.
+ The pre-write JIRA lookups stay on the module-level asf_jira client; the
writes (components,
+ fix versions, the resolve transition, assignment, and the contributor-role
grant) go through
+ here, so DryRunJira can log them. Two Production writes (add_fix_versions,
resolve_issue) also
+ read the issue back afterward to print the updated summary; DryRunJira
skips the write and that
+ read-back alike. main() builds Jira(asf_jira) normally and
DryRunJira(asf_jira) for a dry run,
+ after initialize_jira() sets asf_jira.
"""
- url = getattr(asf_jira, "_get_latest_url")(f"issue/{issue}/assignee")
- payload = {"name": assignee}
- getattr(asf_jira, "_session").put(url, data=json.dumps(payload))
- return True
+
+ def __init__(self, client):
+ self._client = client
+
+ def update_components(self, issue, new_names):
+ try:
+ issue.update(fields={"components": [{"name": n} for n in
new_names]})
+ print("Updated JIRA %s components to: %s" % (issue.key, ",
".join(new_names)))
+ except Exception as e:
+ print_error("Failed to update components on JIRA %s: %s" %
(issue.key, e))
+
+ def add_fix_versions(self, issue, existing_fix_versions,
new_version_jsons):
+ issue.update(
+ fields={"fixVersions": [v.raw for v in existing_fix_versions] +
new_version_jsons}
+ )
+ try:
+ print_jira_issue_summary(self._client.issue(issue.key))
+ except Exception:
+ print("Unable to fetch JIRA issue %s after updating fix versions"
% issue.key)
+ print(
+ "Successfully updated %s with additional fixVersions=%s!"
+ % (issue.key, [v["name"] for v in new_version_jsons])
+ )
+
+ def resolve_issue(
+ self, issue, resolve_id, fix_version_jsons, comment, resolution_id,
fix_version_names
+ ):
+ self._client.transition_issue(
+ issue.key,
+ resolve_id,
+ fixVersions=fix_version_jsons,
+ comment=comment,
+ resolution={"id": resolution_id},
+ )
+ try:
+ print_jira_issue_summary(self._client.issue(issue.key))
+ except Exception:
+ print("Unable to fetch JIRA issue %s after resolving" % issue.key)
+ print("Successfully resolved %s with fixVersions=%s!" % (issue.key,
fix_version_names))
+
+ def assign(self, issue_key, assignee):
+ # Shorthand for jira.client.JIRA.assign_issue. The library's own
assign_issue re-searches
+ # users and blindly picks the first of 20 candidates when unmatched;
here the assignee is
+ # already resolved, so PUT it directly.
+ url = getattr(self._client,
"_get_latest_url")(f"issue/{issue_key}/assignee")
+ getattr(self._client, "_session").put(url, data=json.dumps({"name":
assignee}))
+ return True
+
+ def grant_contributor(self, user):
+ role = self._client.project_role("SPARK", 10010)
+ role.add_user(user)
+ print("Successfully added user '%s' to contributors role" % user)
+
+
+class DryRunJira(Jira):
+ """Logs the intended JIRA writes instead of calling the API; reads still
go through."""
+
+ def update_components(self, issue, new_names):
+ print("DRY-RUN: would set JIRA %s components to: %s" % (issue.key, ",
".join(new_names)))
+
+ def add_fix_versions(self, issue, existing_fix_versions,
new_version_jsons):
+ print(
+ "DRY-RUN: would add fixVersions=%s to JIRA %s."
+ % ([v["name"] for v in new_version_jsons], issue.key)
+ )
+
+ def resolve_issue(
+ self, issue, resolve_id, fix_version_jsons, comment, resolution_id,
fix_version_names
+ ):
+ print(
+ "DRY-RUN: would resolve JIRA %s as Fixed with fixVersions=%s and
add comment:\n%s"
+ % (issue.key, fix_version_names, comment)
+ )
+
+ def assign(self, issue_key, assignee):
+ print("DRY-RUN: would assign JIRA %s to '%s'." % (issue_key, assignee))
+ return True
+
+ def grant_contributor(self, user):
+ print("DRY-RUN: would add user '%s' to the SPARK contributors role." %
user)
+
+
+# Set in main() after initialize_jira(): Jira(asf_jira) normally,
DryRunJira(asf_jira) for a dry
+# run. None until then; only the JIRA-write flow, well after construction,
uses it.
+jira_ops = None
def resolve_jira_issues(title, merge_branches, comment, title_components=()):
@@ -1671,10 +1789,10 @@ def prompt_for_components():
def get_current_ref():
- ref = run_cmd("git rev-parse --abbrev-ref HEAD").strip()
+ ref = git.run("git rev-parse --abbrev-ref HEAD").strip()
if ref == "HEAD":
# The current ref is a detached HEAD, so grab its SHA.
- return run_cmd("git rev-parse HEAD").strip()
+ return git.run("git rev-parse HEAD").strip()
else:
return ref
@@ -1778,10 +1896,49 @@ def check_script_up_to_date():
)
+def build_arg_parser():
+ """CLI parser: an optional PR number plus the --dry-run/-n flag.
+
+ --dry-run defaults to on when the DRY_RUN env var is set to any non-empty
value (see the
+ DRY_RUN_ENV note near the top of this file), so either the flag or the env
var enables a
+ dry run; passing the flag forces it on regardless.
+ """
+ parser = argparse.ArgumentParser(description="Merge a Spark pull request.")
+ parser.add_argument(
+ "pr_num",
+ nargs="?",
+ help="Pull request number to merge; prompted for interactively if
omitted.",
+ )
+ parser.add_argument(
+ "--dry-run",
+ "-n",
+ action="store_true",
+ default=DRY_RUN_ENV,
+ help="Run every read-only step for real but suppress all outbound
effects (git push, "
+ "GitHub PR close/comment, JIRA writes), logging each as a 'DRY-RUN:
would ...' line. "
+ "Also enabled by setting the DRY_RUN environment variable.",
+ )
+ return parser
+
+
def main():
+ global git, github, jira_ops, original_head
+ args = build_arg_parser().parse_args()
+ if args.dry_run:
+ # Swap the outbound clients for their logging-only variants;
everything else is unchanged.
+ git = DryRunGit()
+ github = DryRunGitHub()
+ print(
+ bold(
+ "=== DRY-RUN: read-only steps run for real, but the git push
to %s, the GitHub "
+ "PR close/comment, and all JIRA writes are suppressed and only
logged. ==="
+ % PUSH_REMOTE_NAME
+ )
+ )
+
check_script_up_to_date()
initialize_jira()
- global original_head
+ jira_ops = DryRunJira(asf_jira) if args.dry_run else Jira(asf_jira)
os.chdir(SPARK_HOME)
original_head = get_current_ref()
@@ -1790,10 +1947,10 @@ def main():
branch_names = list(filter(lambda x: x.startswith("branch-"), [x["name"]
for x in branches]))
branch_names = sorted(branch_names, key=semver_branch_rank, reverse=True)
- if len(sys.argv) == 1:
+ if args.pr_num is None:
pr_num = get_input("Which pull request would you like to merge? (e.g.
34): ", r"^\d+$")
else:
- pr_num = sys.argv[1]
+ pr_num = args.pr_num
print("Start to merge pull request #%s" % (pr_num))
pr = get_json("%s/pulls/%s" % (GITHUB_API_BASE, pr_num))
pr_events = get_json("%s/issues/%s/events" % (GITHUB_API_BASE, pr_num))
@@ -1927,7 +2084,7 @@ def main():
if merge_hash is not None:
print("Pull request %s has already been merged, assuming you want to
backport" % pr_num)
commit_is_downloaded = (
- run_cmd(["git", "rev-parse", "--quiet", "--verify", "%s^{commit}"
% merge_hash]).strip()
+ git.run(["git", "rev-parse", "--quiet", "--verify", "%s^{commit}"
% merge_hash]).strip()
!= ""
)
if not commit_is_downloaded:
@@ -2085,7 +2242,7 @@ def main():
if pr_state != "closed":
close_message = "PR #%s is still open after push; closing it
explicitly." % pr_num
print("\n%s\n" % bold(close_message))
- close_pr(pr_num)
+ github.close_pr(pr_num)
# Record every branch that successfully received the change on the
PR.
post_merge_comment(pr_num, merged_commits)
# This is deliberately in the finally block: once the target branch
has been pushed,
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
