This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new 21209b0d1369 Add "maintenance tasks & cherry-picking" guide
21209b0d1369 is described below
commit 21209b0d1369206170137ab5a33c16047a14c5c5
Author: Lari Hotari <[email protected]>
AuthorDate: Tue Oct 8 13:20:14 2024 +0300
Add "maintenance tasks & cherry-picking" guide
---
contribute/maintenance-process.md | 83 +++++++++++++++++++++++++++++++++++++++
contribute/release-process.md | 52 +-----------------------
sidebarsDevelopment.js | 1 +
3 files changed, 85 insertions(+), 51 deletions(-)
diff --git a/contribute/maintenance-process.md
b/contribute/maintenance-process.md
new file mode 100644
index 000000000000..b1c33fe6b2f2
--- /dev/null
+++ b/contribute/maintenance-process.md
@@ -0,0 +1,83 @@
+---
+id: maintenance-process
+title: Cherry-picking
+---
+
+The following maintenance tasks are performed by Apache Pulsar committers
since they have access to these tasks.
+
+## Labeling PRs for Release Branches
+
+Before merging a PR, please ensure you label the releases where the PR should
also be applied. Our current process involves adding release labels for the
next pending releases so that we also cherry-pick fixes to maintenance branches.
+
+Always label non-breaking fixes to the supported maintenance version branches.
Check the supported versions to find out the currently supported branches and
current versions. Pick the label for the next release. If the label is missing
in GitHub, you should first add it.
+
+For example, when the last release is `3.0.7` for `branch-3.0`, the respective
PR label for the next release is `release/3.0.8`.
+
+These labels are crucial for searching PRs that need to be cherry-picked
during the release process. Picking an outdated release label could cause the
PR to be missed.
+
+## Cherry-picking Process
+
+### Current Process and Responsibilities
+
+It's sufficient to add the labels while merging PRs. This is the expectation
for the Apache Pulsar committer who merges a PR. A PR shouldn't be merged
without checking that it contains the necessary labels.
+
+Committers actively working as release managers will handle cherry-picking and
backporting.
+Release managers will request specific backports when a PR cannot be easily
backported due to merge conflicts.
+
+### Cherry-picking
+
+Cherry-picks should be performed in the same order as the PRs were merged into
the master branch.
+This is necessary to reduce unnecessary merge conflicts in maintenance
branches.
+Cherry-picking should retain a reference to the original commit. This can be
achieved by passing the `-x` argument so that cherry-picks are performed with
the `git cherry-pick -x <commit hash>` command.
+
+#### Handling Merge Conflicts
+
+Merge conflicts are common, and it's necessary to understand the tooling and
process to handle conflict resolution. Please refer to suggestions in [setting
up git with proper merge & diff tools for maintaining Pulsar](setup-git.md).
+
+In cases where a PR cannot be applied without substantial backporting effort
or risk of breaking changes, a separate PR to the maintenance branch is
requested from the original PR author. If the author is not willing to do this
work, the release manager attempts to find a volunteer to handle this.
+
+In some cases, a large number of merge conflicts signal that there's a
dependent PR that is also needed in the maintenance branch which hasn't been
cherry-picked. It is useful to review the dependency and consider
cherry-picking it. Only non-breaking bug fixes or minor improvements (excluding
PIP-related changes) can be cherry-picked without discussion on the dev mailing
list. Backporting is necessary when a fix depends on newer PIP-related changes.
+
+#### Cherry-picking Changes Scheduled for the Release
+
+Before proceeding, ensure that you have [set up a Git
mergetool](setup-git.md#mergetool). This tool is essential for resolving merge
conflicts that may arise during the cherry-picking process.
+
+Use a search such as `is:merged is:pr label:release/3.0.8
-label:cherry-picked/branch-3.0` to search for merged PRs that are scheduled
for the release but haven't yet been cherry-picked.
+It is necessary to handle cherry-picks in the same order as they have been
merged into the master branch. Otherwise, there will be unnecessary merge
conflicts to resolve.
+
+#### Cherry-picking CLI tooling
+
+Here's a shell script where the output that will ease cherry-picking from
master branch:
+assumes `gawk` is gnu awk. install `brew install gawk` or `alias gawk=awk` on
Linux.
+
+```shell
+UPSTREAM=origin
+git fetch $UPSTREAM
+RELEASE_NUMBER=3.0.8
+RELEASE_BRANCH=branch-3.0
+PR_QUERY="is:merged label:release/$RELEASE_NUMBER
-label:cherry-picked/$RELEASE_BRANCH"
+PR_NUMBERS=$(gh pr list -L 100 --search "$PR_QUERY" --json number --jq
'["#"+(.[].number|tostring)] | join("|")')
+ALREADY_PICKED=$(git log --oneline -P --grep="$PR_NUMBERS" --reverse
$RELEASE_BRANCH | gawk 'match($0, /\(#([0-9]+)\)/, a) {print substr(a[0], 2,
length(a[0])-2)}' | tr '\n' '|' | sed 's/|$//')
+if [[ -n "$ALREADY_PICKED" ]]; then
+ echo "** Already picked but not tagged as cherry-picked **"
+ git log --color --oneline -P --grep="$PR_NUMBERS" --reverse $RELEASE_BRANCH
| gawk 'match($0, /\(#([0-9]+)\)/, a) {print $0 "
https://github.com/apache/pulsar/pull/"; substr(a[0], 3, length(a[0])-3)}'
+fi
+echo "** Not cherry-picked from $UPSTREAM/master **"
+git log --color --oneline -P --grep="$PR_NUMBERS" --reverse $UPSTREAM/master |
{ [ -n "$ALREADY_PICKED" ] && grep --color -v -E "$ALREADY_PICKED" || cat; } |
gawk 'match($0, /\(#([0-9]+)\)/, a) {print $0 "
https://github.com/apache/pulsar/pull/"; substr(a[0], 3, length(a[0])-3)}'
+echo "Check https://github.com/apache/pulsar/pulls?q=is:pr+$(echo "$PR_QUERY"
| tr ' ' '+')"
+```
+
+this produces an output such as:
+
+```shell
+0fa9572f8af [fix][broker] Fix AvgShedder strategy check (#23156)
https://github.com/apache/pulsar/pull/23156
+** Not cherry-picked from origin/master **
+806fdf86866 [improve][misc] Upgrade Jetty to 9.4.56.v20240826 (#23405)
https://github.com/apache/pulsar/pull/23405
+Check
https://github.com/apache/pulsar/pulls?q=is:pr+is:merged+label:release/3.0.8+-label:cherry-picked/branch-3.0
+```
+
+It will speed up cherry-picking since you commit ids are there and there's
also links to the PRs.
+A cherry-pick should be done in this order with `git cherry-pick -x COMMIT_ID`.
+It's possible that some dependent commits are necessary to be cherry-picked
when you encounter a lot of merge conflicts in a case where they aren't
expected.
+
+There are more advanced versions of the cherry-picking CLI tooling in
[lhotari's `pulsar-contributor-toolbox-functions.sh` shell script function
library](https://github.com/lhotari/pulsar-contributor-toolbox/blob/0272419e70a9fc5f14945717bac964bda355520b/functions/pulsar-contributor-toolbox-functions.sh#L1393-L1455).
\ No newline at end of file
diff --git a/contribute/release-process.md b/contribute/release-process.md
index 061f0991f922..2103f7e6d892 100644
--- a/contribute/release-process.md
+++ b/contribute/release-process.md
@@ -160,57 +160,7 @@ If you created a new branch, update the [CI - OWASP
Dependency Check](https://gi
### Cherry-picking changes scheduled for the release
-Before proceeding, ensure that you have [set up a Git
mergetool](setup-git.md#mergetool). This tool is essential for resolving merge
conflicts that may arise during the cherry-picking process.
-
-Use a search such as `is:merged is:pr label:release/3.0.3
-label:cherry-picked/branch-3.0` to search for merged PRs that are scheduled
for the release, but haven't yet been cherry-picked.
-It is necessary to handle cherry-picks in the same order as they have been
merged in the master branch. Otherwise there will be unnecessary merge
conflicts to resolve.
-
-Here's a shell script where the output that will ease cherry-picking from
master branch:
-assumes `gawk` is gnu awk. install `brew install gawk` or `alias gawk=awk` on
Linux.
-```
-UPSTREAM=origin
-git fetch $UPSTREAM
-RELEASE_NUMBER=3.0.3
-RELEASE_BRANCH=branch-3.0
-PR_QUERY="is:merged label:release/$RELEASE_NUMBER
-label:cherry-picked/$RELEASE_BRANCH"
-PR_NUMBERS=$(gh pr list -L 100 --search "$PR_QUERY" --json number --jq
'["#"+(.[].number|tostring)] | join("|")')
-ALREADY_PICKED=$(git log --oneline -P --grep="$PR_NUMBERS" --reverse
$RELEASE_BRANCH | gawk 'match($0, /\(#([0-9]+)\)/, a) {print substr(a[0], 2,
length(a[0])-2)}' | tr '\n' '|' | sed 's/|$//')
-if [[ -n "$ALREADY_PICKED" ]]; then
- echo "** Already picked but not tagged as cherry-picked **"
- git log --color --oneline -P --grep="$PR_NUMBERS" --reverse $RELEASE_BRANCH
| gawk 'match($0, /\(#([0-9]+)\)/, a) {print $0 "
https://github.com/apache/pulsar/pull/"; substr(a[0], 3, length(a[0])-3)}'
-fi
-echo "** Not cherry-picked from $UPSTREAM/master **"
-git log --color --oneline -P --grep="$PR_NUMBERS" --reverse $UPSTREAM/master |
{ [ -n "$ALREADY_PICKED" ] && grep --color -v -E "$ALREADY_PICKED" || cat; } |
gawk 'match($0, /\(#([0-9]+)\)/, a) {print $0 "
https://github.com/apache/pulsar/pull/"; substr(a[0], 3, length(a[0])-3)}'
-echo "Check https://github.com/apache/pulsar/pulls?q=is:pr+$(echo "$PR_QUERY"
| tr ' ' '+')"
-```
-
-this produces an output such as:
-```
-** Already picked but not tagged as cherry-picked **
-744b7af5fc4 [improve][broker] Support not retaining null-key message during
topic compaction (#21578) (#21662) https://github.com/apache/pulsar/pull/21578
-b41013ba45c [improve][broker] defer the ownership checks if the owner is
inactive (ExtensibleLoadManager) (#21857)
https://github.com/apache/pulsar/pull/21857
-a6fd517ee39 [improve][build] Add a default username in the image (#21695)
https://github.com/apache/pulsar/pull/21695
-bbf6ddf9244 [fix] [client] Do no retrying for error subscription not found
when disabled allowAutoSubscriptionCreation (#22078)
https://github.com/apache/pulsar/pull/22078
-** Not cherry-picked from origin/master **
-ecd16d68e29 [fix][client] fix negative message re-delivery twice issue
(#20750) https://github.com/apache/pulsar/pull/20750
-50007c343ad [fix][txn] Fix getting last message ID when there are ongoing
transactions (#21466) https://github.com/apache/pulsar/pull/21466
-e81a20d667a [fix][broker] Avoid consumers receiving acknowledged messages from
compacted topic after reconnection (#21187)
https://github.com/apache/pulsar/pull/21187
-09559c5e661 [fix] [broker] Fix reader stuck when read from compacted topic
with read compact mode disable (#21969)
https://github.com/apache/pulsar/pull/21969
-48b4481969c [improve] [broker] Do not print an Error log when responding to
`HTTP-404` when calling `Admin API` and the topic does not exist. (#21995)
https://github.com/apache/pulsar/pull/21995
-861618a8120 [fix] [broker] Expire messages according to ledger close time to
avoid client clock skew (#21940) https://github.com/apache/pulsar/pull/21940
-48c7e322fec [improve][admin] Expose the offload threshold in seconds to the
amdin (#22101) https://github.com/apache/pulsar/pull/22101
-1c652f5519e [improve] [broker] Do not try to open ML when the topic meta does
not exist and do not expect to create a new one. #21995 (#22004)
https://github.com/apache/pulsar/pull/22004
-86079059890 [improve][broker] Cache the internal writer when sent to system
topic. (#22099) https://github.com/apache/pulsar/pull/22099
-1b1cfb58f4e [fix] [broker] Enabling batch causes negative unackedMessages due
to ack and delivery concurrency (#22090)
https://github.com/apache/pulsar/pull/22090
-0c49cac105e [fix] [client] fix huge permits if acked a half batched message
(#22091) https://github.com/apache/pulsar/pull/22091
-31ed115d0b5 [fix][sec] Add a check for the input time value (#22023)
https://github.com/apache/pulsar/pull/22023
-30134966a18 [fix][test] fix test testSyncNormalPositionWhenTBRecover (#22120)
https://github.com/apache/pulsar/pull/22120
-91de98ad456 [fix][test] Fix test testAsyncFunctionMaxPending (#22121)
https://github.com/apache/pulsar/pull/22121
-```
-
-It will speed up cherry-picking since you commit ids are there and there's
also links to the PRs.
-A cherry-pick should be done in this order with `git cherry-pick -x COMMIT_ID`.
-It's possible that some dependent commits are necessary to be cherry-picked
when you encounter a lot of merge conflicts in a case where they aren't
expected.
+Please read the [separate guide about maintenance tasks and
cherry-picking](maintenance-process.md).
### Update project version and tag
diff --git a/sidebarsDevelopment.js b/sidebarsDevelopment.js
index 2e7567ddb8ef..7f03176154e8 100644
--- a/sidebarsDevelopment.js
+++ b/sidebarsDevelopment.js
@@ -61,6 +61,7 @@ const sidebars = {
'release-note-guide',
]
},
+ 'maintenance-process',
'validate-release-candidate',
]
},
