bug#70456: Request for merging "core-updates" branch
Hi Chris and all,
Christopher Baines skribis:
> I think keeping the Git commit history clean and representative is
> really important, so to me at least this means core-updates can't be
> merged to master in it's current form, even if the changes overall from
> these 6351 commits are reasonable.
>
> I'm really not sure how to move forward though, I had a go at trying to
> rebuild the branch without introducing the thousands of duplicate
> commits and that produced a branch with 765 commits over master, which
> still seems a lot, but a big improvement over 6351:
>
> https://git.cbaines.net/guix/log/?h=chris-core-updates-no-duplicates-attempt
>
> That was really hard going though, as there's plenty of merge conflicts
> along the way, and I'm pretty sure I solved some of them
> incorrectly. The resulting branch also differs from core-updates.
Woow, impressive. How did you go about finding which commits were
duplicates/cherry-picked from master? Which commit did you start from?
Given everything you’ve explained, it seems to me it’s worth trying to
start from a clean branch like this.
I checked it out (commit da77ea23daa0bfa4a73290dff99b22d6825ff80b) to
get an idea of where we are and got this:
--8<---cut here---start->8---
make[2]: *** No rule to make target
'gnu/packages/patches/glib-networking-gnutls-binding.patch', needed by 'all-am'.
make[2]: *** No rule to make target
'gnu/packages/patches/librecad-support-for-boost-1.76.patch', needed by
'all-am'.
--8<---cut here---end--->8---
It stopped at:
--8<---cut here---start->8---
gnu/packages/sdl.scm:72:2: error: (package (name "sdl2") (version "2.30.1")
(source (origin (method url-fetch) (uri (string-append
"https://libsdl.org/release/SDL2-; version ".tar.gz")) (sha256 (base32
"0fj7gxc7rlzzrafnx9nmf7ws3paxy583fmx7bcbavi6gr3xmy881" (arguments (list
#:tests? #f #:configure-flags (gexp (append (quote ("--disable-wayland-shared"
"--enable-video-kmsdrm" "--disable-kmsdrm-shared")) (quote
("--disable-alsa-shared" "--disable-pulseaudio-shared" "--disable-x11-shared"
"LDFLAGS=-lGL" #:make-flags (gexp (cons* (string-append
"LDFLAGS=-Wl,-rpath," (ungexp (this-package-input "eudev")) "/lib" ",-rpath,"
(ungexp (this-package-input "vulkan-loader")) "/lib") (quote ("V=1"))
(propagated-inputs (list libx11 libcap mesa)) (native-inputs (list pkg-config))
(inputs (list libxrandr glu alsa-lib pulseaudio dbus eudev glib ibus-minimal
libxkbcommon libxcursor vulkan-loader wayland wayland-protocols)) (outputs
(quote ("out" "debug"))) (synopsis "Cross platform game development library")
(description "Simple DirectMedia Layer is a cross-platform development library
designed to\nprovide low level access to audio, keyboard, mouse, joystick, and
graphics\nhardware.") (home-page "https://libsdl.org/;) (license
license:bsd-3)): missing field initializers (build-system)
--8<---cut here---end--->8---
I guess these are merge conflicts that weren’t correctly resolved.
This branch rewrites the entire ‘core-updates’ history. What about
rewriting starting from the first series of “duplicate” commits? That
should solve the immediate issue while keeping the “known good” history?
Thanks,
Ludo’.
bug#70456: Request for merging "core-updates" branch
emacs has a script gitmerge.el, it can skip some commit when merge with different merge rule (ours), maybe can make life easier: https://git.savannah.gnu.org/cgit/emacs.git/tree/admin/gitmerge.el https://git.savannah.gnu.org/cgit/emacs.git/tree/admin/notes/git-workflow -- 发自我的网易邮箱手机智能版
bug#70456: Request for merging "core-updates" branch
Hi Christopher, Christopher Baines writes: [...] > Assuming we merge core-updates without doing anything about these > duplicate commits, and taking the cwltool package as a semi-random > example, if you do: > > git log -p gnu/packages/bioinformatics.scm I trust the 'newest' (appearing first in 'git log --grep='cwltool: Update') would yield the commit having substitutes? If so, the inconvenience is somewhat mitigated, as long as you know to use the newest of duplicated commits. -- Thanks, Maxim
bug#70456: Request for merging "core-updates" branch
Maxim Cournoyer writes: > Hi, > > Christopher Baines writes: > >> Christopher Baines writes: >> >>> I'm also really confused by what commits appear to be on the branch, >>> take 12b15585a75062f3fba09d82861c6fae9a7743b2 which appears to be one >>> core-updates, but it's a duplicate of >>> e2a7c227dea5b361e2ebdbba24b923d1922a79d0 which was pushed to >>> master. Same with this commit 28d14130953d868d4848540d9de8e1ae4a01a467, >>> which is different to f29f80c194d0c534a92354b2bc19022a9b70ecf8 on >>> master. >> >> I've worked out at least when these two werid commits turned up on >> core-updates. >> >> 12b15585a7 is mentioned here: >> https://lists.gnu.org/archive/html/guix-commits/2023-09/msg00955.html >> >> and 28d1413095 is mentioned here: >> https://lists.gnu.org/archive/html/guix-commits/2024-03/msg00381.html >> >> >> With the changes last month in March, I was going to suggest deleting >> the branch and then re-creating from f205179ed2 and trying to re-apply >> the changes that should be on core-updates, while avoiding any >> "duplicate" commits. However, I'm not even sure where to being with the >> ~5000 commits pushed in September, at least one of them is a duplicate >> of a commit on master, but I'm not sure how many of the other ~5000 are. >> >> For comparison, I did a merge of master in to core-updates today, and >> this is what it shows up like on guix-commits: >> >> https://lists.gnu.org/archive/html/guix-commits/2024-04/msg01209.html >> >> There are only two new revisions, the ed update I pushed, and the merge >> commit, which is what a merge should look like as far as I'm aware. > > I think probably what happened is that in the middle of a merge of > master -> core-updates (which entails sometimes painful conflicts > resolution), a new commit pushed to core-updates, and to be able to push > the resulting local branch (including the thousands of commits from the > merge commit) got rebased on the remote core-updates. > > Perhaps another merge commit appeared on the remote around the same > time, which would explain the duplicates. > > While I agree it's messy to have 5000 of duplicated commits, I'm not > sure attempting to rewrite the branch, which has seen a lot of original > commits, is a good idea (it'd be easy to have some good commits fall > into cracks, leading to lost of work). I think it's important to weigh up the cost and risks associated with either merging these commits, or somehow avoiding doing so. I think the potential impact is more than just a bit of messy Git history. Assuming we merge core-updates without doing anything about these duplicate commits, and taking the cwltool package as a semi-random example, if you do: git log -p gnu/packages/bioinformatics.scm You're going to see two commits for the update to 3.1.20240112164112, that's maybe confusing, but not a big issue I guess since they look the same, just different hashes. But say you're looking at the Git history because you want that specific version of cwltool and you're going to use guix time-machine or an inferior looking at that revision. Well, it's a lucky dip. If you pick the original master commit, you're in luck, you'll probably get substitutes for cwltool. But if you pick the other seemingly identical commit, you're effectively checking out core-updates as it was last month and the chance of substitutes is much less likely. I also can't really think how you'd work out which commit is best to use once core-updates is merged? The easiest way would probably be to check the signature, but that will only work most of the time. This isn't a new issue, it's already problematic for substitute availability to use intermediate commits (commits that weren't directly pointed to by master). But there are over 1000 packages who's versions are being changed on core-updates currently, or at least it looks like this because of the duplicate commits, and if I'm correct about how people are using the git history to find commits for specific versions of packages, then having these duplicates in the Git history for master forever more is going to catch people out for as long as those versions remain relevant. signature.asc Description: PGP signature
bug#70456: Request for merging core-updates branch
Let's see where we are with the branch currently. Thanks, Steve / Futurile
bug#70456: Request for merging "core-updates" branch
Hi, Christopher Baines writes: > Christopher Baines writes: > >> I'm also really confused by what commits appear to be on the branch, >> take 12b15585a75062f3fba09d82861c6fae9a7743b2 which appears to be one >> core-updates, but it's a duplicate of >> e2a7c227dea5b361e2ebdbba24b923d1922a79d0 which was pushed to >> master. Same with this commit 28d14130953d868d4848540d9de8e1ae4a01a467, >> which is different to f29f80c194d0c534a92354b2bc19022a9b70ecf8 on >> master. > > I've worked out at least when these two werid commits turned up on > core-updates. > > 12b15585a7 is mentioned here: > https://lists.gnu.org/archive/html/guix-commits/2023-09/msg00955.html > > and 28d1413095 is mentioned here: > https://lists.gnu.org/archive/html/guix-commits/2024-03/msg00381.html > > > With the changes last month in March, I was going to suggest deleting > the branch and then re-creating from f205179ed2 and trying to re-apply > the changes that should be on core-updates, while avoiding any > "duplicate" commits. However, I'm not even sure where to being with the > ~5000 commits pushed in September, at least one of them is a duplicate > of a commit on master, but I'm not sure how many of the other ~5000 are. > > For comparison, I did a merge of master in to core-updates today, and > this is what it shows up like on guix-commits: > > https://lists.gnu.org/archive/html/guix-commits/2024-04/msg01209.html > > There are only two new revisions, the ed update I pushed, and the merge > commit, which is what a merge should look like as far as I'm aware. I think probably what happened is that in the middle of a merge of master -> core-updates (which entails sometimes painful conflicts resolution), a new commit pushed to core-updates, and to be able to push the resulting local branch (including the thousands of commits from the merge commit) got rebased on the remote core-updates. Perhaps another merge commit appeared on the remote around the same time, which would explain the duplicates. While I agree it's messy to have 5000 of duplicated commits, I'm not sure attempting to rewrite the branch, which has seen a lot of original commits, is a good idea (it'd be easy to have some good commits fall into cracks, leading to lost of work). I'd rather we take this experience as a strong reminding that rebasing merge commits should be avoided at all costs (git already issues a warning, IIRC). As you suggested, the next time a situation like this happens (locally prepared merge commit with new commits made to the remote branch), merging the remote into the local branch is probably a nicer solution. -- Thanks, Maxim
bug#70456: Request for merging "core-updates" branch
Christopher Baines writes: > I'm also really confused by what commits appear to be on the branch, > take 12b15585a75062f3fba09d82861c6fae9a7743b2 which appears to be one > core-updates, but it's a duplicate of > e2a7c227dea5b361e2ebdbba24b923d1922a79d0 which was pushed to > master. Same with this commit 28d14130953d868d4848540d9de8e1ae4a01a467, > which is different to f29f80c194d0c534a92354b2bc19022a9b70ecf8 on > master. I've worked out at least when these two werid commits turned up on core-updates. 12b15585a7 is mentioned here: https://lists.gnu.org/archive/html/guix-commits/2023-09/msg00955.html and 28d1413095 is mentioned here: https://lists.gnu.org/archive/html/guix-commits/2024-03/msg00381.html With the changes last month in March, I was going to suggest deleting the branch and then re-creating from f205179ed2 and trying to re-apply the changes that should be on core-updates, while avoiding any "duplicate" commits. However, I'm not even sure where to being with the ~5000 commits pushed in September, at least one of them is a duplicate of a commit on master, but I'm not sure how many of the other ~5000 are. For comparison, I did a merge of master in to core-updates today, and this is what it shows up like on guix-commits: https://lists.gnu.org/archive/html/guix-commits/2024-04/msg01209.html There are only two new revisions, the ed update I pushed, and the merge commit, which is what a merge should look like as far as I'm aware. signature.asc Description: PGP signature
bug#70456: Request for merging "core-updates" branch
Hey, Thanks for raising this issue Steve, given the branch has been going for around 9 months (since [1]) now, I think it's well overdue to start looking at building and merging it. 1: https://lists.gnu.org/archive/html/guix-commits/2023-07/msg00332.html I pushed a single commit plus a merge from master today, and that was pretty difficult. There was plenty of conflicts, and I probably have resolved some wrongly, and there's potentially some things that Git didn't raise as conflicts but might have broken with merging in master. I'm also really confused by what commits appear to be on the branch, take 12b15585a75062f3fba09d82861c6fae9a7743b2 which appears to be one core-updates, but it's a duplicate of e2a7c227dea5b361e2ebdbba24b923d1922a79d0 which was pushed to master. Same with this commit 28d14130953d868d4848540d9de8e1ae4a01a467, which is different to f29f80c194d0c534a92354b2bc19022a9b70ecf8 on master. Putting aside the functional changes on core-updates, it's doesn't seem good to merge these seemingly duplicate commits on to master. I'm not sure how this happened though, or how to fix it. Chris signature.asc Description: PGP signature
