Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On 11/07/2021 08.55, Otto Kekäläinen wrote: Tested Roundcube + default-mysql-server upgrades with both apt and apt-get, and with and without --no-install-recommends in https://salsa.debian.org/mariadb-team/mariadb-10.5/-/commit/c3f53fe664517bbce7f8ae67244e018f9086534c All passed fine. the early upgrade of apt "broke" the test, using apt/buster "works fine", i.e. default-mysql-server gets removed. I'm not sure whether the updated apt resolver works differently, or if just the different set of installed packages makes the difference. And my dependency change in mariadb-server-10.5 does not change the outcome as long as the correspondig change is missing from galera-4 Andreas
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On Sat, Jul 10, 2021 at 12:37 AM Andreas Beckmann wrote: > > On 10/07/2021 09.16, Otto Kekäläinen wrote: > > Seems this issue only happens with apt – not when using apt-get. > > Yuck. Can you ask the apt developers whether this is intended? I always > expected apt-get and apt to be functionally equivalent (i.e. to always > take the same decisions when installing/upgrading/removing packages > since they use the same resolver in the background). > (If they are not, testing apt, too, means doubling the number of > piuparts tests...) Filed https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=990922 > Have you tried default-mysql-server + (roundcube-core (or roundcube) or > cacti)? This yields the unwanted behavior (removing default-mysql-server > + mariadb-server-10.3 without installing mariadb-server-10.5)) with > apt-get in my piuparts tests both with and without --install-recommends. > I can find more examples that only work with --install-recommends. Tested Roundcube + default-mysql-server upgrades with both apt and apt-get, and with and without --no-install-recommends in https://salsa.debian.org/mariadb-team/mariadb-10.5/-/commit/c3f53fe664517bbce7f8ae67244e018f9086534c All passed fine.
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On 10/07/2021 09.16, Otto Kekäläinen wrote: Seems this issue only happens with apt – not when using apt-get. Yuck. Can you ask the apt developers whether this is intended? I always expected apt-get and apt to be functionally equivalent (i.e. to always take the same decisions when installing/upgrading/removing packages since they use the same resolver in the background). (If they are not, testing apt, too, means doubling the number of piuparts tests...) Have you tried default-mysql-server + (roundcube-core (or roundcube) or cacti)? This yields the unwanted behavior (removing default-mysql-server + mariadb-server-10.3 without installing mariadb-server-10.5)) with apt-get in my piuparts tests both with and without --install-recommends. I can find more examples that only work with --install-recommends. Andreas
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
Hello! Seems this issue only happens with apt – not when using apt-get. Olaf's scenario with apt-get upgrades fine: https://salsa.debian.org/mariadb-team/mariadb-10.5/-/jobs/1748579 The exact same with apt-get -> apt fails: https://salsa.debian.org/mariadb-team/mariadb-10.5/-/jobs/1748579 https://salsa.debian.org/mariadb-team/mariadb-10.5/-/commit/c8ed387f91be4f7a98ccb768935a4725a9f10cd4
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On 09/07/2021 09.12, Otto Kekäläinen wrote: Weird. I renamed the test and extended it bit to have logging and be uniform in general, and now it passes (without your patch): https://salsa.debian.org/mariadb-team/mariadb-10.5/-/jobs/1748275 I was expecting that adding the new step in Salsa-CI would show failure, and then adding your patch would fix it. I'll try to switch from apt-get to apt and see if this problem occurs only with apt, but not apt-get. I'm trying to find again some of the failures I encountered in my piuparts instance... (they were all solved by testing with the patched mariadb-10.5 + galera-4 packages) You could try installing in buster default-mysql-server + roundcube-core (still looking for "easier" ones) Upgrading that to buster yields for me (even without --install-recommends) ... The following packages will be REMOVED: ***default-mysql-server*** libapache2-mod-php7.3 mariadb-client-10.3 mariadb-client-core-10.3 mariadb-server-10.3 php7.3-cli php7.3-common php7.3-intl php7.3-json php7.3-mbstring php7.3-mysql php7.3-opcache php7.3-readline php7.3-xml The following NEW packages will be installed: gcc-10-base libapache2-mod-php7.4 libapt-pkg6.0 libbpf0 libcrypt1 libffi7 libgcc-s1 libhogweed6 libicu67 liblua5.3-0 libmariadb3 libmd0 libnettle8 libnsl2 libonig5 libperl5.32 libreadline8 libtirpc-common libtirpc3 libxxhash0 logsave mailcap mariadb-client mariadb-client-10.5 mariadb-client-core-10.5 media-types perl-modules-5.32 php7.4-cli php7.4-common php7.4-intl php7.4-json php7.4-mbstring php7.4-mysql php7.4-opcache php7.4-readline php7.4-xml The following packages have been kept back: ***roundcube-core roundcube-mysql*** The following packages will be upgraded: ... Andreas
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On 09/07/2021 08.37, Andreas Beckmann wrote: On 09/07/2021 07.27, Otto Kekäläinen wrote: In the meantime you might want to think about the galera-4 branch 990708 because Salsa-CI shows it regressed: https://salsa.debian.org/mariadb-team/galera-4/-/jobs/1744367 That should solve itself once the mariadb fix gets uploaded. I've enabled problem resolver debugging for the upgrade job: Investigating (0) galera-4:amd64 < none -> 26.4.8-2+salsaci @un uN Ib > Broken galera-4:amd64 Breaks on galera-3:amd64 < 25.3.25-2 -> 25.3.33-1 @ii umU > (< 26.4) Considering galera-3:amd64 0 as a solution to galera-4:amd64 0 Holding Back galera-4:amd64 rather than change galera-3:amd64 Investigating (0) mariadb-server-10.5:amd64 < none -> 1:10.5.10-2 @un uN Ib > Broken mariadb-server-10.5:amd64 Depends on galera-4:amd64 < none | 26.4.8-2+salsaci @un uH > (>= 26.4) Considering galera-4:amd64 0 as a solution to mariadb-server-10.5:amd64 0 Holding Back mariadb-server-10.5:amd64 rather than change galera-4:amd64 Once the mariadb fix is available, the scores should move 1 point in favor of galera-4 (probably galera-3: -1, galera-4: 0) and this should be solved. Looks like the changelog bump was not needed, since there is the +salsaci suffix, but I wasn't sure beforehand which package version was going to be tested. Andreas
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
Weird. I renamed the test and extended it bit to have logging and be uniform in general, and now it passes (without your patch): https://salsa.debian.org/mariadb-team/mariadb-10.5/-/jobs/1748275 I was expecting that adding the new step in Salsa-CI would show failure, and then adding your patch would fix it. I'll try to switch from apt-get to apt and see if this problem occurs only with apt, but not apt-get.
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On 09/07/2021 06.50, Otto Kekäläinen wrote: ..but this easily gets too verbose. Ideally apt had a mode that if it fails, it would output a proper error message with a "trace", but in normal cases stays short and sweet not to clutter the logs too much. In the current scenario apt does not fail, it just takes a "suboptimal" decision: removing the package we are interested in. You would want a trace here, too. Do you read the successful logs? Does it really matter that they are "too verbose"? Similarily buildd logs should be as verbose as possible, s.t. there is hopefully enough information to investigate failures if the only thing being available is the logfile. Andreas
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On 09/07/2021 07.27, Otto Kekäläinen wrote: In the meantime you might want to think about the galera-4 branch 990708 because Salsa-CI shows it regressed: https://salsa.debian.org/mariadb-team/galera-4/-/jobs/1744367 That should solve itself once the mariadb fix gets uploaded. Andreas
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On Wed, Jul 7, 2021 at 6:37 AM Andreas Beckmann wrote: > > On 06/07/2021 01.25, Otto Kekäläinen wrote: > > Thanks Andreas for testing potential changes! > > So far this looks good, I see no more cases of default-mysql-server > being held at the buster version (and thus mariadb-server-10.3 instead > of mariadb-server-10.5 being installed). > > > If you file them as MRs at > > https://salsa.debian.org/mariadb-team/galera-4 and > > https://salsa.debian.org/mariadb-team/mariadb-10.5 you'll get the > > additional CI testing run on top of the for free for extra validation. > > you can find a 990708 branch in both repositories Thanks! I am currently working on the mariadb-10.5 repo branch 990708 to finalize it. I will force push a new version soon. In the meantime you might want to think about the galera-4 branch 990708 because Salsa-CI shows it regressed: https://salsa.debian.org/mariadb-team/galera-4/-/jobs/1744367
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
> BTW, the following apt option might be helpful for debugging upgrade > issues in your CI jobs: > >Debug::pkgProblemResolver "true"; In https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=988089 we went as far as running apt with: apt dist-upgrade -o Debug::pkgDepCache::AutoInstall=1 -o Debug::pkgDepCache::Marker=1 -o Debug::pkgProblemResolver=1 ..but this easily gets too verbose. Ideally apt had a mode that if it fails, it would output a proper error message with a "trace", but in normal cases stays short and sweet not to clutter the logs too much.
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
> > Some general understanding question: Would mariadb-server-10.5 work with > galera-3, too? (with only the Depends being relaxed to 'Depends: > galera-4 | galera-3, ...', without recompilation or similar actions) > (Bonus question: would mariadb-server-10.3/buster work with > galera-4/bullseye with only the dependencies similarily relaxed?) > That could allow for even easier solutions. > Galera 3 works with MariaDB 10.1-10.3. Galera 4 works with MariaDB 10.4-10.6. Galera 4 has new API, thus does not work with MariaDB 10.3. MariaDB needs Galera only if the feature is used. In future might change Depends to Suggests.
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On 07/07/2021 15.37, Andreas Beckmann wrote: you can find a 990708 branch in both repositories The galera-4 upgrade job failed, but I'd expect that to succeed once the mariadb-server-10.5 change is in as well. BTW, the following apt option might be helpful for debugging upgrade issues in your CI jobs: Debug::pkgProblemResolver "true"; Andreas
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On 06/07/2021 01.25, Otto Kekäläinen wrote: Thanks Andreas for testing potential changes! So far this looks good, I see no more cases of default-mysql-server being held at the buster version (and thus mariadb-server-10.3 instead of mariadb-server-10.5 being installed). If you file them as MRs at https://salsa.debian.org/mariadb-team/galera-4 and https://salsa.debian.org/mariadb-team/mariadb-10.5 you'll get the additional CI testing run on top of the for free for extra validation. you can find a 990708 branch in both repositories There is a mariadb-10.5 Bullseye update pending already, so this is excellent time to put in more changes as long as they are justified (these are). We also plan to make a mariadb-10.3 stable update to Buster, so putting a critical fix is also an option. I don't think anything needs to be fixed in stable. Some general understanding question: Would mariadb-server-10.5 work with galera-3, too? (with only the Depends being relaxed to 'Depends: galera-4 | galera-3, ...', without recompilation or similar actions) (Bonus question: would mariadb-server-10.3/buster work with galera-4/bullseye with only the dependencies similarily relaxed?) That could allow for even easier solutions. Andreas
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
Thanks Andreas for testing potential changes! If you file them as MRs at https://salsa.debian.org/mariadb-team/galera-4 and https://salsa.debian.org/mariadb-team/mariadb-10.5 you'll get the additional CI testing run on top of the for free for extra validation. There is a mariadb-10.5 Bullseye update pending already, so this is excellent time to put in more changes as long as they are justified (these are). We also plan to make a mariadb-10.3 stable update to Buster, so putting a critical fix is also an option.
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
> How would one use galera-3 in bullseye? There is no mariadb-10.3 in > buster, only 10.5 which depends on galera-4 which blocks galera-3 from > being installed. > (Even if galera-3 gets removed from bullseye, it will stay in buster > (and sid if you want)) One would use upstream repositories for MariaDB 10.3 on Bullseye (but get Galera from Debian proper). > What do you need the virtual galera package for? My gut feeling is that > this is the source of our problems because it forms some kind of a > conflicts cycle. Yes, we probably don't need the virtual package for Galera. I guess it has been around since the start and was used because all existing packaging at that time form upstream Codership and Percona had such a package, and the replacement had to offer it as well. I think that by now in 2021 we could probably drop it.
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
Control: tag -1 patch
attached are two patches for mariadb-10.5 and galera-4 that might solve
the upgrade issues, trying them now ...
Andreas
diff -Nru galera-4-26.4.8/debian/changelog galera-4-26.4.8/debian/changelog
--- galera-4-26.4.8/debian/changelog 2021-05-26 05:38:32.0 +0200
+++ galera-4-26.4.8/debian/changelog 2021-07-05 22:14:25.0 +0200
@@ -1,3 +1,10 @@
+galera-4 (26.4.8-2) UNRELEASED; urgency=medium
+
+ * Solve circular Conflicts with galera-3 by no longer providing a virtual
+galera package. (Closes: #990708)
+
+ -- Andreas Beckmann Mon, 05 Jul 2021 22:14:25 +0200
+
galera-4 (26.4.8-1) unstable; urgency=medium
* New upstream release 26.4.8
diff -Nru galera-4-26.4.8/debian/control galera-4-26.4.8/debian/control
--- galera-4-26.4.8/debian/control 2021-05-26 05:38:25.0 +0200
+++ galera-4-26.4.8/debian/control 2021-07-05 22:14:25.0 +0200
@@ -20,8 +20,7 @@
Section: libs
Depends: ${misc:Depends},
${shlibs:Depends}
-Conflicts: galera,
- galera-3,
+Conflicts:
garbd-2,
garbd-3,
garbd2,
@@ -34,13 +33,12 @@
percona-xtradb-cluster-galera-4.x,
percona-xtradb-cluster-garbd-2.x,
percona-xtradb-cluster-garbd-3.x
-Provides: galera,
+Provides:
galera4,
percona-xtradb-cluster-galera-26,
wsrep
-Breaks: galera,
-galera-3
-Replaces: galera
+Breaks: galera-3 (<< 26.4)
+Replaces: galera-3 (<< 26.4)
Description: Replication framework for transactional applications
Galera is a fast synchronous multimaster wsrep provider (replication engine)
for transactional databases and similar applications. For more information
diff -Nru mariadb-10.5-10.5.10/debian/changelog mariadb-10.5-10.5.10/debian/changelog
--- mariadb-10.5-10.5.10/debian/changelog 2021-05-24 06:04:38.0 +0200
+++ mariadb-10.5-10.5.10/debian/changelog 2021-07-05 09:53:32.0 +0200
@@ -1,3 +1,10 @@
+mariadb-10.5 (1:10.5.10-3) UNRELEASED; urgency=medium
+
+ * mariadb-server-10.5: Add Breaks: galera-3 (<< 26.4) to ease switching from
+galera-3 to galera-4 on upgrades from buster. (Closes: #990708)
+
+ -- Andreas Beckmann Mon, 05 Jul 2021 09:53:32 +0200
+
mariadb-10.5 (1:10.5.10-2) unstable; urgency=medium
* Bugfix: Revert upstream code change to fix armhf build (Closes: #988629)
diff -Nru mariadb-10.5-10.5.10/debian/control mariadb-10.5-10.5.10/debian/control
--- mariadb-10.5-10.5.10/debian/control 2021-05-21 07:09:56.0 +0200
+++ mariadb-10.5-10.5.10/debian/control 2021-07-05 09:53:32.0 +0200
@@ -339,7 +339,7 @@
Pre-Depends: adduser (>= 3.40),
debconf,
mariadb-common (>= ${source:Version})
-Depends: galera-4 (>=26.4),
+Depends: galera-4 (>= 26.4),
gawk,
iproute2 [linux-any],
libdbi-perl,
@@ -364,6 +364,7 @@
mysql-server-core-8.0,
virtual-mysql-server
Breaks: cqrlog (<< 1.9.0-5~),
+galera-3 (<< 26.4),
mariadb-galera-server,
mariadb-galera-server-10.0,
mariadb-galera-server-5.5,
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
On 05/07/2021 21.20, Otto Kekäläinen wrote: Hello! I was able to reproduce this, and addressed it in https://salsa.debian.org/mariadb-team/galera-4/-/commit/27a142792b5e02e66b8f1adf9a7b896c5722dd17 + added Salsa-CI testing for this specific scenario so that it would not regress again, but seems there is still some second scenario that does not upgrade properly. I wish somebody could contribute with exact steps on how to reproduce the issue. So far I've gotten some half attempts at that but they haven't been actionable for me. I'm currently looking at upgrading roundcube-core in a minmal chroot with --install-recommends *DISABLED* ... My plan was to keep Galera-3 in Debian stable / Ubuntu stable as for the duration that MariaDB 10.3 is supported upstream, as MariaDB does not offer Galera by themselves, so it is better to have it available in distro. How would one use galera-3 in bullseye? There is no mariadb-10.3 in buster, only 10.5 which depends on galera-4 which blocks galera-3 from being installed. (Even if galera-3 gets removed from bullseye, it will stay in buster (and sid if you want)) What do you need the virtual galera package for? My gut feeling is that this is the source of our problems because it forms some kind of a conflicts cycle. Andreas
Bug#990708: [debian-mysql] Bug#990708: mariadb-server-10.5: upgrade problems due to galera-3 -> galera-4 switch
Hello! I was able to reproduce this, and addressed it in https://salsa.debian.org/mariadb-team/galera-4/-/commit/27a142792b5e02e66b8f1adf9a7b896c5722dd17 + added Salsa-CI testing for this specific scenario so that it would not regress again, but seems there is still some second scenario that does not upgrade properly. I wish somebody could contribute with exact steps on how to reproduce the issue. So far I've gotten some half attempts at that but they haven't been actionable for me. My plan was to keep Galera-3 in Debian stable / Ubuntu stable as for the duration that MariaDB 10.3 is supported upstream, as MariaDB does not offer Galera by themselves, so it is better to have it available in distro. - Otto
