Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
On Fri, Nov 26, 2021 at 12:55:10AM +, Sam James wrote:
>
>
> > On 25 Nov 2021, at 17:07, Thomas Deutschmann wrote:
> >
> > On 2021-11-25 18:01, Piotr Karbowski wrote:
> >>> https://github.com/gentoo/gentoo/blob/master/sys-libs/glibc/glibc-2.34-r2.ebuild#L643
> >> Would you see something like this on more ebuilds, postgres, mysql,
> >> elasticsearch, or have proper FEATURE flag for it instead?
> >> It's all cool and giggles until you realize that even such random variable
> >> is not even prefixed with PORTAGE_ or anything, meaning it could be taken
> >> out of shell and meant for entirely different thing.
> >
> > Yeah, sounds like the I_KNOW_WHAT_I_AM_DOING thing which you end up having
> > enabled globally for various reasons.
>
> Just like updating in a cron job is a not-great idea, setting this globally
> and not per-package via /etc/portage/env sounds rather cavalier.
Generally I don't think there should be a central catch-all variable for
this, ebuilds should use their own unlikely variable as needed if using
this route until a better solution.
e.g. MARIADB_I_ALLOW_THE_DESTRUCTION_OF_MY_DATABASE=yes
Could take it further and require that =yes be =${PV} so that it only
works once if forgotten.
Then users setting this in not-so-great ways are still unlikely to be
bitten by it.
--
ionen
signature.asc
Description: PGP signature
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
> On 25 Nov 2021, at 17:07, Thomas Deutschmann wrote: > > On 2021-11-25 18:01, Piotr Karbowski wrote: >>> https://github.com/gentoo/gentoo/blob/master/sys-libs/glibc/glibc-2.34-r2.ebuild#L643 >> Would you see something like this on more ebuilds, postgres, mysql, >> elasticsearch, or have proper FEATURE flag for it instead? >> It's all cool and giggles until you realize that even such random variable >> is not even prefixed with PORTAGE_ or anything, meaning it could be taken >> out of shell and meant for entirely different thing. > > Yeah, sounds like the I_KNOW_WHAT_I_AM_DOING thing which you end up having > enabled globally for various reasons. Just like updating in a cron job is a not-great idea, setting this globally and not per-package via /etc/portage/env sounds rather cavalier. signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
> On 25 Nov 2021, at 03:21, Thomas Deutschmann wrote: > > Bug: https://bugs.gentoo.org/825234 > Signed-off-by: Thomas Deutschmann > --- > [snip] > +On 2021-11-21, a member of the QA project accidentially de-keyworded > +MariaDB 10.6 to address a file collision, users, who also had latest > +dev-db/mariadb-connector-c installed, experienced (NOTE: The default > +MySQL connector in Gentoo Linux is provided by > +dev-db/mysql-connector-c) [Link 1]. There's no reason to refer to MySQL here (nor am I sure that we're really "defaulting" to either MariaDB or MySQL). > + > +However, downgrades are not supported in MySQL/MariaDB [Link 2]. There's no reason to refer to MySQL here. > +Link 1: https://bugs.gentoo.org/825234#c8 No need to link to the comment. Also, please use the standard [0] and [1] reference notation. signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
> On 25 Nov 2021, at 13:59, Thomas Deutschmann wrote: > If you do a normal world upgrade, this is the default portage behavior, not? > I.e. package manager will downgrade if you don't stop. And especially on > servers, people tend to use cronjobs/scripts to do that... I would really not recommend doing this and I say this regularly. We expect users to: - read elog output; - read news items first; - read pkg_pretend / pkg_postinst output which may be critical and needed to be acted on soon (even if it's accompanied by a news item, although it isn't always). Should we try to improve that situation? Yeah, I think so, but that's really not the case right now. > And forcefully here refers to the undesirable result (at least that was my > intention). Something the user doesn't want. As we've discussed at length during the "Bitcoin consent" issue (wrt Taproot), users are expected to read and verify world upgrade output before proceeding. We have similar issues with e.g. new major glibc versions (you shouldn't do this unless you're in a position to restart at least several services). signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
> On 25 Nov 2021, at 13:37, Thomas Deutschmann wrote:
> For the records:
> I hope that this news item won't get delayed by the recent ComRel bug someone
> filled against me regarding this news item.
No need to reference that here. Please keep that to e.g. -core.
> While we often rush with news items and don't wait the 72h listed in GLEP 42,
> this one should go out as soon as possible.
> Why?
> Because every minute we wait will increase the chance that someone affected
> by this will be unable to recover. This is (for those not familiar with
> database engines) because bin logs are about to expire (getting overwritten)
> in typical setups after 3-5 days. And in case someone will learn about this
> not before next week and has to do a full restore, that user will lose about
> one week of data...
While it seems like the news item may be worth pursuing out of caution, I would
expect people operating sophisticated database setups to do at least one of:
- slot the DB version in their world file;
- mask newer versions and unmask when ready;
- upgrade all machines to the same version at the same time (for clustering);
- read the world upgrade output and not permit the downgrade?
I also echo the other requests to add a downgrade warning based on
${REPLACING_VERSIONS} given this is a
common issue with databases.
Also, while this does not affect the rationale for this news item, I would note
that I would expect
anybody using Gentoo in production to be using stable or at the very least to
have backups
of a server running on ~arch that they're upgrading without reading the
upgrades/downgrades
carefully for software they rely on.
thanks,
sam
signature.asc
Description: Message signed with OpenPGP
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
On Thu, 2021-11-25 at 18:01 +0100, Piotr Karbowski wrote: > > https://github.com/gentoo/gentoo/blob/master/sys-libs/glibc/glibc-2.34-r2.ebuild#L643 > > Would you see something like this on more ebuilds, postgres, mysql, > elasticsearch, or have proper FEATURE flag for it instead? I don't think there's any need for this with PostgreSQL. The major versions are slotted, similar to MariaDB, but the data itself is also installed into versioned locations. Migrating usually involves copying the data to the new location. signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
On Thu, Nov 25, 2021 at 12:01 PM Piotr Karbowski wrote: > > > https://github.com/gentoo/gentoo/blob/master/sys-libs/glibc/glibc-2.34-r2.ebuild#L643 > > Would you see something like this on more ebuilds, postgres, mysql, > elasticsearch, or have proper FEATURE flag for it instead? > > It's all cool and giggles until you realize that even such random > variable is not even prefixed with PORTAGE_ or anything, meaning it > could be taken out of shell and meant for entirely different thing. Another problem with such magic variables is that they are ineffective when installing binpkgs. During pkg_preinst, portage will restore the environment variables that were present when the package was built, and ignore anything you have set in the environment since then.
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
On 2021-11-25 18:01, Piotr Karbowski wrote: https://github.com/gentoo/gentoo/blob/master/sys-libs/glibc/glibc-2.34-r2.ebuild#L643 Would you see something like this on more ebuilds, postgres, mysql, elasticsearch, or have proper FEATURE flag for it instead? It's all cool and giggles until you realize that even such random variable is not even prefixed with PORTAGE_ or anything, meaning it could be taken out of shell and meant for entirely different thing. Yeah, sounds like the I_KNOW_WHAT_I_AM_DOING thing which you end up having enabled globally for various reasons. But thank you Pacho. This is indeed a good suggestion and we will have a look how we can improve (there is also some work happening upstream: dev-db/mysql-8+ for example removed mysql_upgrade command and will take care of everything by itself and prevent startup when it detects a downgrade and there is some discussion for such a feature for 10.7 (probably too late) or 10.8). -- Regards, Thomas Deutschmann / Gentoo Linux Developer fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
https://github.com/gentoo/gentoo/blob/master/sys-libs/glibc/glibc-2.34-r2.ebuild#L643 Would you see something like this on more ebuilds, postgres, mysql, elasticsearch, or have proper FEATURE flag for it instead? It's all cool and giggles until you realize that even such random variable is not even prefixed with PORTAGE_ or anything, meaning it could be taken out of shell and meant for entirely different thing. -- Piotr.
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
On Thu, 2021-11-25 at 17:21 +0100, Piotr Karbowski wrote: > Hi, > > On 25/11/2021 16.50, Pacho Ramos wrote: > > As a side note, maybe ebuild should add sanity checks (like those from > > glibc) to > > prevent downgrades, otherwise people could still accidentally hit the > > issue > > You might have valid use cases to downgrade mysql, if you are okay not > preserving data. I'd assume it's a common knowledge that downgrade of > any data store, be that database or likes like elasticsearch, will > corrupt the data. > > To make it actually useful I think we'd need new EAPI feature > 'downgrade-protection', that unless is explicit ignored like > --ignore-downgrade-protection mysql, it would prevent it from happening. > > -- Piotr. > https://github.com/gentoo/gentoo/blob/master/sys-libs/glibc/glibc-2.34-r2.ebuild#L643
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
Hi, On 25/11/2021 16.50, Pacho Ramos wrote: As a side note, maybe ebuild should add sanity checks (like those from glibc) to prevent downgrades, otherwise people could still accidentally hit the issue You might have valid use cases to downgrade mysql, if you are okay not preserving data. I'd assume it's a common knowledge that downgrade of any data store, be that database or likes like elasticsearch, will corrupt the data. To make it actually useful I think we'd need new EAPI feature 'downgrade-protection', that unless is explicit ignored like --ignore-downgrade-protection mysql, it would prevent it from happening. -- Piotr.
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
El jue, 25-11-2021 a las 09:20 -0500, Mike Gilbert escribió [...] > I don't like the phrase "forcefully downgraded" here. This implies > > > that something happened without the user's consent. emerge would have > > > informed them of the downgrade before it happened. I would suggest > > > removing the word "forcefully" from these paragraphs. > > > > If you do a normal world upgrade, this is the default portage behavior, > > not? I.e. package manager will downgrade if you don't stop. And > > especially on servers, people tend to use cronjobs/scripts to do that... > > Something happening by default is not the same as forcing it to happen. > > Using a cron job or other blind automation to do package upgrades on > any production system is a bad idea. We certainly do not recommend > that to people, nor force them to do it. > > As a side note, maybe ebuild should add sanity checks (like those from glibc) to prevent downgrades, otherwise people could still accidentally hit the issue Thanks signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
On Thu, Nov 25, 2021 at 8:59 AM Thomas Deutschmann wrote: > > Hi, > > On 2021-11-25 04:49, Mike Gilbert wrote: > > On 2021-11-21, keywords for dev-db/mariadb-10.6 were removed to > > address a file collision with dev-db/mariadb-connector-c. This > > unintentionally triggered a version downgrade for users who had > > successfully upgraded to dev-db/mariadb-10.6 already. > > Works for me. However, I would write dev-db/mariadb:10.6. Is that > acceptable for you? Sure. > > I don't like the phrase "forcefully downgraded" here. This implies > > that something happened without the user's consent. emerge would have > > informed them of the downgrade before it happened. I would suggest > > removing the word "forcefully" from these paragraphs. > > If you do a normal world upgrade, this is the default portage behavior, > not? I.e. package manager will downgrade if you don't stop. And > especially on servers, people tend to use cronjobs/scripts to do that... Something happening by default is not the same as forcing it to happen. Using a cron job or other blind automation to do package upgrades on any production system is a bad idea. We certainly do not recommend that to people, nor force them to do it. > And forcefully here refers to the undesirable result (at least that was > my intention). Something the user doesn't want. That's really not what "forcefully" means. It would be fine to use "unintentional" or "unwanted" in its place.
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
Hi, On 2021-11-25 04:49, Mike Gilbert wrote: On 2021-11-21, keywords for dev-db/mariadb-10.6 were removed to address a file collision with dev-db/mariadb-connector-c. This unintentionally triggered a version downgrade for users who had successfully upgraded to dev-db/mariadb-10.6 already. Works for me. However, I would write dev-db/mariadb:10.6. Is that acceptable for you? I don't like the phrase "forcefully downgraded" here. This implies that something happened without the user's consent. emerge would have informed them of the downgrade before it happened. I would suggest removing the word "forcefully" from these paragraphs. If you do a normal world upgrade, this is the default portage behavior, not? I.e. package manager will downgrade if you don't stop. And especially on servers, people tend to use cronjobs/scripts to do that... And forcefully here refers to the undesirable result (at least that was my intention). Something the user doesn't want. -- Regards, Thomas Deutschmann / Gentoo Linux Developer fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
On 25.11.2021 04:21, Thomas Deutschmann wrote:
> Bug: https://bugs.gentoo.org/825234
> Signed-off-by: Thomas Deutschmann
> ---
> ...adb-database-restore-maybe-required.en.txt | 46 +++
> 1 file changed, 46 insertions(+)
> create mode 100644
> 2021-11-23-mariadb-database-restore-maybe-required/2021-11-23-mariadb-database-restore-maybe-required.en.txt
>
> diff --git
> a/2021-11-23-mariadb-database-restore-maybe-required/2021-11-23-mariadb-database-restore-maybe-required.en.txt
>
> b/2021-11-23-mariadb-database-restore-maybe-required/2021-11-23-mariadb-database-restore-maybe-required.en.txt
> new file mode 100644
> index 000..c4a698e
> --- /dev/null
> +++
> b/2021-11-23-mariadb-database-restore-maybe-required/2021-11-23-mariadb-database-restore-maybe-required.en.txt
> @@ -0,0 +1,46 @@
> +Title: Full MariaDB database restore maybe required
> +Author: Thomas Deutschmann
> +Posted: 2021-11-23
> +Revision: 1
> +News-Item-Format: 2.0
> +Display-If-Installed: dev-db/mariadb
> +Display-If-Installed: sys-cluster/galera
> +
> +On 2021-11-21, a member of the QA project accidentially de-keyworded
Agreed with others in the thread, this is not professional.
> +MariaDB 10.6 to address a file collision, users, who also had latest
> +dev-db/mariadb-connector-c installed, experienced (NOTE: The default
> +MySQL connector in Gentoo Linux is provided by
> +dev-db/mysql-connector-c) [Link 1].
This ^ is a very confusing sentence and is way too long. Needs shortening and
re-wording, too much commas.
> +
> +However, downgrades are not supported in MySQL/MariaDB [Link 2].
> +
> +In case you already fully upgraded to MariaDB 10.6 (which includes
> +executing mysql_upgrade command) and forcefully downgraded your
> +MariaDB instance afterwards during the time window when keywords were
> +removed, you maybe experiencing different problems:
> +
> +At best, your forcefully downgraded MariaDB instance prevented startup
> +so all you have to do is upgrade to MariaDB 10.6 again to resume
> +services.
> +
> +In case previous MariaDB version was able to start, you are encouraged
> +to do a full backup as soon as possible using mysqldump command and
> +manually restore each database ("logical downgrade") to prevent any
> +data corruption.
> +
> +Depending on used feature set and from which version you upgraded,
> +it is maybe required to do a full restore from a previous backup before
> +MariaDB 10.6 upgrade to restore services and prevent any data loss or
> +future runtime errors.
> +
> +In case you are using MariaDB in a cluster and/or Galera setup you
> +probably have to rebuild the entire cluster in case the upgrade to
> +MariaDB 10.6 was already replicated (using pt-table-checksum from
> +dev-db/percona-toolkit can help you to validate your cluster).
> +
> +Keep in mind that due to the forced downgraded, point-in-time recovery
> +may not be available to the extent that you are used to.
> +
> +Link 1: https://bugs.gentoo.org/825234#c8
> +
> +Link 2:
> https://mariadb.com/kb/en/downgrading-between-major-versions-of-mariadb/
> --
> 2.34.0
>
>
--
Best regards,
Georgy
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
> On Thu, 25 Nov 2021, Mike Gilbert wrote: > On Wed, Nov 24, 2021 at 10:21 PM Thomas Deutschmann wrote: >> +On 2021-11-21, a member of the QA project accidentially de-keyworded >> +MariaDB 10.6 to address a file collision, users, who also had latest >> +dev-db/mariadb-connector-c installed, experienced (NOTE: The default >> +MySQL connector in Gentoo Linux is provided by >> +dev-db/mysql-connector-c) [Link 1]. > This sentence is very difficult to read. Also, I don't think it is > relevant to call out the mistake by the QA team in a news item > intended for end users. I would rewrite this as: > On 2021-11-21, keywords for dev-db/mariadb-10.6 were removed to > address a file collision with dev-db/mariadb-connector-c. This > unintentionally triggered a version downgrade for users who had > successfully upgraded to dev-db/mariadb-10.6 already. +1 News item are to provide information to users, and the details about who did what in the past aren't necessary here. At most, keep the link to bug 825234 for reference (but not to any specific comment). Ulrich signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
On 2021-11-25 04:49, Mike Gilbert wrote: Also, I don't think it is relevant to call out the mistake by the QA team in a news item intended for end users. My sentiment exactly. No finger pointing in news items, please. I don't like the phrase "forcefully downgraded" here. This implies that something happened without the user's consent. emerge would have informed them of the downgrade before it happened. I would suggest removing the word "forcefully" from these paragraphs. Again, I very much second this suggestion. -- Marecki OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] [PATCH] 2021-11-23-mariadb-database-restore-maybe-required: add item
On Wed, Nov 24, 2021 at 10:21 PM Thomas Deutschmann wrote: > +On 2021-11-21, a member of the QA project accidentially de-keyworded > +MariaDB 10.6 to address a file collision, users, who also had latest > +dev-db/mariadb-connector-c installed, experienced (NOTE: The default > +MySQL connector in Gentoo Linux is provided by > +dev-db/mysql-connector-c) [Link 1]. This sentence is very difficult to read. Also, I don't think it is relevant to call out the mistake by the QA team in a news item intended for end users. I would rewrite this as: On 2021-11-21, keywords for dev-db/mariadb-10.6 were removed to address a file collision with dev-db/mariadb-connector-c. This unintentionally triggered a version downgrade for users who had successfully upgraded to dev-db/mariadb-10.6 already. > +However, downgrades are not supported in MySQL/MariaDB [Link 2]. > + > +In case you already fully upgraded to MariaDB 10.6 (which includes > +executing mysql_upgrade command) and forcefully downgraded your > +MariaDB instance afterwards during the time window when keywords were > +removed, you maybe experiencing different problems: > + > +At best, your forcefully downgraded MariaDB instance prevented startup > +so all you have to do is upgrade to MariaDB 10.6 again to resume > +services. I don't like the phrase "forcefully downgraded" here. This implies that something happened without the user's consent. emerge would have informed them of the downgrade before it happened. I would suggest removing the word "forcefully" from these paragraphs.
