Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Fri, 2022-09-02 at 22:43 +0200, Fabio Valentini wrote: > On Thu, Sep 1, 2022 at 8:02 PM Demi Marie Obenour > wrote: > > > > On 9/1/22 13:47, Jaroslav Mracek wrote: > > > With loading or not loading file list it is not so easy or in general - > > > performance optimization is allways a trade one resource for another one > > > or for some features. DNF5 will provide a setting to not load file list, > > > as well to not create disk cache, or not using zchunk (already > > > implemented in DNF). How it can help? It allows to trade download for > > > disk space and CPU, or RAM for HDD, or RAM, HDD, CPU for features. And > > > each user prefers or requires different different things. > > > > > > Without file lists there will be not available some functionality: > > > 1 dnf install/repoquery/whatprovides > > /usr/(s)bin> => will not work > > > 2. resolve transaction can fail - keep in mind that people are not only > > > using packages from Fedora distribution. > > > > From my perspective, the main problem here is that DNF (possibly via > > libsolv) loads all metadata into memory, instead of only the parts > > it actually needs. This should not be necessary, and I consider it > > a significant limitation of DNF. > > I wonder, would it be better for memory-constrained systems to memmap > (and possibly uncompress on disk before doing that, if necessary) > repository data instead of loading it into memory directly? Or is the > limitation not the size of the repository data itself, but the > in-memory data structures that are generated *from* it? Before this goes too far into the weeds, please remember DNF is already being revised to do this a lot better. The numbers about that are in the original bug report. Whenever DNF 5 lands in Fedora, the problem should get a lot smaller. -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Thu, Sep 1, 2022 at 8:02 PM Demi Marie Obenour wrote: > > On 9/1/22 13:47, Jaroslav Mracek wrote: > > With loading or not loading file list it is not so easy or in general - > > performance optimization is allways a trade one resource for another one or > > for some features. DNF5 will provide a setting to not load file list, as > > well to not create disk cache, or not using zchunk (already implemented in > > DNF). How it can help? It allows to trade download for disk space and CPU, > > or RAM for HDD, or RAM, HDD, CPU for features. And each user prefers or > > requires different different things. > > > > Without file lists there will be not available some functionality: > > 1 dnf install/repoquery/whatprovides > > => will not work > > 2. resolve transaction can fail - keep in mind that people are not only > > using packages from Fedora distribution. > > From my perspective, the main problem here is that DNF (possibly via > libsolv) loads all metadata into memory, instead of only the parts > it actually needs. This should not be necessary, and I consider it > a significant limitation of DNF. I wonder, would it be better for memory-constrained systems to memmap (and possibly uncompress on disk before doing that, if necessary) repository data instead of loading it into memory directly? Or is the limitation not the size of the repository data itself, but the in-memory data structures that are generated *from* it? Fabio ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Thu, Sep 01, 2022 at 05:47:21PM -, Jaroslav Mracek wrote: > With loading or not loading file list it is not so easy or in general - > performance optimization is allways a trade one resource for another one or > for some features. DNF5 will provide a setting to not load file list, as well > to not create disk cache, or not using zchunk (already implemented in DNF). > How it can help? It allows to trade download for disk space and CPU, or RAM > for HDD, or RAM, HDD, CPU for features. And each user prefers or requires > different different things. > > Without file lists there will be not available some functionality: > 1 dnf install/repoquery/whatprovides > => will not work > 2. resolve transaction can fail - keep in mind that people are not only using > packages from Fedora distribution. Hmm, when this was discussed previously, the idea was that DNF would not load the extra metadata initially, but when in encounters a requirement for it, i.e. something refers to the paths outside of core list, it'd load the extra metadata, possibly downloading it, and restart the transaction. Having this predetermined by a config option doesn't sound nearly as nice… Zbyszek > For some users such a change can be a problem, for others not and they will > see benefits. DNF must provide a solution for all of them and DNF5 will but > there is only one default setting therefore some users will need to modify > dnf5 configuration to achieve what they need. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On 9/1/22 13:47, Jaroslav Mracek wrote: > With loading or not loading file list it is not so easy or in general - > performance optimization is allways a trade one resource for another one or > for some features. DNF5 will provide a setting to not load file list, as well > to not create disk cache, or not using zchunk (already implemented in DNF). > How it can help? It allows to trade download for disk space and CPU, or RAM > for HDD, or RAM, HDD, CPU for features. And each user prefers or requires > different different things. > > Without file lists there will be not available some functionality: > 1 dnf install/repoquery/whatprovides > => will not work > 2. resolve transaction can fail - keep in mind that people are not only using > packages from Fedora distribution. From my perspective, the main problem here is that DNF (possibly via libsolv) loads all metadata into memory, instead of only the parts it actually needs. This should not be necessary, and I consider it a significant limitation of DNF. -- Sincerely, Demi Marie Obenour (she/her/hers) ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
With loading or not loading file list it is not so easy or in general - performance optimization is allways a trade one resource for another one or for some features. DNF5 will provide a setting to not load file list, as well to not create disk cache, or not using zchunk (already implemented in DNF). How it can help? It allows to trade download for disk space and CPU, or RAM for HDD, or RAM, HDD, CPU for features. And each user prefers or requires different different things. Without file lists there will be not available some functionality: 1 dnf install/repoquery/whatprovides => will not work 2. resolve transaction can fail - keep in mind that people are not only using packages from Fedora distribution. For some users such a change can be a problem, for others not and they will see benefits. DNF must provide a solution for all of them and DNF5 will but there is only one default setting therefore some users will need to modify dnf5 configuration to achieve what they need. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Thu, Sep 1, 2022 at 8:51 AM Dan Čermák wrote: > > Casey Jao via devel writes: > > > Old rpm-ostree thread about this: > > https://github.com/coreos/rpm-ostree/issues/1127 > > > > 1. Debian provides a comparably sized package catalog using one-tenth the > > size of Fedora's metadata. Are there any lessons that Fedora can learn from > > Debian? > > 2. Does SUSE have the same problem? > > No, I don't think so. openSUSE and SUSE use zypper + libzypp instead of > dnf and iirc zypper is lighter on memory usage (but lacks the more "fancy" > features like `dnf repoquery`). > > Also, the openSUSE Tumbleweed repository is smaller than the Fedora > repository (and only contains the latest rebuilds, i.e. even less > metadata). I am not 100% sure about Leap though, as I have heard horror > stories about the size of the Leap Update repository size (the > repository contains every package build for the whole release)… Zypper is affected too, because the problem is at the libsolv level. However, it's harder for Zypper to hit it because there are fewer cases where all the metadata files need to be loaded and SUSE distribution repos are generally smaller (for reasons I don't want to get into here). DNF5 will improve the situation by only loading some metadata into the solver on-demand as it determines it is needed. -- 真実はいつも一つ！/ Always, there's only one truth! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
Casey Jao via devel writes: > Old rpm-ostree thread about this: > https://github.com/coreos/rpm-ostree/issues/1127 > > 1. Debian provides a comparably sized package catalog using one-tenth the > size of Fedora's metadata. Are there any lessons that Fedora can learn from > Debian? > 2. Does SUSE have the same problem? No, I don't think so. openSUSE and SUSE use zypper + libzypp instead of dnf and iirc zypper is lighter on memory usage (but lacks the more "fancy" features like `dnf repoquery`). Also, the openSUSE Tumbleweed repository is smaller than the Fedora repository (and only contains the latest rebuilds, i.e. even less metadata). I am not 100% sure about Leap though, as I have heard horror stories about the size of the Leap Update repository size (the repository contains every package build for the whole release)… ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
Old rpm-ostree thread about this: https://github.com/coreos/rpm-ostree/issues/1127 1. Debian provides a comparably sized package catalog using one-tenth the size of Fedora's metadata. Are there any lessons that Fedora can learn from Debian? 2. Does SUSE have the same problem? ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
> The reason this bug isn't hitting RHEL right now is simply just because the > default RHEL repositories are much smaller - also crucially things like many -devel packages are in a separate repository. RHEL actually is hitting this issue in different contexts for an entirely different reason (a bug that caused changelog metadata to inflate over time). https://bugzilla.redhat.com/show_bug.cgi?id=2040170 It is in the progress of being fixed. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On 30/08/2022 15:48, Chris Murphy wrote: The built-in default for cryptsetup on Fedora is LUKS2 which uses argon2id with parameters: Iteration time: 2000, Memory required: 1048576kB, Parallel threads: 4 This is maximum, it is decreased according to benchmark during format. It is possible to specify a different memory requirement at luksFormat time. But I'm not sure if there's any warning emitted by cryptsetup if there's significant memory pressure, i.e. high potential that a future Fedora might fail to open this volume. But yeah, if we have use cases where an encrypted image is created on a system or in a container with plenty of memory, thus the high memory requirement of the default applies, but then is later used with libguestfs in a container or VM with less memory available, it's a possible problem. I'm not really sure what the resolution for this is though. We'd either have to change the cryptsetup default to something like 512MB Fedora wide, or the burden is on the create time sysadmin to override the Fedora default. Please do not change defaults. If you prepare an image on some different system, you have to always think about setting parameters for the target. Cryptsetup will never use more that half of physical memory, it works perfectly fine with small memory systems like RPi - just you need to format it there (or decrease parameters if formatted elsewhere). Also you can use luksChangeKey later to decrease costs. For very limited systems it makes more sense to switch to older PBKDF2 anyway (losing some bruteforce resistance). What is the real problem with cryptsetup on systems with low memory is that it locks all used memory and that includes all loaded libraries - and this often kicks OOM killer. That will change in the next release where we removed that problematic mlockall() behavior. ... but the thread was about dnf and there it really does not make sense to eat such amount of memory... :) Milan ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Tue, Aug 30, 2022, at 9:43 AM, Chris Murphy wrote: > On Tue, Aug 30, 2022, at 8:27 AM, Richard W.M. Jones wrote: >> Another is that LUKS filesystem decryption uses a deliberately >> "memory-hard" algorithm called Argon2 which requires loads of RAM and >> sometimes has problems running on a machine with 1GB and no swap. > > The built-in default for cryptsetup on Fedora is LUKS2 which uses > argon2id with parameters: > Iteration time: 2000, Memory required: 1048576kB, Parallel threads: 4 > > It is possible to specify a different memory requirement at luksFormat > time. But I'm not sure if there's any warning emitted by cryptsetup if > there's significant memory pressure, i.e. high potential that a future > Fedora might fail to open this volume. But yeah, if we have use cases where an encrypted image is created on a system or in a container with plenty of memory, thus the high memory requirement of the default applies, but then is later used with libguestfs in a container or VM with less memory available, it's a possible problem. I'm not really sure what the resolution for this is though. We'd either have to change the cryptsetup default to something like 512MB Fedora wide, or the burden is on the create time sysadmin to override the Fedora default. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Tue, Aug 30, 2022, at 8:27 AM, Richard W.M. Jones wrote: > Another is that LUKS filesystem decryption uses a deliberately > "memory-hard" algorithm called Argon2 which requires loads of RAM and > sometimes has problems running on a machine with 1GB and no swap. The built-in default for cryptsetup on Fedora is LUKS2 which uses argon2id with parameters: Iteration time: 2000, Memory required: 1048576kB, Parallel threads: 4 It is possible to specify a different memory requirement at luksFormat time. But I'm not sure if there's any warning emitted by cryptsetup if there's significant memory pressure, i.e. high potential that a future Fedora might fail to open this volume. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
[I can't find the original message, so sorry about incorrect threading] This bug is another symptom of issues which make it hard to run Fedora in "small" systems with "only" 1GB of RAM. Another is that LUKS filesystem decryption uses a deliberately "memory-hard" algorithm called Argon2 which requires loads of RAM and sometimes has problems running on a machine with 1GB and no swap. Libguestfs is affected here because our appliance has limited RAM. So when we operate on disk images eg through virt-customize, some features like --install packages and fs decryption can fail unnecessarily. I do think we should block the Fedora release if basic features don't work on such machines. A machine with 1GB of RAM should even be able to run a full system with GUI, and certainly should be usable from the command line. (Insert statement here about how back in the day I ran multiuser Minix on a PC with 640K of RAM and a steady hand ...) Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On 29-08-2022 19:32, Chris Murphy wrote: On Mon, Aug 29, 2022, at 11:12 AM, Chris Murphy wrote: On Mon, Aug 29, 2022, at 11:10 AM, Adam Williamson wrote: There *is* a workaround, BTW - I didn't mention this in my original mail, and probably should have. At least according to discussion in the bug, microdnf works OK. So you can use that instead. Yes, but will you be able to install it? Yes you could go to koji, download the correct RPMs, and have rpm do it without dnf but... that'd be a pretty saucy work around. Can microdnf and dnf co-exist? If so, maybe the non-desktops should just include microdnf along side dnf? Or is this potentially a trap where microdnf can also fail for the same reason (i.e. it's not dnf, it's the size of the repo metadata)? Yes, booth can co-exist. It was proposed at the blocker review meeting[1] to ship microdnf by default for all spins aimed at potentially low memory machines. [1] https://meetbot-raw.fedoraproject.org//fedora-blocker-review/2022-08-29/f37-blocker-review.2022-08-29-16.01.txt -- Sandro ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On 8/22/22 20:44, Adam Williamson wrote: > Hey folks! I apologize for the wide distribution, but this seemed like > a bug it'd be appropriate to get a wide range of input on. > > There's a bug that was proposed as an F37 Beta blocker: > https://bugzilla.redhat.com/show_bug.cgi?id=1907030 > > it's quite an old bug, but up until recently, the summary was > apparently accurate - dnf would run out of memory with 512M of RAM, but > was OK with 1G. However, as of quite recently, on F36 at least (not > sure if anyone's explicitly tested F37), dnf operations are commonly > failing on VMs/containers with 1G of RAM due to running out of RAM and > getting OOM-killed. > > There's some discussion in the bug about what might be causing this and > potential ways to resolve it, and please do dig into/contribute to that > if you can, but the other question here I guess is: how much do we care > about this? How bad is it that you can't reliably run dnf operations on > top of a minimal Fedora environment with 1G of RAM? I think we need to stop thinking about this as just a single computer problem. In a containerized world you could have numerous containers running dnf operations on a machine at any given time. Making the dnf operations take less memory means we'll hit many fewer issues as single machines are able to run many dnf operations at a given moment. > > This obviously has some overlap with our stated hardware requirements, > so here they are for the record: > > https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/welcome/Hardware_Overview/ > > that specifies 2GB as the minimum memory for "the default > installation", by which I think it's referring to a default Workstation > install, though this should be clarified. But then there's a "Low > memory installations" boxout, which suggests that "users with less than > 768MB of system memory may have better results performing a minimal > install and adding to it afterward", which kinda is recommending that > people do exactly the thing that doesn't work (do a minimal install > then use dnf on it), and implying it'll work. > > After some consideration I don't think it makes sense to take this bug > as an F37 blocker, since it already affects F36, and that's what I'll > be suggesting at the next blocker review meeting. However, it does seem > a perfect candidate for prioritized bug status, and I've nominated it > for that. > > I guess if folks can chime in with thoughts here and/or in the bug > report, maybe a consensus will emerge on just how big of an issue this > is (and how likely it is to get fixed). There will presumably be a > FESCo ticket related to prioritized bug status too. > > Thanks folks! ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On 8/22/22 20:44, Adam Williamson wrote: > Hey folks! I apologize for the wide distribution, but this seemed like > a bug it'd be appropriate to get a wide range of input on. > > There's a bug that was proposed as an F37 Beta blocker: > https://bugzilla.redhat.com/show_bug.cgi?id=1907030 > > it's quite an old bug, but up until recently, the summary was > apparently accurate - dnf would run out of memory with 512M of RAM, but > was OK with 1G. However, as of quite recently, on F36 at least (not > sure if anyone's explicitly tested F37), dnf operations are commonly > failing on VMs/containers with 1G of RAM due to running out of RAM and > getting OOM-killed. > > There's some discussion in the bug about what might be causing this and > potential ways to resolve it, and please do dig into/contribute to that > if you can, but the other question here I guess is: how much do we care > about this? How bad is it that you can't reliably run dnf operations on > top of a minimal Fedora environment with 1G of RAM? > > This obviously has some overlap with our stated hardware requirements, > so here they are for the record: > > https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/welcome/Hardware_Overview/ > > that specifies 2GB as the minimum memory for "the default > installation", by which I think it's referring to a default Workstation > install, though this should be clarified. But then there's a "Low > memory installations" boxout, which suggests that "users with less than > 768MB of system memory may have better results performing a minimal > install and adding to it afterward", which kinda is recommending that > people do exactly the thing that doesn't work (do a minimal install > then use dnf on it), and implying it'll work. > > After some consideration I don't think it makes sense to take this bug > as an F37 blocker, since it already affects F36, and that's what I'll > be suggesting at the next blocker review meeting. However, it does seem > a perfect candidate for prioritized bug status, and I've nominated it > for that. > > I guess if folks can chime in with thoughts here and/or in the bug > report, maybe a consensus will emerge on just how big of an issue this > is (and how likely it is to get fixed). There will presumably be a > FESCo ticket related to prioritized bug status too. > > Thanks folks! This could be a problem for Qubes OS, so adding Marek in case he has something to say about the subject. Qubes OS does have 1G of swap by default, which might help. -- Sincerely, Demi Marie Obenour (she/her/hers) OpenPGP_0xB288B55FFF9C22C1.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Mon, Aug 29, 2022, at 11:12 AM, Chris Murphy wrote: > On Mon, Aug 29, 2022, at 11:10 AM, Adam Williamson wrote: >> There *is* a workaround, BTW - I didn't mention this in my original >> mail, and probably should have. At least according to discussion in the >> bug, microdnf works OK. So you can use that instead. > > Yes, but will you be able to install it? Yes you could go to koji, > download the correct RPMs, and have rpm do it without dnf but... that'd > be a pretty saucy work around. Can microdnf and dnf co-exist? If so, maybe the non-desktops should just include microdnf along side dnf? Or is this potentially a trap where microdnf can also fail for the same reason (i.e. it's not dnf, it's the size of the repo metadata)? -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
> On 29 Aug 2022, at 04:25, Adam Williamson wrote: > > Hey folks! I apologize for the wide distribution, but this seemed like > a bug it'd be appropriate to get a wide range of input on. > > There's a bug that was proposed as an F37 Beta blocker: > https://bugzilla.redhat.com/show_bug.cgi?id=1907030 > > it's quite an old bug, but up until recently, the summary was > apparently accurate - dnf would run out of memory with 512M of RAM, but > was OK with 1G. However, as of quite recently, on F36 at least (not > sure if anyone's explicitly tested F37), dnf operations are commonly > failing on VMs/containers with 1G of RAM due to running out of RAM and > getting OOM-killed. > > There's some discussion in the bug about what might be causing this and > potential ways to resolve it, and please do dig into/contribute to that > if you can, but the other question here I guess is: how much do we care > about this? How bad is it that you can't reliably run dnf operations on > top of a minimal Fedora environment with 1G of RAM? > > This obviously has some overlap with our stated hardware requirements, > so here they are for the record: > > https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/welcome/Hardware_Overview/ > > that specifies 2GB as the minimum memory for "the default > installation", by which I think it's referring to a default Workstation > install, though this should be clarified. But then there's a "Low > memory installations" boxout, which suggests that "users with less than > 768MB of system memory may have better results performing a minimal > install and adding to it afterward", which kinda is recommending that > people do exactly the thing that doesn't work (do a minimal install > then use dnf on it), and implying it'll work. I have seen dnf fail to work on my 2GiB Rpi 4 with f36. What I did to workaround this was install the kernel on its own, then dnf update the rest. So it’s only 1GiB systems that are effected. I also have a 1GiB digital ocean VM that happens to not see this issue. I suspect it may be certain packages that make this more likely to fail. Barry > > After some consideration I don't think it makes sense to take this bug > as an F37 blocker, since it already affects F36, and that's what I'll > be suggesting at the next blocker review meeting. However, it does seem > a perfect candidate for prioritized bug status, and I've nominated it > for that. > > I guess if folks can chime in with thoughts here and/or in the bug > report, maybe a consensus will emerge on just how big of an issue this > is (and how likely it is to get fixed). There will presumably be a > FESCo ticket related to prioritized bug status too. > > Thanks folks! > -- > Adam Williamson > Fedora QA > IRC: adamw | Twitter: adamw_ha > https://www.happyassassin.net > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
What is the contents of /etc/dnf/dnf.conf It might help to see if there is a setting therein that is causing the mentioned issue. Regards Leslie Leslie Satenstein Montréal Québec, Canada On Sunday, August 28, 2022 at 11:24:30 p.m. EDT, Adam Williamson wrote: Hey folks! I apologize for the wide distribution, but this seemed like a bug it'd be appropriate to get a wide range of input on. There's a bug that was proposed as an F37 Beta blocker: https://bugzilla.redhat.com/show_bug.cgi?id=1907030 it's quite an old bug, but up until recently, the summary was apparently accurate - dnf would run out of memory with 512M of RAM, but was OK with 1G. However, as of quite recently, on F36 at least (not sure if anyone's explicitly tested F37), dnf operations are commonly failing on VMs/containers with 1G of RAM due to running out of RAM and getting OOM-killed. There's some discussion in the bug about what might be causing this and potential ways to resolve it, and please do dig into/contribute to that if you can, but the other question here I guess is: how much do we care about this? How bad is it that you can't reliably run dnf operations on top of a minimal Fedora environment with 1G of RAM? This obviously has some overlap with our stated hardware requirements, so here they are for the record: https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/welcome/Hardware_Overview/ that specifies 2GB as the minimum memory for "the default installation", by which I think it's referring to a default Workstation install, though this should be clarified. But then there's a "Low memory installations" boxout, which suggests that "users with less than 768MB of system memory may have better results performing a minimal install and adding to it afterward", which kinda is recommending that people do exactly the thing that doesn't work (do a minimal install then use dnf on it), and implying it'll work. After some consideration I don't think it makes sense to take this bug as an F37 blocker, since it already affects F36, and that's what I'll be suggesting at the next blocker review meeting. However, it does seem a perfect candidate for prioritized bug status, and I've nominated it for that. I guess if folks can chime in with thoughts here and/or in the bug report, maybe a consensus will emerge on just how big of an issue this is (and how likely it is to get fixed). There will presumably be a FESCo ticket related to prioritized bug status too. Thanks folks! -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net ___ kde mailing list -- k...@lists.fedoraproject.org To unsubscribe send an email to kde-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/k...@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Mon, Aug 29, 2022, at 11:10 AM, Adam Williamson wrote: > On Mon, 2022-08-29 at 10:53 +0200, Fabio Valentini wrote: >> On Mon, Aug 29, 2022 at 9:53 AM Brian (bex) Exelbierd >> wrote: >> > >> > I wonder if we should take another approach here. Assuming no serious bugs >> > in dnf, rather than tuning dnf for low memory environments could we >> > suggest those folks use Fedora Silverblue, CoreOS, or IoT? >> >> Just speaking for myself: I have Fedora installations on 1G RAM VM >> servers that I've been upgrading for many years now, and they just >> keep chugging along as of Fedora 35. If I would need to either 1) pay >> twice the money per month to keep this setup, or 2) spend a few days >> reprovisioning the servers with Fedora CoreOS 36 (assuming it doesn't >> suffer from the same problem), then I'd consider neither of these >> options a desirable outcome. > > There *is* a workaround, BTW - I didn't mention this in my original > mail, and probably should have. At least according to discussion in the > bug, microdnf works OK. So you can use that instead. Yes, but will you be able to install it? Yes you could go to koji, download the correct RPMs, and have rpm do it without dnf but... that'd be a pretty saucy work around. -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Mon, 2022-08-29 at 10:53 +0200, Fabio Valentini wrote: > On Mon, Aug 29, 2022 at 9:53 AM Brian (bex) Exelbierd > wrote: > > > > I wonder if we should take another approach here. Assuming no serious bugs > > in dnf, rather than tuning dnf for low memory environments could we suggest > > those folks use Fedora Silverblue, CoreOS, or IoT? > > Just speaking for myself: I have Fedora installations on 1G RAM VM > servers that I've been upgrading for many years now, and they just > keep chugging along as of Fedora 35. If I would need to either 1) pay > twice the money per month to keep this setup, or 2) spend a few days > reprovisioning the servers with Fedora CoreOS 36 (assuming it doesn't > suffer from the same problem), then I'd consider neither of these > options a desirable outcome. There *is* a workaround, BTW - I didn't mention this in my original mail, and probably should have. At least according to discussion in the bug, microdnf works OK. So you can use that instead. -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
Because it's typically not an option in these kinds of VMs – you'd need to create a swapfile, enable it, before you could launch the first DNF command. Ideally, disable it after, as chances are a certain bookseller is billing you per IOPS. It's completely out of the questions for (e.g. cgroups) limited containers, which can't even tell the kernel to enable swap. Best, Marcus On 8/29/22 12:12, Roberto Ragusa wrote: On 8/29/22 09:19, Zbigniew Jędrzejewski-Szmek wrote: On Mon, Aug 22, 2022 at 05:44:26PM -0700, Adam Williamson wrote: it's quite an old bug, but up until recently, the summary was apparently accurate - dnf would run out of memory with 512M of RAM, but was OK with 1G. However, as of quite recently, on F36 at least (not sure if anyone's explicitly tested F37), dnf operations are commonly failing on VMs/containers with 1G of RAM due to running out of RAM and getting OOM-killed. The discussion in the bug indicates that this memory growth is related to loading of the full filepath dataset. We have been discussing splitting out the non-primary-filepath-data (i.e. paths that are not /etc, /usr/bin, /usr/sbin), out into a separate lazilly-loaded file. If it is currently loading stuff that is not really used in the computation, this is a perfect case where swap could help (things would be written to swap and then forgot on deallocation). Unfortunately, swap is for some reason considered out-of-fashion, even in these memory constrained set ups. Regards. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Mon, Aug 29, 2022, at 3:52 AM, Brian (bex) Exelbierd wrote: > I use Fedora IoT on GCPs free tier offering and it is fine. I a, > assuming `rpm-ostree install` doesn’t have this issue. It does have the issue. rpm-ostree links to libdnf which is doing all the same things. As I commented in the bug though, to do default image based updates, rpm-ostree very intentionally *does not* load the repo metadata, it's just replicating the filesystem tree, which takes very little memory. The issue really has nothing to do with the client side - it's all about the *repository metadata size*. That's why I reassigned the bug to "distribution". The reason this bug isn't hitting RHEL right now is simply just because the default RHEL repositories are much smaller - also crucially things like many -devel packages are in a separate repository. I don't see a need to rehash all of this *again* - the linked BZ already contains everything said in this thread, and the BZ itself is mainly rehashing a 4 year old thread https://pagure.io/fesco/issue/1955 ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On 8/29/22 09:19, Zbigniew Jędrzejewski-Szmek wrote: On Mon, Aug 22, 2022 at 05:44:26PM -0700, Adam Williamson wrote: it's quite an old bug, but up until recently, the summary was apparently accurate - dnf would run out of memory with 512M of RAM, but was OK with 1G. However, as of quite recently, on F36 at least (not sure if anyone's explicitly tested F37), dnf operations are commonly failing on VMs/containers with 1G of RAM due to running out of RAM and getting OOM-killed. The discussion in the bug indicates that this memory growth is related to loading of the full filepath dataset. We have been discussing splitting out the non-primary-filepath-data (i.e. paths that are not /etc, /usr/bin, /usr/sbin), out into a separate lazilly-loaded file. If it is currently loading stuff that is not really used in the computation, this is a perfect case where swap could help (things would be written to swap and then forgot on deallocation). Unfortunately, swap is for some reason considered out-of-fashion, even in these memory constrained set ups. Regards. -- Roberto Ragusamail at robertoragusa.it ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Mon, Aug 29, 2022 at 9:53 AM Brian (bex) Exelbierd wrote: > > I wonder if we should take another approach here. Assuming no serious bugs in > dnf, rather than tuning dnf for low memory environments could we suggest > those folks use Fedora Silverblue, CoreOS, or IoT? Just speaking for myself: I have Fedora installations on 1G RAM VM servers that I've been upgrading for many years now, and they just keep chugging along as of Fedora 35. If I would need to either 1) pay twice the money per month to keep this setup, or 2) spend a few days reprovisioning the servers with Fedora CoreOS 36 (assuming it doesn't suffer from the same problem), then I'd consider neither of these options a desirable outcome. > I use Fedora IoT on GCPs free tier offering and it is fine. I a, assuming > `rpm-ostree install` doesn’t have this issue. rpm-ostree uses libdnf under the hood, so it might suffer from the same issue ... I wonder, what actually caused this issue? Are there just so many packages with so many files now that the metadata has just grown beyond a specific threshold (again)? Or are there underlying bugs / inefficiencies in dnf that lead to more RAM use that would be necessary? If it's the former, then "fixing" the problem will probably need to wait for "DNF5", but if it's the latter, that's something we could actually fix *now*? Fabio ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Mon, Aug 29, 2022 at 5:24 AM Adam Williamson wrote: > Hey folks! I apologize for the wide distribution, but this seemed like > a bug it'd be appropriate to get a wide range of input on. > > There's a bug that was proposed as an F37 Beta blocker: > https://bugzilla.redhat.com/show_bug.cgi?id=1907030 > > it's quite an old bug, but up until recently, the summary was > apparently accurate - dnf would run out of memory with 512M of RAM, but > was OK with 1G. However, as of quite recently, on F36 at least (not > sure if anyone's explicitly tested F37), dnf operations are commonly > failing on VMs/containers with 1G of RAM due to running out of RAM and > getting OOM-killed. I wonder if we should take another approach here. Assuming no serious bugs in dnf, rather than tuning dnf for low memory environments could we suggest those folks use Fedora Silverblue, CoreOS, or IoT? I use Fedora IoT on GCPs free tier offering and it is fine. I a, assuming `rpm-ostree install` doesn’t have this issue. Regards, bex > > There's some discussion in the bug about what might be causing this and > potential ways to resolve it, and please do dig into/contribute to that > if you can, but the other question here I guess is: how much do we care > about this? How bad is it that you can't reliably run dnf operations on > top of a minimal Fedora environment with 1G of RAM? > > This obviously has some overlap with our stated hardware requirements, > so here they are for the record: > > > https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/welcome/Hardware_Overview/ > > that specifies 2GB as the minimum memory for "the default > installation", by which I think it's referring to a default Workstation > install, though this should be clarified. But then there's a "Low > memory installations" boxout, which suggests that "users with less than > 768MB of system memory may have better results performing a minimal > install and adding to it afterward", which kinda is recommending that > people do exactly the thing that doesn't work (do a minimal install > then use dnf on it), and implying it'll work. > > After some consideration I don't think it makes sense to take this bug > as an F37 blocker, since it already affects F36, and that's what I'll > be suggesting at the next blocker review meeting. However, it does seem > a perfect candidate for prioritized bug status, and I've nominated it > for that. > > I guess if folks can chime in with thoughts here and/or in the bug > report, maybe a consensus will emerge on just how big of an issue this > is (and how likely it is to get fixed). There will presumably be a > FESCo ticket related to prioritized bug status too. > > Thanks folks! > -- > Adam Williamson > Fedora QA > IRC: adamw | Twitter: adamw_ha > https://www.happyassassin.net > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue > -- Don't rush to reply to this email. Enjoy work/life balance. I read email once a day and am in the CE(S)T timezone. If you have an urgent email, ping me, and consider other mediums in the future. Brian "bex" Exelbierd (he/him/his) Business Strategist for Communities and Developers Red Hat Enterprise Linux @bexelbie | http://www.winglemeyer.org bexel...@redhat.com ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Mon, Aug 22, 2022 at 05:44:26PM -0700, Adam Williamson wrote: > Hey folks! I apologize for the wide distribution, but this seemed like > a bug it'd be appropriate to get a wide range of input on. > > There's a bug that was proposed as an F37 Beta blocker: > https://bugzilla.redhat.com/show_bug.cgi?id=1907030 > > it's quite an old bug, but up until recently, the summary was > apparently accurate - dnf would run out of memory with 512M of RAM, but > was OK with 1G. However, as of quite recently, on F36 at least (not > sure if anyone's explicitly tested F37), dnf operations are commonly > failing on VMs/containers with 1G of RAM due to running out of RAM and > getting OOM-killed. The discussion in the bug indicates that this memory growth is related to loading of the full filepath dataset. We have been discussing splitting out the non-primary-filepath-data (i.e. paths that are not /etc, /usr/bin, /usr/sbin), out into a separate lazilly-loaded file. If we manage to do that, we'll kill two birds with one stone: - initial download of repo metadata on every freakin' dnf operation can go down from 80 to 20 MB - the peak memory use will go down Apparently DNF5 makes this possible. My vote is: yes, this is an issue. No, we shouldn't block F37 on this. Apparently the only reasonable way to tackle this is with a major rework of DNF, so let's get it right with DNF5. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
Once upon a time, Adam Williamson said: > There's some discussion in the bug about what might be causing this and > potential ways to resolve it, and please do dig into/contribute to that > if you can, but the other question here I guess is: how much do we care > about this? How bad is it that you can't reliably run dnf operations on > top of a minimal Fedora environment with 1G of RAM? At $LASTJOB, I had a number of 1G VMs for low-overhead things like caching DNS, DHCP, and RADIUS. Those were mostly CentOS, but what happens in Fedora makes its way there eventually. Having the package manager take more RAM than all the operational services combined is not so good IMHO. -- Chris Adams ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: Heads-up / for discussion: dnf not working with 1G of RAM or less
On Sun, Aug 28, 2022 at 11:24 PM Adam Williamson wrote: > > Hey folks! I apologize for the wide distribution, but this seemed like > a bug it'd be appropriate to get a wide range of input on. > > There's a bug that was proposed as an F37 Beta blocker: > https://bugzilla.redhat.com/show_bug.cgi?id=1907030 > > it's quite an old bug, but up until recently, the summary was > apparently accurate - dnf would run out of memory with 512M of RAM, but > was OK with 1G. However, as of quite recently, on F36 at least (not > sure if anyone's explicitly tested F37), dnf operations are commonly > failing on VMs/containers with 1G of RAM due to running out of RAM and > getting OOM-killed. > > There's some discussion in the bug about what might be causing this and > potential ways to resolve it, and please do dig into/contribute to that > if you can, but the other question here I guess is: how much do we care > about this? How bad is it that you can't reliably run dnf operations on > top of a minimal Fedora environment with 1G of RAM? > DigitalOcean's (probably most popular?) formerly-5-now-$6 droplet is 1GB of RAM, and they also added a $4 with 512MB of RAM. They also provide Fedora as an image to provision your droplet. Getting an install and then being unable to install anything or update it seems... less than ideal for a somewhat-public-facing server, assuming that's what people use it for. I don't know how many other cloud providers have Fedora images that you can provision with, but breaking them is not great. > This obviously has some overlap with our stated hardware requirements, > so here they are for the record: > > https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/welcome/Hardware_Overview/ > > that specifies 2GB as the minimum memory for "the default > installation", by which I think it's referring to a default Workstation > install, though this should be clarified. That's "for the default installation", so about Workstation. > But then there's a "Low > memory installations" boxout, which suggests that "users with less than > 768MB of system memory may have better results performing a minimal > install and adding to it afterward", which kinda is recommending that > people do exactly the thing that doesn't work (do a minimal install > then use dnf on it), and implying it'll work. > > After some consideration I don't think it makes sense to take this bug > as an F37 blocker, since it already affects F36, and that's what I'll > be suggesting at the next blocker review meeting. However, it does seem > a perfect candidate for prioritized bug status, and I've nominated it > for that. > > I guess if folks can chime in with thoughts here and/or in the bug > report, maybe a consensus will emerge on just how big of an issue this > is (and how likely it is to get fixed). There will presumably be a > FESCo ticket related to prioritized bug status too. > > Thanks folks! > -- > Adam Williamson > Fedora QA > IRC: adamw | Twitter: adamw_ha > https://www.happyassassin.net > -- Elliott ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
