Re: An idea for swap partition size vs. swap space size in use handling
On Feb 25, 2023, at 21:00, Mark Millard wrote: > On Feb 25, 2023, at 20:01, Jamie Landeg-Jones wrote: > >> Mark Millard wrote: >> >>> On Jan 21, 2023, at 23:17, Poul-Henning Kamp wrote: Last I looked at that code, that is precisely what happens if you add a too big swap-device ? >>> >>> It produces a notice reporting how much bigger what it is >>> using is than what is recommended, if I understand the >>> message right. Here is an example were the difference was >>> small for an armv7 context: >>> >>> warning: total configured swap (1003519 pages) exceeds maximum recommended >>> amount (1003072 pages). >>> >>> Another from a context with a much bigger difference: >>> >>> warning: total configured swap (2097152 pages) exceeds maximum recommended >>> amount (916632 pages). >>> >>> These sort of messages are followed by: >>> >>> warning: increase kern.maxswzone or reduce amount of swap. >> >> I thought the same as phk. And I always thought those messages were >> just informational warnings on what to do to stop that excess swap >> space being unused and effectively wasted (i.e. suggestions to change >> settings, or reduce the swap size!), but you say this isn't correct: >> >>> As I understand, the 2097152 pages vs. 916632 pages example means >>> that it was operating with the referenced fragmentation problems >>> being more likely. That would not be true if it was just using >>> more like the 916632 pages and ignoring the rest. >> >> Are you, phk, or anyone else, able to provide further pointers or >> clarification on this? > > In: > > https://lists.freebsd.org/archives/freebsd-current/2023-January/003086.html > > I wrote text between the places the places that you quote: > > "warning: increase kern.maxswzone or reduce amount of swap." > > and: > > "As I understand, the 2097152 pages vs. 916632 pages example means" > > that included quotes from man 8 loader, specifically its > kern.maxswzone material, that indicated there are resource > tradeoffs in adjusting kern.maxswzone and also indicated that > "care should be taken to not configure more swap than approximately > half of the theoretical maximum". It also says that kern.maxswzone > "directly governs the maximum amount of swap the system can support". > > I'll also note that adjusting kern.maxswzone does not, of itself, > notably change the amount of swap that would be put to use for the > same workload. > > So a message reporting to either adjust kern.maxswzone or > reduce the amount of swap is not documented to just be about > avoiding unused swap space, so long as the system does not > reach the "unrecoverable state" that was referenced in the > same quoted material. > > If what I quoted is insufficient evidence for you, I'm not > likely to find other evidence that would be sufficient > for you. (Not that I've ever found other material about > the issue.) One adjustment: I forgot that these days the material that I originally quoted has moved around and where to find it depends on the vintage of FreeBSD in question. For 13.1-RELEASE: https://man.freebsd.org/cgi/man.cgi?query=loader=0=8=FreeBSD+13.1-RELEASE=default=html shows it in man 8 loader . (Similarly for older.) But for 13.2-STABLE: https://man.freebsd.org/cgi/man.cgi?query=loader_simp=0=8=FreeBSD+13.2-STABLE=default=html shows it in man 8 loader_simp . (Similarly for 14.0-CURRENT.) === Mark Millard marklmi at yahoo.com
Re: An idea for swap partition size vs. swap space size in use handling
On Feb 25, 2023, at 20:01, Jamie Landeg-Jones wrote: > Mark Millard wrote: > >> On Jan 21, 2023, at 23:17, Poul-Henning Kamp wrote: >>> >>> Last I looked at that code, that is precisely what happens >>> if you add a too big swap-device ? >> >> It produces a notice reporting how much bigger what it is >> using is than what is recommended, if I understand the >> message right. Here is an example were the difference was >> small for an armv7 context: >> >> warning: total configured swap (1003519 pages) exceeds maximum recommended >> amount (1003072 pages). >> >> Another from a context with a much bigger difference: >> >> warning: total configured swap (2097152 pages) exceeds maximum recommended >> amount (916632 pages). >> >> These sort of messages are followed by: >> >> warning: increase kern.maxswzone or reduce amount of swap. > > I thought the same as phk. And I always thought those messages were > just informational warnings on what to do to stop that excess swap > space being unused and effectively wasted (i.e. suggestions to change > settings, or reduce the swap size!), but you say this isn't correct: > >> As I understand, the 2097152 pages vs. 916632 pages example means >> that it was operating with the referenced fragmentation problems >> being more likely. That would not be true if it was just using >> more like the 916632 pages and ignoring the rest. > > Are you, phk, or anyone else, able to provide further pointers or > clarification on this? In: https://lists.freebsd.org/archives/freebsd-current/2023-January/003086.html I wrote text between the places the places that you quote: "warning: increase kern.maxswzone or reduce amount of swap." and: "As I understand, the 2097152 pages vs. 916632 pages example means" that included quotes from man 8 loader, specifically its kern.maxswzone material, that indicated there are resource tradeoffs in adjusting kern.maxswzone and also indicated that "care should be taken to not configure more swap than approximately half of the theoretical maximum". It also says that kern.maxswzone "directly governs the maximum amount of swap the system can support". I'll also note that adjusting kern.maxswzone does not, of itself, notably change the amount of swap that would be put to use for the same workload. So a message reporting to either adjust kern.maxswzone or reduce the amount of swap is not documented to just be about avoiding unused swap space, so long as the system does not reach the "unrecoverable state" that was referenced in the same quoted material. If what I quoted is insufficient evidence for you, I'm not likely to find other evidence that would be sufficient for you. (Not that I've ever found other material about the issue.) === Mark Millard marklmi at yahoo.com
Re: An idea for swap partition size vs. swap space size in use handling
Mark Millard wrote: > On Jan 21, 2023, at 23:17, Poul-Henning Kamp wrote: > > > > Last I looked at that code, that is precisely what happens > > if you add a too big swap-device ? > > It produces a notice reporting how much bigger what it is > using is than what is recommended, if I understand the > message right. Here is an example were the difference was > small for an armv7 context: > > warning: total configured swap (1003519 pages) exceeds maximum recommended > amount (1003072 pages). > > Another from a context with a much bigger difference: > > warning: total configured swap (2097152 pages) exceeds maximum recommended > amount (916632 pages). > > These sort of messages are followed by: > > warning: increase kern.maxswzone or reduce amount of swap. I thought the same as phk. And I always thought those messages were just informational warnings on what to do to stop that excess swap space being unused and effectively wasted (i.e. suggestions to change settings, or reduce the swap size!), but you say this isn't correct: > As I understand, the 2097152 pages vs. 916632 pages example means > that it was operating with the referenced fragmentation problems > being more likely. That would not be true if it was just using > more like the 916632 pages and ignoring the rest. Are you, phk, or anyone else, able to provide further pointers or clarification on this? Cheers, Jamie
Re: An idea for swap partition size vs. swap space size in use handling
On Jan 22, 2023, at 09:46, Mark Millard wrote: > >> On Jan 22, 2023, at 05:15, Mike Karels wrote: >> >>> On 22 Jan 2023, at 2:42, Mark Millard wrote: >>> On Jan 22, 2023, at 00:21, Mark Millard wrote: > On Jan 21, 2023, at 23:17, Poul-Henning Kamp wrote: > >> >> Mark Millard writes: >> >>> It would be nice if I could have just one swap partition >>> on a given boot media, one that is more than sufficient >>> in size for all but the biggest RAM system --but to then >>> be able to tell the system to just use up to the >>> recommended swap space size and to ignore any extra swap >>> space in the swap partition. >>> >>> Why not just reduce the size of the swap partition to the desired size >>> with “gpart resize”? Granted, that requires manual intervention. > > On Jan 22, 2023, at 05:15, Mike Karels wrote: > >> On 22 Jan 2023, at 2:42, Mark Millard wrote: >> >>> On Jan 22, 2023, at 00:21, Mark Millard wrote: >>> On Jan 21, 2023, at 23:17, Poul-Henning Kamp wrote: > > Mark Millard writes: > >> It would be nice if I could have just one swap partition >> on a given boot media, one that is more than sufficient >> in size for all but the biggest RAM system --but to then >> be able to tell the system to just use up to the >> recommended swap space size and to ignore any extra swap >> space in the swap partition. >> >> Why not just reduce the size of the swap partition to the desired size >> with “gpart resize”? Granted, that requires manual intervention. > > Why would I use the size for a 1 GiByte aarch64 system > (prior boot) when I'm using the media to boot the 64 > GiByte system? So increases too. Manual intervention > every time I move the media between systems, going in > both directions over time? > > That gets me into the business of independently > calculating a reasonable lower bound for the recommended > swap space every time I move the media between systems > with differing amounts of RAM. I've noticed that the > recommendation varies some across the OS updates. (My > guess is variations in the kernel space use is involved > in the recommendation.) I'm looking for something > avoiding such a redundant calculation, something that > just works given the variability across OS updates. (And across platforms: armv7 and aarch64 have very different recommendations for the same amount of RAM. And across differing amounts of RAM from system to system.) Took a while for me to think of it, but there is also that I use paths based on labels to identify the swap partition independent of the system it is plugged into and what other devices are present in that system. (Some of the media here are USB media.) Resizing (or subranging) partitions invalidates labeling because the labeling information goes at the end of the (logical) partition, if I understand right. So labeling would have to be re-established for the new size as well. === Mark Millard marklmi at yahoo.com
Re: An idea for swap partition size vs. swap space size in use handling
On Jan 22, 2023, at 06:11, Ronald Klop wrote: > > > Van: Mark Millard > Datum: zondag, 22 januari 2023 05:41 > Aan: freebsd-current > Onderwerp: An idea for swap partition size vs. swap space size in use handling > I have boot media that are each set up to boot a variety > of systems that have widely different RAM sizes, from > 1 GiBytes to 64 GiBytes for the aarch64 examples of this. > > This has lead to having multiple swap partitions of > various sizes so that I can have total swap spaces that > are somewhat under the recommended maximum sizes for the > amount of RAM in each of those systems that a given media > can boot. > > It would be nice if I could have just one swap partition > on a given boot media, one that is more than sufficient > in size for all but the biggest RAM system --but to then > be able to tell the system to just use up to the > recommended swap space size and to ignore any extra swap > space in the swap partition. > > If such could be done, I'd no longer use multiple swap > partitions at the same time in order to get to a desired > total for the system at hand at the time. > > Of course, that still leaves what to do when multiple > swap partitions are enabled if such a "ignore what > would be extra" mode was also enabled. As I'd not use > such, I've no specific recommendations to make that > would make any difference to my use. > > === > Mark Millard > marklmi at yahoo.com > > Â > > > Hi Mark, > > Do you mean you would like to be able to add a "size" parameter to the fstab > swap line? > > /dev/da0p1 noneswapsw,size=1g 0 0 No. That would use 1g for even the 64 GiByte RAM system when I move the media to that 64 GiByte system and boot it --and similarly as I move the media from system to system in general. For my style of use, I want the swap to be (a lower bound approximation of) the recommended figure for the amount of RAM for the specific system the media is currently use in. That means that the figure changes when I move the media to a system with a different amount of RAM. > Another option I can think of is using the "gnop" geom provider. It has a -s > parameter to set the size. I classically use a label based path to identify the swap partition, not a device name. This fits with the variations in what system I'm booting and what other devices might be around. (Some of the media involved here are USB media.) Labels go at the end of partitions, as I remember. This would appear to make forms of resizing or subranging the partition also mean adjusting the labeling. That is something I'd like to avoid. I'd also like to avoid needing to provide my own lower bound approximation of the recommended figure for the live amount of RAM. For one, it is platform dependent, for example armv7 and aarch64 are very different for the same amount of RAM. === Mark Millard marklmi at yahoo.com
Re: An idea for swap partition size vs. swap space size in use handling
On Jan 22, 2023, at 05:15, Mike Karels wrote: > On 22 Jan 2023, at 2:42, Mark Millard wrote: > >> On Jan 22, 2023, at 00:21, Mark Millard wrote: >> >>> On Jan 21, 2023, at 23:17, Poul-Henning Kamp wrote: >>> Mark Millard writes: > It would be nice if I could have just one swap partition > on a given boot media, one that is more than sufficient > in size for all but the biggest RAM system --but to then > be able to tell the system to just use up to the > recommended swap space size and to ignore any extra swap > space in the swap partition. > > Why not just reduce the size of the swap partition to the desired size > with “gpart resize”? Granted, that requires manual intervention. Why would I use the size for a 1 GiByte aarch64 system (prior boot) when I'm using the media to boot the 64 GiByte system? So increases too. Manual intervention every time I move the media between systems, going in both directions over time? That gets me into the business of independently calculating a reasonable lower bound for the recommended swap space every time I move the media between systems with differing amounts of RAM. I've noticed that the recommendation varies some across the OS updates. (My guess is variations in the kernel space use is involved in the recommendation.) I'm looking for something avoiding such a redundant calculation, something that just works given the variability across OS updates. === Mark Millard marklmi at yahoo.com
Re: An idea for swap partition size vs. swap space size in use handling
Van: Mark Millard Datum: zondag, 22 januari 2023 05:41 Aan: freebsd-current Onderwerp: An idea for swap partition size vs. swap space size in use handling I have boot media that are each set up to boot a variety of systems that have widely different RAM sizes, from 1 GiBytes to 64 GiBytes for the aarch64 examples of this. This has lead to having multiple swap partitions of various sizes so that I can have total swap spaces that are somewhat under the recommended maximum sizes for the amount of RAM in each of those systems that a given media can boot. It would be nice if I could have just one swap partition on a given boot media, one that is more than sufficient in size for all but the biggest RAM system --but to then be able to tell the system to just use up to the recommended swap space size and to ignore any extra swap space in the swap partition. If such could be done, I'd no longer use multiple swap partitions at the same time in order to get to a desired total for the system at hand at the time. Of course, that still leaves what to do when multiple swap partitions are enabled if such a "ignore what would be extra" mode was also enabled. As I'd not use such, I've no specific recommendations to make that would make any difference to my use. === Mark Millard marklmi at yahoo.com  Hi Mark, Do you mean you would like to be able to add a "size" parameter to the fstab swap line? /dev/da0p1 noneswapsw,size=1g 0 0 Another option I can think of is using the "gnop" geom provider. It has a -s parameter to set the size. Regards, Ronald. Â
Re: An idea for swap partition size vs. swap space size in use handling
On 22 Jan 2023, at 2:42, Mark Millard wrote: > On Jan 22, 2023, at 00:21, Mark Millard wrote: > >> On Jan 21, 2023, at 23:17, Poul-Henning Kamp wrote: >> >>> >>> Mark Millard writes: >>> It would be nice if I could have just one swap partition on a given boot media, one that is more than sufficient in size for all but the biggest RAM system --but to then be able to tell the system to just use up to the recommended swap space size and to ignore any extra swap space in the swap partition. Why not just reduce the size of the swap partition to the desired size with “gpart resize”? Granted, that requires manual intervention. Mike >>> Last I looked at that code, that is precisely what happens >>> if you add a too big swap-device ? >> >> It produces a notice reporting how much bigger what it is >> using is than what is recommended, if I understand the >> message right. Here is an example were the difference was >> small for an armv7 context: >> >> warning: total configured swap (1003519 pages) exceeds maximum recommended >> amount (1003072 pages). >> >> Another from a context with a much bigger difference: >> >> warning: total configured swap (2097152 pages) exceeds maximum recommended >> amount (916632 pages). >> >> These sort of messages are followed by: >> >> warning: increase kern.maxswzone or reduce amount of swap. >> >> But, as I understand, increasing kern.maxswzone makes >> tradoffs with other kernel memory use. man 8 loader >> reports: > > All my references to "man 8 loader" should have been to > "man 8 loader_simp" these days. (Old habit, not yet > replaced.) > >> kern.maxswzone >> Limits the amount of KVM to be used to hold swap metadata, >> which directly governs the maximum amount of swap the >> system can support . . . >> . . . >> Note that swap metadata can be fragmented, which means that >> the system can run out of space before it reaches the >> theoretical limit. Therefore, care should be taken to not >> configure more swap than approximately half of the >> theoretical maximum. >> >> (Note: My understanding is that an "approximately half" is the >> figure shown as the "recommended amount" in the warnings.) >> >> Running out of space for swap metadata can leave the system >> in an unrecoverable state. Therefore, you should only >> change this parameter if you need to greatly extend the KVM >> reservation for other resources such as the buffer cache or >> kern.ipc.nmbclusters. Modifies kernel option >> VM_SWZONE_SIZE_MAX. >> >> The wording in man 8 loader is about decreasing kern.maxswzone > > Again. > >> in order to make room for other resources. But the implication >> is that increases leave less room than normal for other >> resources. I try to avoid getting the warnings as I do not have >> knowledge/context to make well-guided tradeoffs for the >> resources. >> >> As I understand, the 2097152 pages vs. 916632 pages example means >> that it was operating with the referenced fragmentation problems >> being more likely. That would not be true if it was just using >> more like the 916632 pages and ignoring the rest. >> >> (I was not suggesting changes to default behavior. I was only >> suggesting being able to put it in a mode where it would have >> used, for example, just around 916632 pages of the swap space.) >> >> >> (Note: Some of the detailed man 8 loader claims that I left out > > Again. > >> seem to not be general to all platforms, despite the wording >> giving no hint of that issue.) > > > > > === > Mark Millard > marklmi at yahoo.com
Re: An idea for swap partition size vs. swap space size in use handling
On Jan 22, 2023, at 00:21, Mark Millard wrote: > On Jan 21, 2023, at 23:17, Poul-Henning Kamp wrote: > >> >> Mark Millard writes: >> >>> It would be nice if I could have just one swap partition >>> on a given boot media, one that is more than sufficient >>> in size for all but the biggest RAM system --but to then >>> be able to tell the system to just use up to the >>> recommended swap space size and to ignore any extra swap >>> space in the swap partition. >> >> Last I looked at that code, that is precisely what happens >> if you add a too big swap-device ? > > It produces a notice reporting how much bigger what it is > using is than what is recommended, if I understand the > message right. Here is an example were the difference was > small for an armv7 context: > > warning: total configured swap (1003519 pages) exceeds maximum recommended > amount (1003072 pages). > > Another from a context with a much bigger difference: > > warning: total configured swap (2097152 pages) exceeds maximum recommended > amount (916632 pages). > > These sort of messages are followed by: > > warning: increase kern.maxswzone or reduce amount of swap. > > But, as I understand, increasing kern.maxswzone makes > tradoffs with other kernel memory use. man 8 loader > reports: All my references to "man 8 loader" should have been to "man 8 loader_simp" these days. (Old habit, not yet replaced.) > kern.maxswzone > Limits the amount of KVM to be used to hold swap metadata, > which directly governs the maximum amount of swap the > system can support . . . > . . . > Note that swap metadata can be fragmented, which means that > the system can run out of space before it reaches the > theoretical limit. Therefore, care should be taken to not > configure more swap than approximately half of the > theoretical maximum. > > (Note: My understanding is that an "approximately half" is the > figure shown as the "recommended amount" in the warnings.) > > Running out of space for swap metadata can leave the system > in an unrecoverable state. Therefore, you should only > change this parameter if you need to greatly extend the KVM > reservation for other resources such as the buffer cache or > kern.ipc.nmbclusters. Modifies kernel option > VM_SWZONE_SIZE_MAX. > > The wording in man 8 loader is about decreasing kern.maxswzone Again. > in order to make room for other resources. But the implication > is that increases leave less room than normal for other > resources. I try to avoid getting the warnings as I do not have > knowledge/context to make well-guided tradeoffs for the > resources. > > As I understand, the 2097152 pages vs. 916632 pages example means > that it was operating with the referenced fragmentation problems > being more likely. That would not be true if it was just using > more like the 916632 pages and ignoring the rest. > > (I was not suggesting changes to default behavior. I was only > suggesting being able to put it in a mode where it would have > used, for example, just around 916632 pages of the swap space.) > > > (Note: Some of the detailed man 8 loader claims that I left out Again. > seem to not be general to all platforms, despite the wording > giving no hint of that issue.) === Mark Millard marklmi at yahoo.com
Re: An idea for swap partition size vs. swap space size in use handling
On Jan 21, 2023, at 23:17, Poul-Henning Kamp wrote: > > Mark Millard writes: > >> It would be nice if I could have just one swap partition >> on a given boot media, one that is more than sufficient >> in size for all but the biggest RAM system --but to then >> be able to tell the system to just use up to the >> recommended swap space size and to ignore any extra swap >> space in the swap partition. > > Last I looked at that code, that is precisely what happens > if you add a too big swap-device ? It produces a notice reporting how much bigger what it is using is than what is recommended, if I understand the message right. Here is an example were the difference was small for an armv7 context: warning: total configured swap (1003519 pages) exceeds maximum recommended amount (1003072 pages). Another from a context with a much bigger difference: warning: total configured swap (2097152 pages) exceeds maximum recommended amount (916632 pages). These sort of messages are followed by: warning: increase kern.maxswzone or reduce amount of swap. But, as I understand, increasing kern.maxswzone makes tradoffs with other kernel memory use. man 8 loader reports: kern.maxswzone Limits the amount of KVM to be used to hold swap metadata, which directly governs the maximum amount of swap the system can support . . . . . . Note that swap metadata can be fragmented, which means that the system can run out of space before it reaches the theoretical limit. Therefore, care should be taken to not configure more swap than approximately half of the theoretical maximum. (Note: My understanding is that an "approximately half" is the figure shown as the "recommended amount" in the warnings.) Running out of space for swap metadata can leave the system in an unrecoverable state. Therefore, you should only change this parameter if you need to greatly extend the KVM reservation for other resources such as the buffer cache or kern.ipc.nmbclusters. Modifies kernel option VM_SWZONE_SIZE_MAX. The wording in man 8 loader is about decreasing kern.maxswzone in order to make room for other resources. But the implication is that increases leave less room than normal for other resources. I try to avoid getting the warnings as I do not have knowledge/context to make well-guided tradeoffs for the resources. As I understand, the 2097152 pages vs. 916632 pages example means that it was operating with the referenced fragmentation problems being more likely. That would not be true if it was just using more like the 916632 pages and ignoring the rest. (I was not suggesting changes to default behavior. I was only suggesting being able to put it in a mode where it would have used, for example, just around 916632 pages of the swap space.) (Note: Some of the detailed man 8 loader claims that I left out seem to not be general to all platforms, despite the wording giving no hint of that issue.) === Mark Millard marklmi at yahoo.com
Re: An idea for swap partition size vs. swap space size in use handling
Mark Millard writes: > It would be nice if I could have just one swap partition > on a given boot media, one that is more than sufficient > in size for all but the biggest RAM system --but to then > be able to tell the system to just use up to the > recommended swap space size and to ignore any extra swap > space in the swap partition. Last I looked at that code, that is precisely what happens if you add a too big swap-device ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.