Bug#645052: kernel only recognizes 32G of memory
On Thu, 2012-01-12 at 14:46 +0300, Dmitry Musatov wrote: > Hi Ian, Hi Ben, > > >> Yes, that seems reasonable. Let's do it (but after -39). > > > > Sounds like a plan. I'll wait for -40 to begin then check that in. > > What's status on this? I see -40 in squeeze-proposed-updates, but > according to the changelog > (http://packages.debian.org/changelogs/pool/main/l/linux-2.6/linux-2.6_2.6.32-40/changelog.html) > it still stuck with 32GB. I completely forgot about this, sorry. I'll put it in SVN for -41. Ian. -- Ian Campbell Uncertain fortune is thoroughly mastered by the equity of the calculation. -- Blaise Pascal -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#645052: kernel only recognizes 32G of memory
Hi Ian, Hi Ben, >> Yes, that seems reasonable. Let's do it (but after -39). > > Sounds like a plan. I'll wait for -40 to begin then check that in. What's status on this? I see -40 in squeeze-proposed-updates, but according to the changelog (http://packages.debian.org/changelogs/pool/main/l/linux-2.6/linux-2.6_2.6.32-40/changelog.html) it still stuck with 32GB. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#645052: kernel only recognizes 32G of memory
On Thu, 2011-10-20 at 05:30 +0100, Ben Hutchings wrote: > On Tue, 2011-10-18 at 10:57 +0100, Ian Campbell wrote: > > On Tue, 2011-10-18 at 04:40 +0100, Ben Hutchings wrote: > [...] > > > and since some > > > people like to run domains with much less memory, I'm inclined to say > > > that this is 'wontfix' for squeeze. But I'm not sure just how small > > > they are likely to be (while still running Debian). Maybe the cost > > > isn't that significant. > > > > http://www.debian.org/releases/stable/amd64/ch03s04.html.en and > > http://www.debian.org/releases/stable/i386/ch03s04.html.en both say the > > minimum is 64M. > > > > We are talking about going from 128KB to 280KB reserved for the p2m. > > Which for a 64M machine is going from 0.20% to 0.43% of RAM overhead. > > > > I'm not sure if <64M is realistic. I have a (32-bit, physical) machine I > > use as a firewall which has 32M and apt-get and friends really do grind > > along (it's also an old Pentium with a tiny disk, so there are other > > factors in that). > > > > I think we are only talking about the limit for a 64 bit guest? I would > > guess that those are more unlikely to be given tiny amounts of RAM > > compared with 32 bit. > > Yes, that seems reasonable. Let's do it (but after -39). Sounds like a plan. I'll wait for -40 to begin then check that in. Cheers, Ian. -- Ian Campbell Better hope you get what you want before you stop wanting it. signature.asc Description: This is a digitally signed message part
Bug#645052: kernel only recognizes 32G of memory
On Tue, 2011-10-18 at 10:57 +0100, Ian Campbell wrote: > On Tue, 2011-10-18 at 04:40 +0100, Ben Hutchings wrote: [...] > > and since some > > people like to run domains with much less memory, I'm inclined to say > > that this is 'wontfix' for squeeze. But I'm not sure just how small > > they are likely to be (while still running Debian). Maybe the cost > > isn't that significant. > > http://www.debian.org/releases/stable/amd64/ch03s04.html.en and > http://www.debian.org/releases/stable/i386/ch03s04.html.en both say the > minimum is 64M. > > We are talking about going from 128KB to 280KB reserved for the p2m. > Which for a 64M machine is going from 0.20% to 0.43% of RAM overhead. > > I'm not sure if <64M is realistic. I have a (32-bit, physical) machine I > use as a firewall which has 32M and apt-get and friends really do grind > along (it's also an old Pentium with a tiny disk, so there are other > factors in that). > > I think we are only talking about the limit for a 64 bit guest? I would > guess that those are more unlikely to be given tiny amounts of RAM > compared with 32 bit. Yes, that seems reasonable. Let's do it (but after -39). Ben. -- Ben Hutchings 73.46% of all statistics are made up. signature.asc Description: This is a digitally signed message part
Bug#645052: kernel only recognizes 32G of memory
On Tue, 2011-10-18 at 04:40 +0100, Ben Hutchings wrote: > On Thu, 2011-10-13 at 07:05 +0100, Ian Campbell wrote: > > On Thu, 2011-10-13 at 03:27 +0100, Ben Hutchings wrote: > > > On Wed, 2011-10-12 at 14:58 +0100, Ian Campbell wrote: > > > > On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote: > > > > > On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote: > > > > > > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote: > > > > > > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory > > > > > > > a > > > > > > > Xen instance is seeing. The default for 64bit is 32GB, which is > > > > > > > the > > > > > > > reason that m2.4xlarge Amazon EC2 instances only report this > > > > > > > amount of > > > > > > > memory. > > > > > > > Please set this limit to 70GB as there is a known restriction for > > > > > > > t1.micro instances at about 80GB. > > > > > > > Similar bug exists and Ubuntu where it's already fixed > > > > > > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796) > > > > > > > > > > > > Is this the sort of change we can consider making in a stable > > > > > > update? > > > > > > I'm not at all sure, although my gut feeling is that it would be > > > > > > safe. > > > > > [...] > > > > > > > > > > I think so. But what is the trade-off? There must be some reason why > > > > > this isn't set to however many TB the kernel can support. > > > > > > > > It effects the amount of space set aside for the P2M table (the mapping > > > > of physical to machine addresses). In the kernel in Squeeze this space > > > > is statically reserved in BSS so increasing it will waste some more > > > > memory, according to the Kconfig comment it is 1 page per GB. > > > > > > > > In a more up to date kernel the space comes from BRK and is reclaimed if > > > > it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the > > > > same change. > > > > > > How intrusive is the change? Could we reasonably backport it? > > > > It was 58e05027b530 "xen: convert p2m to a 3 level tree" which I think > > is too big. IIRC there was a bunch of subsequent fixups to it as well, > > it was quite a subtle change. > > You didn't directly answer the questions, but that sounds like 'fairly' > and 'no'. Correct. Sorry, I thought "too big" covered both. > If I understand correctly, the memory cost of expanding the table to > cover 70GB is (70GB - 32GB) * 4KB / 1GB = 156KB. Is that right? Yes. > Since we don't have a specific flavour to support EC2, One option might be to increase the limit only for the xen flavour and leave the normal flavour where it is. That adds a rather unfortunate matrix to the selection of which flavour to use though, ideally I would prefer folks to be using the regular flavour in domU wherever possible. > and since some > people like to run domains with much less memory, I'm inclined to say > that this is 'wontfix' for squeeze. But I'm not sure just how small > they are likely to be (while still running Debian). Maybe the cost > isn't that significant. http://www.debian.org/releases/stable/amd64/ch03s04.html.en and http://www.debian.org/releases/stable/i386/ch03s04.html.en both say the minimum is 64M. We are talking about going from 128KB to 280KB reserved for the p2m. Which for a 64M machine is going from 0.20% to 0.43% of RAM overhead. I'm not sure if <64M is realistic. I have a (32-bit, physical) machine I use as a firewall which has 32M and apt-get and friends really do grind along (it's also an old Pentium with a tiny disk, so there are other factors in that). I think we are only talking about the limit for a 64 bit guest? I would guess that those are more unlikely to be given tiny amounts of RAM compared with 32 bit. Ian. -- Ian Campbell Current Noise: Monster Magnet - Bored With Sorcery Your picture of the world often changes just before you get it into focus. signature.asc Description: This is a digitally signed message part
Bug#645052: kernel only recognizes 32G of memory
On Thu, 2011-10-13 at 07:05 +0100, Ian Campbell wrote: > On Thu, 2011-10-13 at 03:27 +0100, Ben Hutchings wrote: > > On Wed, 2011-10-12 at 14:58 +0100, Ian Campbell wrote: > > > On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote: > > > > On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote: > > > > > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote: > > > > > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a > > > > > > Xen instance is seeing. The default for 64bit is 32GB, which is the > > > > > > reason that m2.4xlarge Amazon EC2 instances only report this amount > > > > > > of > > > > > > memory. > > > > > > Please set this limit to 70GB as there is a known restriction for > > > > > > t1.micro instances at about 80GB. > > > > > > Similar bug exists and Ubuntu where it's already fixed > > > > > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796) > > > > > > > > > > Is this the sort of change we can consider making in a stable update? > > > > > I'm not at all sure, although my gut feeling is that it would be safe. > > > > [...] > > > > > > > > I think so. But what is the trade-off? There must be some reason why > > > > this isn't set to however many TB the kernel can support. > > > > > > It effects the amount of space set aside for the P2M table (the mapping > > > of physical to machine addresses). In the kernel in Squeeze this space > > > is statically reserved in BSS so increasing it will waste some more > > > memory, according to the Kconfig comment it is 1 page per GB. > > > > > > In a more up to date kernel the space comes from BRK and is reclaimed if > > > it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the > > > same change. > > > > How intrusive is the change? Could we reasonably backport it? > > It was 58e05027b530 "xen: convert p2m to a 3 level tree" which I think > is too big. IIRC there was a bunch of subsequent fixups to it as well, > it was quite a subtle change. You didn't directly answer the questions, but that sounds like 'fairly' and 'no'. If I understand correctly, the memory cost of expanding the table to cover 70GB is (70GB - 32GB) * 4KB / 1GB = 156KB. Is that right? Since we don't have a specific flavour to support EC2, and since some people like to run domains with much less memory, I'm inclined to say that this is 'wontfix' for squeeze. But I'm not sure just how small they are likely to be (while still running Debian). Maybe the cost isn't that significant. Ben. -- Ben Hutchings No political challenge can be met by shopping. - George Monbiot signature.asc Description: This is a digitally signed message part
Bug#645052: kernel only recognizes 32G of memory
On Thu, 2011-10-13 at 03:27 +0100, Ben Hutchings wrote: > On Wed, 2011-10-12 at 14:58 +0100, Ian Campbell wrote: > > On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote: > > > On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote: > > > > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote: > > > > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a > > > > > Xen instance is seeing. The default for 64bit is 32GB, which is the > > > > > reason that m2.4xlarge Amazon EC2 instances only report this amount of > > > > > memory. > > > > > Please set this limit to 70GB as there is a known restriction for > > > > > t1.micro instances at about 80GB. > > > > > Similar bug exists and Ubuntu where it's already fixed > > > > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796) > > > > > > > > Is this the sort of change we can consider making in a stable update? > > > > I'm not at all sure, although my gut feeling is that it would be safe. > > > [...] > > > > > > I think so. But what is the trade-off? There must be some reason why > > > this isn't set to however many TB the kernel can support. > > > > It effects the amount of space set aside for the P2M table (the mapping > > of physical to machine addresses). In the kernel in Squeeze this space > > is statically reserved in BSS so increasing it will waste some more > > memory, according to the Kconfig comment it is 1 page per GB. > > > > In a more up to date kernel the space comes from BRK and is reclaimed if > > it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the > > same change. > > How intrusive is the change? Could we reasonably backport it? It was 58e05027b530 "xen: convert p2m to a 3 level tree" which I think is too big. IIRC there was a bunch of subsequent fixups to it as well, it was quite a subtle change. Ian. > > Ben. > -- Ian Campbell Aliquid melius quam pessimum optimum non est. signature.asc Description: This is a digitally signed message part
Bug#645052: kernel only recognizes 32G of memory
On Wed, 2011-10-12 at 14:58 +0100, Ian Campbell wrote: > On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote: > > On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote: > > > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote: > > > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a > > > > Xen instance is seeing. The default for 64bit is 32GB, which is the > > > > reason that m2.4xlarge Amazon EC2 instances only report this amount of > > > > memory. > > > > Please set this limit to 70GB as there is a known restriction for > > > > t1.micro instances at about 80GB. > > > > Similar bug exists and Ubuntu where it's already fixed > > > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796) > > > > > > Is this the sort of change we can consider making in a stable update? > > > I'm not at all sure, although my gut feeling is that it would be safe. > > [...] > > > > I think so. But what is the trade-off? There must be some reason why > > this isn't set to however many TB the kernel can support. > > It effects the amount of space set aside for the P2M table (the mapping > of physical to machine addresses). In the kernel in Squeeze this space > is statically reserved in BSS so increasing it will waste some more > memory, according to the Kconfig comment it is 1 page per GB. > > In a more up to date kernel the space comes from BRK and is reclaimed if > it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the > same change. How intrusive is the change? Could we reasonably backport it? Ben. -- Ben Hutchings Quantity is no substitute for quality, but it's the only one we've got. signature.asc Description: This is a digitally signed message part
Bug#645052: kernel only recognizes 32G of memory
On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote: > On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote: > > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote: > > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a > > > Xen instance is seeing. The default for 64bit is 32GB, which is the > > > reason that m2.4xlarge Amazon EC2 instances only report this amount of > > > memory. > > > Please set this limit to 70GB as there is a known restriction for > > > t1.micro instances at about 80GB. > > > Similar bug exists and Ubuntu where it's already fixed > > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796) > > > > Is this the sort of change we can consider making in a stable update? > > I'm not at all sure, although my gut feeling is that it would be safe. > [...] > > I think so. But what is the trade-off? There must be some reason why > this isn't set to however many TB the kernel can support. It effects the amount of space set aside for the P2M table (the mapping of physical to machine addresses). In the kernel in Squeeze this space is statically reserved in BSS so increasing it will waste some more memory, according to the Kconfig comment it is 1 page per GB. In a more up to date kernel the space comes from BRK and is reclaimed if it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the same change. Ian. -- Ian Campbell Current Noise: Devin Townsend - Namaste The only way to learn a new programming language is by writing programs in it. -- Brian Kernighan -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#645052: kernel only recognizes 32G of memory
On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote: > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote: > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a > > Xen instance is seeing. The default for 64bit is 32GB, which is the > > reason that m2.4xlarge Amazon EC2 instances only report this amount of > > memory. > > Please set this limit to 70GB as there is a known restriction for > > t1.micro instances at about 80GB. > > Similar bug exists and Ubuntu where it's already fixed > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796) > > Is this the sort of change we can consider making in a stable update? > I'm not at all sure, although my gut feeling is that it would be safe. [...] I think so. But what is the trade-off? There must be some reason why this isn't set to however many TB the kernel can support. Ben. -- Ben Hutchings If at first you don't succeed, you're doing about average. signature.asc Description: This is a digitally signed message part
Bug#645052: kernel only recognizes 32G of memory
On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote: > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a > Xen instance is seeing. The default for 64bit is 32GB, which is the > reason that m2.4xlarge Amazon EC2 instances only report this amount of > memory. > Please set this limit to 70GB as there is a known restriction for > t1.micro instances at about 80GB. > Similar bug exists and Ubuntu where it's already fixed > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796) Is this the sort of change we can consider making in a stable update? I'm not at all sure, although my gut feeling is that it would be safe. The 3.0 kernels in unstable (and testing?) right now have XEN_MAX_DOMAIN_MEMORY=128, I think they use a more dynamic scheme for this limit than the 2.6.32 kernels did too. Ian. -- Ian Campbell Captain Penny's Law: You can fool all of the people some of the time, and some of the people all of the time, but you can't fool mom. signature.asc Description: This is a digitally signed message part
Bug#645052: kernel only recognizes 32G of memory
Package: linux-2.6 Version: 2.6.32-38 Severity: important Tags: squeeze -- Package-specific info: ** Version: Linux version 2.6.32-5-amd64 (Debian 2.6.32-38) (b...@decadent.org.uk) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 SMP Mon Oct 3 03:59:20 UTC 2011 -- System Information: Debian Release: 6.0.3 APT prefers proposed-updates APT policy: (992, 'proposed-updates'), (991, 'stable-updates'), (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.32-5-amd64 (SMP w/4 CPU cores) Locale: LANG=ru_RU.UTF-8, LC_CTYPE=ru_RU.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash -- debconf information excluded The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a Xen instance is seeing. The default for 64bit is 32GB, which is the reason that m2.4xlarge Amazon EC2 instances only report this amount of memory. Please set this limit to 70GB as there is a known restriction for t1.micro instances at about 80GB. Similar bug exists and Ubuntu where it's already fixed (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796) -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
