Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 2012-05-30 at 20:18 -0500, Stan Hoeppner wrote: On 5/30/2012 8:55 AM, Seyyed Mohtadin Hashemi wrote: On Tue, 2012-05-29 at 15:48 -0500, Stan Hoeppner wrote: On 5/29/2012 4:08 PM, Seyyed Mohtadin Hashemi wrote: On Tue, 2012-05-15 at 21:26 +0200, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 8:51 PM, Stan Hoeppner s...@hardwarefreak.com wrote: On 5/15/2012 12:26 PM, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 4:30 AM, Henrique de Moraes Holschuh h...@debian.org wrote: On Mon, 14 May 2012, Stan Hoeppner wrote: On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Well, it is the first time I've seen a BIOS screw it up so badly as to have someone lose 61GiB of RAM over it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. We _really_ need to have that enabled by default on the Debian kernels IMO, if we don't enable it already. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Thank you for the tips Henrique and Stan, unfortunately i don't have time to build/test new kernels this week because i have to finish my thesis. I will have time next week to look at it and report back the results. In that case you could simply install the latest backport kernel image and see if that does the trick. Should be quick 'n painless. Add to /etc/apt/sources.list deb http://backports.debian.org/debian-backports squeeze-backports \ main contrib non-free $ aptitude update $ aptitude -t squeeze-backports install linux-image-3.2.0-0.bpo.1-amd64 $ shutdown -r now Should take less than 5 minutes. -- Stan Funny you should mention that, I did actually try the exact kernel you mentioned yesterday - it did not go well, i got kernel panic. I didn't do many tests because i didn't have much time, i went back to the old kernel, and though i'm not happy with the situation the computer at least works and i can use the CPU to do calculations. Hi Stan, I RMA'd the MB and with the replacement I received I am able to run the 3.2 kernel and all installed RAM is usable. However, I have to use noapic irqpoll acpi=force boot flags. Needing some boot flags with some main boards isn't uncommon. And in fact using various boot flags used to be (maybe still is) needed to get Linux VMs running properly on VMWare ESX, specifically the system clock. So the boot flags are just a bare metal hardware issue. I did have a small problem, sometimes I would get RAM R/W test fail at BIOS POST. I had done extensive memtest on the DIMMs earlier so I only tested if the individual DIMMs could POST, only one gave
SOLVED: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
Summary: Debian Squeeze 64-bit was not able to use more than 3.6GB RAM. Linux showed the problem as MTRR not covering all available RAM. MTRR problems can be fixed by kernel if MTRR repair is enabled, e.g. in the 3.2 kernel. However, the using the backported 3.2 kernel resulted in kernel panic. As RAM had been tested extensively with memtest, the problem was hypothesized to be because of faulty motherboard or BIOS. One RMA of MB later, and everything is working with the 3.2 kernel and noapic irqpoll acpi=force boot flags. I did have a small problem: Sometimes I would get RAM R/W test fail at BIOS POST. I tested if the individual DIMMs could POST, only one gave the RAM R/W test fail. After removing the 'faulty' DIMM + a healthy DIMM, the system works smoothly. I have not had time to test the 'faulty' DIMM in another computer yet to see if it really is faulty. Thank you all for your help! -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1338522341.4810.23.camel@smh-tablet
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Tue, 2012-05-29 at 15:48 -0500, Stan Hoeppner wrote: On 5/29/2012 4:08 PM, Seyyed Mohtadin Hashemi wrote: On Tue, 2012-05-15 at 21:26 +0200, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 8:51 PM, Stan Hoeppner s...@hardwarefreak.com wrote: On 5/15/2012 12:26 PM, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 4:30 AM, Henrique de Moraes Holschuh h...@debian.org wrote: On Mon, 14 May 2012, Stan Hoeppner wrote: On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Well, it is the first time I've seen a BIOS screw it up so badly as to have someone lose 61GiB of RAM over it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. We _really_ need to have that enabled by default on the Debian kernels IMO, if we don't enable it already. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Thank you for the tips Henrique and Stan, unfortunately i don't have time to build/test new kernels this week because i have to finish my thesis. I will have time next week to look at it and report back the results. In that case you could simply install the latest backport kernel image and see if that does the trick. Should be quick 'n painless. Add to /etc/apt/sources.list deb http://backports.debian.org/debian-backports squeeze-backports \ main contrib non-free $ aptitude update $ aptitude -t squeeze-backports install linux-image-3.2.0-0.bpo.1-amd64 $ shutdown -r now Should take less than 5 minutes. -- Stan Funny you should mention that, I did actually try the exact kernel you mentioned yesterday - it did not go well, i got kernel panic. I didn't do many tests because i didn't have much time, i went back to the old kernel, and though i'm not happy with the situation the computer at least works and i can use the CPU to do calculations. Hi Stan, I RMA'd the MB and with the replacement I received I am able to run the 3.2 kernel and all installed RAM is usable. However, I have to use noapic irqpoll acpi=force boot flags. Needing some boot flags with some main boards isn't uncommon. And in fact using various boot flags used to be (maybe still is) needed to get Linux VMs running properly on VMWare ESX, specifically the system clock. So the boot flags are just a bare metal hardware issue. I did have a small problem, sometimes I would get RAM R/W test fail at BIOS POST. I had done extensive memtest on the DIMMs earlier so I only tested if the individual DIMMs could POST, only one gave the RAM R/W test fail. After removing the faulty DIMM + a healthy DIMM the system works smoothly. What
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 5/30/2012 8:55 AM, Seyyed Mohtadin Hashemi wrote: On Tue, 2012-05-29 at 15:48 -0500, Stan Hoeppner wrote: On 5/29/2012 4:08 PM, Seyyed Mohtadin Hashemi wrote: On Tue, 2012-05-15 at 21:26 +0200, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 8:51 PM, Stan Hoeppner s...@hardwarefreak.com wrote: On 5/15/2012 12:26 PM, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 4:30 AM, Henrique de Moraes Holschuh h...@debian.org wrote: On Mon, 14 May 2012, Stan Hoeppner wrote: On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Well, it is the first time I've seen a BIOS screw it up so badly as to have someone lose 61GiB of RAM over it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. We _really_ need to have that enabled by default on the Debian kernels IMO, if we don't enable it already. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Thank you for the tips Henrique and Stan, unfortunately i don't have time to build/test new kernels this week because i have to finish my thesis. I will have time next week to look at it and report back the results. In that case you could simply install the latest backport kernel image and see if that does the trick. Should be quick 'n painless. Add to /etc/apt/sources.list deb http://backports.debian.org/debian-backports squeeze-backports \ main contrib non-free $ aptitude update $ aptitude -t squeeze-backports install linux-image-3.2.0-0.bpo.1-amd64 $ shutdown -r now Should take less than 5 minutes. -- Stan Funny you should mention that, I did actually try the exact kernel you mentioned yesterday - it did not go well, i got kernel panic. I didn't do many tests because i didn't have much time, i went back to the old kernel, and though i'm not happy with the situation the computer at least works and i can use the CPU to do calculations. Hi Stan, I RMA'd the MB and with the replacement I received I am able to run the 3.2 kernel and all installed RAM is usable. However, I have to use noapic irqpoll acpi=force boot flags. Needing some boot flags with some main boards isn't uncommon. And in fact using various boot flags used to be (maybe still is) needed to get Linux VMs running properly on VMWare ESX, specifically the system clock. So the boot flags are just a bare metal hardware issue. I did have a small problem, sometimes I would get RAM R/W test fail at BIOS POST. I had done extensive memtest on the DIMMs earlier so I only tested if the individual DIMMs could POST, only one gave the RAM R/W test fail. After removing the faulty DIMM + a healthy DIMM the system works smoothly. What replacement board board did you get? Another ASUS or a SuperMicro? I
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Tue, 2012-05-15 at 21:26 +0200, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 8:51 PM, Stan Hoeppner s...@hardwarefreak.com wrote: On 5/15/2012 12:26 PM, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 4:30 AM, Henrique de Moraes Holschuh h...@debian.org wrote: On Mon, 14 May 2012, Stan Hoeppner wrote: On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Well, it is the first time I've seen a BIOS screw it up so badly as to have someone lose 61GiB of RAM over it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. We _really_ need to have that enabled by default on the Debian kernels IMO, if we don't enable it already. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Thank you for the tips Henrique and Stan, unfortunately i don't have time to build/test new kernels this week because i have to finish my thesis. I will have time next week to look at it and report back the results. In that case you could simply install the latest backport kernel image and see if that does the trick. Should be quick 'n painless. Add to /etc/apt/sources.list deb http://backports.debian.org/debian-backports squeeze-backports \ main contrib non-free $ aptitude update $ aptitude -t squeeze-backports install linux-image-3.2.0-0.bpo.1-amd64 $ shutdown -r now Should take less than 5 minutes. -- Stan Funny you should mention that, I did actually try the exact kernel you mentioned yesterday - it did not go well, i got kernel panic. I didn't do many tests because i didn't have much time, i went back to the old kernel, and though i'm not happy with the situation the computer at least works and i can use the CPU to do calculations. Hi Stan, I RMA'd the MB and with the replacement I received I am able to run the 3.2 kernel and all installed RAM is usable. However, I have to use noapic irqpoll acpi=force boot flags. I did have a small problem, sometimes I would get RAM R/W test fail at BIOS POST. I had done extensive memtest on the DIMMs earlier so I only tested if the individual DIMMs could POST, only one gave the RAM R/W test fail. After removing the faulty DIMM + a healthy DIMM the system works smoothly. regards, Mohtadin -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1338325696.2251.68.camel@smh-tablet
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 5/29/2012 4:08 PM, Seyyed Mohtadin Hashemi wrote: On Tue, 2012-05-15 at 21:26 +0200, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 8:51 PM, Stan Hoeppner s...@hardwarefreak.com wrote: On 5/15/2012 12:26 PM, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 4:30 AM, Henrique de Moraes Holschuh h...@debian.org wrote: On Mon, 14 May 2012, Stan Hoeppner wrote: On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Well, it is the first time I've seen a BIOS screw it up so badly as to have someone lose 61GiB of RAM over it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. We _really_ need to have that enabled by default on the Debian kernels IMO, if we don't enable it already. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Thank you for the tips Henrique and Stan, unfortunately i don't have time to build/test new kernels this week because i have to finish my thesis. I will have time next week to look at it and report back the results. In that case you could simply install the latest backport kernel image and see if that does the trick. Should be quick 'n painless. Add to /etc/apt/sources.list deb http://backports.debian.org/debian-backports squeeze-backports \ main contrib non-free $ aptitude update $ aptitude -t squeeze-backports install linux-image-3.2.0-0.bpo.1-amd64 $ shutdown -r now Should take less than 5 minutes. -- Stan Funny you should mention that, I did actually try the exact kernel you mentioned yesterday - it did not go well, i got kernel panic. I didn't do many tests because i didn't have much time, i went back to the old kernel, and though i'm not happy with the situation the computer at least works and i can use the CPU to do calculations. Hi Stan, I RMA'd the MB and with the replacement I received I am able to run the 3.2 kernel and all installed RAM is usable. However, I have to use noapic irqpoll acpi=force boot flags. Needing some boot flags with some main boards isn't uncommon. And in fact using various boot flags used to be (maybe still is) needed to get Linux VMs running properly on VMWare ESX, specifically the system clock. So the boot flags are just a bare metal hardware issue. I did have a small problem, sometimes I would get RAM R/W test fail at BIOS POST. I had done extensive memtest on the DIMMs earlier so I only tested if the individual DIMMs could POST, only one gave the RAM R/W test fail. After removing the faulty DIMM + a healthy DIMM the system works smoothly. What replacement board board did you get? Another ASUS or a SuperMicro? -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble?
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 5/29/2012 3:48 PM, Stan Hoeppner wrote: So the boot flags are just a bare metal hardware issue. This should have read are NOT just a bare metal issue. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fc57d2f.8040...@hardwarefreak.com
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Tue, May 15, 2012 at 4:30 AM, Henrique de Moraes Holschuh h...@debian.org wrote: On Mon, 14 May 2012, Stan Hoeppner wrote: On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Well, it is the first time I've seen a BIOS screw it up so badly as to have someone lose 61GiB of RAM over it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. We _really_ need to have that enabled by default on the Debian kernels IMO, if we don't enable it already. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Thank you for the tips Henrique and Stan, unfortunately i don't have time to build/test new kernels this week because i have to finish my thesis. I will have time next week to look at it and report back the results. best regards, Mohtadin
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 5/15/2012 12:26 PM, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 4:30 AM, Henrique de Moraes Holschuh h...@debian.org wrote: On Mon, 14 May 2012, Stan Hoeppner wrote: On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Well, it is the first time I've seen a BIOS screw it up so badly as to have someone lose 61GiB of RAM over it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. We _really_ need to have that enabled by default on the Debian kernels IMO, if we don't enable it already. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Thank you for the tips Henrique and Stan, unfortunately i don't have time to build/test new kernels this week because i have to finish my thesis. I will have time next week to look at it and report back the results. In that case you could simply install the latest backport kernel image and see if that does the trick. Should be quick 'n painless. Add to /etc/apt/sources.list deb http://backports.debian.org/debian-backports squeeze-backports \ main contrib non-free $ aptitude update $ aptitude -t squeeze-backports install linux-image-3.2.0-0.bpo.1-amd64 $ shutdown -r now Should take less than 5 minutes. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fb2a59f.6010...@hardwarefreak.com
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Tue, May 15, 2012 at 8:51 PM, Stan Hoeppner s...@hardwarefreak.comwrote: On 5/15/2012 12:26 PM, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 4:30 AM, Henrique de Moraes Holschuh h...@debian.org wrote: On Mon, 14 May 2012, Stan Hoeppner wrote: On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Well, it is the first time I've seen a BIOS screw it up so badly as to have someone lose 61GiB of RAM over it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. We _really_ need to have that enabled by default on the Debian kernels IMO, if we don't enable it already. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Thank you for the tips Henrique and Stan, unfortunately i don't have time to build/test new kernels this week because i have to finish my thesis. I will have time next week to look at it and report back the results. In that case you could simply install the latest backport kernel image and see if that does the trick. Should be quick 'n painless. Add to /etc/apt/sources.list deb http://backports.debian.org/debian-backports squeeze-backports \ main contrib non-free $ aptitude update $ aptitude -t squeeze-backports install linux-image-3.2.0-0.bpo.1-amd64 $ shutdown -r now Should take less than 5 minutes. -- Stan Funny you should mention that, I did actually try the exact kernel you mentioned yesterday - it did not go well, i got kernel panic. I didn't do many tests because i didn't have much time, i went back to the old kernel, and though i'm not happy with the situation the computer at least works and i can use the CPU to do calculations.
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 5/15/2012 2:26 PM, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 8:51 PM, Stan Hoeppner s...@hardwarefreak.comwrote: On 5/15/2012 12:26 PM, Seyyed Mohtadin Hashemi wrote: On Tue, May 15, 2012 at 4:30 AM, Henrique de Moraes Holschuh h...@debian.org wrote: On Mon, 14 May 2012, Stan Hoeppner wrote: On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Well, it is the first time I've seen a BIOS screw it up so badly as to have someone lose 61GiB of RAM over it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. We _really_ need to have that enabled by default on the Debian kernels IMO, if we don't enable it already. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Thank you for the tips Henrique and Stan, unfortunately i don't have time to build/test new kernels this week because i have to finish my thesis. I will have time next week to look at it and report back the results. In that case you could simply install the latest backport kernel image and see if that does the trick. Should be quick 'n painless. Add to /etc/apt/sources.list deb http://backports.debian.org/debian-backports squeeze-backports \ main contrib non-free $ aptitude update $ aptitude -t squeeze-backports install linux-image-3.2.0-0.bpo.1-amd64 $ shutdown -r now Should take less than 5 minutes. -- Stan Funny you should mention that, I did actually try the exact kernel you mentioned yesterday - it did not go well, i got kernel panic. I didn't do many tests because i didn't have much time, i went back to the old kernel, and though i'm not happy with the situation the computer at least works and i can use the CPU to do calculations. That Asus board just doesn't seem to want to cooperate. At this point I'd suggest swapping it for a Supermicro H8DGi. IIRC you were already prepared to send it back at the point I entered this thread, and that you acquired this Asus because your preferred board wasn't available at your preferred vendor. My apologies for causing you to delay. I thought we might be able to get the Asus board working. It's worth noting that Asus has a total of only *3* socket G34 mobos on the market, only one is a standard form factor, and all 3 are dual socket boards. This shows that their volume is very low, and that Asus doesn't have much experience with the socket G34 platform. This tends to explain the development immaturity of this board, as well as the large number of BIOS updates since it was released, each one likely the result of a single customer report of a problem, with the exception of the update to support Bulldozer CPUs. For comparison, Supermicro has 22 socket G34 boards on the market, including single, dual, and quad socket boards, supporting from 128GB to 1TB RAM (32x32GB DDR-1066 RDIMMs) in max configurations. They offer many packaged servers based on these boards. This demonstrates they are shipping a large volume of G34 systems and have mature product. This product maturity and quality is the reason I've been a fan of Supermicro for 15 years. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fb2d8a5.3010...@hardwarefreak.com
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. .config - Linux/i386 3.2.6 Kernel Configuration ... Saying Y here also fixes a problem with buggy SMP BIOSes which only set the MTRRs for the boot CPU and not for the secondary CPUs. This can lead to all sorts of problems, so it's good to say Y here. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fb17636.8000...@hardwarefreak.com
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Mon, 14 May 2012, Stan Hoeppner wrote: On 5/13/2012 7:02 PM, Henrique de Moraes Holschuh wrote: On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: On 5/10/2012 1:16 PM, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. I'm not surprised I was correct WRT a BIOS bug, but I am a little embarrassed I didn't know and suggest this would be reported in dmesg. I admit I just don't see this very often--this being the 1st time actually seeing this WARNING. Well, it is the first time I've seen a BIOS screw it up so badly as to have someone lose 61GiB of RAM over it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: Yep. In vanilla 3.2.6 it's selected by default in menuconfig, and you can't un-select it. We _really_ need to have that enabled by default on the Debian kernels IMO, if we don't enable it already. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120515023008.ga6...@khazad-dum.debian.net
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Fri, 11 May 2012, Seyyed Mohtadin Hashemi wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM. There you have it. Any of the latest versions of the longterm kernels (2.6.32, 3.0), or latest 3.2 should be able to repair MTRRs properly, but you have to compile the kernel with that option enabled. It might be already available, but not enabled by default. In that case, this might help you: http://hawknotes.blogspot.com.br/2010/05/ubuntu-1004-fixing-mtrr-errors.html and http://phoronix.com/forums/showthread.php?16940-bad-MTRR-setup-according-to-DRMp=74238#post74238 File a bug with the mainboard vendor, anyway. Give them the Linux dmesg bootlog you sent me, up to and including the warning above, so that they know what they're doing wrong. Also, run BITS on that mainboard and complain to the motherboard vendor about anything BITS reports as broken. http://www.biosbits.org/ -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120514000253.ga30...@khazad-dum.debian.net
Re: Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Thu, 2012-05-10 at 10:49 +0200, Seyyed Mohtadin Hashemi wrote: PS. I have unsubscribed from the main list and am only getting the digest, so everyone: When replying please CC to my mail also, otherwise You will have to wait till i get Your answer/question via the digest. Something I did prefer too, but it's not handy, since a default reply to a mailing list, should be reply to the mailing list only ;). OT: Is digest repaired? OT: Unfortunately no, it show all as Unidentified subject! and does not link sent/to addresses.
Re: Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Thu, 10 May 2012, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working case. BTW, you originally mentioned you pulled all 16 sticks from a production server that had run fine for months/years, and installed them in this system. Now you tell use you ordered the 16 Mushkin sticks for this build. This conflicting story leads me to wonder exactly what memory you have, where you're being straight with us, and where you're not. Makes it really hard to assist in this troubleshooting effort when we don't know what information is reliable. Indeed. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh Thanks for the info about dmesg! It says this: [0.00] BIOS-provided physical RAM map: [0.00] BIOS-e820: 0100 - 0009e800 (usable) [0.00] BIOS-e820: 0009e800 - 000a (reserved) [0.00] BIOS-e820: 000e6000 - 0010 (reserved) [0.00] BIOS-e820: 0010 - dff8 (usable) [0.00] BIOS-e820: dff8e000 - dff9 (reserved) [0.00] BIOS-e820: dff9 - dffa2000 (ACPI data) [0.00] BIOS-e820: dffa2000 - dffe (ACPI NVS) [0.00] BIOS-e820: dffe - f000 (reserved) [0.00] BIOS-e820: ffe0 - 0001 (reserved) [0.00] BIOS-e820: 0001 - 00101f00 (usable) [0.00] DMI present. [0.00] AMI BIOS detected: BIOS may corrupt low RAM, working around it. [0.00] e820 update range: - 0001 (usable) == (reserved) [0.00] last_pfn = 0x101f000 max_arch_pfn = 0x4 [0.00] MTRR default type: uncachable [0.00] MTRR fixed ranges enabled: [0.00] 0-9 write-back [0.00] A-E uncachable [0.00] F-F write-protect [0.00] MTRR variable ranges enabled: [0.00] 0 base mask 8000 write-back [0.00] 1 base 8000 mask C000 write-back [0.00] 2 base C000 mask E000 write-back [0.00] 3 disabled [0.00] 4 disabled [0.00] 5 disabled [0.00] 6 disabled [0.00] 7 disabled [0.00] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 [0.00] e820 update range: e000 - 00101f00 (usable) == (reserved) [0.00] WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 61936MB of RAM.
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
Sorry for double and top post. Pushed the wrong button. I was saying that the RAM are all Mushkin 996770, i was also going to buy a SuperMicro board as i have very good experience with those but i had to wait 1 mo th for it, time that i dont have. Again i went with the Mushkin dimms because i could get them quickly, otherwise i would have bought kingston rdimms which i use in all my other computers. I now know that my kingston dimms don't work on the board, so it must be a BIOS issue. I say this because in the BIOS changelog there is an entry for update concerning RAM not recognized by OS, which they claim has been fixed. Regards, Mohtadin Stan Hoeppner s...@hardwarefreak.com wrote: On 5/9/2012 11:43 AM, Seyyed Mohtadin Hashemi wrote: An update: I tried CentOS 6.1 (the only one i had at hand) and it gave the exact same result as Debian. I have in the meantime tried to use other RAM modules, and unfortunately they also did not give more than 3.5gb. Considering that i pulled the ram from a 24/7 stable system i assume that it is indeed the motherboard/BIOS that does not function properly. That is not a safe assumption given how picky socket G34 is with memory channel configuration. I will return the motherboard and see if i can get one that works. Before you return the board, would you mind posting the manufacturer, part number, and specs of each of the 16 DIMMs you have installed? They should all match. Are they all indeed the same part# from the same vendor? Are they unbuffered or registered? Are they single, dual, or quad rank modules? All of these things matter, greatly. Unfortunately, Asus doesn't provide the proper table in the manual for this board that shows which combinations of speed, density, rank, and un/buffered DIMMs work in socket G34. SuperMicro does, and frankly they're the only vendor of AMD server boards I'd ever use because of their attention to all the details. Socket G34 is fairly picky regarding DIMM configurations, especially with high density (8/16GB) modules, though not as picky with low density (1/2/4GB) modules. 16 identical unbuffered, or registered, 4GB DDR3-1333 SR or DR DIMMs *should* work, so long as you don't put both unbuffered and buffered DIMMs in the same two slot channel. -- Stan
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
10.05.2012 08:36, Mohtadin Hashemi kirjoitti: Sorry for top post, stupid phone wont let me do anything else. You seem to be using Android. There is email client called K9 Mail in the Google Play Store, which can bottom post (and also sign and decrypt clearsigned emails). -- [Mika Suomalainen](https://mkaysi.github.com/) || [gpg --keyserver pool.sks-keyservers.net --recv-keys 4DB53CFE82A46728](http://mkaysi.github.com/PGP/key.txt) || [Why do I sign my emails?](http://mkaysi.github.com/PGP/WhyDoISignEmails.html) || [Please don't send HTML.](http://mkaysi.github.com/articles/complaining/HTML.html) || [Please don't toppost](http://mkaysi.github.com/articles/complaining/topposting.html) || [This signature](https://gist.github.com/2643070) || signature.asc Description: OpenPGP digital signature
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 09 May 2012 17:35:51 -0500, Stan wrote in message 4faaf147.7010...@hardwarefreak.com: On 5/9/2012 11:43 AM, Seyyed Mohtadin Hashemi wrote: An update: I tried CentOS 6.1 (the only one i had at hand) and it gave the exact same result as Debian. I have in the meantime tried to use other RAM modules, and unfortunately they also did not give more than 3.5gb. Considering that i pulled the ram from a 24/7 stable system i assume that it is indeed the motherboard/BIOS that does not function properly. That is not a safe assumption given how picky socket G34 is with memory channel configuration. I will return the motherboard and see if i can get one that works. Before you return the board, would you mind posting the manufacturer, part number, and specs of each of the 16 DIMMs you have installed? They should all match. Are they all indeed the same part# from the same vendor? Are they unbuffered or registered? Are they single, dual, or quad rank modules? All of these things matter, greatly. Unfortunately, Asus doesn't provide the proper table in the manual for this board that shows which combinations of speed, density, rank, and un/buffered DIMMs work in socket G34. SuperMicro does, and frankly they're the only vendor of AMD server boards I'd ever use because of their attention to all the details. Socket G34 is fairly picky regarding DIMM configurations, especially with high density (8/16GB) modules, though not as picky with low density (1/2/4GB) modules. 16 identical unbuffered, or registered, 4GB DDR3-1333 SR or DR DIMMs *should* work, so long as you don't put both unbuffered and buffered DIMMs in the same two slot channel. ..a test suggestion: remove 2 of the 16 ram sticks and make sure the remaining 14 is set up according to the main board manual's ram stick map, I saw it posted in this thread. If this test works, you should see 56G once booted up. -- ..med vennlig hilsen = with Kind Regards from Arnt Karlsen ...with a number of polar bear hunters in his ancestry... Scenarios always come in sets of three: best case, worst case, and just in case. Sorry for not replying to the earlier suggestion, i missed the mail because i was being bombarded by mails. I did as you suggested and it did not work. In the meantime a bigger problem has arisen: During the tests i most have done something because now 48GB is not working either. I have not done anything to the OS but i have been resetting the CMOS every time i changed the RAM DIMMS. PS. I have unsubscribed from the main list and am only getting the digest, so everyone: When replying please CC to my mail also, otherwise You will have to wait till i get Your answer/question via the digest.
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Thu, 2012-05-10 at 10:49 +0200, Seyyed Mohtadin Hashemi wrote: PS. I have unsubscribed from the main list and am only getting the digest, so everyone: When replying please CC to my mail also, otherwise You will have to wait till i get Your answer/question via the digest. Something I did prefer too, but it's not handy, since a default reply to a mailing list, should be reply to the mailing list only ;). OT: Is digest repaired? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336652087.2307.10.camel@precise
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 5/10/2012 12:47 AM, Mohtadin Hashemi wrote: Sorry for double and top post. Pushed the wrong button. I was saying that the RAM are all Mushkin 996770, i was also going to buy a SuperMicro board as i have very good experience with those but i had to wait 1 mo th for it, time that i dont have. Again i went with the Mushkin dimms because i could get them quickly, otherwise i would have bought kingston rdimms which i use in all my other computers. I now know that my kingston dimms don't work on the board, so it must be a BIOS issue. I say this because in the BIOS changelog there is an entry for update concerning RAM not recognized by OS, which they claim has been fixed. Socket G34 is able to work with 8x 4GB SR or DR unbuffered DIMMs at 1333MHz at 1.5v, even 1600Mhz at 1.5v with 6200 series CPUs. So this may be a board specific problem. You mentioned that removing 4 DIMMs allowed Debian to recognize 48GB. Try this... With all 16 DIMMs installed, enter the system BIOS and change Memory CLK in the Advanced/Northbridge menu to 533MHz, save and reboot. This should drop the memory bus frequency from 1333 to 1066. If the problem is due to electrical bus loading at 1333MHz, lowering the freq may help. This is probably a log shot, but worth testing as it's quick/simple. If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. BTW, you originally mentioned you pulled all 16 sticks from a production server that had run fine for months/years, and installed them in this system. Now you tell use you ordered the 16 Mushkin sticks for this build. This conflicting story leads me to wonder exactly what memory you have, where you're being straight with us, and where you're not. Makes it really hard to assist in this troubleshooting effort when we don't know what information is reliable. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fac060f.1080...@hardwarefreak.com
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Thu, May 10, 2012 at 8:16 PM, Stan Hoeppner s...@hardwarefreak.comwrote: On 5/10/2012 12:47 AM, Mohtadin Hashemi wrote: Sorry for double and top post. Pushed the wrong button. I was saying that the RAM are all Mushkin 996770, i was also going to buy a SuperMicro board as i have very good experience with those but i had to wait 1 mo th for it, time that i dont have. Again i went with the Mushkin dimms because i could get them quickly, otherwise i would have bought kingston rdimms which i use in all my other computers. I now know that my kingston dimms don't work on the board, so it must be a BIOS issue. I say this because in the BIOS changelog there is an entry for update concerning RAM not recognized by OS, which they claim has been fixed. Socket G34 is able to work with 8x 4GB SR or DR unbuffered DIMMs at 1333MHz at 1.5v, even 1600Mhz at 1.5v with 6200 series CPUs. So this may be a board specific problem. You mentioned that removing 4 DIMMs allowed Debian to recognize 48GB. Try this... With all 16 DIMMs installed, enter the system BIOS and change Memory CLK in the Advanced/Northbridge menu to 533MHz, save and reboot. This should drop the memory bus frequency from 1333 to 1066. If the problem is due to electrical bus loading at 1333MHz, lowering the freq may help. This is probably a log shot, but worth testing as it's quick/simple. If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. BTW, you originally mentioned you pulled all 16 sticks from a production server that had run fine for months/years, and installed them in this system. Now you tell use you ordered the 16 Mushkin sticks for this build. This conflicting story leads me to wonder exactly what memory you have, where you're being straight with us, and where you're not. Makes it really hard to assist in this troubleshooting effort when we don't know what information is reliable. -- Stan I have not been clear then: The RAM that was pulled from a working environment was/is 6 Kingston RDIMMs with 8GB each. The Mushkin RAM was ordered specifically for this server/board. As for lowering the frequency, I did actually try; both 533 and 400 MHz - neither helped. I wrote a few e-mails back that the 48GB config does not work anymore - lending more credential to the BIOS is at fault theory. I have not touched the OS, just reset CMOS every time I have changed RAM modules. Other things I have tried are: 1) Disable the 2nd CPU via BIOS and install only one 8GB DIMM - this did not work, again only 3.5GB was shown. 1.a) Disable the 2nd CPU and install two 4GB DIMMs - does not work. 2) Gradually increase 4GB DIMM count from 2 to 16 - the same result as all the other times. I don't know any other utilities that do a similar job as memtest, otherwise I would have tried it. As it is now, only memtest and BIOS can recognize all 64GB RAM.
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Thu, 2012-05-10 at 21:30 +0200, Seyyed Mohtadin Hashemi wrote: As it is now, only memtest and BIOS can recognize all 64GB RAM. Again, 32-bit PAE ;)? Unfortunately I couldn't find a live media with a PAE. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336679964.5199.53.camel@precise
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Thu, 10 May 2012, Stan Hoeppner wrote: If this doesn't fix the issue, and memtest and other utils can see all 64GB just fine, then I'd say you're dealing with a BIOS bug. The very top of /var/log/dmesg has the kernel debug output about the memory map. It might well tell us very quickly who is the culprit, if the user with the problem can post it for the best working case and the non-working case. BTW, you originally mentioned you pulled all 16 sticks from a production server that had run fine for months/years, and installed them in this system. Now you tell use you ordered the 16 Mushkin sticks for this build. This conflicting story leads me to wonder exactly what memory you have, where you're being straight with us, and where you're not. Makes it really hard to assist in this troubleshooting effort when we don't know what information is reliable. Indeed. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120510211052.ga21...@khazad-dum.debian.net
Only 3.6gb of 64gb RAM recognized by 64bit squeeze
Hello, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 As the subject suggest I have a box that does not utilize the available RAM installed. I noticed that only 3.6gb RAM was recognized when I got segmentation faults during a simulation. The funny thing is that when I remove dims so that only 48gb RAM is available then it works fine, I did a 'lshw -C memory' and it shows all the dims at the correct spot (the output is attached). BIOS and memtest show and successfully test all 64gb. cat /proc/meminfo shows: # cat /proc/meminfo MemTotal:3607688 kB MemFree: 2802508 kB Buffers: 0 kB Cached: 704164 kB SwapCached:0 kB Active:25724 kB Inactive: 684068 kB Active(anon): 5720 kB Inactive(anon):0 kB Active(file): 20004 kB Inactive(file): 684068 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 5716 kB Mapped: 4504 kB Shmem:92 kB Slab: 49784 kB SReclaimable: 29840 kB SUnreclaim:19944 kB KernelStack:3464 kB PageTables: 1196 kB NFS_Unstable: 0 kB Bounce:0 kB WritebackTmp: 0 kB CommitLimit: 1803844 kB Committed_AS: 37792 kB VmallocTotal: 34359738367 kB VmallocUsed: 283296 kB VmallocChunk: 34359449396 kB HardwareCorrupted: 0 kB HugePages_Total: 0 HugePages_Free:0 HugePages_Rsvd:0 HugePages_Surp:0 Hugepagesize: 2048 kB DirectMap4k:5632 kB DirectMap2M: 1566720 kB DirectMap1G: 2097152 kB Hope somebody can help me fix this, I cannot think of anything that could cause this Best regards, Mohtadin *-firmware description: BIOS vendor: American Megatrends Inc. physical id: 0 version: 2102 (12/29/2011) size: 64KiB capacity: 1984KiB capabilities: isa pci pnp upgrade shadowing escd cdboot bootselect socketedrom edd int13floppy1200 int13floppy720 int13floppy2880 int5printscreen int9keyboard int14serial int17printer int10video acpi usb ls120boot zipboot biosbootspecification *-cache:0 description: L1 cache physical id: 5 slot: L1-Cache size: 768KiB capacity: 768KiB clock: 1GHz (1.0ns) capabilities: pipeline-burst internal write-back unified *-cache:1 description: L2 cache physical id: 6 slot: L2-Cache size: 16MiB capacity: 16MiB clock: 1GHz (1.0ns) capabilities: pipeline-burst internal write-back unified *-cache:2 description: L3 cache physical id: 7 slot: L3-Cache size: 12MiB capacity: 12MiB clock: 1GHz (1.0ns) capabilities: pipeline-burst internal write-back unified *-cache:0 description: L1 cache physical id: 9 slot: L1-Cache size: 768KiB capacity: 768KiB clock: 1GHz (1.0ns) capabilities: pipeline-burst internal write-back unified *-cache:1 description: L2 cache physical id: a slot: L2-Cache size: 16MiB capacity: 16MiB clock: 1GHz (1.0ns) capabilities: pipeline-burst internal write-back unified *-cache:2 description: L3 cache physical id: b slot: L3-Cache size: 12MiB capacity: 12MiB clock: 1GHz (1.0ns) capabilities: pipeline-burst internal write-back unified *-memory description: System Memory physical id: 3f slot: System board or motherboard size: 64GiB *-bank:0 description: DIMM Synchronous 1333 MHz (0.8 ns) product: ModulePartNumber00 vendor: Manufacturer00 physical id: 0 serial: SerNum00 slot: DIMM_A1 size: 4GiB width: 64 bits clock: 1333MHz (0.8ns) *-bank:1 description: DIMM Synchronous 1333 MHz (0.8 ns) product: ModulePartNumber01 vendor: Manufacturer01 physical id: 1 serial: SerNum01 slot: DIMM_A2 size: 4GiB width: 64 bits clock: 1333MHz (0.8ns) *-bank:2 description: DIMM Synchronous 1333 MHz (0.8 ns) product: ModulePartNumber02 vendor: Manufacturer02 physical id: 2 serial: SerNum02 slot: DIMM_B1 size: 4GiB width: 64 bits clock: 1333MHz (0.8ns) *-bank:3 description: DIMM Synchronous 1333 MHz (0.8 ns) product: ModulePartNumber03 vendor: Manufacturer03 physical id: 3 serial: SerNum03 slot: DIMM_B2 size: 4GiB width: 64 bits clock: 1333MHz (0.8ns) *-bank:4
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 09/05/12 19:19, Seyyed Mohtadin Hashemi wrote: Hello, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03 mailto:root@n03:~# uname -r 2.6.32-3-amd64 root@n03 mailto:root@n03:~# uname -m x86_64 As the subject suggest I have a box that does not utilize the available RAM installed. I noticed that only 3.6gb RAM was recognized when I got segmentation faults during a simulation. The funny thing is that when I remove dims so that only 48gb RAM is available then it works fine, I did a 'lshw -C memory' and it shows all the dims at the correct spot (the output is attached). BIOS and memtest show and successfully test all 64gb. Weird. What is the RAM slot arrangement? 8xdouble slots? What was the arrangement of the sticks when you got 48GB to show? Did it matter which sticks you used (for the 48GB)? Did it matter where those sticks were (for the 48GB)? What is your motherboard? Is this problem recent - or has it always been like this? snipped It looks like all the RAM sticks are identical - is that the case? Kind regards -- Iceweasel/Firefox/Chrome/Chromium/Iceape/IE extensions for finding answers to questions about Debian:- https://addons.mozilla.org/en-US/firefox/collections/Scott_Ferguson/debian/ -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4faa3b69.1010...@gmail.com
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 2012-05-09 at 11:19 +0200, Seyyed Mohtadin Hashemi wrote: Hello, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 As the subject suggest I have a box that does not utilize the available RAM installed. I noticed that only 3.6gb RAM was recognized when I got segmentation faults during a simulation. The funny thing is that when I remove dims so that only 48gb RAM is available then it works fine, I did a 'lshw -C memory' and it shows all the dims at the correct spot (the output is attached). BIOS and memtest show and successfully test all 64gb. A lot of people do have this issue on different distros. I noticed that 256MB of my 4GB where missing on 64-bit, while for a 32-bit PAE there where 4GB. I couldn't find a cause or solution in the Internet. On my machine I found out, that I'm using a NVIDIA graphics that has 256MB own RAM, but the proprietary settings thingy does show that it has got 512MB available. Perhaps your framebuffer is 60GB large :D. I've got no idea why this happened. Regards, Ralf -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336556652.2171.380.camel@precise
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, May 09, 2012 at 11:19:10AM +0200, Seyyed Mohtadin Hashemi wrote: Hello, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: [1]root@n03:~# uname -r 2.6.32-3-amd64 [2]root@n03:~# uname -m x86_64 [cut] Hope somebody can help me fix this, I cannot think of anything that could cause this This may be an issue with your kernel. Can you post the output of the following here? $ grep CONFIG_HIGHMEM /boot/config-$(uname -r) If you get # CONFIG_HIGHMEM64G is not set in the list, then you'll need to look at re-building your kernel to support RAM 64Gb. signature.asc Description: Digital signature
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
Did you check settings for the BIOS? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336558068.2171.385.camel@precise
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 2012-05-09 at 11:07 +0100, Darac Marjal wrote: On Wed, May 09, 2012 at 11:19:10AM +0200, Seyyed Mohtadin Hashemi wrote: Hello, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: [1]root@n03:~# uname -r 2.6.32-3-amd64 [2]root@n03:~# uname -m x86_64 [cut] Hope somebody can help me fix this, I cannot think of anything that could cause this This may be an issue with your kernel. Can you post the output of the following here? $ grep CONFIG_HIGHMEM /boot/config-$(uname -r) If you get # CONFIG_HIGHMEM64G is not set in the list, then you'll need to look at re-building your kernel to support RAM 64Gb. For a 64-bit kernel, there's no need to run $ grep CONFIG_HIGHMEM /boot/config-$(uname -r) this is nonsense. You're referring to 32-bit PAE kernels. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336558561.2171.388.camel@precise
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 9 May 2012 11:19:10 +0200, Seyyed wrote in message CAKJMja+mrpWmo8uhakjGfwsuk3W0=+a6lmbhfvxkkfcterf...@mail.gmail.com: Hello, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 As the subject suggest I have a box that does not utilize the available RAM installed. I noticed that only 3.6gb RAM was recognized when I got segmentation faults during a simulation. The funny thing is that when I remove dims so that only 48gb RAM is available then it works fine, I did a 'lshw -C memory' and it shows all the dims at the correct spot (the output is attached). BIOS and memtest show and successfully test all 64gb. ..post a one core snip of your /proc/cpuinfo, I'm wondering if you've hit some 36 bit address space ceiling. I have ... root@celsius:~# cat /proc/cpuinfo processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Core(TM)2 CPU T7200 @ 2.00GHz ... bogomips: 3989.93 clflush size: 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: ...and: root@celsius:~# qalc 2^36bytes to GiBytes (2^36) * byte = 64 gibibytes Hope somebody can help me fix this, I cannot think of anything that could cause this Best regards, Mohtadin -- ..med vennlig hilsen = with Kind Regards from Arnt Karlsen ...with a number of polar bear hunters in his ancestry... Scenarios always come in sets of three: best case, worst case, and just in case. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120509123221.5fbba...@celsius.lan
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 09/05/12 20:03, Seyyed Mohtadin Hashemi wrote: The motherboard is ASUS KGPED16 fitted with 2xOpteron 6200 series CPUs. For the 48gb configuration I just did what the motherboard manual suggested (i have attached the table), I have tried to shuffle the ram dimms and it works no matter which dimms are installed. The dimms are identical. Unfortunately i don't know if the problem was present before or not, I have just now discovered that only 3.6gb RAM is recognized. When I was building the box I did the customary memtest and it checked so I did not check anything else - i have not had any problems with my other boxes that use 48gb RAM so i though that this box also worked. I have not had time to do more than install OS and the simulation environment on the box. I suspect you've proved the RAM working - though to be totally sure you'd have to test each stick individually. If you did test each individual stick, and you're not running an Intel GPU [*1] then the problem should be the OS. I'd test each individual stick, ensure firmware is updated, check BIOS settings (memory remap) and also try a 3.2x kernel? [*1] AMD boards, large amounts of RAM, and Intel video have been problematic in the past. snipped Kind regards -- Iceweasel/Firefox/Chrome/Chromium/Iceape/IE extensions for finding answers to questions about Debian:- https://addons.mozilla.org/en-US/firefox/collections/Scott_Ferguson/debian/ -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4faa4871.3030...@gmail.com
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
Here you go: processor : 31 vendor_id : AuthenticAMD cpu family : 21 model : 1 model name : AMD Opteron(TM) Processor 6274 stepping: 2 cpu MHz : 2500.286 cache size : 2048 KB physical id : 1 siblings: 16 core id : 15 cpu cores : 16 apicid : 79 initial apicid : 47 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc nonstop_tsc extd_apicid pni pclmulqdq monitor ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs sse5 skinit wdt bogomips: 4399.96 TLB size: 1536 4K pages clflush size: 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm 100mhzsteps hwpstate [9] On Wed, May 9, 2012 at 12:32 PM, Arnt Karlsen a...@c2i.net wrote: On Wed, 9 May 2012 11:19:10 +0200, Seyyed wrote in message CAKJMja+mrpWmo8uhakjGfwsuk3W0=+a6lmbhfvxkkfcterf...@mail.gmail.com: Hello, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 As the subject suggest I have a box that does not utilize the available RAM installed. I noticed that only 3.6gb RAM was recognized when I got segmentation faults during a simulation. The funny thing is that when I remove dims so that only 48gb RAM is available then it works fine, I did a 'lshw -C memory' and it shows all the dims at the correct spot (the output is attached). BIOS and memtest show and successfully test all 64gb. ..post a one core snip of your /proc/cpuinfo, I'm wondering if you've hit some 36 bit address space ceiling. I have ... root@celsius:~# cat /proc/cpuinfo processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Core(TM)2 CPU T7200 @ 2.00GHz ... bogomips: 3989.93 clflush size: 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: ...and: root@celsius:~# qalc 2^36bytes to GiBytes (2^36) * byte = 64 gibibytes Hope somebody can help me fix this, I cannot think of anything that could cause this Best regards, Mohtadin -- ..med vennlig hilsen = with Kind Regards from Arnt Karlsen ...with a number of polar bear hunters in his ancestry... Scenarios always come in sets of three: best case, worst case, and just in case. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120509123221.5fbba...@celsius.lan -- De venligste hilsner/I am, yours most sincerely Seyyed Mohtadin Hashemi
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
Which settings? I did check the BIOS to see if all the correct CPU settings were set, and that CPU2 was enabled - all was as it should be, I can't to much to the RAM other than timings. And i did update the BIOS to the latest version. On Wed, May 9, 2012 at 12:07 PM, Ralf Mardorf ralf.mard...@alice-dsl.netwrote: Did you check settings for the BIOS? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336558068.2171.385.camel@precise -- De venligste hilsner/I am, yours most sincerely Seyyed Mohtadin Hashemi
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 2012-05-09 at 13:04 +0200, Seyyed Mohtadin Hashemi wrote: Which settings? I did check the BIOS to see if all the correct CPU settings were set, and that CPU2 was enabled - all was as it should be, I can't to much to the RAM other than timings. And i did update the BIOS to the latest version. On Wed, May 9, 2012 at 12:07 PM, Ralf Mardorf ralf.mard...@alice-dsl.net wrote: Did you check settings for the BIOS? I suspect this was send randomly off-list, so I'll send it to the list. I don't know. My BIOS don't has relevant settings regarding to this issue, but some BIOS do have settings about managing the RAM. Here it's the same, I only can handle stuff like timing. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336563548.7752.29.camel@precise
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
Hallo Seyyed, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 I have a similar problem on a computer I bought a few years ago and only recently found that the cause might be a buggy bios. I don't have 48Gb ram but only 4Gb and can use about 3.3Gb of it. Regards Johann -- Johann SpiesTelefoon: 021-808 4699 Databestuurder / Data manager Sentrum vir Navorsing oor Evaluasie, Wetenskap en Tegnologie Centre for Research on Evaluation, Science and Technology Universiteit Stellenbosch. Therefore, my beloved brethren, be ye stedfast, unmoveable, always abounding in the work of the Lord, forasmuch as ye know that your labour is not in vain in the Lord. I Corinthians 15:58 signature.asc Description: Digital signature
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 09/05/12 08:09 AM, Johann Spies wrote: Hallo Seyyed, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 I have a similar problem on a computer I bought a few years ago and only recently found that the cause might be a buggy bios. I don't have 48Gb ram but only 4Gb and can use about 3.3Gb of it. Regards Johann Both of you: Have you tried booting from a live disk for a different distro. I'd suggest something really different, like Fedora or OpenSUSE, and not one based on Debian. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4faa67fa.1090...@rogers.com
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, May 9, 2012 at 2:50 PM, Gary Dale garyd...@rogers.com wrote: On 09/05/12 08:09 AM, Johann Spies wrote: Hallo Seyyed, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 I have a similar problem on a computer I bought a few years ago and only recently found that the cause might be a buggy bios. I don't have 48Gb ram but only 4Gb and can use about 3.3Gb of it. Regards Johann Both of you: Have you tried booting from a live disk for a different distro. I'd suggest something really different, like Fedora or OpenSUSE, and not one based on Debian. -- To UNSUBSCRIBE, email to debian-user-REQUEST@lists.**debian.orgdebian-user-requ...@lists.debian.orgwith a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/**4faa67fa.1090...@rogers.comhttp://lists.debian.org/4faa67fa.1090...@rogers.com I did try Ubuntu 12.04 rescue remix, and it behaved exactly the same. I used to use Fedora but switched to Debian some time ago - I will try CentOS later. An update on the RAM, I did test each dimm and all passed a quick memtest.
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 09/05/12 09:28 AM, Seyyed Mohtadin Hashemi wrote: On Wed, May 9, 2012 at 2:50 PM, Gary Dale garyd...@rogers.com mailto:garyd...@rogers.com wrote: On 09/05/12 08:09 AM, Johann Spies wrote: Hallo Seyyed, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 I have a similar problem on a computer I bought a few years ago and only recently found that the cause might be a buggy bios. I don't have 48Gb ram but only 4Gb and can use about 3.3Gb of it. Regards Johann Both of you: Have you tried booting from a live disk for a different distro. I'd suggest something really different, like Fedora or OpenSUSE, and not one based on Debian. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org mailto:debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org mailto:listmas...@lists.debian.org Archive: http://lists.debian.org/4faa67fa.1090...@rogers.com I did try Ubuntu 12.04 rescue remix, and it behaved exactly the same. I used to use Fedora but switched to Debian some time ago - I will try CentOS later. An update on the RAM, I did test each dimm and all passed a quick memtest. Ubuntu is based on Debian and may have the same bug(s). CentOS would be a good test since it comes from an entirely different chain.
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
If you're wanting to run 64 bit with all that ram, have you checked you've not got some sort of 32 bit compatibility mode enabled in the bios? 3.6Gb sounds like the bios has limited it and saved the window above 3.6Gb for device mapping. From: Seyyed Mohtadin Hashemi haa...@gmail.com To: g...@dalefamily.org Cc: debian-user@lists.debian.org Sent: Wednesday, 9 May 2012, 14:28 Subject: Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze On Wed, May 9, 2012 at 2:50 PM, Gary Dale garyd...@rogers.com wrote: On 09/05/12 08:09 AM, Johann Spies wrote: Hallo Seyyed, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 I have a similar problem on a computer I bought a few years ago and only recently found that the cause might be a buggy bios. I don't have 48Gb ram but only 4Gb and can use about 3.3Gb of it. Regards Johann Both of you: Have you tried booting from a live disk for a different distro. I'd suggest something really different, like Fedora or OpenSUSE, and not one based on Debian. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4faa67fa.1090...@rogers.com I did try Ubuntu 12.04 rescue remix, and it behaved exactly the same. I used to use Fedora but switched to Debian some time ago - I will try CentOS later. An update on the RAM, I did test each dimm and all passed a quick memtest.
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 2012-05-09 at 09:47 -0400, Gary Dale wrote: Ubuntu is based on Debian and may have the same bug(s). CentOS would be a good test since it comes from an entirely different chain. A 32-bit Debian with a PAE kernel might also give the wanted result. If so, it would ensure that the hardware isn't broken and that there seems to be an issue regarding to Debain and 64-bit for some machines. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336571585.2994.13.camel@precise
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 09 May 2012 11:19:10 +0200, Seyyed Mohtadin Hashemi wrote: Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 Well put, facts are what matters :-) As the subject suggest I have a box that does not utilize the available RAM installed. I noticed that only 3.6gb RAM was recognized when I got segmentation faults during a simulation. The funny thing is that when I remove dims so that only 48gb RAM is available then it works fine, I did a 'lshw -C memory' and it shows all the dims at the correct spot (the output is attached). BIOS and memtest show and successfully test all 64gb. (...) Ensure that all of the RAM modules are identical and have been approved/ tested by your motherboard's manufacturer. You can also try to load a LiveCD of your choice (e.g., SystemRescueCD) and check from there, just to discard/confirm this is something Debian- specific. Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/joe2gi$bci$8...@dough.gmane.org
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
Might also help to make sure your bios is recognizing all of the memory. If not you might need to check the limitations of the MB or see if there is a bios update that will make the system see all the memory. Shane On Wed, May 9, 2012 at 9:29 AM, Camaleón noela...@gmail.com wrote: On Wed, 09 May 2012 11:19:10 +0200, Seyyed Mohtadin Hashemi wrote: Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 Well put, facts are what matters :-) As the subject suggest I have a box that does not utilize the available RAM installed. I noticed that only 3.6gb RAM was recognized when I got segmentation faults during a simulation. The funny thing is that when I remove dims so that only 48gb RAM is available then it works fine, I did a 'lshw -C memory' and it shows all the dims at the correct spot (the output is attached). BIOS and memtest show and successfully test all 64gb. (...) Ensure that all of the RAM modules are identical and have been approved/ tested by your motherboard's manufacturer. You can also try to load a LiveCD of your choice (e.g., SystemRescueCD) and check from there, just to discard/confirm this is something Debian- specific. Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/joe2gi$bci$8...@dough.gmane.org -- Shane D. Johnson IT Administrator Rasmussen Equipment
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 09 May 2012 09:44:19 -0600, Shane Johnson wrote: Might also help to make sure your bios is recognizing all of the memory. The OP already said that yes, the BIOS is recognizing the full 64 GB of available RAM. Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/joe48s$bci$1...@dough.gmane.org
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, May 9, 2012 at 3:53 PM, Ralf Mardorf ralf.mard...@alice-dsl.netwrote: On Wed, 2012-05-09 at 09:47 -0400, Gary Dale wrote: Ubuntu is based on Debian and may have the same bug(s). CentOS would be a good test since it comes from an entirely different chain. A 32-bit Debian with a PAE kernel might also give the wanted result. If so, it would ensure that the hardware isn't broken and that there seems to be an issue regarding to Debain and 64-bit for some machines. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336571585.2994.13.camel@precise I need 64-bit for some of my programs. But nevertheless i will give the PAE kernel a try.
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 2012-05-09 at 15:29 +, Camaleón wrote: Ensure that all of the RAM modules are identical and have been approved/ tested by your motherboard's manufacturer. Usually just pairs should be identically. Slot A and B should use the same RAM and slot C and D could use two of the same different RAM. IMO it's seldom that you really can buy some manufacturer of the RAM that really don't work with a mobo. Sure! It's possible that this cause an issue. - Ralf -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336579260.2994.31.camel@precise
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 2012-05-09 at 18:01 +0200, Seyyed Mohtadin Hashemi wrote: I need 64-bit for some of my programs. But nevertheless i will give the PAE kernel a try. Just for testing purpose. I agree that WE should use 64-bit Linux. - Ralf -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1336579609.2994.34.camel@precise
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 09 May 2012 18:01:00 +0200, Ralf Mardorf wrote: On Wed, 2012-05-09 at 15:29 +, Camaleón wrote: Ensure that all of the RAM modules are identical and have been approved/ tested by your motherboard's manufacturer. Usually just pairs should be identically. Slot A and B should use the same RAM and slot C and D could use two of the same different RAM. YMMV. I have SuperMicro boards that do not boot when using RAM modules which only differ from the serial number, go figure :-/ IMO it's seldom that you really can buy some manufacturer of the RAM that really don't work with a mobo. Again, that's usually true for standard/user targeted boards but there are manufacturers that are more picky with this, mostly when it comes to motherboards aimed to a professional/high-end sector. Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/joe5do$bci$1...@dough.gmane.org
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
An update: I tried CentOS 6.1 (the only one i had at hand) and it gave the exact same result as Debian. I have in the meantime tried to use other RAM modules, and unfortunately they also did not give more than 3.5gb. Considering that i pulled the ram from a 24/7 stable system i assume that it is indeed the motherboard/BIOS that does not function properly. I will return the motherboard and see if i can get one that works. Thank you all for your ideas and comments, I will post an update later on to tell how the new motherboard functions. best regards, Mohtadin
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 9 May 2012 13:01:25 +0200, Seyyed wrote in message CAKJMjaJM8PnwFpknfZ+1C62rhFR=szdvjukulc_pa3c_z+w...@mail.gmail.com: On Wed, May 9, 2012 at 12:32 PM, Arnt Karlsen a...@c2i.net wrote: On Wed, 9 May 2012 11:19:10 +0200, Seyyed wrote in message CAKJMja+mrpWmo8uhakjGfwsuk3W0=+a6lmbhfvxkkfcterf...@mail.gmail.com: Hello, Before anybody starts arguing that I don't have 64-bit, this is uname -r and uname -m: root@n03:~# uname -r 2.6.32-3-amd64 root@n03:~# uname -m x86_64 As the subject suggest I have a box that does not utilize the available RAM installed. I noticed that only 3.6gb RAM was recognized when I got segmentation faults during a simulation. The funny thing is that when I remove dims so that only 48gb RAM is available then it works fine, I did a 'lshw -C memory' and it shows all the dims at the correct spot (the output is attached). BIOS and memtest show and successfully test all 64gb. ..post a one core snip of your /proc/cpuinfo, I'm wondering if you've hit some 36 bit address space ceiling. I have ... root@celsius:~# cat /proc/cpuinfo processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Core(TM)2 CPU T7200 @ 2.00GHz ... bogomips: 3989.93 clflush size: 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: ...and: root@celsius:~# qalc 2^36bytes to GiBytes (2^36) * byte = 64 gibibytes Here you go: processor : 31 vendor_id : AuthenticAMD cpu family : 21 model : 1 model name : AMD Opteron(TM) Processor 6274 ... address sizes : 48 bits physical, 48 bits virtual ..ok, your cpus have 48 bit address space, is there anything else that could force you into a 36 bit straitjacket? ..what happens if you remove 2 ram sticks? -- ..med vennlig hilsen = with Kind Regards from Arnt Karlsen ...with a number of polar bear hunters in his ancestry... Scenarios always come in sets of three: best case, worst case, and just in case. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120509235625.4d3c2...@celsius.lan
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On 5/9/2012 11:43 AM, Seyyed Mohtadin Hashemi wrote: An update: I tried CentOS 6.1 (the only one i had at hand) and it gave the exact same result as Debian. I have in the meantime tried to use other RAM modules, and unfortunately they also did not give more than 3.5gb. Considering that i pulled the ram from a 24/7 stable system i assume that it is indeed the motherboard/BIOS that does not function properly. That is not a safe assumption given how picky socket G34 is with memory channel configuration. I will return the motherboard and see if i can get one that works. Before you return the board, would you mind posting the manufacturer, part number, and specs of each of the 16 DIMMs you have installed? They should all match. Are they all indeed the same part# from the same vendor? Are they unbuffered or registered? Are they single, dual, or quad rank modules? All of these things matter, greatly. Unfortunately, Asus doesn't provide the proper table in the manual for this board that shows which combinations of speed, density, rank, and un/buffered DIMMs work in socket G34. SuperMicro does, and frankly they're the only vendor of AMD server boards I'd ever use because of their attention to all the details. Socket G34 is fairly picky regarding DIMM configurations, especially with high density (8/16GB) modules, though not as picky with low density (1/2/4GB) modules. 16 identical unbuffered, or registered, 4GB DDR3-1333 SR or DR DIMMs *should* work, so long as you don't put both unbuffered and buffered DIMMs in the same two slot channel. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4faaf147.7010...@hardwarefreak.com
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
On Wed, 09 May 2012 17:35:51 -0500, Stan wrote in message 4faaf147.7010...@hardwarefreak.com: On 5/9/2012 11:43 AM, Seyyed Mohtadin Hashemi wrote: An update: I tried CentOS 6.1 (the only one i had at hand) and it gave the exact same result as Debian. I have in the meantime tried to use other RAM modules, and unfortunately they also did not give more than 3.5gb. Considering that i pulled the ram from a 24/7 stable system i assume that it is indeed the motherboard/BIOS that does not function properly. That is not a safe assumption given how picky socket G34 is with memory channel configuration. I will return the motherboard and see if i can get one that works. Before you return the board, would you mind posting the manufacturer, part number, and specs of each of the 16 DIMMs you have installed? They should all match. Are they all indeed the same part# from the same vendor? Are they unbuffered or registered? Are they single, dual, or quad rank modules? All of these things matter, greatly. Unfortunately, Asus doesn't provide the proper table in the manual for this board that shows which combinations of speed, density, rank, and un/buffered DIMMs work in socket G34. SuperMicro does, and frankly they're the only vendor of AMD server boards I'd ever use because of their attention to all the details. Socket G34 is fairly picky regarding DIMM configurations, especially with high density (8/16GB) modules, though not as picky with low density (1/2/4GB) modules. 16 identical unbuffered, or registered, 4GB DDR3-1333 SR or DR DIMMs *should* work, so long as you don't put both unbuffered and buffered DIMMs in the same two slot channel. ..a test suggestion: remove 2 of the 16 ram sticks and make sure the remaining 14 is set up according to the main board manual's ram stick map, I saw it posted in this thread. If this test works, you should see 56G once booted up. -- ..med vennlig hilsen = with Kind Regards from Arnt Karlsen ...with a number of polar bear hunters in his ancestry... Scenarios always come in sets of three: best case, worst case, and just in case. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120510013854.63b36...@celsius.lan
Re: Only 3.6gb of 64gb RAM recognized by 64bit squeeze
Sorry for top post, stupid phone wont let me do anything else. The RAM are as dar as i can see identical, they are all Mushkin Stan Hoeppner s...@hardwarefreak.com wrote: On 5/9/2012 11:43 AM, Seyyed Mohtadin Hashemi wrote: An update: I tried CentOS 6.1 (the only one i had at hand) and it gave the exact same result as Debian. I have in the meantime tried to use other RAM modules, and unfortunately they also did not give more than 3.5gb. Considering that i pulled the ram from a 24/7 stable system i assume that it is indeed the motherboard/BIOS that does not function properly. That is not a safe assumption given how picky socket G34 is with memory channel configuration. I will return the motherboard and see if i can get one that works. Before you return the board, would you mind posting the manufacturer, part number, and specs of each of the 16 DIMMs you have installed? They should all match. Are they all indeed the same part# from the same vendor? Are they unbuffered or registered? Are they single, dual, or quad rank modules? All of these things matter, greatly. Unfortunately, Asus doesn't provide the proper table in the manual for this board that shows which combinations of speed, density, rank, and un/buffered DIMMs work in socket G34. SuperMicro does, and frankly they're the only vendor of AMD server boards I'd ever use because of their attention to all the details. Socket G34 is fairly picky regarding DIMM configurations, especially with high density (8/16GB) modules, though not as picky with low density (1/2/4GB) modules. 16 identical unbuffered, or registered, 4GB DDR3-1333 SR or DR DIMMs *should* work, so long as you don't put both unbuffered and buffered DIMMs in the same two slot channel. -- Stan
