Re: [gentoo-user] Dual or Quad CPU complications?
Something _really_ weird happened to your quoting; you quoted my email, but your email client said you wrote it. On Mon, Dec 17, 2012 at 2:00 AM, J. Roeleveld jo...@antarean.org wrote: On Sun, Dec 16, 2012 at 10:39 AM, J. Roeleveld jo...@antarean.org wrote: ^-- weird --^ On Sunday, December 16, 2012 01:52:46 PM Volker Armin Hemmann wrote: Am Samstag, 15. Dezember 2012, 20:57:24 schrieb J. Roeleveld: Even on a system with only 2 sockets, it can be useful to have NUMA available. or not, because it costs you performance. When does it cost performance? In all situations? It adds some additional logic to memory allocation (put an allocation near the process that uses it) and to process scheduling (keep the process near its memory, but bump it to a more distant idle core if necessary). That's the way it's supposed to work, yes :) In all honestly, it's not a performance loss you're likely to notice, unless you're so in need of squeezing out every spare cycle that you most definitely _have_ hardware where there are disconnected memory banks. I'm not convinced it's even measurable for us mundanes and our hardware. I don't think I would notice it either, but as the system I have supports it, I want to use it. And then I want to be certain it actually supports it correctly. The system I'm talking about is used for testing purposes. Running multiple VMs. As far as I know, Xen has support for it, just need to configure it properly. And for this usecase, I think NUMA with only 2 physical CPUs should make a positive difference. Don't get me wrong; I was arguing that it shouldn't hurt to have it enabled. :) -- :wq
Re: [gentoo-user] Dual or Quad CPU complications?
Something _really_ weird happened to your quoting; you quoted my email, but your email client said you wrote it. On Mon, Dec 17, 2012 at 2:00 AM, J. Roeleveld jo...@antarean.org wrote: On Sun, Dec 16, 2012 at 10:39 AM, J. Roeleveld jo...@antarean.org wrote: ^-- weird --^ Very weird, especially as I am using the same client now (squirrelmail) and I don't see that line at all now. On Sunday, December 16, 2012 01:52:46 PM Volker Armin Hemmann wrote: Am Samstag, 15. Dezember 2012, 20:57:24 schrieb J. Roeleveld: Even on a system with only 2 sockets, it can be useful to have NUMA available. or not, because it costs you performance. When does it cost performance? In all situations? It adds some additional logic to memory allocation (put an allocation near the process that uses it) and to process scheduling (keep the process near its memory, but bump it to a more distant idle core if necessary). That's the way it's supposed to work, yes :) In all honestly, it's not a performance loss you're likely to notice, unless you're so in need of squeezing out every spare cycle that you most definitely _have_ hardware where there are disconnected memory banks. I'm not convinced it's even measurable for us mundanes and our hardware. I don't think I would notice it either, but as the system I have supports it, I want to use it. And then I want to be certain it actually supports it correctly. The system I'm talking about is used for testing purposes. Running multiple VMs. As far as I know, Xen has support for it, just need to configure it properly. And for this usecase, I think NUMA with only 2 physical CPUs should make a positive difference. Don't get me wrong; I was arguing that it shouldn't hurt to have it enabled. :) I know, just wanted to add the use-case for considering NUMA a usefull option even with only 2 physical CPUs :) -- Joost
Re: [gentoo-user] Dual or Quad CPU complications?
Am Samstag, 15. Dezember 2012, 20:57:24 schrieb J. Roeleveld: On Saturday, December 15, 2012 11:46:36 AM Grant wrote: You have to buy NUMA hardware. If the hardware you buys does not scream NUMA at you, you don't have it. It is really that simple. Multicore, multisocket systems MIGHT be NUMA systems - but that is not a guarantee. Now can this stupid thread please die away? I guess the question seems stupid if you already know the answer. There are no stupid questions, only stupid answers... Even on a system with only 2 sockets, it can be useful to have NUMA available. or not, because it costs you performance. And while the starting questions were not stupid this thread is overflowing with stupid answers. -- #163933
Re: [gentoo-user] Dual or Quad CPU complications?
On Sunday, December 16, 2012 01:52:46 PM Volker Armin Hemmann wrote: Am Samstag, 15. Dezember 2012, 20:57:24 schrieb J. Roeleveld: Even on a system with only 2 sockets, it can be useful to have NUMA available. or not, because it costs you performance. When does it cost performance? In all situations? And while the starting questions were not stupid this thread is overflowing with stupid answers. Matter of opinion...
Re: [gentoo-user] Dual or Quad CPU complications?
On Sun, Dec 16, 2012 at 10:39 AM, J. Roeleveld jo...@antarean.org wrote: On Sunday, December 16, 2012 01:52:46 PM Volker Armin Hemmann wrote: Am Samstag, 15. Dezember 2012, 20:57:24 schrieb J. Roeleveld: Even on a system with only 2 sockets, it can be useful to have NUMA available. or not, because it costs you performance. When does it cost performance? In all situations? It adds some additional logic to memory allocation (put an allocation near the process that uses it) and to process scheduling (keep the process near its memory, but bump it to a more distant idle core if necessary). In all honestly, it's not a performance loss you're likely to notice, unless you're so in need of squeezing out every spare cycle that you most definitely _have_ hardware where there are disconnected memory banks. I'm not convinced it's even measurable for us mundanes and our hardware. And while the starting questions were not stupid this thread is overflowing with stupid answers. Matter of opinion... Indeed. -- :wq
Re: [gentoo-user] Dual or Quad CPU complications?
On Sun, Dec 16, 2012 at 10:39 AM, J. Roeleveld jo...@antarean.org wrote: On Sunday, December 16, 2012 01:52:46 PM Volker Armin Hemmann wrote: Am Samstag, 15. Dezember 2012, 20:57:24 schrieb J. Roeleveld: Even on a system with only 2 sockets, it can be useful to have NUMA available. or not, because it costs you performance. When does it cost performance? In all situations? It adds some additional logic to memory allocation (put an allocation near the process that uses it) and to process scheduling (keep the process near its memory, but bump it to a more distant idle core if necessary). That's the way it's supposed to work, yes :) In all honestly, it's not a performance loss you're likely to notice, unless you're so in need of squeezing out every spare cycle that you most definitely _have_ hardware where there are disconnected memory banks. I'm not convinced it's even measurable for us mundanes and our hardware. I don't think I would notice it either, but as the system I have supports it, I want to use it. And then I want to be certain it actually supports it correctly. The system I'm talking about is used for testing purposes. Running multiple VMs. As far as I know, Xen has support for it, just need to configure it properly. And for this usecase, I think NUMA with only 2 physical CPUs should make a positive difference. -- Joost
Re: [gentoo-user] Dual or Quad CPU complications?
Am 15.12.2012 01:40, schrieb Mick: On Thursday 13 Dec 2012 14:13:56 Bruce Hill wrote: On Thu, Dec 13, 2012 at 08:44:45AM +0100, J. Roeleveld wrote: NUMA is also an option in the kernel. Should also be fully transparent. I got one machine with NUMA and only had to set an option for it. Does anyone know how to check it's working properly? dmesg | grep NUMA Hmm ... it seems that it can't find NUMA configuration: $ dmesg | grep UMA No NUMA configuration found Am I supposed to configure something in userspace? This is what the kernel has: $ uname -a Linux dell_xps 3.5.7-gentoo #2 SMP PREEMPT Mon Nov 26 10:36:47 GMT 2012 x86_64 Intel(R) Core(TM) i7 CPU Q 720 @ 1.60GHz GenuineIntel GNU/Linux [...] dell_xps as in XPS laptop? There are no NUMA laptops. Despite all the stuff about terminology, we are basically talking about multi-socket systems. Things with mainboards like these [1] as opposed to these [2]. [1] http://www.newegg.com/Product/Product.aspx?Item=N82E16813131378 [2] http://www.newegg.com/Product/Product.aspx?Item=N82E16813131725 Regards, Florian Philipp signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] Dual or Quad CPU complications?
Am 15.12.2012 04:16, schrieb Grant: So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm using 8 cores? Can NUMA be either enabled or disabled when using more than one physical CPU, or is it required? NUMA is a hardware architecture. It's how you access memory on a hardware level: NUMA = Non Uniform Memory Access vs a UMA architecture of typical (old/legacy) SMP systems (UMA = Uniform Memory Access). In a UMA system, all the memory belongs to all the sockets. In a NUMA system, each socket has it's own local memory. In modern (x86-64) processors, each socket has it's own memory controller so each socket controls its own local memory. If one socket runs out of memory it can ask another socket to lend him some memory. In a UMA system, no socket has to ask since memory is global and belongs to all sockets so if one socket uses up all the memory ... the rest starve. In NUMA, there's more control over who uses what (be it cores or RAM). If you have a modern dual or quad (or higher #) socket system ... you've got NUMA architecture and you can't get rid of it, it's hardware, not software. So I must enable CONFIG_NUMA for more than one physical CPU, and disable it for only one physical CPU? Yup. But ... Why would you want to disable a socket (CPU)? If you disable a socket (CPU) ... you lose the memory attached to that socket (CPU) not to mention you lose those cores ;) Sure but it sounds like if my system only has one CPU socket, CONFIG_NUMA should be disabled. I read this in make menuconfig: The kernel will try to allocate memory used by a CPU on the local memory controller of the CPU and add some more NUMA awareness to the kernel. For 64-bit this is recommended if the system is Intel Core i7 (or later), AMD Opteron, or EM64T NUMA. To be sure I have this right, I should disable CONFIG_NUMA on any system with a single physical CPU, even if it's an AMD Opteron? - Grant Disable it. You only have one memory controller. There is nothing the kernel could do wrong without. Regards, Florian Philipp signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] Dual or Quad CPU complications?
Am Freitag, 14. Dezember 2012, 08:55:08 schrieb Rafa Griman: On Thu, Dec 13, 2012 at 7:22 PM, Volker Armin Hemmann volkerar...@googlemail.com wrote: Am Mittwoch, 12. Dezember 2012, 22:12:18 schrieb Grant: I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? none also, forget numa. You won't deal with douzends of cores each using local memory and acccession the memory managed by the other cores. It depends on his application, maybe his application does benefit on NUMA architecture. Until we don't know what he's running, we can't really say this or that architecture/technology is of no use ;) So Volker, what applications are you running (and BTW: what volume of data are you managing, how many users, ...)? This will helps us help you :) you don't get NUMA just for free. You have to buy NUMA hardware. If the hardware you buys does not scream NUMA at you, you don't have it. It is really that simple. Multicore, multisocket systems MIGHT be NUMA systems - but that is not a guarantee. Now can this stupid thread please die away? There are no caveats going from single to multicore on consumer hardware. and even on non consumer hardware there aren't many. -- #163933
Re: [gentoo-user] Dual or Quad CPU complications?
Am Freitag, 14. Dezember 2012, 01:44:26 schrieb Grant: So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm using 8 cores? Can NUMA be either enabled or disabled when using more than one physical CPU, or is it required? NUMA is a hardware architecture. It's how you access memory on a hardware level: NUMA = Non Uniform Memory Access vs a UMA architecture of typical (old/legacy) SMP systems (UMA = Uniform Memory Access). In a UMA system, all the memory belongs to all the sockets. In a NUMA system, each socket has it's own local memory. In modern (x86-64) processors, each socket has it's own memory controller so each socket controls its own local memory. If one socket runs out of memory it can ask another socket to lend him some memory. In a UMA system, no socket has to ask since memory is global and belongs to all sockets so if one socket uses up all the memory ... the rest starve. In NUMA, there's more control over who uses what (be it cores or RAM). If you have a modern dual or quad (or higher #) socket system ... you've got NUMA architecture and you can't get rid of it, it's hardware, not software. So I must enable CONFIG_NUMA for more than one physical CPU, and disable it for only one physical CPU? you never need numa for one cpu. Ok? And even if you have several, you will probably never need it. -- #163933
Re: [gentoo-user] Dual or Quad CPU complications?
You have to buy NUMA hardware. If the hardware you buys does not scream NUMA at you, you don't have it. It is really that simple. Multicore, multisocket systems MIGHT be NUMA systems - but that is not a guarantee. Now can this stupid thread please die away? I guess the question seems stupid if you already know the answer. - Grant
Re: [gentoo-user] Dual or Quad CPU complications?
On Saturday, December 15, 2012 11:46:36 AM Grant wrote: You have to buy NUMA hardware. If the hardware you buys does not scream NUMA at you, you don't have it. It is really that simple. Multicore, multisocket systems MIGHT be NUMA systems - but that is not a guarantee. Now can this stupid thread please die away? I guess the question seems stupid if you already know the answer. There are no stupid questions, only stupid answers... Even on a system with only 2 sockets, it can be useful to have NUMA available.
Re: [gentoo-user] Dual or Quad CPU complications?
I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? none also, forget numa. You won't deal with douzends of cores each using local memory and acccession the memory managed by the other cores. It depends on his application, maybe his application does benefit on NUMA architecture. Until we don't know what he's running, we can't really say this or that architecture/technology is of no use ;) So Volker, what applications are you running (and BTW: what volume of data are you managing, how many users, ...)? This will helps us help you :) Rafa It's an apache2/mysql server with a medium amount of traffic. - Grant
Re: [gentoo-user] Dual or Quad CPU complications?
On Fri, Dec 14, 2012 at 8:43 AM, Grant emailgr...@gmail.com wrote: [...] So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm using 8 cores? Can NUMA be either enabled or disabled when using more than one physical CPU, or is it required? NUMA is a hardware architecture. It's how you access memory on a hardware level: NUMA = Non Uniform Memory Access vs a UMA architecture of typical (old/legacy) SMP systems (UMA = Uniform Memory Access). In a UMA system, all the memory belongs to all the sockets. In a NUMA system, each socket has it's own local memory. In modern (x86-64) processors, each socket has it's own memory controller so each socket controls its own local memory. If one socket runs out of memory it can ask another socket to lend him some memory. In a UMA system, no socket has to ask since memory is global and belongs to all sockets so if one socket uses up all the memory ... the rest starve. In NUMA, there's more control over who uses what (be it cores or RAM). If you have a modern dual or quad (or higher #) socket system ... you've got NUMA architecture and you can't get rid of it, it's hardware, not software. NUMA is not bad nor good. It's transparent to you. If your SW supports threads, OpenMP, ... you'll be using it without knowing. That doesn't mean you can't tweak performance and use numactl tools, cgroups, ... to increase performance. You can :) HTH Rafa
Re: [gentoo-user] Dual or Quad CPU complications?
So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm using 8 cores? Can NUMA be either enabled or disabled when using more than one physical CPU, or is it required? NUMA is a hardware architecture. It's how you access memory on a hardware level: NUMA = Non Uniform Memory Access vs a UMA architecture of typical (old/legacy) SMP systems (UMA = Uniform Memory Access). In a UMA system, all the memory belongs to all the sockets. In a NUMA system, each socket has it's own local memory. In modern (x86-64) processors, each socket has it's own memory controller so each socket controls its own local memory. If one socket runs out of memory it can ask another socket to lend him some memory. In a UMA system, no socket has to ask since memory is global and belongs to all sockets so if one socket uses up all the memory ... the rest starve. In NUMA, there's more control over who uses what (be it cores or RAM). If you have a modern dual or quad (or higher #) socket system ... you've got NUMA architecture and you can't get rid of it, it's hardware, not software. So I must enable CONFIG_NUMA for more than one physical CPU, and disable it for only one physical CPU? - Grant NUMA is not bad nor good. It's transparent to you. If your SW supports threads, OpenMP, ... you'll be using it without knowing. That doesn't mean you can't tweak performance and use numactl tools, cgroups, ... to increase performance. You can :) HTH Rafa
Re: [gentoo-user] Dual or Quad CPU complications?
On Fri, Dec 14, 2012 at 10:44 AM, Grant emailgr...@gmail.com wrote: So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm using 8 cores? Can NUMA be either enabled or disabled when using more than one physical CPU, or is it required? NUMA is a hardware architecture. It's how you access memory on a hardware level: NUMA = Non Uniform Memory Access vs a UMA architecture of typical (old/legacy) SMP systems (UMA = Uniform Memory Access). In a UMA system, all the memory belongs to all the sockets. In a NUMA system, each socket has it's own local memory. In modern (x86-64) processors, each socket has it's own memory controller so each socket controls its own local memory. If one socket runs out of memory it can ask another socket to lend him some memory. In a UMA system, no socket has to ask since memory is global and belongs to all sockets so if one socket uses up all the memory ... the rest starve. In NUMA, there's more control over who uses what (be it cores or RAM). If you have a modern dual or quad (or higher #) socket system ... you've got NUMA architecture and you can't get rid of it, it's hardware, not software. So I must enable CONFIG_NUMA for more than one physical CPU, and disable it for only one physical CPU? Yup. But ... Why would you want to disable a socket (CPU)? If you disable a socket (CPU) ... you lose the memory attached to that socket (CPU) not to mention you lose those cores ;) A better solution would be to use cgroups or numactl tools to pin a certain process to a set of cores and a memory region. If you really want to deactivate cores (but not the whole socket), you can type: echo 0 /sys/devices/system/cpu/cpu1/online This would deactivate core #1. You can deactivate as many cores as you wish, except for core #0. This can be done without rebooting your server (aka during run time). Your memory will not be affected, but you will have less cores (and theoretically more memory bandwidth). I say theoretically because you always have to benchmark these things with YOUR application (remember logic NEVER applies to real life ;) If you want to check the # of cores you've got: cat /proc/interrupts | grep CPU Other possibilities such as cat /proc/cpuinfo or dmesg, ... can be useful too for this: your choice, FLOSS gives you options. If you want to activate the previously deactivated core, you can run: echo 1 /sys/devices/system/cpu/cpu1/online Now ... be sure your core numbering is the expected core numbering. IOW, not all server vendors follow the same numbering scheme so core #1 in vendor A's server could be core #2 in vendor B's server. Never trust logic ;) As I mentioned previously: test/benchmark YOUR software. DON'T trust logic or generic benchmarks or web pages with results. Trust YOUR results only. HTH Rafa
Re: [gentoo-user] Dual or Quad CPU complications?
So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm using 8 cores? Can NUMA be either enabled or disabled when using more than one physical CPU, or is it required? NUMA is a hardware architecture. It's how you access memory on a hardware level: NUMA = Non Uniform Memory Access vs a UMA architecture of typical (old/legacy) SMP systems (UMA = Uniform Memory Access). In a UMA system, all the memory belongs to all the sockets. In a NUMA system, each socket has it's own local memory. In modern (x86-64) processors, each socket has it's own memory controller so each socket controls its own local memory. If one socket runs out of memory it can ask another socket to lend him some memory. In a UMA system, no socket has to ask since memory is global and belongs to all sockets so if one socket uses up all the memory ... the rest starve. In NUMA, there's more control over who uses what (be it cores or RAM). If you have a modern dual or quad (or higher #) socket system ... you've got NUMA architecture and you can't get rid of it, it's hardware, not software. So I must enable CONFIG_NUMA for more than one physical CPU, and disable it for only one physical CPU? Yup. But ... Why would you want to disable a socket (CPU)? If you disable a socket (CPU) ... you lose the memory attached to that socket (CPU) not to mention you lose those cores ;) Sure but it sounds like if my system only has one CPU socket, CONFIG_NUMA should be disabled. - Grant A better solution would be to use cgroups or numactl tools to pin a certain process to a set of cores and a memory region. If you really want to deactivate cores (but not the whole socket), you can type: echo 0 /sys/devices/system/cpu/cpu1/online This would deactivate core #1. You can deactivate as many cores as you wish, except for core #0. This can be done without rebooting your server (aka during run time). Your memory will not be affected, but you will have less cores (and theoretically more memory bandwidth). I say theoretically because you always have to benchmark these things with YOUR application (remember logic NEVER applies to real life ;) If you want to check the # of cores you've got: cat /proc/interrupts | grep CPU Other possibilities such as cat /proc/cpuinfo or dmesg, ... can be useful too for this: your choice, FLOSS gives you options. If you want to activate the previously deactivated core, you can run: echo 1 /sys/devices/system/cpu/cpu1/online Now ... be sure your core numbering is the expected core numbering. IOW, not all server vendors follow the same numbering scheme so core #1 in vendor A's server could be core #2 in vendor B's server. Never trust logic ;) As I mentioned previously: test/benchmark YOUR software. DON'T trust logic or generic benchmarks or web pages with results. Trust YOUR results only. HTH Rafa
Re: [gentoo-user] Dual or Quad CPU complications?
On Thursday 13 Dec 2012 14:13:56 Bruce Hill wrote: On Thu, Dec 13, 2012 at 08:44:45AM +0100, J. Roeleveld wrote: NUMA is also an option in the kernel. Should also be fully transparent. I got one machine with NUMA and only had to set an option for it. Does anyone know how to check it's working properly? dmesg | grep NUMA Hmm ... it seems that it can't find NUMA configuration: $ dmesg | grep UMA No NUMA configuration found Am I supposed to configure something in userspace? This is what the kernel has: $ uname -a Linux dell_xps 3.5.7-gentoo #2 SMP PREEMPT Mon Nov 26 10:36:47 GMT 2012 x86_64 Intel(R) Core(TM) i7 CPU Q 720 @ 1.60GHz GenuineIntel GNU/Linux $ cat /usr/src/linux/.config | grep -i NUMA CONFIG_NUMA=y # CONFIG_AMD_NUMA is not set CONFIG_X86_64_ACPI_NUMA=y # CONFIG_NUMA_EMU is not set CONFIG_USE_PERCPU_NUMA_NODE_ID=y CONFIG_ACPI_NUMA=y -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Dual or Quad CPU complications?
So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm using 8 cores? Can NUMA be either enabled or disabled when using more than one physical CPU, or is it required? NUMA is a hardware architecture. It's how you access memory on a hardware level: NUMA = Non Uniform Memory Access vs a UMA architecture of typical (old/legacy) SMP systems (UMA = Uniform Memory Access). In a UMA system, all the memory belongs to all the sockets. In a NUMA system, each socket has it's own local memory. In modern (x86-64) processors, each socket has it's own memory controller so each socket controls its own local memory. If one socket runs out of memory it can ask another socket to lend him some memory. In a UMA system, no socket has to ask since memory is global and belongs to all sockets so if one socket uses up all the memory ... the rest starve. In NUMA, there's more control over who uses what (be it cores or RAM). If you have a modern dual or quad (or higher #) socket system ... you've got NUMA architecture and you can't get rid of it, it's hardware, not software. So I must enable CONFIG_NUMA for more than one physical CPU, and disable it for only one physical CPU? Yup. But ... Why would you want to disable a socket (CPU)? If you disable a socket (CPU) ... you lose the memory attached to that socket (CPU) not to mention you lose those cores ;) Sure but it sounds like if my system only has one CPU socket, CONFIG_NUMA should be disabled. I read this in make menuconfig: The kernel will try to allocate memory used by a CPU on the local memory controller of the CPU and add some more NUMA awareness to the kernel. For 64-bit this is recommended if the system is Intel Core i7 (or later), AMD Opteron, or EM64T NUMA. To be sure I have this right, I should disable CONFIG_NUMA on any system with a single physical CPU, even if it's an AMD Opteron? - Grant
Re: [gentoo-user] Dual or Quad CPU complications?
On Fri, Dec 14, 2012 at 10:16 PM, Grant emailgr...@gmail.com wrote: So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm using 8 cores? Can NUMA be either enabled or disabled when using more than one physical CPU, or is it required? NUMA is a hardware architecture. It's how you access memory on a hardware level: NUMA = Non Uniform Memory Access vs a UMA architecture of typical (old/legacy) SMP systems (UMA = Uniform Memory Access). In a UMA system, all the memory belongs to all the sockets. In a NUMA system, each socket has it's own local memory. In modern (x86-64) processors, each socket has it's own memory controller so each socket controls its own local memory. If one socket runs out of memory it can ask another socket to lend him some memory. In a UMA system, no socket has to ask since memory is global and belongs to all sockets so if one socket uses up all the memory ... the rest starve. In NUMA, there's more control over who uses what (be it cores or RAM). If you have a modern dual or quad (or higher #) socket system ... you've got NUMA architecture and you can't get rid of it, it's hardware, not software. So I must enable CONFIG_NUMA for more than one physical CPU, and disable it for only one physical CPU? Yup. But ... Why would you want to disable a socket (CPU)? If you disable a socket (CPU) ... you lose the memory attached to that socket (CPU) not to mention you lose those cores ;) Sure but it sounds like if my system only has one CPU socket, CONFIG_NUMA should be disabled. I read this in make menuconfig: The kernel will try to allocate memory used by a CPU on the local memory controller of the CPU and add some more NUMA awareness to the kernel. For 64-bit this is recommended if the system is Intel Core i7 (or later), AMD Opteron, or EM64T NUMA. To be sure I have this right, I should disable CONFIG_NUMA on any system with a single physical CPU, even if it's an AMD Opteron? No harm done if you enable NUMA on a system where it's not necessary. -- :wq
Re: [gentoo-user] Dual or Quad CPU complications?
I believe NUMA is only used on multiprocessor machine and not on only multicore. http://en.wikipedia.org/wiki/Non-Uniform_Memory_Access 2012/12/13 J. Roeleveld jo...@antarean.org Florian Philipp li...@binarywings.net wrote: Am 13.12.2012 07:23, schrieb Alan McKinnon: On Wed, 12 Dec 2012 22:12:18 -0800 Grant emailgr...@gmail.com wrote: I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? No complication. Configure CONFIG_SMP in the the kernel for multicore. Everything else is transparent. Cores make threads work better, so you'd want to investigate if USE=threads is useful for you. I think he's looking for advice on NUMA, not SMP. NUMA is also an option in the kernel. Should also be fully transparent. I got one machine with NUMA and only had to set an option for it. Does anyone know how to check it's working properly? -- Joost -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Re: [gentoo-user] Dual or Quad CPU complications?
On Thu, Dec 13, 2012 at 2:01 PM, Luis Gustavo Vilela de Oliveira luisgustavo.vil...@gmail.com wrote: I believe NUMA is only used on multiprocessor machine and not on only multicore. NUMA's about memory access so it's about cores/CPUs/processors/whatever_you_want_to_call_it and how they access memory. [...] NUMA is also an option in the kernel. Should also be fully transparent. I got one machine with NUMA and only had to set an option for it. Does anyone know how to check it's working properly? You could use numactl and place your binary on certain nodes (cores) and check whether that works. You should also check your code uses OpenMP, threading, ... for example. Monitor your CPU and memory also while your software is running ;) What SW are you going to be running? Do you know if your software is SMP and/or NUMA aware? Rafa
Re: [gentoo-user] Dual or Quad CPU complications?
On Thu, Dec 13, 2012 at 08:44:45AM +0100, J. Roeleveld wrote: NUMA is also an option in the kernel. Should also be fully transparent. I got one machine with NUMA and only had to set an option for it. Does anyone know how to check it's working properly? dmesg | grep NUMA -- Happy Penguin Computers ') 126 Fenco Drive ( \ Tupelo, MS 38801 ^^ supp...@happypenguincomputers.com 662-269-2706 662-205-6424 http://happypenguincomputers.com/ Don't top-post: http://en.wikipedia.org/wiki/Top_post#Top-posting
Re: [gentoo-user] Dual or Quad CPU complications?
Am 13.12.2012 07:12, schrieb Grant: I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? - Grant If you want to run mysql with high memory usage on that machine, you might want to read http://blog.jcole.us/2010/09/28/mysql-swap-insanity-and-the-numa-architecture/ Everything else that I can think of already has beed said. Oh, tweak MAKEOPTS for a faster compile time, you also might want to look at emerges --jobs and --load-average parameters
Re: [gentoo-user] Dual or Quad CPU complications?
Am Mittwoch, 12. Dezember 2012, 22:12:18 schrieb Grant: I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? none also, forget numa. You won't deal with douzends of cores each using local memory and acccession the memory managed by the other cores. - Grant -- #163933
Re: [gentoo-user] Dual or Quad CPU complications?
I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? No complication. Configure CONFIG_SMP in the the kernel for multicore. Everything else is transparent. Cores make threads work better, so you'd want to investigate if USE=threads is useful for you. I think he's looking for advice on NUMA, not SMP. So if I have 2 physical CPU's with 4 cores each and I enable SMP, I'm using 8 cores? Can NUMA be either enabled or disabled when using more than one physical CPU, or is it required? - Grant
Re: [gentoo-user] Dual or Quad CPU complications?
On Thu, Dec 13, 2012 at 7:22 PM, Volker Armin Hemmann volkerar...@googlemail.com wrote: Am Mittwoch, 12. Dezember 2012, 22:12:18 schrieb Grant: I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? none also, forget numa. You won't deal with douzends of cores each using local memory and acccession the memory managed by the other cores. It depends on his application, maybe his application does benefit on NUMA architecture. Until we don't know what he's running, we can't really say this or that architecture/technology is of no use ;) So Volker, what applications are you running (and BTW: what volume of data are you managing, how many users, ...)? This will helps us help you :) Rafa
[gentoo-user] Dual or Quad CPU complications?
I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? - Grant
Re: [gentoo-user] Dual or Quad CPU complications?
On Wed, 12 Dec 2012 22:12:18 -0800 Grant emailgr...@gmail.com wrote: I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? No complication. Configure CONFIG_SMP in the the kernel for multicore. Everything else is transparent. Cores make threads work better, so you'd want to investigate if USE=threads is useful for you. -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Dual or Quad CPU complications?
Am 13.12.2012 07:23, schrieb Alan McKinnon: On Wed, 12 Dec 2012 22:12:18 -0800 Grant emailgr...@gmail.com wrote: I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? No complication. Configure CONFIG_SMP in the the kernel for multicore. Everything else is transparent. Cores make threads work better, so you'd want to investigate if USE=threads is useful for you. I think he's looking for advice on NUMA, not SMP. signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] Dual or Quad CPU complications?
Florian Philipp li...@binarywings.net wrote: Am 13.12.2012 07:23, schrieb Alan McKinnon: On Wed, 12 Dec 2012 22:12:18 -0800 Grant emailgr...@gmail.com wrote: I've only ever used systems with a single CPU. I'm looking for a new host for a dedicated server (suggestions?) and it looks like I'll probably choose a machine with two or four CPUs. What sort of complications does that add to set up and/or maintenance with Gentoo? No complication. Configure CONFIG_SMP in the the kernel for multicore. Everything else is transparent. Cores make threads work better, so you'd want to investigate if USE=threads is useful for you. I think he's looking for advice on NUMA, not SMP. NUMA is also an option in the kernel. Should also be fully transparent. I got one machine with NUMA and only had to set an option for it. Does anyone know how to check it's working properly? -- Joost -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
