Re: OT: Can lot of RAM can slow down a calculation workstation?
On 07/02/2015 09:32 AM, Dan wrote: I can buy 8x 32 Gb or 16 x 16Gb. The first option is more expensive than the second one, but I will have only one DIMM per channel. Any suggestions or experience with this? I have also seen cases where one memory module per channel is faster than two of the same modules per channel, as reported by the BIOS and/ or memtest86+. But, I can't remember a single case where I could tell the operational difference between faster and slower memory modules of the same capacity, even with significant differences in frequency specifications. Ideally, you want enough memory to hold your program and all it's data in memory at the same time, plus whatever memory the rest of your computer needs, plus a decent amount of extra memory. My experience has been that more memory is always better than less memory, even when the larger memory is slower. Whenever I build a new computer, the largest, fastest memory modules are usually hard to get and/ or expensive. So, I buy one module of 1/2 or 1/4 maximum capacity and 100% maximum speed per channel. If/ when I decide to add memory later, I prefer to buy maximum size and speed modules. Once the memory slots (and CPU sockets) are maxed out and that's still not enough, it's time sell or recycle the computer. The reality for many programs is that all the data is far too big to hold in memory, so disk I/O can become the bottleneck. (The kernel uses that extra memory for caching, which is vital.) The standard solution in this case is faster drives. You might want to spend some time benchmarking your program on whatever computers you have available. Older, slower machines can be very revealing. See if you can tease out the various factors -- program size, CPU(s), core(s), memory, disk I/O, etc.. This will allow you to make informed decisions for upgrades. You also might want to enlist the help of a computer scientist/ programmer familiar with numerical methods and/ or the library/ package you are using. Smarter algorithms, data structures, etc., can yield dramatic improvements in efficiency, alleviating the need for more hardware. David -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/5595f141.2020...@holgerdanske.com
Re: OT: Can lot of RAM can slow down a calculation workstation?
On Fri, Jun 26, 2015 at 2:34 PM, to...@tuxteam.de wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, Jun 26, 2015 at 11:58:40AM +0200, Dan wrote: On Fri, Jun 26, 2015 at 11:46 AM, to...@tuxteam.de wrote: [...] No I do not know that. I am a scientist, and I use the computers as a tool to do simulations that I write in C++ with (Threading Building Blocks). I have a limited knowledge of the computer architecture. That would mean that a calculation that fits in 64GB will run slower with 256GB? or that means that when I increase the size the calculation will be slower. Note that I'm deep in hand-waving territory here. Suppose you could reduce your problem size to one-fourth its current size, so that it fits in your 64 GB. Let's say it needs now a time T_0. Now consider your (real) four-fold problem. Leaving out all effects of swapping and all that (and only considering the pure algorithm), your run time will be T_1, which most probably is (depending on the algorithm's properties) bigger than T_0. For a linear algorithm: T_1 = 4 * T_0 (you're lucky!), for a quadratic one T_1 = 16 * T_0 and so on (if you're _very_ lucky, you have a sublinear algorithm, but given all you've written before I wouldn't bet on that). Taking into account the effects of RAM, you get for 64 GB and your problem (real size) some time T_1_64 which is most probably significantly bigger than T_1: T_1_64 T_1, due to all the caching overhead. How much bigger will depend on how cache-friendly the algorithm is: if it is random-accessing data from all over the place, the slowdown will be horrible (in the limit, of the order of magnitude of the relation of the SSD speed to the RAM speed (yeah, latency, bandwidth. Some combination of both. Pick the worst of them ;-) Now to the 256 GB case. Ideally, the thing fits in there: ideally the time would be T_1_256 =~ T_1, since no swapping overhead, etc. What I was saying is that you might quite well get T_1_256 T_1, because there are other factors (the CPU has a whole hierarchy of caches between itself and the RAM, because the RAM is horribly slow from the POV of the CPU). Those caches might be more overwhelmed by the bigger addressable memory. Now how much bigger, that is a tough question. Most probably you get T_1_64 T_1_256 T_1 so the extra RAM will help, but in some cases the slowdown from the ideal T_1 to the real T_1_256 might prove disappointing. Sometimes, partitioning the problem might give you more speed than throwing RAM at it. Sometimes! I think you have no choice but to try it out. Hi, I took a closer look into this. I found out that that it is important to have at least one DIMM per channel. But if you have a several DIMMs per channel there can be a hit in the performance. Many servers clock down the memory to a lower speed when you add a second or a third DPC. https://marchamilton.wordpress.com/2012/02/07/optimizing-hpc-server-memory-configurations/ http://frankdenneman.nl/2015/02/20/memory-deep-dive/ I did some research and it seems that in general there is no drop with 2 dimms per channel but there is a drop with 3 dimms per channel. I can buy 8x 32 Gb or 16 x 16Gb. The first option is more expensive than the second one, but I will have only one DIMM per channel. Any suggestions or experience with this? Thanks, Dan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAK00fOKAGqoWp7SbXS7dwNL1kDO=fiw77erhhhqpxjcz2_y...@mail.gmail.com
Re: OT: Can lot of RAM can slow down a calculation workstation?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, Jun 26, 2015 at 11:33:48AM +0200, Dan wrote: Yes the calculation is memory/cpu instensive. I can not consider a faster (multi-core) processor because I do not have the budget to buy a new machine. But I need more RAM, the calculation do not fit in 64 GB of RAM. In that case, if the computer has to resort to disk (swap), most probably more RAM might help. How is the access pattern? Does the calculation need access to all of its address space all of the time? In that case, there seems you have no choice. OTOH this will be the pessimal access pattern wrt memory caches (which will be stretched half as thinly with the more RAM), but probably you know all of that. Probably I will increase the RAM, I will also compare the calculation time. I will post the results in the list. Looking forward to your results. Thanks! - -- t -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlWNH2wACgkQBcgs9XrR2kab0gCcDT6HRfv8VtW82Xvj4wEhTv9w B88AnR3m/LkRqg+nTQsNmsvzUA/wtVfA =8/EZ -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150626094620.ga11...@tuxteam.de
Re: OT: Can lot of RAM can slow down a calculation workstation?
On Fri, Jun 26, 2015 at 11:46 AM, to...@tuxteam.de wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, Jun 26, 2015 at 11:33:48AM +0200, Dan wrote: Yes the calculation is memory/cpu instensive. I can not consider a faster (multi-core) processor because I do not have the budget to buy a new machine. But I need more RAM, the calculation do not fit in 64 GB of RAM. In that case, if the computer has to resort to disk (swap), most probably more RAM might help. How is the access pattern? Does the calculation need access to all of its address space all of the time? Right now the calculation fits in 64 GB, but I want to increase the size. I could use the swap (SDD), but it gets very hot, maybe I could buy a fan for the SSD. I do not know if that exists. The calculation needs previous results. It does not need all the addess space at the exact same time. I could use the swap, but I suppose that there is a limit for the temperature that the SSD can get. In that case, there seems you have no choice. OTOH this will be the pessimal access pattern wrt memory caches (which will be stretched half as thinly with the more RAM), but probably you know all of that. No I do not know that. I am a scientist, and I use the computers as a tool to do simulations that I write in C++ with (Threading Building Blocks). I have a limited knowledge of the computer architecture. That would mean that a calculation that fits in 64GB will run slower with 256GB? or that means that when I increase the size the calculation will be slower. Thanks! Dan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAK00fOKy_2f57oNtOG=eshb2djr-wmwwwmdh4c5hg7trynw...@mail.gmail.com
Re: OT: Can lot of RAM can slow down a calculation workstation?
On Wed, Jun 24, 2015 at 11:51 AM, rajiv chavan rc214...@gmail.com wrote: On 6/23/15, to...@tuxteam.de to...@tuxteam.de wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, Jun 23, 2015 at 06:24:52PM +0530, rajiv chavan wrote: Tue, 23 Jun 2015 18:16:51 +0530 Is calculation memory/cpu cycle intensive? Consider a faster (and multi-core) processor with more cache? Agreed -- but I think you wanted to reply to list or to the original poster, Dan. regards - -- t -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlWJjlEACgkQBcgs9XrR2kZHxQCfaeDakuB+6QOlYLS8nkQ9DNae q84An3xu7OoKkfTSrpdyRL+Nbj8ApCyI =/S4F -END PGP SIGNATURE- Thank you Tomas. Hi, Yes the calculation is memory/cpu instensive. I can not consider a faster (multi-core) processor because I do not have the budget to buy a new machine. But I need more RAM, the calculation do not fit in 64 GB of RAM. Probably I will increase the RAM, I will also compare the calculation time. I will post the results in the list. Thanks, Dan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAK00fO+VNBAfeJ_08CQ0uB3ZG+y_O=k6px0rayjomu5znkn...@mail.gmail.com
Re: OT: Can lot of RAM can slow down a calculation workstation?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, Jun 26, 2015 at 11:58:40AM +0200, Dan wrote: On Fri, Jun 26, 2015 at 11:46 AM, to...@tuxteam.de wrote: [...] No I do not know that. I am a scientist, and I use the computers as a tool to do simulations that I write in C++ with (Threading Building Blocks). I have a limited knowledge of the computer architecture. That would mean that a calculation that fits in 64GB will run slower with 256GB? or that means that when I increase the size the calculation will be slower. Note that I'm deep in hand-waving territory here. Suppose you could reduce your problem size to one-fourth its current size, so that it fits in your 64 GB. Let's say it needs now a time T_0. Now consider your (real) four-fold problem. Leaving out all effects of swapping and all that (and only considering the pure algorithm), your run time will be T_1, which most probably is (depending on the algorithm's properties) bigger than T_0. For a linear algorithm: T_1 = 4 * T_0 (you're lucky!), for a quadratic one T_1 = 16 * T_0 and so on (if you're _very_ lucky, you have a sublinear algorithm, but given all you've written before I wouldn't bet on that). Taking into account the effects of RAM, you get for 64 GB and your problem (real size) some time T_1_64 which is most probably significantly bigger than T_1: T_1_64 T_1, due to all the caching overhead. How much bigger will depend on how cache-friendly the algorithm is: if it is random-accessing data from all over the place, the slowdown will be horrible (in the limit, of the order of magnitude of the relation of the SSD speed to the RAM speed (yeah, latency, bandwidth. Some combination of both. Pick the worst of them ;-) Now to the 256 GB case. Ideally, the thing fits in there: ideally the time would be T_1_256 =~ T_1, since no swapping overhead, etc. What I was saying is that you might quite well get T_1_256 T_1, because there are other factors (the CPU has a whole hierarchy of caches between itself and the RAM, because the RAM is horribly slow from the POV of the CPU). Those caches might be more overwhelmed by the bigger addressable memory. Now how much bigger, that is a tough question. Most probably you get T_1_64 T_1_256 T_1 so the extra RAM will help, but in some cases the slowdown from the ideal T_1 to the real T_1_256 might prove disappointing. Sometimes, partitioning the problem might give you more speed than throwing RAM at it. Sometimes! I think you have no choice but to try it out. Regards - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlWNRukACgkQBcgs9XrR2kZurQCfZQFOLgVKll3RDQU6ONGPxIrR dOwAmwdnvY/SOhYxVW+nT/UsQlTHjVHO =DRQU -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150626123449.ga15...@tuxteam.de
Re: OT: Can lot of RAM can slow down a calculation workstation?
On 6/23/15, to...@tuxteam.de to...@tuxteam.de wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, Jun 23, 2015 at 06:24:52PM +0530, rajiv chavan wrote: Tue, 23 Jun 2015 18:16:51 +0530 Is calculation memory/cpu cycle intensive? Consider a faster (and multi-core) processor with more cache? Agreed -- but I think you wanted to reply to list or to the original poster, Dan. regards - -- t -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlWJjlEACgkQBcgs9XrR2kZHxQCfaeDakuB+6QOlYLS8nkQ9DNae q84An3xu7OoKkfTSrpdyRL+Nbj8ApCyI =/S4F -END PGP SIGNATURE- Thank you Tomas. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/calm+mzs8urxfwkgvzkqbhd3o_eoq_bccgu8vtj1-_pn2q2h...@mail.gmail.com
Re: OT: Can lot of RAM can slow down a calculation workstation?
On Tue, Jun 23, 2015 at 10:52:50AM +0200, Dan wrote: Hi, I have a workstation Dell Precision T7610 that I use for scientific calculations. It has two Xeon CPU E5-2650 v2 @ 2.60GHz with 64GB DDR3 1866MHz. I would like to upgrade the machine and buy 256 GB of RAM. In the case of a calculation that can fit in 64GB, Does too much RAM can slow down the computer? I suppose that it shouldn't as long as the RAM is also DDR3 1866MHz. I would suggest this is a question for the kernel people. I have a vague memory (pardon the pun) of there being some issue such that if you did have vast amounts of RAM, some data structure grew rather large and the kernel spent a disproprtionate amount of time servicing that. The fact that I've seen that, though, PROBABLY means that it was mentioned as part of a fix. In short, make sure your kernel is reasonably new (Something 4.0 or newer should be fine, maybe even 3.0 or newer). Oh, and from a hardware perspective, read the manual of your motherboard for how best to load the RAM; you will probably be instructed to load sticks of RAM in a balanced configuration in order to fully exploit parallel access by the memory controller. Thanks, Dan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/cak00folm5-jbl0mwhtorth-59-klhkeyaq-ntmvtktg12rt...@mail.gmail.com -- For more information, please reread. signature.asc Description: Digital signature
Re: OT: Can lot of RAM can slow down a calculation workstation?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, Jun 23, 2015 at 10:52:50AM +0200, Dan wrote: Hi, [...] I would like to upgrade the machine and buy 256 GB of RAM [...] Tough question. Apart from Darac's considerations, that'll depend much on your calculations: if the working set of your application can grow into more RAM, the cache hit ratio might diminish. Whether this results in a slowdown overall will depend on processor, memory architecture and application in pretty complex ways. regards - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlWJJWQACgkQBcgs9XrR2ka1yACdHjYjsP8GstFgDzc4Mb5cfnBX KqYAn1ec5iA/G9H8Vugbr2i6GPFffRsP =wJsi -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150623092244.gc19...@tuxteam.de
