Re: [gmx-users] Xeon Gold + RTX 5000
Hi Szilárd, Thanks for the interesting observations on recent hardware. I was wondering if you could comment on the use of somewhat older server cpus and motherboards (versus more cutting edge consumer parts). I recently noticed that Haswell era Xeon cpus (E5 v3) are quite affordable now (~$400 for 12 core models with 40 pcie lanes) and so are the corresponding 2 cpu socket server motherboards. Of course the RAM is slower than what can be used with the latest Ryzen or i7/i9 cpus. Are there any other bottlenecks with this somewhat older server hardware that I might not be aware of? Thanks again for the interesting information and practical advice on this topic. Mike > On Jul 18, 2019, at 2:21 AM, Szilárd Páll wrote: > > PS: You will get more PCIe lanes without motherboard trickery -- and note > that consumer motherboards with PCIe switches can sometimes cause > instabilities when under heavy compute load -- if you buy the aging and > quite overpriced i9 X-series like the i9-7920 with 12 cores or the > Threadripper 2950x 16 cores and 60 PCIe lanes. > > Also note that, but more cores always win when the CPU performance matters > and while 8 cores are generally sufficient, in some use-cases it may not be > (like runs with free energy). > > -- > Szilárd > > > On Thu, Jul 18, 2019 at 10:08 AM Szilárd Páll > wrote: > >> On Wed, Jul 17, 2019 at 7:00 PM Moir, Michael (MMoir) >> wrote: >> >>> This is not quite true. I certainly observed this degradation in >>> performance using the 9900K with two GPUs as Szilárd states using a >>> motherboard with one PCIe controller, but the limitation is from the >>> motherboard not from the CPU. >> >> >> Sorry, but that's not the case. PCIe controllers have been integrated into >> CPUs for many years; see >> >> https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ia-introduction-basics-paper.pdf >> >> https://www.microway.com/hpc-tech-tips/common-pci-express-myths-gpu-computing/ >> >> So no, the limitation is the CPU itself. Consumer CPUs these days have 24 >> lanes total, some of which are used to connect the CPU to the chipset, and >> effectively you get 16-20 lanes (BTW here too the new AMD CPUs win as they >> provide 16 lanes for GPUs and similar devices and 4 lanes for NVMe, all on >> PCIe 4.0). >> >> >>> It is possible to obtain a motherboard that contains two PCIe >>> controllers which overcomes this obstacle for not a whole lot more money. >>> >> >> It is possibly to buy motherboards with PCIe switches. These don't >> increase the number of lanes just do what a swtich does: as long as not all >> connected devices try to use the full capacity of the CPU (!) at the same >> time, you can get full speed on all connected devices. >> e.g.: >> https://techreport.com/r.x/2015_11_19_Gigabytes_Z170XGaming_G1_motherboard_reviewed/05-diagram_pcie_routing.gif >> >> Cheers, >> -- >> Szilárd >> >> Mike >>> >>> -Original Message- >>> From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se < >>> gromacs.org_gmx-users-boun...@maillist.sys.kth.se> On Behalf Of Szilárd >>> Páll >>> Sent: Wednesday, July 17, 2019 8:14 AM >>> To: Discussion list for GROMACS users >>> Subject: [**EXTERNAL**] Re: [gmx-users] Xeon Gold + RTX 5000 >>> >>> Hi Alex, >>> >>> I've not had a chance to test the new 3rd gen Ryzen CPUs, but all >>> public benchmarks out there point to the fact that they are a major >>> improvement over the previous generation Ryzen -- which were already >>> quite competitive for GPU-accelerated GROMACS runs compared to Intel, >>> especially in perf/price. >>> >>> One caveat for dual-GPU setups on the i9 9900 or the Ryzen 3900X is >>> that they don't have enough PCI lanes for peak CPU-GPU transfer (x8 >>> for both of the GPUs) which will lead to a slightly less performance >>> (I'd estimate <5-10%) in particular compared to i) having a single GPU >>> plugged in into the machine ii) compare to CPUs like Threadripper or >>> the i9 79xx series processors which have more PCIe lanes. >>> >>> However, if throughput is the goal, the ideal use-case especially for >>> small simulation systems like <=50k atoms is to run e.g. 2 runs / GPU, >>> hence 4 runs on a 2-GPU system case in which the impact of the >>> aforementioned limitation will be further decreased. >>> >>> Cheers, >>> -- >>> Szilárd >>> >>> >
Re: [gmx-users] Xeon Gold + RTX 5000
PS: You will get more PCIe lanes without motherboard trickery -- and note that consumer motherboards with PCIe switches can sometimes cause instabilities when under heavy compute load -- if you buy the aging and quite overpriced i9 X-series like the i9-7920 with 12 cores or the Threadripper 2950x 16 cores and 60 PCIe lanes. Also note that, but more cores always win when the CPU performance matters and while 8 cores are generally sufficient, in some use-cases it may not be (like runs with free energy). -- Szilárd On Thu, Jul 18, 2019 at 10:08 AM Szilárd Páll wrote: > On Wed, Jul 17, 2019 at 7:00 PM Moir, Michael (MMoir) > wrote: > >> This is not quite true. I certainly observed this degradation in >> performance using the 9900K with two GPUs as Szilárd states using a >> motherboard with one PCIe controller, but the limitation is from the >> motherboard not from the CPU. > > > Sorry, but that's not the case. PCIe controllers have been integrated into > CPUs for many years; see > > https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ia-introduction-basics-paper.pdf > > https://www.microway.com/hpc-tech-tips/common-pci-express-myths-gpu-computing/ > > So no, the limitation is the CPU itself. Consumer CPUs these days have 24 > lanes total, some of which are used to connect the CPU to the chipset, and > effectively you get 16-20 lanes (BTW here too the new AMD CPUs win as they > provide 16 lanes for GPUs and similar devices and 4 lanes for NVMe, all on > PCIe 4.0). > > >> It is possible to obtain a motherboard that contains two PCIe >> controllers which overcomes this obstacle for not a whole lot more money. >> > > It is possibly to buy motherboards with PCIe switches. These don't > increase the number of lanes just do what a swtich does: as long as not all > connected devices try to use the full capacity of the CPU (!) at the same > time, you can get full speed on all connected devices. > e.g.: > https://techreport.com/r.x/2015_11_19_Gigabytes_Z170XGaming_G1_motherboard_reviewed/05-diagram_pcie_routing.gif > > Cheers, > -- > Szilárd > > Mike >> >> -Original Message- >> From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se < >> gromacs.org_gmx-users-boun...@maillist.sys.kth.se> On Behalf Of Szilárd >> Páll >> Sent: Wednesday, July 17, 2019 8:14 AM >> To: Discussion list for GROMACS users >> Subject: [**EXTERNAL**] Re: [gmx-users] Xeon Gold + RTX 5000 >> >> Hi Alex, >> >> I've not had a chance to test the new 3rd gen Ryzen CPUs, but all >> public benchmarks out there point to the fact that they are a major >> improvement over the previous generation Ryzen -- which were already >> quite competitive for GPU-accelerated GROMACS runs compared to Intel, >> especially in perf/price. >> >> One caveat for dual-GPU setups on the i9 9900 or the Ryzen 3900X is >> that they don't have enough PCI lanes for peak CPU-GPU transfer (x8 >> for both of the GPUs) which will lead to a slightly less performance >> (I'd estimate <5-10%) in particular compared to i) having a single GPU >> plugged in into the machine ii) compare to CPUs like Threadripper or >> the i9 79xx series processors which have more PCIe lanes. >> >> However, if throughput is the goal, the ideal use-case especially for >> small simulation systems like <=50k atoms is to run e.g. 2 runs / GPU, >> hence 4 runs on a 2-GPU system case in which the impact of the >> aforementioned limitation will be further decreased. >> >> Cheers, >> -- >> Szilárd >> >> >> On Tue, Jul 16, 2019 at 7:18 PM Alex wrote: >> > >> > That is excellent information, thank you. None of us have dealt with AMD >> > CPUs in a while, so would the combination of a Ryzen 3900X and two >> > Quadro 2080 Ti be a good choice? >> > >> > Again, thanks! >> > >> > Alex >> > >> > >> > On 7/16/2019 8:41 AM, Szilárd Páll wrote: >> > > Hi Alex, >> > > >> > > On Mon, Jul 15, 2019 at 8:53 PM Alex wrote: >> > >> Hi all and especially Szilard! >> > >> >> > >> My glorious management asked me to post this here. One of our group >> > >> members, an ex-NAMD guy, wants to use Gromacs for biophysics and the >> > >> following basics have been spec'ed for him: >> > >> >> > >> CPU: Xeon Gold 6244 >> > >> GPU: RTX 5000 or 6000 >> > >> >> > >> I'll be surprised if he runs systems with more than 50K particles. >> Could >> > >> y
Re: [gmx-users] Xeon Gold + RTX 5000
On Wed, Jul 17, 2019 at 7:00 PM Moir, Michael (MMoir) wrote: > This is not quite true. I certainly observed this degradation in > performance using the 9900K with two GPUs as Szilárd states using a > motherboard with one PCIe controller, but the limitation is from the > motherboard not from the CPU. Sorry, but that's not the case. PCIe controllers have been integrated into CPUs for many years; see https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ia-introduction-basics-paper.pdf https://www.microway.com/hpc-tech-tips/common-pci-express-myths-gpu-computing/ So no, the limitation is the CPU itself. Consumer CPUs these days have 24 lanes total, some of which are used to connect the CPU to the chipset, and effectively you get 16-20 lanes (BTW here too the new AMD CPUs win as they provide 16 lanes for GPUs and similar devices and 4 lanes for NVMe, all on PCIe 4.0). > It is possible to obtain a motherboard that contains two PCIe > controllers which overcomes this obstacle for not a whole lot more money. > It is possibly to buy motherboards with PCIe switches. These don't increase the number of lanes just do what a swtich does: as long as not all connected devices try to use the full capacity of the CPU (!) at the same time, you can get full speed on all connected devices. e.g.: https://techreport.com/r.x/2015_11_19_Gigabytes_Z170XGaming_G1_motherboard_reviewed/05-diagram_pcie_routing.gif Cheers, -- Szilárd Mike > > -Original Message- > From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se < > gromacs.org_gmx-users-boun...@maillist.sys.kth.se> On Behalf Of Szilárd > Páll > Sent: Wednesday, July 17, 2019 8:14 AM > To: Discussion list for GROMACS users > Subject: [**EXTERNAL**] Re: [gmx-users] Xeon Gold + RTX 5000 > > Hi Alex, > > I've not had a chance to test the new 3rd gen Ryzen CPUs, but all > public benchmarks out there point to the fact that they are a major > improvement over the previous generation Ryzen -- which were already > quite competitive for GPU-accelerated GROMACS runs compared to Intel, > especially in perf/price. > > One caveat for dual-GPU setups on the i9 9900 or the Ryzen 3900X is > that they don't have enough PCI lanes for peak CPU-GPU transfer (x8 > for both of the GPUs) which will lead to a slightly less performance > (I'd estimate <5-10%) in particular compared to i) having a single GPU > plugged in into the machine ii) compare to CPUs like Threadripper or > the i9 79xx series processors which have more PCIe lanes. > > However, if throughput is the goal, the ideal use-case especially for > small simulation systems like <=50k atoms is to run e.g. 2 runs / GPU, > hence 4 runs on a 2-GPU system case in which the impact of the > aforementioned limitation will be further decreased. > > Cheers, > -- > Szilárd > > > On Tue, Jul 16, 2019 at 7:18 PM Alex wrote: > > > > That is excellent information, thank you. None of us have dealt with AMD > > CPUs in a while, so would the combination of a Ryzen 3900X and two > > Quadro 2080 Ti be a good choice? > > > > Again, thanks! > > > > Alex > > > > > > On 7/16/2019 8:41 AM, Szilárd Páll wrote: > > > Hi Alex, > > > > > > On Mon, Jul 15, 2019 at 8:53 PM Alex wrote: > > >> Hi all and especially Szilard! > > >> > > >> My glorious management asked me to post this here. One of our group > > >> members, an ex-NAMD guy, wants to use Gromacs for biophysics and the > > >> following basics have been spec'ed for him: > > >> > > >> CPU: Xeon Gold 6244 > > >> GPU: RTX 5000 or 6000 > > >> > > >> I'll be surprised if he runs systems with more than 50K particles. > Could > > >> you please comment on whether this is a cost-efficient and reasonably > > >> powerful setup? Your past suggestions have been invaluable for us. > > > That will be reasonably fast, but cost efficiency will be awful, to be > honest: > > > - that CPU is a ~$3000 part and won't perform much better than a > > > $4-500 desktop CPU like an i9 9900, let alone a Ryzen 3900X which > > > would be significantly faster. > > > - Quadro cards also pretty low in bang for buck: a 2080 Ti will be > > > close to the RTX 6000 for ~5x less and the 2080 or 2070 Super a bit > > > slower for at least another 1.5x less. > > > > > > Single run at a time or possibly multiple? The proposed (or any 8+ > > > core) workstation CPU is fast enough in the majority of the > > > simulations to pair well with two of those GPUs if used for two > > > concurrent simulations. If that's a relevant use-case,
Re: [gmx-users] Xeon Gold + RTX 5000
Perfect, thanks a lot! We are less constrained by cost, so we'll go straight to 2080Ti. You guys already saved us a few grand here. ;) Alex On 7/17/2019 7:34 PM, Moir, Michael (MMoir) wrote: Alex, The motherboard I am using with the 9900K is the ASUS WS Z390 PRO. The PRO version has the extra PCIe controller. I am using two RTX 1070ti GPUs, and I can hear everyone snorting with derision, but with this configuration I get performance with 100,000 atoms of about 72 ns/day with 2019.1 which is adequate for my needs. I’ll upgrade to the 2080ti when the price drops to <$1000. Another tip is to get the highest speed memory that the motherboard will handle. It doesn’t make a huge difference, maybe 2-3% over the cheapest memory but it is something that is easy and low cost to do. Mike Sent from my iPhone On Jul 17, 2019, at 11:16 AM, Alex wrote: Gentlemen, thank you both! Michael, would you be able to suggest a specific motherboard that removes the bottleneck? We aren't really limited by price in this case and would prefer to get every bit of benefit out of the processing components, if possible. Thanks, Alex On 7/17/2019 10:44 AM, Moir, Michael (MMoir) wrote: This is not quite true. I certainly observed this degradation in performance using the 9900K with two GPUs as Szilárd states using a motherboard with one PCIe controller, but the limitation is from the motherboard not from the CPU. It is possible to obtain a motherboard that contains two PCIe controllers which overcomes this obstacle for not a whole lot more money. Mike -Original Message- From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se On Behalf Of Szilárd Páll Sent: Wednesday, July 17, 2019 8:14 AM To: Discussion list for GROMACS users Subject: [**EXTERNAL**] Re: [gmx-users] Xeon Gold + RTX 5000 Hi Alex, I've not had a chance to test the new 3rd gen Ryzen CPUs, but all public benchmarks out there point to the fact that they are a major improvement over the previous generation Ryzen -- which were already quite competitive for GPU-accelerated GROMACS runs compared to Intel, especially in perf/price. One caveat for dual-GPU setups on the i9 9900 or the Ryzen 3900X is that they don't have enough PCI lanes for peak CPU-GPU transfer (x8 for both of the GPUs) which will lead to a slightly less performance (I'd estimate <5-10%) in particular compared to i) having a single GPU plugged in into the machine ii) compare to CPUs like Threadripper or the i9 79xx series processors which have more PCIe lanes. However, if throughput is the goal, the ideal use-case especially for small simulation systems like <=50k atoms is to run e.g. 2 runs / GPU, hence 4 runs on a 2-GPU system case in which the impact of the aforementioned limitation will be further decreased. Cheers, -- Szilárd On Tue, Jul 16, 2019 at 7:18 PM Alex wrote: That is excellent information, thank you. None of us have dealt with AMD CPUs in a while, so would the combination of a Ryzen 3900X and two Quadro 2080 Ti be a good choice? Again, thanks! Alex On 7/16/2019 8:41 AM, Szilárd Páll wrote: Hi Alex, On Mon, Jul 15, 2019 at 8:53 PM Alex wrote: Hi all and especially Szilard! My glorious management asked me to post this here. One of our group members, an ex-NAMD guy, wants to use Gromacs for biophysics and the following basics have been spec'ed for him: CPU: Xeon Gold 6244 GPU: RTX 5000 or 6000 I'll be surprised if he runs systems with more than 50K particles. Could you please comment on whether this is a cost-efficient and reasonably powerful setup? Your past suggestions have been invaluable for us. That will be reasonably fast, but cost efficiency will be awful, to be honest: - that CPU is a ~$3000 part and won't perform much better than a $4-500 desktop CPU like an i9 9900, let alone a Ryzen 3900X which would be significantly faster. - Quadro cards also pretty low in bang for buck: a 2080 Ti will be close to the RTX 6000 for ~5x less and the 2080 or 2070 Super a bit slower for at least another 1.5x less. Single run at a time or possibly multiple? The proposed (or any 8+ core) workstation CPU is fast enough in the majority of the simulations to pair well with two of those GPUs if used for two concurrent simulations. If that's a relevant use-case, I'd recommend two 2070 Super or 2080 cards. Cheers, -- Szilárd Thank you, Alex -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests vi
Re: [gmx-users] Xeon Gold + RTX 5000
Alex, The motherboard I am using with the 9900K is the ASUS WS Z390 PRO. The PRO version has the extra PCIe controller. I am using two RTX 1070ti GPUs, and I can hear everyone snorting with derision, but with this configuration I get performance with 100,000 atoms of about 72 ns/day with 2019.1 which is adequate for my needs. I’ll upgrade to the 2080ti when the price drops to <$1000. Another tip is to get the highest speed memory that the motherboard will handle. It doesn’t make a huge difference, maybe 2-3% over the cheapest memory but it is something that is easy and low cost to do. Mike Sent from my iPhone > On Jul 17, 2019, at 11:16 AM, Alex wrote: > > Gentlemen, thank you both! > > Michael, would you be able to suggest a specific motherboard that removes the > bottleneck? We aren't really limited by price in this case and would prefer > to get every bit of benefit out of the processing components, if possible. > > Thanks, > > Alex > >> On 7/17/2019 10:44 AM, Moir, Michael (MMoir) wrote: >> This is not quite true. I certainly observed this degradation in >> performance using the 9900K with two GPUs as Szilárd states using a >> motherboard with one PCIe controller, but the limitation is from the >> motherboard not from the CPU. It is possible to obtain a motherboard that >> contains two PCIe controllers which overcomes this obstacle for not a whole >> lot more money. >> >> Mike >> >> -Original Message- >> From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se >> On Behalf Of Szilárd Páll >> Sent: Wednesday, July 17, 2019 8:14 AM >> To: Discussion list for GROMACS users >> Subject: [**EXTERNAL**] Re: [gmx-users] Xeon Gold + RTX 5000 >> >> Hi Alex, >> >> I've not had a chance to test the new 3rd gen Ryzen CPUs, but all >> public benchmarks out there point to the fact that they are a major >> improvement over the previous generation Ryzen -- which were already >> quite competitive for GPU-accelerated GROMACS runs compared to Intel, >> especially in perf/price. >> >> One caveat for dual-GPU setups on the i9 9900 or the Ryzen 3900X is >> that they don't have enough PCI lanes for peak CPU-GPU transfer (x8 >> for both of the GPUs) which will lead to a slightly less performance >> (I'd estimate <5-10%) in particular compared to i) having a single GPU >> plugged in into the machine ii) compare to CPUs like Threadripper or >> the i9 79xx series processors which have more PCIe lanes. >> >> However, if throughput is the goal, the ideal use-case especially for >> small simulation systems like <=50k atoms is to run e.g. 2 runs / GPU, >> hence 4 runs on a 2-GPU system case in which the impact of the >> aforementioned limitation will be further decreased. >> >> Cheers, >> -- >> Szilárd >> >> >>> On Tue, Jul 16, 2019 at 7:18 PM Alex wrote: >>> That is excellent information, thank you. None of us have dealt with AMD >>> CPUs in a while, so would the combination of a Ryzen 3900X and two >>> Quadro 2080 Ti be a good choice? >>> >>> Again, thanks! >>> >>> Alex >>> >>> >>>> On 7/16/2019 8:41 AM, Szilárd Páll wrote: >>>> Hi Alex, >>>> >>>>> On Mon, Jul 15, 2019 at 8:53 PM Alex wrote: >>>>> Hi all and especially Szilard! >>>>> >>>>> My glorious management asked me to post this here. One of our group >>>>> members, an ex-NAMD guy, wants to use Gromacs for biophysics and the >>>>> following basics have been spec'ed for him: >>>>> >>>>> CPU: Xeon Gold 6244 >>>>> GPU: RTX 5000 or 6000 >>>>> >>>>> I'll be surprised if he runs systems with more than 50K particles. Could >>>>> you please comment on whether this is a cost-efficient and reasonably >>>>> powerful setup? Your past suggestions have been invaluable for us. >>>> That will be reasonably fast, but cost efficiency will be awful, to be >>>> honest: >>>> - that CPU is a ~$3000 part and won't perform much better than a >>>> $4-500 desktop CPU like an i9 9900, let alone a Ryzen 3900X which >>>> would be significantly faster. >>>> - Quadro cards also pretty low in bang for buck: a 2080 Ti will be >>>> close to the RTX 6000 for ~5x less and the 2080 or 2070 Super a bit >>>> slower for at least another 1.5x less. >>>> >>>> Single run at a time or possibly multiple? The proposed (or any 8+ >>>&
Re: [gmx-users] Xeon Gold + RTX 5000
Certainly. When I get home this evening I will post the information. Mike -Original Message- From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se On Behalf Of Alex Sent: Wednesday, July 17, 2019 11:16 AM To: gmx-us...@gromacs.org Subject: [**EXTERNAL**] Re: [gmx-users] Xeon Gold + RTX 5000 Gentlemen, thank you both! Michael, would you be able to suggest a specific motherboard that removes the bottleneck? We aren't really limited by price in this case and would prefer to get every bit of benefit out of the processing components, if possible. Thanks, Alex On 7/17/2019 10:44 AM, Moir, Michael (MMoir) wrote: > This is not quite true. I certainly observed this degradation in performance > using the 9900K with two GPUs as Szilárd states using a motherboard with one > PCIe controller, but the limitation is from the motherboard not from the CPU. > It is possible to obtain a motherboard that contains two PCIe controllers > which overcomes this obstacle for not a whole lot more money. > > Mike > > -Original Message- > From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se > On Behalf Of Szilárd Páll > Sent: Wednesday, July 17, 2019 8:14 AM > To: Discussion list for GROMACS users > Subject: [**EXTERNAL**] Re: [gmx-users] Xeon Gold + RTX 5000 > > Hi Alex, > > I've not had a chance to test the new 3rd gen Ryzen CPUs, but all > public benchmarks out there point to the fact that they are a major > improvement over the previous generation Ryzen -- which were already > quite competitive for GPU-accelerated GROMACS runs compared to Intel, > especially in perf/price. > > One caveat for dual-GPU setups on the i9 9900 or the Ryzen 3900X is > that they don't have enough PCI lanes for peak CPU-GPU transfer (x8 > for both of the GPUs) which will lead to a slightly less performance > (I'd estimate <5-10%) in particular compared to i) having a single GPU > plugged in into the machine ii) compare to CPUs like Threadripper or > the i9 79xx series processors which have more PCIe lanes. > > However, if throughput is the goal, the ideal use-case especially for > small simulation systems like <=50k atoms is to run e.g. 2 runs / GPU, > hence 4 runs on a 2-GPU system case in which the impact of the > aforementioned limitation will be further decreased. > > Cheers, > -- > Szilárd > > > On Tue, Jul 16, 2019 at 7:18 PM Alex wrote: >> That is excellent information, thank you. None of us have dealt with AMD >> CPUs in a while, so would the combination of a Ryzen 3900X and two >> Quadro 2080 Ti be a good choice? >> >> Again, thanks! >> >> Alex >> >> >> On 7/16/2019 8:41 AM, Szilárd Páll wrote: >>> Hi Alex, >>> >>> On Mon, Jul 15, 2019 at 8:53 PM Alex wrote: >>>> Hi all and especially Szilard! >>>> >>>> My glorious management asked me to post this here. One of our group >>>> members, an ex-NAMD guy, wants to use Gromacs for biophysics and the >>>> following basics have been spec'ed for him: >>>> >>>> CPU: Xeon Gold 6244 >>>> GPU: RTX 5000 or 6000 >>>> >>>> I'll be surprised if he runs systems with more than 50K particles. Could >>>> you please comment on whether this is a cost-efficient and reasonably >>>> powerful setup? Your past suggestions have been invaluable for us. >>> That will be reasonably fast, but cost efficiency will be awful, to be >>> honest: >>> - that CPU is a ~$3000 part and won't perform much better than a >>> $4-500 desktop CPU like an i9 9900, let alone a Ryzen 3900X which >>> would be significantly faster. >>> - Quadro cards also pretty low in bang for buck: a 2080 Ti will be >>> close to the RTX 6000 for ~5x less and the 2080 or 2070 Super a bit >>> slower for at least another 1.5x less. >>> >>> Single run at a time or possibly multiple? The proposed (or any 8+ >>> core) workstation CPU is fast enough in the majority of the >>> simulations to pair well with two of those GPUs if used for two >>> concurrent simulations. If that's a relevant use-case, I'd recommend >>> two 2070 Super or 2080 cards. >>> >>> Cheers, >>> -- >>> Szilárd >>> >>> >>>> Thank you, >>>> >>>> Alex >>>> -- >>>> Gromacs Users mailing list >>>> >>>> * Please search the archive at >>>> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! >>>> >>>> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists >>>> >>>&
Re: [gmx-users] Xeon Gold + RTX 5000
Gentlemen, thank you both! Michael, would you be able to suggest a specific motherboard that removes the bottleneck? We aren't really limited by price in this case and would prefer to get every bit of benefit out of the processing components, if possible. Thanks, Alex On 7/17/2019 10:44 AM, Moir, Michael (MMoir) wrote: This is not quite true. I certainly observed this degradation in performance using the 9900K with two GPUs as Szilárd states using a motherboard with one PCIe controller, but the limitation is from the motherboard not from the CPU. It is possible to obtain a motherboard that contains two PCIe controllers which overcomes this obstacle for not a whole lot more money. Mike -Original Message- From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se On Behalf Of Szilárd Páll Sent: Wednesday, July 17, 2019 8:14 AM To: Discussion list for GROMACS users Subject: [**EXTERNAL**] Re: [gmx-users] Xeon Gold + RTX 5000 Hi Alex, I've not had a chance to test the new 3rd gen Ryzen CPUs, but all public benchmarks out there point to the fact that they are a major improvement over the previous generation Ryzen -- which were already quite competitive for GPU-accelerated GROMACS runs compared to Intel, especially in perf/price. One caveat for dual-GPU setups on the i9 9900 or the Ryzen 3900X is that they don't have enough PCI lanes for peak CPU-GPU transfer (x8 for both of the GPUs) which will lead to a slightly less performance (I'd estimate <5-10%) in particular compared to i) having a single GPU plugged in into the machine ii) compare to CPUs like Threadripper or the i9 79xx series processors which have more PCIe lanes. However, if throughput is the goal, the ideal use-case especially for small simulation systems like <=50k atoms is to run e.g. 2 runs / GPU, hence 4 runs on a 2-GPU system case in which the impact of the aforementioned limitation will be further decreased. Cheers, -- Szilárd On Tue, Jul 16, 2019 at 7:18 PM Alex wrote: That is excellent information, thank you. None of us have dealt with AMD CPUs in a while, so would the combination of a Ryzen 3900X and two Quadro 2080 Ti be a good choice? Again, thanks! Alex On 7/16/2019 8:41 AM, Szilárd Páll wrote: Hi Alex, On Mon, Jul 15, 2019 at 8:53 PM Alex wrote: Hi all and especially Szilard! My glorious management asked me to post this here. One of our group members, an ex-NAMD guy, wants to use Gromacs for biophysics and the following basics have been spec'ed for him: CPU: Xeon Gold 6244 GPU: RTX 5000 or 6000 I'll be surprised if he runs systems with more than 50K particles. Could you please comment on whether this is a cost-efficient and reasonably powerful setup? Your past suggestions have been invaluable for us. That will be reasonably fast, but cost efficiency will be awful, to be honest: - that CPU is a ~$3000 part and won't perform much better than a $4-500 desktop CPU like an i9 9900, let alone a Ryzen 3900X which would be significantly faster. - Quadro cards also pretty low in bang for buck: a 2080 Ti will be close to the RTX 6000 for ~5x less and the 2080 or 2070 Super a bit slower for at least another 1.5x less. Single run at a time or possibly multiple? The proposed (or any 8+ core) workstation CPU is fast enough in the majority of the simulations to pair well with two of those GPUs if used for two concurrent simulations. If that's a relevant use-case, I'd recommend two 2070 Super or 2080 cards. Cheers, -- Szilárd Thank you, Alex -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Xeon Gold + RTX 5000
This is not quite true. I certainly observed this degradation in performance using the 9900K with two GPUs as Szilárd states using a motherboard with one PCIe controller, but the limitation is from the motherboard not from the CPU. It is possible to obtain a motherboard that contains two PCIe controllers which overcomes this obstacle for not a whole lot more money. Mike -Original Message- From: gromacs.org_gmx-users-boun...@maillist.sys.kth.se On Behalf Of Szilárd Páll Sent: Wednesday, July 17, 2019 8:14 AM To: Discussion list for GROMACS users Subject: [**EXTERNAL**] Re: [gmx-users] Xeon Gold + RTX 5000 Hi Alex, I've not had a chance to test the new 3rd gen Ryzen CPUs, but all public benchmarks out there point to the fact that they are a major improvement over the previous generation Ryzen -- which were already quite competitive for GPU-accelerated GROMACS runs compared to Intel, especially in perf/price. One caveat for dual-GPU setups on the i9 9900 or the Ryzen 3900X is that they don't have enough PCI lanes for peak CPU-GPU transfer (x8 for both of the GPUs) which will lead to a slightly less performance (I'd estimate <5-10%) in particular compared to i) having a single GPU plugged in into the machine ii) compare to CPUs like Threadripper or the i9 79xx series processors which have more PCIe lanes. However, if throughput is the goal, the ideal use-case especially for small simulation systems like <=50k atoms is to run e.g. 2 runs / GPU, hence 4 runs on a 2-GPU system case in which the impact of the aforementioned limitation will be further decreased. Cheers, -- Szilárd On Tue, Jul 16, 2019 at 7:18 PM Alex wrote: > > That is excellent information, thank you. None of us have dealt with AMD > CPUs in a while, so would the combination of a Ryzen 3900X and two > Quadro 2080 Ti be a good choice? > > Again, thanks! > > Alex > > > On 7/16/2019 8:41 AM, Szilárd Páll wrote: > > Hi Alex, > > > > On Mon, Jul 15, 2019 at 8:53 PM Alex wrote: > >> Hi all and especially Szilard! > >> > >> My glorious management asked me to post this here. One of our group > >> members, an ex-NAMD guy, wants to use Gromacs for biophysics and the > >> following basics have been spec'ed for him: > >> > >> CPU: Xeon Gold 6244 > >> GPU: RTX 5000 or 6000 > >> > >> I'll be surprised if he runs systems with more than 50K particles. Could > >> you please comment on whether this is a cost-efficient and reasonably > >> powerful setup? Your past suggestions have been invaluable for us. > > That will be reasonably fast, but cost efficiency will be awful, to be > > honest: > > - that CPU is a ~$3000 part and won't perform much better than a > > $4-500 desktop CPU like an i9 9900, let alone a Ryzen 3900X which > > would be significantly faster. > > - Quadro cards also pretty low in bang for buck: a 2080 Ti will be > > close to the RTX 6000 for ~5x less and the 2080 or 2070 Super a bit > > slower for at least another 1.5x less. > > > > Single run at a time or possibly multiple? The proposed (or any 8+ > > core) workstation CPU is fast enough in the majority of the > > simulations to pair well with two of those GPUs if used for two > > concurrent simulations. If that's a relevant use-case, I'd recommend > > two 2070 Super or 2080 cards. > > > > Cheers, > > -- > > Szilárd > > > > > >> Thank you, > >> > >> Alex > >> -- > >> Gromacs Users mailing list > >> > >> * Please search the archive at > >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! > >> > >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > >> > >> * For (un)subscribe requests visit > >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send > >> a mail to gmx-users-requ...@gromacs.org. > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a > mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Xeon Gold + RTX 5000
Hi Alex, I've not had a chance to test the new 3rd gen Ryzen CPUs, but all public benchmarks out there point to the fact that they are a major improvement over the previous generation Ryzen -- which were already quite competitive for GPU-accelerated GROMACS runs compared to Intel, especially in perf/price. One caveat for dual-GPU setups on the i9 9900 or the Ryzen 3900X is that they don't have enough PCI lanes for peak CPU-GPU transfer (x8 for both of the GPUs) which will lead to a slightly less performance (I'd estimate <5-10%) in particular compared to i) having a single GPU plugged in into the machine ii) compare to CPUs like Threadripper or the i9 79xx series processors which have more PCIe lanes. However, if throughput is the goal, the ideal use-case especially for small simulation systems like <=50k atoms is to run e.g. 2 runs / GPU, hence 4 runs on a 2-GPU system case in which the impact of the aforementioned limitation will be further decreased. Cheers, -- Szilárd On Tue, Jul 16, 2019 at 7:18 PM Alex wrote: > > That is excellent information, thank you. None of us have dealt with AMD > CPUs in a while, so would the combination of a Ryzen 3900X and two > Quadro 2080 Ti be a good choice? > > Again, thanks! > > Alex > > > On 7/16/2019 8:41 AM, Szilárd Páll wrote: > > Hi Alex, > > > > On Mon, Jul 15, 2019 at 8:53 PM Alex wrote: > >> Hi all and especially Szilard! > >> > >> My glorious management asked me to post this here. One of our group > >> members, an ex-NAMD guy, wants to use Gromacs for biophysics and the > >> following basics have been spec'ed for him: > >> > >> CPU: Xeon Gold 6244 > >> GPU: RTX 5000 or 6000 > >> > >> I'll be surprised if he runs systems with more than 50K particles. Could > >> you please comment on whether this is a cost-efficient and reasonably > >> powerful setup? Your past suggestions have been invaluable for us. > > That will be reasonably fast, but cost efficiency will be awful, to be > > honest: > > - that CPU is a ~$3000 part and won't perform much better than a > > $4-500 desktop CPU like an i9 9900, let alone a Ryzen 3900X which > > would be significantly faster. > > - Quadro cards also pretty low in bang for buck: a 2080 Ti will be > > close to the RTX 6000 for ~5x less and the 2080 or 2070 Super a bit > > slower for at least another 1.5x less. > > > > Single run at a time or possibly multiple? The proposed (or any 8+ > > core) workstation CPU is fast enough in the majority of the > > simulations to pair well with two of those GPUs if used for two > > concurrent simulations. If that's a relevant use-case, I'd recommend > > two 2070 Super or 2080 cards. > > > > Cheers, > > -- > > Szilárd > > > > > >> Thank you, > >> > >> Alex > >> -- > >> Gromacs Users mailing list > >> > >> * Please search the archive at > >> http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! > >> > >> * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > >> > >> * For (un)subscribe requests visit > >> https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send > >> a mail to gmx-users-requ...@gromacs.org. > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a > mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Xeon Gold + RTX 5000
That is excellent information, thank you. None of us have dealt with AMD CPUs in a while, so would the combination of a Ryzen 3900X and two Quadro 2080 Ti be a good choice? Again, thanks! Alex On 7/16/2019 8:41 AM, Szilárd Páll wrote: Hi Alex, On Mon, Jul 15, 2019 at 8:53 PM Alex wrote: Hi all and especially Szilard! My glorious management asked me to post this here. One of our group members, an ex-NAMD guy, wants to use Gromacs for biophysics and the following basics have been spec'ed for him: CPU: Xeon Gold 6244 GPU: RTX 5000 or 6000 I'll be surprised if he runs systems with more than 50K particles. Could you please comment on whether this is a cost-efficient and reasonably powerful setup? Your past suggestions have been invaluable for us. That will be reasonably fast, but cost efficiency will be awful, to be honest: - that CPU is a ~$3000 part and won't perform much better than a $4-500 desktop CPU like an i9 9900, let alone a Ryzen 3900X which would be significantly faster. - Quadro cards also pretty low in bang for buck: a 2080 Ti will be close to the RTX 6000 for ~5x less and the 2080 or 2070 Super a bit slower for at least another 1.5x less. Single run at a time or possibly multiple? The proposed (or any 8+ core) workstation CPU is fast enough in the majority of the simulations to pair well with two of those GPUs if used for two concurrent simulations. If that's a relevant use-case, I'd recommend two 2070 Super or 2080 cards. Cheers, -- Szilárd Thank you, Alex -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Xeon Gold + RTX 5000
Hi Alex, On Mon, Jul 15, 2019 at 8:53 PM Alex wrote: > > Hi all and especially Szilard! > > My glorious management asked me to post this here. One of our group > members, an ex-NAMD guy, wants to use Gromacs for biophysics and the > following basics have been spec'ed for him: > > CPU: Xeon Gold 6244 > GPU: RTX 5000 or 6000 > > I'll be surprised if he runs systems with more than 50K particles. Could > you please comment on whether this is a cost-efficient and reasonably > powerful setup? Your past suggestions have been invaluable for us. That will be reasonably fast, but cost efficiency will be awful, to be honest: - that CPU is a ~$3000 part and won't perform much better than a $4-500 desktop CPU like an i9 9900, let alone a Ryzen 3900X which would be significantly faster. - Quadro cards also pretty low in bang for buck: a 2080 Ti will be close to the RTX 6000 for ~5x less and the 2080 or 2070 Super a bit slower for at least another 1.5x less. Single run at a time or possibly multiple? The proposed (or any 8+ core) workstation CPU is fast enough in the majority of the simulations to pair well with two of those GPUs if used for two concurrent simulations. If that's a relevant use-case, I'd recommend two 2070 Super or 2080 cards. Cheers, -- Szilárd > Thank you, > > Alex > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a > mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
[gmx-users] Xeon Gold + RTX 5000
Hi all and especially Szilard! My glorious management asked me to post this here. One of our group members, an ex-NAMD guy, wants to use Gromacs for biophysics and the following basics have been spec'ed for him: CPU: Xeon Gold 6244 GPU: RTX 5000 or 6000 I'll be surprised if he runs systems with more than 50K particles. Could you please comment on whether this is a cost-efficient and reasonably powerful setup? Your past suggestions have been invaluable for us. Thank you, Alex -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
