Re: [gmx-users] Worse GROMACS performance with better specs?
Dear Carsten, Look's like we're seeing the same thing here, but only when using gcc 4.5.3: Original performance (gcc 5.3.1, AVX512, no hwloc support): 49 ns/day With hwloc support: gcc 4.5.3, AVX2_256 = 67 ns/day gcc 4.5.3, AVX512 = can't compile gcc 5.3.1, AVX2_256 = 36 ns/day gcc 5.3.1, AVX512 = 60 ns/day At the very least it's now comparable to my previous workstation. Will have to try with the gromacs 2018 next. Cheers for the info. Jason -- Message: 2 Date: Wed, 10 Jan 2018 08:39:22 + From: "Kutzner, Carsten" <ckut...@gwdg.de> To: "<gmx-us...@gromacs.org> GROMACS users" <gmx-us...@gromacs.org> Subject: Re: [gmx-users] Worse GROMACS performance with better specs? Message-ID: <816dea54-1e01-42ac-a78a-3680fdef0...@gwdg.de> Content-Type: text/plain; charset="us-ascii" Dear Jason, 1.) we have observed a similar behavior comparing Intel Silver 4114 against E5-2630v4 processors in a server with one GTX 1080Ti. Both CPUs have 10 cores and run at 2.2 GHz. Using our standard benchmark systems (see https://arxiv.org/abs/1507.00898) we were able to get 74.4 ns/day on the E5 but only 65.7 ns/day on the Silver machine for the MEM (80k atoms), and 4.4 vs. 4.2 ns/day for the RIB (2M atoms) system. Although the 4114 supports AVX512 SIMD instructions, compiling with AVX2_256 yielded higher performance (this is already included in the above numbers, so all benchmarks were run with the exact same mdrun executable). 2.) We recently bought a bunch of workstations with 2x Gold 6146 CPUs and 2x GTX1080Ti GPUs and were very surprised to find that initial GROMACS 2016 performances differ a lot between the identical machines (some were up to 40% slower). This was the reported difference in the md.log files: > - mdrun logs show Hardware Topology: Basic versus Hardware Topology: Only > logical processor count For some reason, the logical processor count was reported in a mixed-up way for the slower machines so that mdrun is unable to correctly determine the hardware topology. We solved this CPU problem by installing GROMACS with hwloc support. Then performances were equal over all 6146 nodes. Best, Carsten > On 10. Jan 2018, at 07:45, Jason Loo Siau Ee <jasonsiauee@taylors.edu.my> > wrote: > > Dear gmx users, > > > > I recently purchased a second GPU workstation and tried compiling GROMACS on > it, but despite the better (and more expensive) specs the performance is > significantly worse on my test system. To test things I standardized the > version (2016.4). Some details below: > > > > Workstation 1: > > 2 x Intel Xeon E5-2680v4 (14 cores), 2 x GTX 1080 > > gmx mdrun -ntmpi 8 -ntomp 7 -gpu_id > > Performance: 61ns/day > > > > Workstation 2: > > 2 x Intel Xeon Gold 6126 (12 cores), 2 x GTX 1080Ti > > gmx mdrun -ntmpi 8 -ntomp 6 -gpu_id > > Performance: 49ns/day > > > > I'm guessing it's an issue during compilation but I can't figure it out. I > wouldn't claim to have any knowledge about how GROMACS interacts with the > hardware, so some observations as below (not sure idea which is actually > relevant): > > > > - Compilation command for both: cmake .. -DGMX_BUILD_OWN_FFTW=ON > -DREGRESSIONTEST_DOWNLOAD=ON -DGMX_GPU=ON > -DCMAKE_INSTALL_PREFIX=/opt/gromacs > > > > - When compiling on Workstation 2 I originally got a CMake error "Cannot find > AVX512F compiler flag". Updated my gcc version to 5.3.1 to solve this. > > > > - Some regression tests fail for Workstation 2 during compilation: 4 > -FTUnitTest (SEGFAULT), 16 - CorrelationTest > > > > - mdrun logs show Hardware Topology: Basic versus Hardware Topology: Only > logical processor count > > > > - Running CPU-only (export CUDA_VISIBLE_DEVICES="") I get 21ns/day versus 23 > ns/day , so the CPUs in Workstation 2 are definitely faster. > > > > - Upgraded both to 2018.rc1 (used cmake3) I get a regression test fail for > Workstation 1 (9 - GPUUtilUnitTest) and Workstation 2 (8 - FFTUnitTests, 9 - > GPUUtilsUnitTest, 26 - CorrelationTest). Performance is 66.5ns/day versus > 51.95ns/day. The GPU load actually looks similar to previous versions (~70% > for Workstation 1 and 50-60% for Workstation 2). I actually got the best > performance with Workstation 1 running 2016.1 (69ns/day). > > > > Any help on how I can optimize performance on Workstation 2 would be > appreciated. If there are certain files that would be helpful let me know and > I'll send a link. > > > > Cheers, > Jason > > > > > > > > Jason Loo > > PhD, MPharm, RPh > > Lecturer > > School of Phar
[gmx-users] Worse GROMACS performance with better specs?
Dear gmx users, I recently purchased a second GPU workstation and tried compiling GROMACS on it, but despite the better (and more expensive) specs the performance is significantly worse on my test system. To test things I standardized the version (2016.4). Some details below: Workstation 1: 2 x Intel Xeon E5-2680v4 (14 cores), 2 x GTX 1080 gmx mdrun -ntmpi 8 -ntomp 7 -gpu_id Performance: 61ns/day Workstation 2: 2 x Intel Xeon Gold 6126 (12 cores), 2 x GTX 1080Ti gmx mdrun -ntmpi 8 -ntomp 6 -gpu_id Performance: 49ns/day I'm guessing it's an issue during compilation but I can't figure it out. I wouldn't claim to have any knowledge about how GROMACS interacts with the hardware, so some observations as below (not sure idea which is actually relevant): - Compilation command for both: cmake .. -DGMX_BUILD_OWN_FFTW=ON -DREGRESSIONTEST_DOWNLOAD=ON -DGMX_GPU=ON -DCMAKE_INSTALL_PREFIX=/opt/gromacs - When compiling on Workstation 2 I originally got a CMake error "Cannot find AVX512F compiler flag". Updated my gcc version to 5.3.1 to solve this. - Some regression tests fail for Workstation 2 during compilation: 4 -FTUnitTest (SEGFAULT), 16 - CorrelationTest - mdrun logs show Hardware Topology: Basic versus Hardware Topology: Only logical processor count - Running CPU-only (export CUDA_VISIBLE_DEVICES="") I get 21ns/day versus 23 ns/day , so the CPUs in Workstation 2 are definitely faster. - Upgraded both to 2018.rc1 (used cmake3) I get a regression test fail for Workstation 1 (9 - GPUUtilUnitTest) and Workstation 2 (8 - FFTUnitTests, 9 - GPUUtilsUnitTest, 26 - CorrelationTest). Performance is 66.5ns/day versus 51.95ns/day. The GPU load actually looks similar to previous versions (~70% for Workstation 1 and 50-60% for Workstation 2). I actually got the best performance with Workstation 1 running 2016.1 (69ns/day). Any help on how I can optimize performance on Workstation 2 would be appreciated. If there are certain files that would be helpful let me know and I'll send a link. Cheers, Jason Jason Loo PhD, MPharm, RPh Lecturer School of Pharmacy Faculty of Health and Medical Sciences Taylor's University This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and (i) destroy this message if a facsimile or (ii) delete this message immediately if this is an electronic communication. Thank you. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
[gmx-users] NA or SOD in CHARMM36
Hi all, I'm trying to simulate a protein-ligand complex in POPC membrane using the CHARMM36 forcefield, which I got from the Mackerell site and included in /share/gromacs/top/. Now when I run grompp I get the error "Fatal error: No such moleculetype SOD" despite the fact that I have #include "charmm36.ff/ions.itp" in my topology. A quick check on the ions.itp in charmm36.ff/ shows the following: [ moleculetype ] ; molname nrexcl NA 1 [ atoms ] ; idat type res nr residu name at name cg nr charge 1 SOD 1 NA NA 1 1 Now as I understand sodium is called SOD in CHARMM, so shouldn't all the NA be SOD instead? As this is from the Mackerell site I'm going to assume there isn't an error here, so am I about the do something silly? Is there a reason for this discrepancy (like because genion only adds NA and not SOD)? I've noticed similar issues with other ions and the water models (SOL instead of TI3P). I'm guessing I should create a local copy of the modified itp files then? Regards, Jason Loo Confidentiality Disclaimer: This e-mail and any attachments are confidential and intended solely for the intended addressee and may also be privileged or exempt from disclosure under applicable law. If you are not the intended addressee, or have received this e-mail in error, please notify the sender immediately, delete it from your system and do not copy, disclose, distribute or otherwise act in reliance upon any part of this e-mail or its attachments. Taylor's Education Group does not accept responsibility for any loss arising from unauthorized access to, or interference with, any internet communications by any third party in reliance to this email, or from the transmission of any viruses. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Taylor's Education Group. Replies to this e-mail may be monitored by Taylor's Education Group for operational or business reasons. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Expected Performance of a GPU workstation
Dear Carsten, That pretty much covers all my questions and then some. Thanks for that! Cheers, Jason -- Hi Jason, you might want to take a look at this study: http://arxiv.org/abs/1507.00898 Best, Carsten On 15 Jul 2015, at 06:44, Jason Loo Siau Ee jasonsiauee@taylors.edu.my wrote: Dear Gromacs users, I'm thinking about purchasing a GPU workstation for some simulation work, and was wondering about the optimal CPU-GPU combination for running MD as I'm pretty clueless about the hardware side of things. Typical systems simulated would be membrane protein systems of around 100k atoms, with PME electrostatics. Assuming I have a budget of approximately 7000USD for the machine, what sort of CPU-GPU combination would you recommend? If I went with something like 2x Intel(r) Xeon(r) Processor E5-2695 v2, 12 cores + 2x GTX980s, how many ns/day can I reasonably expect? Has anyone had experience with similar systems? Thanks in advance for the info. Cheers, Jason Confidentiality Disclaimer: This e-mail and any attachments are confidential and intended solely for the intended addressee and may also be privileged or exempt from disclosure under applicable law. If you are not the intended addressee, or have received this e-mail in error, please notify the sender immediately, delete it from your system and do not copy, disclose, distribute or otherwise act in reliance upon any part of this e-mail or its attachments. Taylor's Education Group does not accept responsibility for any loss arising from unauthorized access to, or interference with, any internet communications by any third party in reliance to this email, or from the transmission of any viruses. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Taylor's Education Group. Replies to this e-mail may be monitored by Taylor's Education Group for operational or business reasons. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
[gmx-users] Expected Performance of a GPU workstation
Dear Gromacs users, I'm thinking about purchasing a GPU workstation for some simulation work, and was wondering about the optimal CPU-GPU combination for running MD as I'm pretty clueless about the hardware side of things. Typical systems simulated would be membrane protein systems of around 100k atoms, with PME electrostatics. Assuming I have a budget of approximately 7000USD for the machine, what sort of CPU-GPU combination would you recommend? If I went with something like 2x Intel(r) Xeon(r) Processor E5-2695 v2, 12 cores + 2x GTX980s, how many ns/day can I reasonably expect? Has anyone had experience with similar systems? Thanks in advance for the info. Cheers, Jason Confidentiality Disclaimer: This e-mail and any attachments are confidential and intended solely for the intended addressee and may also be privileged or exempt from disclosure under applicable law. If you are not the intended addressee, or have received this e-mail in error, please notify the sender immediately, delete it from your system and do not copy, disclose, distribute or otherwise act in reliance upon any part of this e-mail or its attachments. Taylor's Education Group does not accept responsibility for any loss arising from unauthorized access to, or interference with, any internet communications by any third party in reliance to this email, or from the transmission of any viruses. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Taylor's Education Group. Replies to this e-mail may be monitored by Taylor's Education Group for operational or business reasons. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
