Re: [gmx-users] Fatal Error when launching gromacs 2019.2 on GPU.
Hi, Indeed, the standard way provided by CUDA to expose a subset of GPUs to an application is the CUDA_VISIBLE_DEVICES (note the "S" ending); I did not realize that is something you were interested in, I thought you wanted to avoid using GPUs. Also note (for anyone interested), a when using a queue system, if node sharing is allowed, the scheduler should be set up to a correct CUDA_VISIBLE_DEVICES variable as well. Cheers, -- Szilárd On Mon, Oct 28, 2019 at 1:43 PM Artem Shekhovtsov wrote: > Hi, > > Thanks, setting this variable allowed me to start GROMACS without errors > using the CPU. > The problem is that this method prevents me from using other free GPUs on > the host, but I would like to do this. > I also found out that setting the CUDA_VISIBLE_DEVICE variable to available > GPUs at the time of launch allows us to solve this problem. > > Artem > > > On Sat, Oct 26, 2019 at 1:50 AM Szilárd Páll > wrote: > > > Hi, > > > > This is an issue in one of pre-detection checks that trips due to > > encountering exclusive / prohibited mode devices. > > > > You can work around this by entirely disabling the detection using the > > GMX_DISABLE_GPU_DETECTION environment variable. > > > > Cheers, > > -- > > Szilárd > > > > > > On Thu, Oct 17, 2019 at 5:01 PM Artem Shekhovtsov < > > job.shekhovt...@gmail.com> > > wrote: > > > > > Hello! > > > Problem: The launch of mdrun that does not require video cards exit > with > > > fatal error if at least one video card is busy on the host at that > time. > > > gmx mdrun -deffnm test -ntmpi 1 -ntomp 1 -nb cpu -bonded cpu > > > --- > > > Program: gmx mdrun, version 2019.2 > > > Source file: src/gromacs/gpu_utils/gpu_utils.cu (line 100) > > > > > > Fatal error: > > > cudaFuncGetAttributes failed: all CUDA-capable devices are busy or > > > unavailable > > > > > > For more information and tips for troubleshooting, please check the > > GROMACS > > > website at http://www.gromacs.org/Documentation/Errors > > > --- > > > > > > I have this error in gromacs version 2019.2, 2019.3, 2020.beta. > > > Version - 2018.6 is not affected. > > > All version builds with the same flags. > > > > > > Archive with log files and gromacs build files > > > > > > > > > https://drive.google.com/file/d/1ahn7S69CU5yvAPlLWHryXmMzcGfdWVxP/view?usp=sharing > > > > > > > > > I would appreciate any help. > > > > > > Thanks, > > > Artem Shekhovtsov. > > > -- > > > Gromacs Users mailing list > > > > > > * Please search the archive at > > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > > posting! > > > > > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > > > > > * For (un)subscribe requests visit > > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > > send a mail to gmx-users-requ...@gromacs.org. > > > > > -- > > Gromacs Users mailing list > > > > * Please search the archive at > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > posting! > > > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > > > * For (un)subscribe requests visit > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > send a mail to gmx-users-requ...@gromacs.org. > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > send a mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Fatal Error when launching gromacs 2019.2 on GPU.
Hi, Thanks, setting this variable allowed me to start GROMACS without errors using the CPU. The problem is that this method prevents me from using other free GPUs on the host, but I would like to do this. I also found out that setting the CUDA_VISIBLE_DEVICE variable to available GPUs at the time of launch allows us to solve this problem. Artem On Sat, Oct 26, 2019 at 1:50 AM Szilárd Páll wrote: > Hi, > > This is an issue in one of pre-detection checks that trips due to > encountering exclusive / prohibited mode devices. > > You can work around this by entirely disabling the detection using the > GMX_DISABLE_GPU_DETECTION environment variable. > > Cheers, > -- > Szilárd > > > On Thu, Oct 17, 2019 at 5:01 PM Artem Shekhovtsov < > job.shekhovt...@gmail.com> > wrote: > > > Hello! > > Problem: The launch of mdrun that does not require video cards exit with > > fatal error if at least one video card is busy on the host at that time. > > gmx mdrun -deffnm test -ntmpi 1 -ntomp 1 -nb cpu -bonded cpu > > --- > > Program: gmx mdrun, version 2019.2 > > Source file: src/gromacs/gpu_utils/gpu_utils.cu (line 100) > > > > Fatal error: > > cudaFuncGetAttributes failed: all CUDA-capable devices are busy or > > unavailable > > > > For more information and tips for troubleshooting, please check the > GROMACS > > website at http://www.gromacs.org/Documentation/Errors > > --- > > > > I have this error in gromacs version 2019.2, 2019.3, 2020.beta. > > Version - 2018.6 is not affected. > > All version builds with the same flags. > > > > Archive with log files and gromacs build files > > > > > https://drive.google.com/file/d/1ahn7S69CU5yvAPlLWHryXmMzcGfdWVxP/view?usp=sharing > > > > > > I would appreciate any help. > > > > Thanks, > > Artem Shekhovtsov. > > -- > > Gromacs Users mailing list > > > > * Please search the archive at > > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > > posting! > > > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > > > * For (un)subscribe requests visit > > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > > send a mail to gmx-users-requ...@gromacs.org. > > > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > send a mail to gmx-users-requ...@gromacs.org. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Fatal Error when launching gromacs 2019.2 on GPU.
Hi, This is an issue in one of pre-detection checks that trips due to encountering exclusive / prohibited mode devices. You can work around this by entirely disabling the detection using the GMX_DISABLE_GPU_DETECTION environment variable. Cheers, -- Szilárd On Thu, Oct 17, 2019 at 5:01 PM Artem Shekhovtsov wrote: > Hello! > Problem: The launch of mdrun that does not require video cards exit with > fatal error if at least one video card is busy on the host at that time. > gmx mdrun -deffnm test -ntmpi 1 -ntomp 1 -nb cpu -bonded cpu > --- > Program: gmx mdrun, version 2019.2 > Source file: src/gromacs/gpu_utils/gpu_utils.cu (line 100) > > Fatal error: > cudaFuncGetAttributes failed: all CUDA-capable devices are busy or > unavailable > > For more information and tips for troubleshooting, please check the GROMACS > website at http://www.gromacs.org/Documentation/Errors > --- > > I have this error in gromacs version 2019.2, 2019.3, 2020.beta. > Version - 2018.6 is not affected. > All version builds with the same flags. > > Archive with log files and gromacs build files > > https://drive.google.com/file/d/1ahn7S69CU5yvAPlLWHryXmMzcGfdWVxP/view?usp=sharing > > > I would appreciate any help. > > Thanks, > Artem Shekhovtsov. > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > send a mail to gmx-users-requ...@gromacs.org. > -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
Re: [gmx-users] Fatal Error when launching gromacs 2019.2 on GPU.
Hi, Can you please file an issue on redmine.gromacs.org with the description you gave here? Thanks, -- Szilárd On Thu, Oct 17, 2019 at 5:01 PM Artem Shekhovtsov wrote: > Hello! > Problem: The launch of mdrun that does not require video cards exit with > fatal error if at least one video card is busy on the host at that time. > gmx mdrun -deffnm test -ntmpi 1 -ntomp 1 -nb cpu -bonded cpu > --- > Program: gmx mdrun, version 2019.2 > Source file: src/gromacs/gpu_utils/gpu_utils.cu (line 100) > > Fatal error: > cudaFuncGetAttributes failed: all CUDA-capable devices are busy or > unavailable > > For more information and tips for troubleshooting, please check the GROMACS > website at http://www.gromacs.org/Documentation/Errors > --- > > I have this error in gromacs version 2019.2, 2019.3, 2020.beta. > Version - 2018.6 is not affected. > All version builds with the same flags. > > Archive with log files and gromacs build files > > https://drive.google.com/file/d/1ahn7S69CU5yvAPlLWHryXmMzcGfdWVxP/view?usp=sharing > > > I would appreciate any help. > > Thanks, > Artem Shekhovtsov. > -- > Gromacs Users mailing list > > * Please search the archive at > http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before > posting! > > * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists > > * For (un)subscribe requests visit > https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or > send a mail to gmx-users-requ...@gromacs.org. > -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
[gmx-users] Fatal Error when launching gromacs 2019.2 on GPU.
Hello! Problem: The launch of mdrun that does not require video cards exit with fatal error if at least one video card is busy on the host at that time. gmx mdrun -deffnm test -ntmpi 1 -ntomp 1 -nb cpu -bonded cpu --- Program: gmx mdrun, version 2019.2 Source file: src/gromacs/gpu_utils/gpu_utils.cu (line 100) Fatal error: cudaFuncGetAttributes failed: all CUDA-capable devices are busy or unavailable For more information and tips for troubleshooting, please check the GROMACS website at http://www.gromacs.org/Documentation/Errors --- I have this error in gromacs version 2019.2, 2019.3, 2020.beta. Version - 2018.6 is not affected. All version builds with the same flags. Archive with log files and gromacs build files https://drive.google.com/file/d/1ahn7S69CU5yvAPlLWHryXmMzcGfdWVxP/view?usp=sharing I would appreciate any help. Thanks, Artem Shekhovtsov. -- Gromacs Users mailing list * Please search the archive at http://www.gromacs.org/Support/Mailing_Lists/GMX-Users_List before posting! * Can't post? Read http://www.gromacs.org/Support/Mailing_Lists * For (un)subscribe requests visit https://maillist.sys.kth.se/mailman/listinfo/gromacs.org_gmx-users or send a mail to gmx-users-requ...@gromacs.org.
