Thank you! I do not think OpenMPI with cuda support is particularly relevant for us. I will read the docs and try to understand what --minmal-toolchains does.
Thanks for your suggestions Jakob > On 26 Mar 2019, at 14:41, Jack Perdue <j-per...@tamu.edu> wrote: > > Howdy Jakob, > > The primary difference between fosscuda and > foss+CUDA is that fosscuda has an OpenMPI built > with CUDA support where as the latter does not. > > We run with: > > EASYBUILD_MINIMAL_TOOLCHAINS > > which cuts down on the number of things that > have be rebuilt here For example for > > TensorFlow/1.10.1-fosscuda-2018b-Python-3.6.6 > > we only had to rebuild these packages: > > Python/3.6.6-fosscuda-2018b > protobuf-python/3.6.0-fosscuda-2018b-Python-3.6.6 > cuDNN/7.1.4.18-fosscuda-2018b > > so you might want look at the --minmal-toolchains option. > > > Jack Perdue > Lead Systems Administrator > High Performance Research Computing > TAMU Division of Research > j-per...@tamu.edu http://hprc.tamu.edu > HPRC Helpdesk: h...@hprc.tamu.edu > > On 3/26/19 8:26 AM, Jakob Schiøtz wrote: >> Dear EasyBuilders, >> >> I would like to build a TensorFlow module supporting GPUs. Currently, that >> looks to be TensorFlow-1.12.0-fosscuda-2018b-Python-3.6.6.eb, but this >> requires building a new toolchain (fosscuda), including rebuilding both >> OpenMPI and Python with GPU support. In addition, any other software that >> the user may need alongside TensorFlow will also have to be rebuilt with the >> fosscuda toolchain to prevent mixing toolchains. That seems to be overkill >> to me - after all little if anything is gained by rebuilding Python and >> stuff with GPU support unless the scripts are actually going to use it. I >> don’t know if numpy will begin offloading computations to the GPU, but >> presumably moving data back and forth will be expensive, and TensorFlow will >> allocate all the GPU memory anyway. >> >> There used to be a tensorflow variant for the normal toolchain which just >> depended on CUDA explicitly. Is there a reason not to do it this way? I >> guess I could try to make a .eb file inspired on the existing ones doing >> just that - or have I overlooked something? >> >> Best regards >> >> Jakob >> >> >> -- >> Jakob Schiøtz, professor, Ph.D. >> Department of Physics >> Technical University of Denmark >> DK-2800 Kongens Lyngby, Denmark >> http://www.fysik.dtu.dk/~schiotz/ >> >> >> > -- Jakob Schiøtz, professor, Ph.D. Department of Physics Technical University of Denmark DK-2800 Kongens Lyngby, Denmark http://www.fysik.dtu.dk/~schiotz/
