Re: [easybuild] building TensorFlow doesn't respect job-cores

[Kenneth Hoste]

Dear Yann,

On 30/04/2018 13:31, Yann Sagon wrote:


Dear Kenneth,


2018-04-30 11:32 GMT+02:00 Kenneth Hoste >:


Dear Yann,

On 30/04/2018 11:28, Yann Sagon wrote:
> I just noticed that building TF17 with cuda compute 6.0 and 6.1 doesn't 
> respect the job-cores = 12 that I have in my config file. According to 
> htop, it's using ~28 cores


Did you submit the TF build as a job using "eb --job"?
The --job-cores configuration setting only applies to build jobs
submitted using --job.


duhhh! Sorry about that, I'm dumb. Indeed I should use the parallel flag 
in my case.


But to be honest, I  don't see why there are two different flag for 
almost the same purpose?


They're not exactly the same purpose, but the difference is subtle.

Maybe a good example that combines both helps:

eb --job --job=cores=J --parallel=P ...

This will make EasyBuild submit a job using J cores, to build something 
using (exactly) P cores.


With P

Re: [easybuild] building TensorFlow doesn't respect job-cores

[Yann Sagon]

Dear Kenneth,


2018-04-30 11:32 GMT+02:00 Kenneth Hoste :

> Dear Yann,
>
> On 30/04/2018 11:28, Yann Sagon wrote:
> > I just noticed that building TF17 with cuda compute 6.0 and 6.1 doesn't
> > respect the job-cores = 12 that I have in my config file. According to
> > htop, it's using ~28 cores
>
> Did you submit the TF build as a job using "eb --job"?
> The --job-cores configuration setting only applies to build jobs
> submitted using --job.
>

duhhh! Sorry about that, I'm dumb. Indeed I should use the parallel flag in
my case.

But to be honest, I  don't see why there are two different flag for almost
the same purpose?

Re: [easybuild] building TensorFlow doesn't respect job-cores

[Kenneth Hoste]

Dear Yann,

On 30/04/2018 11:28, Yann Sagon wrote:
I just noticed that building TF17 with cuda compute 6.0 and 6.1 doesn't 
respect the job-cores = 12 that I have in my config file. According to 
htop, it's using ~28 cores


Did you submit the TF build as a job using "eb --job"?
The --job-cores configuration setting only applies to build jobs 
submitted using --job.


If you want to control the number of cores that EasyBuild uses for 
building, you should use "eb --parallel=12" (or set 'parallel' in your 
EasyBuild configuration file).


Also, do make sure you verify your currently active configuration using 
"eb --show-config".



regards,

Kenneth




Best


--
Logo UNIGE  Yann Sagon
Chef d'équipe HPC

Division du système et des technologies de l'information et de la 
communication

Université de Genève | 24 rue Général-Dufour
Tél 022 379 77 37 | Bureau 151

www.unige.ch/stic