Dear Arun,

thank you for the report. I agree with the analysis, detectCores() will 
only report logical processors in the NUMA group in which R is running. 
I don't have a system to test on, could you please check these 
workarounds for me on your systems?

# number of logical processors - what detectCores() should return

out <- system("wmic cpu get numberoflogicalprocessors", intern=TRUE)
sum(as.numeric(gsub("([0-9]+).*", "\\1", grep("[0-9]+[ \t]*", out, 
value=TRUE))))

# number of cores - what detectCores(FALSE) should return

out <- system("wmic cpu get numberofcores", intern=TRUE)
sum(as.numeric(gsub("([0-9]+).*", "\\1", grep("[0-9]+[ \t]*", out, 
value=TRUE))))

# number of physical processors - as a sanity check

system("wmic computersystem get numberofprocessors")

Thanks,
Tomas

On 08/17/2018 05:11 PM, Srinivasan, Arunkumar wrote:
> Dear R-devel list,
>
> R's detectCores() function internally calls "ncpus" function to get the total 
> number of logical processors. However, this doesnot seem to take NUMA into 
> account on Windows machines.
>
> On a machine having 48 processors (24 cores) in total and windows server 2012 
> installed, if NUMA is enabled and has 2 nodes (node 0 and node 1 each having 
> 24 CPUs), then R's detectCores() only detects 24 instead of the total 48. If 
> NUMA is disabled, detectCores() returns 48.
>
> Similarly, on a machine with 88 cores (176 processors) and windows server 
> 2012, detectCores() with NUMA disabled only returns the maximum value of 64. 
> If NUMA is enabled with 4 nodes (44 processors each), then detectCores() will 
> only return 44. This is particularly limiting since we cannot get to use all 
> processors by enabling/disabling NUMA in this case.
>
> We think this is because R's ncpus.c file uses 
> "PSYSTEM_LOGICAL_PROCESSOR_INFORMATION" 
> (https://msdn.microsoft.com/en-us/library/windows/desktop/ms683194(v=vs.85).aspx)
>  instead of "PSYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX" 
> (https://msdn.microsoft.com/en-us/library/windows/desktop/dd405488(v=vs.85).aspx).
>  Specifically, quoting from the first link:
>
> "On systems with more than 64 logical processors, the 
> GetLogicalProcessorInformation function retrieves logical processor 
> information about processors in the 
> https://msdn.microsoft.com/en-us/library/windows/desktop/dd405503(v=vs.85).aspx
>  to which the calling thread is currently assigned. Use the 
> https://msdn.microsoft.com/en-us/library/windows/desktop/dd405488(v=vs.85).aspx
>  function to retrieve information about processors in all processor groups on 
> the system."
>
> Therefore, it might be possible to get the right count of total processors 
> even with NUMA enabled by using "GetLogicalProcessorInformationEX".  It'd be 
> nice to know what you think.
>
> Thank you very much,
> Arun.
>
> --
> Arun Srinivasan
> Analyst, Millennium Management LLC
> 50 Berkeley Street | London, W1J 8HD
>
>
> ######################################################################
>
> The information contained in this communication is con...{{dropped:11}}

______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Reply via email to