Re: [OMPI users] Excessive Use of CPU System Resources with OpenMPI 1.2.4 using TCP only ..

2008-01-23 Thread Terry Frankcombe 
On Tue, 2008-01-22 at 20:19 +0100, Pignot Geoffroy wrote:
> You could try the following MCA setting in your mpirun command
> --mca mpi_yield_when_idle 1

Yes, but to repeat what was said above, it is first essential that you
read:

and the related

Re: [OMPI users] Excessive Use of CPU System Resources with OpenMPI 1.2.4 using TCP only ..

2008-01-22 Thread Pignot Geoffroy 
You could try the following MCA setting in your mpirun command
--mca mpi_yield_when_idle 1

Re: [OMPI users] Excessive Use of CPU System Resources with OpenMPI 1.2.4 using TCP only ..

2008-01-22 Thread Brock Palen 

See this FAQ:

http://icl.cs.utk.edu/open-mpi/faq/?category=running#oversubscribing

OMPI by default assumes that each process has its own cpu, thus  
aggressively looks for data to receive.  I think there is a way to  
yield the processor if you want, but I think what your seeing is  
behavior you want.
If your application is spending allot of time waiting to receive data  
it might be unbalanced, check out opt (http://www.allinea.com/ 
index.php?page=74) works with OMPI, lets you see what your process  
are doing,  I think Tau (http://www.cs.uoregon.edu/research/tau/users/ 
samrai.php) would be a free alternative but i do not know if it works  
with OMPI.


Brock Palen
Center for Advanced Computing
bro...@umich.edu
(734)936-1985


On Jan 22, 2008, at 12:54 AM, Terry Frankcombe wrote:


Well, I have noticed that when a process is waiting for communication
from another process the reported CPU usage remains around 100%.  Is
that what you mean?  I haven't explored whether these processes  
give way

to other active processes under the linux scheduler, nor whether I
should expect anything different.  This has been with 1.2.3 and 1.2.4.



On Tue, 2008-01-22 at 16:48 +1100, Graham Jenkins wrote:

We've observed an excessive use of CPU system resources with OpenMPI
1.2.4 using TCP connections only on our SL5 x86_64 Cluster.  
Typically,

for a simple Canonical Ring Program, we're seeing between 30 and 70%
system usage.

Has anybody else noticed this sort of behaviour?
And does anybody have some suggestions for resolving the issue?

Present values we have are:
--
ompi_info --param btl tcp |grep MCA
 MCA btl: parameter "btl_base_debug" (current  
value: "0")

 MCA btl: parameter "btl" (current value: )
 MCA btl: parameter "btl_base_verbose" (current  
value: "0")
 MCA btl: parameter "btl_tcp_if_include" (current  
value:

"eth0")
 MCA btl: parameter "btl_tcp_if_exclude" (current  
value:

"lo")
 MCA btl: parameter "btl_tcp_free_list_num" (current
value: "8")
 MCA btl: parameter "btl_tcp_free_list_max" (current
value: "-1")
 MCA btl: parameter "btl_tcp_free_list_inc" (current
value: "32")
 MCA btl: parameter "btl_tcp_sndbuf" (current value:
"131072")
 MCA btl: parameter "btl_tcp_rcvbuf" (current value:
"131072")
 MCA btl: parameter "btl_tcp_endpoint_cache" (current
value: "30720")
 MCA btl: parameter "btl_tcp_exclusivity" (current
value: "0")
 MCA btl: parameter "btl_tcp_eager_limit" (current
value: "65536")
 MCA btl: parameter "btl_tcp_min_send_size" (current
value: "65536")
 MCA btl: parameter "btl_tcp_max_send_size" (current
value: "131072")
 MCA btl: parameter "btl_tcp_min_rdma_size" (current
value: "131072")
 MCA btl: parameter "btl_tcp_max_rdma_size" (current
value: "2147483647")
 MCA btl: parameter "btl_tcp_flags" (current  
value: "122")
 MCA btl: parameter "btl_tcp_priority" (current  
value: "0")

 MCA btl: parameter "btl_base_warn_component_unused"
(current value: "1")



___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Excessive Use of CPU System Resources with OpenMPI 1.2.4 using TCP only ..

2008-01-22 Thread Terry Frankcombe 
Well, I have noticed that when a process is waiting for communication
from another process the reported CPU usage remains around 100%.  Is
that what you mean?  I haven't explored whether these processes give way
to other active processes under the linux scheduler, nor whether I
should expect anything different.  This has been with 1.2.3 and 1.2.4.



On Tue, 2008-01-22 at 16:48 +1100, Graham Jenkins wrote:
> We've observed an excessive use of CPU system resources with OpenMPI
> 1.2.4 using TCP connections only on our SL5 x86_64 Cluster. Typically,
> for a simple Canonical Ring Program, we're seeing between 30 and 70%
> system usage.
> 
> Has anybody else noticed this sort of behaviour?
> And does anybody have some suggestions for resolving the issue?
> 
> Present values we have are:
> --
> ompi_info --param btl tcp |grep MCA
>  MCA btl: parameter "btl_base_debug" (current value: "0")
>  MCA btl: parameter "btl" (current value: )
>  MCA btl: parameter "btl_base_verbose" (current value: "0")
>  MCA btl: parameter "btl_tcp_if_include" (current value:
> "eth0")
>  MCA btl: parameter "btl_tcp_if_exclude" (current value:
> "lo")
>  MCA btl: parameter "btl_tcp_free_list_num" (current
> value: "8")
>  MCA btl: parameter "btl_tcp_free_list_max" (current
> value: "-1")
>  MCA btl: parameter "btl_tcp_free_list_inc" (current
> value: "32")
>  MCA btl: parameter "btl_tcp_sndbuf" (current value:
> "131072")
>  MCA btl: parameter "btl_tcp_rcvbuf" (current value:
> "131072")
>  MCA btl: parameter "btl_tcp_endpoint_cache" (current
> value: "30720")
>  MCA btl: parameter "btl_tcp_exclusivity" (current
> value: "0")
>  MCA btl: parameter "btl_tcp_eager_limit" (current
> value: "65536")
>  MCA btl: parameter "btl_tcp_min_send_size" (current
> value: "65536")
>  MCA btl: parameter "btl_tcp_max_send_size" (current
> value: "131072")
>  MCA btl: parameter "btl_tcp_min_rdma_size" (current
> value: "131072")
>  MCA btl: parameter "btl_tcp_max_rdma_size" (current
> value: "2147483647")
>  MCA btl: parameter "btl_tcp_flags" (current value: "122")
>  MCA btl: parameter "btl_tcp_priority" (current value: "0")
>  MCA btl: parameter "btl_base_warn_component_unused"
> (current value: "1")
>

[OMPI users] Excessive Use of CPU System Resources with OpenMPI 1.2.4 using TCP only ..

2008-01-22 Thread Graham Jenkins 
We've observed an excessive use of CPU system resources with OpenMPI
1.2.4 using TCP connections only on our SL5 x86_64 Cluster. Typically,
for a simple Canonical Ring Program, we're seeing between 30 and 70%
system usage.

Has anybody else noticed this sort of behaviour?
And does anybody have some suggestions for resolving the issue?

Present values we have are:
--
ompi_info --param btl tcp |grep MCA
 MCA btl: parameter "btl_base_debug" (current value: "0")
 MCA btl: parameter "btl" (current value: )
 MCA btl: parameter "btl_base_verbose" (current value: "0")
 MCA btl: parameter "btl_tcp_if_include" (current value:
"eth0")
 MCA btl: parameter "btl_tcp_if_exclude" (current value:
"lo")
 MCA btl: parameter "btl_tcp_free_list_num" (current
value: "8")
 MCA btl: parameter "btl_tcp_free_list_max" (current
value: "-1")
 MCA btl: parameter "btl_tcp_free_list_inc" (current
value: "32")
 MCA btl: parameter "btl_tcp_sndbuf" (current value:
"131072")
 MCA btl: parameter "btl_tcp_rcvbuf" (current value:
"131072")
 MCA btl: parameter "btl_tcp_endpoint_cache" (current
value: "30720")
 MCA btl: parameter "btl_tcp_exclusivity" (current
value: "0")
 MCA btl: parameter "btl_tcp_eager_limit" (current
value: "65536")
 MCA btl: parameter "btl_tcp_min_send_size" (current
value: "65536")
 MCA btl: parameter "btl_tcp_max_send_size" (current
value: "131072")
 MCA btl: parameter "btl_tcp_min_rdma_size" (current
value: "131072")
 MCA btl: parameter "btl_tcp_max_rdma_size" (current
value: "2147483647")
 MCA btl: parameter "btl_tcp_flags" (current value: "122")
 MCA btl: parameter "btl_tcp_priority" (current value: "0")
 MCA btl: parameter "btl_base_warn_component_unused"
(current value: "1")

-- 
Graham Jenkins
Senior Software Specialist, eResearch
Monash University

Email: graham.jenk...@its.monash.edu.au
Tel:   +613 9905-5942 (office)   +614 4850-2491 (mobile)