Re: [OMPI users] Accessing Process Affinity within MPI Program

[Jeff Squyres (jsquyres)]

Sorry for the delay in answering this; this mail came after I disappeared for 
the US holidays.

Yes -- through an Open MPI extension (you must configure Open MPI with 
--enable-mpi-ext=affinity or --enable-mpi-ext=all).  See:

http://www.open-mpi.org/doc/v1.8/man3/OMPI_Affinity_str.3.php



On Dec 21, 2014, at 1:57 AM, Saliya Ekanayake  wrote:

> Hi,
> 
> Is it possible to get information on the process affinity that's set in 
> mpirun command within the MPI program? For example I'd like to know the 
> number of cores that a given rank is bound to.
> 
> Thank you
> 
> -- 
> Saliya Ekanayake
> Ph.D. Candidate | Research Assistant
> School of Informatics and Computing | Digital Science Center
> Indiana University, Bloomington
> Cell 812-391-4914
> http://saliya.org
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2014/12/26050.php


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI users] Icreasing OFED registerable memory

[Gus Correa]

Hi Waleed

As Devendar said (and I tried to explain before),
you need to allow the locked memory limit to be unlimited for
user processes (in /etc/security/limits.conf),
*AND* somehow the daemon/job_script/whatever that launches the mpiexec
command must request "ulimit -l unlimited" (directly or indirectly).
The latter part depends on how your system's details.
I am not familiar to SGE (I use Torque), but presumably you can
add "ulimit -l unlimited" when you launch
the SGE daemons on the nodes.
Presumably this will make the processes launched by that daemon
(i.e. your mpiexec) inherit those limits,
and that is how I do it on Torque.
A more brute force way is just to include "ulimit -l unlimited"
in you job script before mpiexec.
Inserting a "ulimit -a" in your jobscript may help diagnose what you
actually have.
Please, see the OMPI FAQ that I sent you before for more details.

I hope this helps,
Gus Correa

On 01/06/2015 01:37 PM, Deva wrote:

Hi Waleed,

--
   Memlock limit: 65536
--

such a low limit should be due to per-user lock memory limit . Can you
make sure it is  set to "unlimited" on all nodes ( "ulimit -l unlimited")?

-Devendar

On Tue, Jan 6, 2015 at 3:42 AM, Waleed Lotfy > wrote:

Hi guys,

Sorry for getting back so late, but we ran into some problems during
the installation process and as soon as the system came up I tested
the new versions for the problem but it showed another memory
related warning.

--
The OpenFabrics (openib) BTL failed to initialize while trying to
allocate some locked memory.  This typically can indicate that the
memlock limits are set too low.  For most HPC installations, the
memlock limits should be set to "unlimited".  The failure occured
here:

   Local host:comp003.local
   OMPI source:   btl_openib_component.c:1200
   Function:  ompi_free_list_init_ex_new()
   Device:mlx4_0
   Memlock limit: 65536

You may need to consult with your system administrator to get this
problem fixed.  This FAQ entry on the Open MPI web site may also be
helpful:

http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
--
--
WARNING: There was an error initializing an OpenFabrics device.

   Local host:   comp003.local
   Local device: mlx4_0
--

<<>>

My current running versions:

OpenMPI: 1.6.4
OFED-internal-2.3-2

I checked /etc/security/limits.d/, the scheduler's configurations
(grid engine) and tried adding the following line to
/etc/modprobe.d/mlx4_core: 'options mlx4_core log_num_mtt=22
log_mtts_per_seg=1' as suggested by Gus.

I am running out of ideas here, so please any help is appreciated.

P.S. I am not sure if I should open a new thread with this issue or
continue with the current one, so please advice.

Waleed Lotfy
Bibliotheca Alexandrina
___
users mailing list
us...@open-mpi.org 
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2015/01/26107.php




--


-Devendar


___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2015/01/26109.php

Re: [OMPI users] Icreasing OFED registerable memory

[Deva]

Can you read this thread and see if following grid engine param setting is
missing?

http://marc.info/?l=npaci-rocks-discussion=135844781420954=2

---
Check that your GridEngine configuration has the following:

execd_params H_MEMORYLOCKED=infinity

The command qconf -sconf will display the current configuration.
--

-Devendar

On Tue, Jan 6, 2015 at 1:37 PM, Deva  wrote:

> Hi Waleed,
>
> --
>Memlock limit: 65536
> --
>
> such a low limit should be due to per-user lock memory limit . Can you
> make sure it is  set to "unlimited" on all nodes ( "ulimit -l unlimited")?
>
> -Devendar
>
> On Tue, Jan 6, 2015 at 3:42 AM, Waleed Lotfy 
> wrote:
>
>> Hi guys,
>>
>> Sorry for getting back so late, but we ran into some problems during the
>> installation process and as soon as the system came up I tested the new
>> versions for the problem but it showed another memory related warning.
>>
>> --
>> The OpenFabrics (openib) BTL failed to initialize while trying to
>> allocate some locked memory.  This typically can indicate that the
>> memlock limits are set too low.  For most HPC installations, the
>> memlock limits should be set to "unlimited".  The failure occured
>> here:
>>
>>   Local host:comp003.local
>>   OMPI source:   btl_openib_component.c:1200
>>   Function:  ompi_free_list_init_ex_new()
>>   Device:mlx4_0
>>   Memlock limit: 65536
>>
>> You may need to consult with your system administrator to get this
>> problem fixed.  This FAQ entry on the Open MPI web site may also be
>> helpful:
>>
>> http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
>> --
>> --
>> WARNING: There was an error initializing an OpenFabrics device.
>>
>>   Local host:   comp003.local
>>   Local device: mlx4_0
>> --
>>
>> <<>>
>>
>> My current running versions:
>>
>> OpenMPI: 1.6.4
>> OFED-internal-2.3-2
>>
>> I checked /etc/security/limits.d/, the scheduler's configurations (grid
>> engine) and tried adding the following line to /etc/modprobe.d/mlx4_core:
>> 'options mlx4_core log_num_mtt=22 log_mtts_per_seg=1' as suggested by Gus.
>>
>> I am running out of ideas here, so please any help is appreciated.
>>
>> P.S. I am not sure if I should open a new thread with this issue or
>> continue with the current one, so please advice.
>>
>> Waleed Lotfy
>> Bibliotheca Alexandrina
>> ___
>> users mailing list
>> us...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> Link to this post:
>> http://www.open-mpi.org/community/lists/users/2015/01/26107.php
>>
>
>
>
> --
>
>
> -Devendar
>



-- 


-Devendar

Re: [OMPI users] Icreasing OFED registerable memory

[Deva]

Hi Waleed,

--
   Memlock limit: 65536
--

such a low limit should be due to per-user lock memory limit . Can you make
sure it is  set to "unlimited" on all nodes ( "ulimit -l unlimited")?

-Devendar

On Tue, Jan 6, 2015 at 3:42 AM, Waleed Lotfy 
wrote:

> Hi guys,
>
> Sorry for getting back so late, but we ran into some problems during the
> installation process and as soon as the system came up I tested the new
> versions for the problem but it showed another memory related warning.
>
> --
> The OpenFabrics (openib) BTL failed to initialize while trying to
> allocate some locked memory.  This typically can indicate that the
> memlock limits are set too low.  For most HPC installations, the
> memlock limits should be set to "unlimited".  The failure occured
> here:
>
>   Local host:comp003.local
>   OMPI source:   btl_openib_component.c:1200
>   Function:  ompi_free_list_init_ex_new()
>   Device:mlx4_0
>   Memlock limit: 65536
>
> You may need to consult with your system administrator to get this
> problem fixed.  This FAQ entry on the Open MPI web site may also be
> helpful:
>
> http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
> --
> --
> WARNING: There was an error initializing an OpenFabrics device.
>
>   Local host:   comp003.local
>   Local device: mlx4_0
> --
>
> <<>>
>
> My current running versions:
>
> OpenMPI: 1.6.4
> OFED-internal-2.3-2
>
> I checked /etc/security/limits.d/, the scheduler's configurations (grid
> engine) and tried adding the following line to /etc/modprobe.d/mlx4_core:
> 'options mlx4_core log_num_mtt=22 log_mtts_per_seg=1' as suggested by Gus.
>
> I am running out of ideas here, so please any help is appreciated.
>
> P.S. I am not sure if I should open a new thread with this issue or
> continue with the current one, so please advice.
>
> Waleed Lotfy
> Bibliotheca Alexandrina
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2015/01/26107.php
>



-- 


-Devendar

Re: [hwloc-users] Hwloc on windows does not show pci devices

[Uffe Jakobsen]

Ah, that explains - thanks for clarifying

/Uffe

On 2015-01-06 15:26, Brice Goglin wrote:

Hello

We don't have PCI support on Windows unfortunately. And on non-Linux
platforms, you would have PCI devices without their locality, not really
useful.

The hwloc I/O doc says:
"Note that I/O discovery requires significant help from the operating
system. The pciaccess library (the development package is usually
|libpciaccess-devel| or |libpciaccess-dev|) is needed to fully detect
PCI devices and bridges, and the actual locality of these devices is
only currently detected on Linux. Also, some operating systems require
privileges for probing PCI devices, see Does hwloc require privileged
access?

for details."
(http://www.open-mpi.org/projects/hwloc/doc/v1.10.0/a00010.php)

There's a ticket about I/O locality on Windows
(https://github.com/open-mpi/hwloc/issues/108). IIRC we need a recent
Windows to make this work, and I don't have any multisocket Windows
machine to test the code anyway :/

Brice




Le 06/01/2015 15:19, Uffe Jakobsen a écrit :



Hello,

Question: the hwloc binaries (1.10.0) for Windows found on the website
does not output pci devices even when supplying "--whole-io" option.

hwloc on both linux and freebsd produces the expected output - with
pci devices.

I haven't been able to find any info about reduced functionality in
the Windows release of hwloc - it on purpose or an error ?

PS: I'm new to hwloc - so bear with me if I'm asking an utterly stupid
question here.

Thanks in advance.

/Uffe

Re: [hwloc-users] Hwloc on windows does not show pci devices

[Brice Goglin]

Hello

We don't have PCI support on Windows unfortunately. And on non-Linux
platforms, you would have PCI devices without their locality, not really
useful.

The hwloc I/O doc says:
"Note that I/O discovery requires significant help from the operating
system. The pciaccess library (the development package is usually
|libpciaccess-devel| or |libpciaccess-dev|) is needed to fully detect
PCI devices and bridges, and the actual locality of these devices is
only currently detected on Linux. Also, some operating systems require
privileges for probing PCI devices, see Does hwloc require privileged
access?

for details."
(http://www.open-mpi.org/projects/hwloc/doc/v1.10.0/a00010.php)

There's a ticket about I/O locality on Windows
(https://github.com/open-mpi/hwloc/issues/108). IIRC we need a recent
Windows to make this work, and I don't have any multisocket Windows
machine to test the code anyway :/

Brice




Le 06/01/2015 15:19, Uffe Jakobsen a écrit :
>
>
> Hello,
>
> Question: the hwloc binaries (1.10.0) for Windows found on the website
> does not output pci devices even when supplying "--whole-io" option.
>
> hwloc on both linux and freebsd produces the expected output - with
> pci devices.
>
> I haven't been able to find any info about reduced functionality in
> the Windows release of hwloc - it on purpose or an error ?
>
> PS: I'm new to hwloc - so bear with me if I'm asking an utterly stupid
> question here.
>
> Thanks in advance.
>
> /Uffe
>
>
> ___
> hwloc-users mailing list
> hwloc-us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
> Link to this post:
> http://www.open-mpi.org/community/lists/hwloc-users/2015/01/1143.php

[hwloc-users] Hwloc on windows does not show pci devices

[Uffe Jakobsen]

Hello,

Question: the hwloc binaries (1.10.0) for Windows found on the website 
does not output pci devices even when supplying "--whole-io" option.


hwloc on both linux and freebsd produces the expected output - with pci 
devices.


I haven't been able to find any info about reduced functionality in the 
Windows release of hwloc - it on purpose or an error ?


PS: I'm new to hwloc - so bear with me if I'm asking an utterly stupid 
question here.


Thanks in advance.

/Uffe

[OMPI users] [ICCS/Alchemy] 2nd CfP: Architecture, Languages, Compilation and Hardware support for Emerging ManYcore systems

[CUDENNEC Loic]

Please accept our apologies if you receive multiple copies of this CfP.



***
* ALCHEMY Workshop 2015
* Architecture, Languages, Compilation and Hardware support for Emerging 
ManYcore systems
*
* Held in conjunction with the International Conference on Computational 
Science (ICCS 2015)
* Reykjavik, Iceland, 1-3 June 2015
*
* http://sites.google.com/site/alchemyworkshop
* Important dates are synchronized with the ICCS meeting
***

The International Conference on Computational Science is an annual
conference that brings together researchers and scientists from mathematics
and computer science as basic computing disciplines, researchers from various
application areas who are pioneering computational methods in sciences such
as physics, chemistry, life sciences, and engineering, as well as in arts and
humanitarian fields, to discuss problems and solutions in the area, to identify 
new
issues, and to shape future directions for research.


Call for Papers
***

Massively parallel processors have entered high performance computing
architectures, as well as embedded systems. In June 2014, the TOP500
number one system (Tianhe-2) features the 57-core Intel Xeon Phi
processor. The increase of the number of cores on a chip is expected
to rise in the next years, as shown by the ITRS trends: other examples
include the Kalray MPPA 256-core chip, the 63-core Tilera GX processor
and even the crowd-funded 64-core Parallella Epiphany chip. In this
context, developers of parallel applications, including heavy
simulations and scientific calculations will undoubtedly have to cope
with many-core processors at the early design steps.

In the two past sessions of the Alchemy workshop, held together with
the ICCS meeting, we have presented significant contributions on the
design of many-core processors, both in the hardware and the software
programming environment sides, as well as some industrial-grade
application case studies. In this 2015 session, we seek academic
and industrial works that contribute to the design and the
programmability of many-core processors.

Topics include, but are not limited to:
* Programming models and languages for many-cores
* Compilers for programming languages
* Runtime generation for parallel programming on manycores
* Architecture support for massive parallelism management
* Enhanced communications for CMP/manycores
* Shared memory, data consistency models and protocols
* New operating systems, or dedicated OS
* Security, crypto systems for manycores
* User feedback on existing manycore architectures
(experiments with Adapteva Epiphany, Intel Phi, Kalray MPPA, ST
STHorm, Tilera Gx, TSAR..etc)


Submission
**

This year, there will be two formats for the presentation at the
workshop. The usual full-length paper is 10 pages according to the ICCS
format, and the short-paper format well fitted for works in progress,
with a maximum of 2 pages. The accepted papers for full-length paper
will be published alongside with the ICCS proceedings in Procedia
Computer Science, whereas the short-papers will be presentation and
poster only at the conference (with proceedings and presentations
available from the workshop website).

The manuscripts of up to 10 pages, written in English and formatted according to
the EasyChair templates, should be submitted electronically.
Templates are available for download in the Easychair right-hand-side menu
in a “New submission” mode.
https://easychair.org/conferences/?conf=iccs20150


Important Dates
*

When - Jun 1, 2015 - Jun 3, 2015
Where - Reykjavik, Iceland
Submission Deadline - Jan 15, 2015
Notification Due - Feb 15, 2015
Final Version Due - Mar 15, 2015


Program Committee
*
(to be extended)

Akram BEN AHMED, University of Aizu, Fukushima, Japan
Jeronimo CASTRILLON, CFAED / TU Dresden, Germany
Camille COTI, Université de Paris-Nord, France
Loïc CUDENNEC, CEA, LIST, France
Stephan DIESTELHORST, ARM Ltd; Cambridge, UK
Aleksandar DRAGOJEVIC, Microsoft Research Cambridge, UK
Daniel ETIEMBLE, Université de Paris-Sud, France
Bernard GOOSSENS, Université de Perpignan, France
Vincent GRAMOLI, NICTA / University of Sydney, Australia
Jorn W. JANNECK, Lund University, Sweden
Vianney LAPOTRE, Université de Bretagne-Sud, France
Eric LENORMAND, Thales TRT, France
Stéphane LOUISE, CEA, LIST, France
Vania MARANGOZOVA-MARTIN, Université Joseph-Fourier Grenoble, France
Marco MATTAVELLI, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland
Eric PETIT, Université de Versailles Saint Quentin-en-Yvelines, France
Erwan PIRIOU, CEA, LIST, France
Antoniu POP, University of Manchester, UK
Mickaël RAULET, IETR / INSA de Rennes, France
Jason RIEDY, Georgia Institute of Technology, USA
Etienne RIVIERE, Université de Neuchâtel, Switzerland
Thomas ROPARS, École 

Re: [OMPI users] Icreasing OFED registerable memory

[Waleed Lotfy]

Hi guys,

Sorry for getting back so late, but we ran into some problems during the 
installation process and as soon as the system came up I tested the new 
versions for the problem but it showed another memory related warning.

--
The OpenFabrics (openib) BTL failed to initialize while trying to
allocate some locked memory.  This typically can indicate that the
memlock limits are set too low.  For most HPC installations, the
memlock limits should be set to "unlimited".  The failure occured
here:

  Local host:comp003.local
  OMPI source:   btl_openib_component.c:1200
  Function:  ompi_free_list_init_ex_new()
  Device:mlx4_0
  Memlock limit: 65536

You may need to consult with your system administrator to get this
problem fixed.  This FAQ entry on the Open MPI web site may also be
helpful:

http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
--
--
WARNING: There was an error initializing an OpenFabrics device.

  Local host:   comp003.local
  Local device: mlx4_0
--

<<>>

My current running versions:

OpenMPI: 1.6.4
OFED-internal-2.3-2

I checked /etc/security/limits.d/, the scheduler's configurations (grid engine) 
and tried adding the following line to /etc/modprobe.d/mlx4_core: 'options 
mlx4_core log_num_mtt=22 log_mtts_per_seg=1' as suggested by Gus.

I am running out of ideas here, so please any help is appreciated.

P.S. I am not sure if I should open a new thread with this issue or continue 
with the current one, so please advice.

Waleed Lotfy
Bibliotheca Alexandrina