Re: [OMPI users] ompe support for filesystems
Edgar,
Thanks - this is exactly what I was looking for. I started going through
the code in earnest and trying to figure out exactly where the magic
happens. I may have more questions :-)
Ognen
Gabriel, Edgar via users writes:
> the ompio software infrastructure has multiple frameworks.
>
> fs framework: abstracts out file system level operations (open, close, etc)
>
> fbtl framework: provides the abstractions and implementations of *individual*
> file I/O operations (seek,read,write, iread,iwrite)
>
> fcoll framework: provides the abstractions and implementations of
> *collective* file I/O operations ( read_all, write_all, etc.)
>
> sharedfp framework: provides the abstractions and implementations *shared
> file pointer* file I/O operations (read_shared, write_shared, read_ordered,
> write_ordered).
>
> Feel free to ping me also directly if you need more assistance. If you are
> looking for a reference and more explanations, please have a look at the
> following paper:
>
> Mohamad Chaarawi, Edgar Gabriel, Rainer Keller, Richard Graham, George
> Bosilca and Jack Dongarra, 'OMPIO: A Modular Software Architecture for MPI
> I/O', in Y. Cotronis, A. Danalis, D. Nikolopoulos, J. Dongarra, (Eds.)
> 'Recent Advances in Message Passing Interface', LNCS vol. 6960, pp. 81-89,
> Springer, 2011.
>
> http://www2.cs.uh.edu/~gabriel/publications/EuroMPI11_OMPIO.pdf
>
> Best regards
> Edgar
>
> -Original Message-
> From: users On Behalf Of Ognen Duzlevski
> via users
> Sent: Monday, November 2, 2020 7:54 AM
> To: Open MPI Users
> Cc: Ognen Duzlevski
> Subject: Re: [OMPI users] ompe support for filesystems
>
> Gilles,
>
> Thank you for replying.
>
> I took a look at the code and am curious to understand where the actual
> read/write/seek etc. operations are implemented. From what I can
> see/understand - what you pointed me to implements file open/close etc.
> operations that pertain to particular filesystems.
>
> I then tried to figure out the read/write/seek etc. operations and can see
> that a MPI_File structure appears to have a f_io_selected_module member,
> whose v_2_0_0 member seems to have the list of pointers to all the
> functionals dealing with the actual file write/read/seek functionality. Is
> this correct?
>
> What I would like to figure out is where the actual writes or reads happen
> (as in the underlying filesystem's implementations). I imagine for some
> filesystems a write, for example, is not just a simple call to the write onto
> disk but involves a bit more logic/magic.
>
> Thanks!
> Ognen
>
>
> Gilles Gouaillardet via users writes:
>
>> Hi Ognen,
>>
>> MPI-IO is implemented by two components:
>> - ROMIO (from MPICH)
>> - ompio ("native" Open MPI MPI-IO, default component unless running
>> on Lustre)
>>
>> Assuming you want to add support for a new filesystem in ompio, first
>> step is to implement a new component in the fs framework the framework
>> is in /ompi/mca/fs, and each component is in its own directory (for
>> example ompi/mca/fs/gpfs)
>>
>> There are a some configury tricks (create a configure.m4, add Makefile
>> to autoconf, ...) to make sure your component is even compiled.
>> If you are struggling with these, feel free to open a Pull Request to
>> get some help fixing the missing bits.
>>
>> Cheers,
>>
>> Gilles
>>
>> On Sun, Nov 1, 2020 at 12:18 PM Ognen Duzlevski via users
>> wrote:
>>>
>>> Hello!
>>>
>>> If I wanted to support a specific filesystem in open mpi, how is this
>>> done? What code in the source tree does it?
>>>
>>> Thanks!
>>> Ognen
Re: [OMPI users] ompe support for filesystems
the ompio software infrastructure has multiple frameworks.
fs framework: abstracts out file system level operations (open, close, etc)
fbtl framework: provides the abstractions and implementations of *individual*
file I/O operations (seek,read,write, iread,iwrite)
fcoll framework: provides the abstractions and implementations of *collective*
file I/O operations ( read_all, write_all, etc.)
sharedfp framework: provides the abstractions and implementations *shared file
pointer* file I/O operations (read_shared, write_shared, read_ordered,
write_ordered).
Feel free to ping me also directly if you need more assistance. If you are
looking for a reference and more explanations, please have a look at the
following paper:
Mohamad Chaarawi, Edgar Gabriel, Rainer Keller, Richard Graham, George Bosilca
and Jack Dongarra, 'OMPIO: A Modular Software Architecture for MPI I/O', in Y.
Cotronis, A. Danalis, D. Nikolopoulos, J. Dongarra, (Eds.) 'Recent Advances in
Message Passing Interface', LNCS vol. 6960, pp. 81-89, Springer, 2011.
http://www2.cs.uh.edu/~gabriel/publications/EuroMPI11_OMPIO.pdf
Best regards
Edgar
-Original Message-
From: users On Behalf Of Ognen Duzlevski via
users
Sent: Monday, November 2, 2020 7:54 AM
To: Open MPI Users
Cc: Ognen Duzlevski
Subject: Re: [OMPI users] ompe support for filesystems
Gilles,
Thank you for replying.
I took a look at the code and am curious to understand where the actual
read/write/seek etc. operations are implemented. From what I can see/understand
- what you pointed me to implements file open/close etc. operations that
pertain to particular filesystems.
I then tried to figure out the read/write/seek etc. operations and can see that
a MPI_File structure appears to have a f_io_selected_module member, whose
v_2_0_0 member seems to have the list of pointers to all the functionals
dealing with the actual file write/read/seek functionality. Is this correct?
What I would like to figure out is where the actual writes or reads happen (as
in the underlying filesystem's implementations). I imagine for some filesystems
a write, for example, is not just a simple call to the write onto disk but
involves a bit more logic/magic.
Thanks!
Ognen
Gilles Gouaillardet via users writes:
> Hi Ognen,
>
> MPI-IO is implemented by two components:
> - ROMIO (from MPICH)
> - ompio ("native" Open MPI MPI-IO, default component unless running
> on Lustre)
>
> Assuming you want to add support for a new filesystem in ompio, first
> step is to implement a new component in the fs framework the framework
> is in /ompi/mca/fs, and each component is in its own directory (for
> example ompi/mca/fs/gpfs)
>
> There are a some configury tricks (create a configure.m4, add Makefile
> to autoconf, ...) to make sure your component is even compiled.
> If you are struggling with these, feel free to open a Pull Request to
> get some help fixing the missing bits.
>
> Cheers,
>
> Gilles
>
> On Sun, Nov 1, 2020 at 12:18 PM Ognen Duzlevski via users
> wrote:
>>
>> Hello!
>>
>> If I wanted to support a specific filesystem in open mpi, how is this
>> done? What code in the source tree does it?
>>
>> Thanks!
>> Ognen
Re: [OMPI users] ompe support for filesystems
Gilles,
Thank you for replying.
I took a look at the code and am curious to understand where the actual
read/write/seek etc. operations are implemented. From what I can
see/understand - what you pointed me to implements file open/close
etc. operations that pertain to particular filesystems.
I then tried to figure out the read/write/seek etc. operations and can
see that a MPI_File structure appears to have a f_io_selected_module
member, whose v_2_0_0 member seems to have the list of pointers to all
the functionals dealing with the actual file write/read/seek
functionality. Is this correct?
What I would like to figure out is where the actual writes or reads
happen (as in the underlying filesystem's implementations). I imagine
for some filesystems a write, for example, is not just a simple call to
the write onto disk but involves a bit more logic/magic.
Thanks!
Ognen
Gilles Gouaillardet via users writes:
> Hi Ognen,
>
> MPI-IO is implemented by two components:
> - ROMIO (from MPICH)
> - ompio ("native" Open MPI MPI-IO, default component unless running on
> Lustre)
>
> Assuming you want to add support for a new filesystem in ompio, first
> step is to implement a new component in the fs framework
> the framework is in /ompi/mca/fs, and each component is in its own
> directory (for example ompi/mca/fs/gpfs)
>
> There are a some configury tricks (create a configure.m4, add Makefile
> to autoconf, ...) to make sure your component is even compiled.
> If you are struggling with these, feel free to open a Pull Request to
> get some help fixing the missing bits.
>
> Cheers,
>
> Gilles
>
> On Sun, Nov 1, 2020 at 12:18 PM Ognen Duzlevski via users
> wrote:
>>
>> Hello!
>>
>> If I wanted to support a specific filesystem in open mpi, how is this
>> done? What code in the source tree does it?
>>
>> Thanks!
>> Ognen
Re: [OMPI users] ompe support for filesystems
Gilles,
Thank you very much!
Ognen
Gilles Gouaillardet via users writes:
> Hi Ognen,
>
> MPI-IO is implemented by two components:
> - ROMIO (from MPICH)
> - ompio ("native" Open MPI MPI-IO, default component unless running on
> Lustre)
>
> Assuming you want to add support for a new filesystem in ompio, first
> step is to implement a new component in the fs framework
> the framework is in /ompi/mca/fs, and each component is in its own
> directory (for example ompi/mca/fs/gpfs)
>
> There are a some configury tricks (create a configure.m4, add Makefile
> to autoconf, ...) to make sure your component is even compiled.
> If you are struggling with these, feel free to open a Pull Request to
> get some help fixing the missing bits.
>
> Cheers,
>
> Gilles
>
> On Sun, Nov 1, 2020 at 12:18 PM Ognen Duzlevski via users
> wrote:
>>
>> Hello!
>>
>> If I wanted to support a specific filesystem in open mpi, how is this
>> done? What code in the source tree does it?
>>
>> Thanks!
>> Ognen
Re: [OMPI users] ompe support for filesystems
Hi Ognen,
MPI-IO is implemented by two components:
- ROMIO (from MPICH)
- ompio ("native" Open MPI MPI-IO, default component unless running on Lustre)
Assuming you want to add support for a new filesystem in ompio, first
step is to implement a new component in the fs framework
the framework is in /ompi/mca/fs, and each component is in its own
directory (for example ompi/mca/fs/gpfs)
There are a some configury tricks (create a configure.m4, add Makefile
to autoconf, ...) to make sure your component is even compiled.
If you are struggling with these, feel free to open a Pull Request to
get some help fixing the missing bits.
Cheers,
Gilles
On Sun, Nov 1, 2020 at 12:18 PM Ognen Duzlevski via users
wrote:
>
> Hello!
>
> If I wanted to support a specific filesystem in open mpi, how is this
> done? What code in the source tree does it?
>
> Thanks!
> Ognen
