Just a reminder that this change is headed for the trunk tomorrow afternoon/evening. We will discuss it on the teleconf, but if you have cycles to test please do so.
Thanks, Josh On Fri, Jun 22, 2012 at 1:45 PM, Josh Hursey <jjhur...@open-mpi.org> wrote: > In response to some early feedback, I fixed the following in the > branch and updated the wiki: > * Fixed 'make distcheck' (missing a file) > * Cleanup some warnings from autogen and configure for -no-orte builds > * Added an ./autogen.pl -no-pineapple option (give you just the > ORTE/OPAl stack) - see wiki > > Thanks for the feedback and keep it coming. > > Also - The name will be changed before the commit to the trunk, I'm > working on some suggestions, but if you have any that you want to > lobby for let me know. > > Thanks, > Josh > > On Fri, Jun 22, 2012 at 8:41 AM, Josh Hursey <jjhur...@open-mpi.org> wrote: >> The pineapple branch is ready for testing. It was last sync'ed to the >> trunk at r26620. I'll try to update that later today, and keep current >> with the trunk moving forward. >> >> I created a wiki page to discuss some of the build options, for those >> interested. At the bottom of the page are some remaining todo items >> that I will probably need some help resolving. >> https://svn.open-mpi.org/trac/ompi/wiki/Runtime_Interposition >> >> The branch is at: >> https://bitbucket.org/jjhursey/ompi-pineapple >> >> I would like to bring this into the trunk the evening after the >> teleconf on Tuesday, June 26, 2012. So if you have cycles to test this >> branch I would appreciate it. >> >> Thanks, >> Josh >> >> On Fri, Jun 15, 2012 at 2:55 PM, Josh Hursey <jjhur...@open-mpi.org> wrote: >>> What: A Runtime Interposition Project - Codename Pineapple >>> >>> Why: Define clear API and semantics for runtime requirements of the OMPI >>> layer. >>> >>> When: >>> - F June 22, 2012 - Work completed >>> - T June 26, 2012 - Discuss on teleconf >>> - R June 28, 2012 - Commit to trunk >>> >>> Where: Trunk (development BitBucket branch below) >>> https://bitbucket.org/jjhursey/ompi-pineapple >>> >>> Attached: >>> PDF of slides presented on the June 12, 2012 teleconf. Note that the >>> timeline was slightly adjusted above (work completed date moved >>> ealier). >>> >>> >>> Description: Short Version >>> -------------------------- >>> Define, in an 'rte.h', the interfaces and semantics that the OMPI >>> layer requires of a runtime environment. Currently this interface >>> matches the subset of ORTE functionality that is used by the OMPI >>> layer. Runtime symbols (e.g., orte_ess.proc_get_locality) are isolated >>> to a framework inside this project to provide linker-level protection >>> against accidental breakage of the pineapple interposition layer. >>> >>> The interposition project provides researchers working on side >>> projects above and below the 'rte.h' interface a single location in >>> the code base to watch for interface and semantic changes that they >>> need to be concerned about. Researchers working above the pineapple >>> layer might explore something other than (or in addition to) OMPI >>> (e.g., Extended OMPI, UPC+OMPI). Researchers working below the >>> pineapple layer might explore something other than (or in addition to) >>> ORTE under OMPI (e.g., specialized runtimes for specific >>> environments). >>> >>> >>> Description: Other notes >>> ------------------------ >>> The pineapple interface provides OMPI developers with a runtime API to >>> program against without requiring detailed knowledge of the layout of >>> ORTE and its frameworks. In some places in OMPI a single source file >>> needs to include >5 (up to 12 in one place) different header files to >>> get all of the necessary symbols. Developers must not only know where >>> these headers are, but must also understand the differences between >>> the various frameworks in ORTE to use ORTE. The developer must also be >>> aware that there are certain APIs and data structure fields that are >>> not available to the MPI process, so should not be used. The pineapple >>> project provides an API representing the small subset of ORTE that is >>> used by OMPI. With this API a developer only needs to look at a single >>> location in the code base to understand what is provided by the >>> runtime for use in the OMPI layer. >>> >>> A similar statement could be made for runtime developers trying to >>> figure out what the OMPI layer requires from the a runtime >>> environment. Currently they need a deep understanding of the behavior >>> of ORTE to understand the semantics of various calls to ORTE from the >>> OMPI layer. Then they must develop a custom patch for the OMPI layer >>> that extracts the ORTE symbols, and replaces them with their own >>> symbols. This process is messy, error prone, and tedious to say the >>> least. Having a single set of interfaces and semantics will allow such >>> developers to focus their efforts on supporting the Open MPI community >>> defined API, and not necessarily the evolution of the ORTE or OMPI >>> project internals. This is advantageous when porting Open MPI to an >>> environment with a full featured runtime already running on the >>> machine, and for researchers exploring radical runtime designs for >>> future systems. The pineapple API allows such projects to develop >>> beside the mainline Open MPI trunk a little more easily than without >>> the pineapple API. >>> >>> >>> FAQ: >>> ---- >>> (1) Why is this a separate project and not a framework of OMPI? or a >>> framework of ORTE? >>> >>> After much deliberation between the developers, from a software >>> engineering perspective, making the pineapple rte.h interface a >>> separate project was the most flexible solution. So neither the OMPI >>> layer nor the ORTE layer 'own' the interface, but it is 'owned' by the >>> Open MPI project primarily to support the interaction between these >>> two layers. >>> >>> Consider that if we decided to place the interface in the OMPI layer >>> as a framework then we would be able to place something other than (or >>> in addition to) ORTE underneath OMPI, but we would be limited in our >>> ability to place something other than (or in addition to) OMPI over >>> ORTE. Alternatively, if we decided to place the rte.h interface in the >>> ORTE layer then we would be able to place something other than (or in >>> addition to) OMPI over ORTE, but we would be limited in our ability to >>> place something other than (or in addition to) ORTE under OMPI. >>> Defining the interposition layer as a separate project between these >>> two layers allows maximal flexibility for the project and researchers >>> working on side branches. >>> >>> >>> (2) What if another project outside of Open MPI needs interface >>> changes to the pineapple 'rte.h'? >>> >>> The rule of thumb is that 'The OMPI/ORTE/OPAL stack is king!'. This >>> means that the pineapple project should always err on the side of >>> supporting the OMPI/ORTE/OPAL stack, as that is the flagship product >>> of the Open MPI project. Interface suggestions are always welcome and >>> the rte.h may be extended/modified in the future as a result of those >>> suggestions. However, if a suggested change negatively impacts the >>> OMPI/ORTE/OPAL stack then it is unlikely to be accepted upstream by >>> the Open MPI community. >>> >>> >>> -- >>> Joshua Hursey >>> Postdoctoral Research Associate >>> Oak Ridge National Laboratory >>> http://users.nccs.gov/~jjhursey >> >> >> >> -- >> Joshua Hursey >> Postdoctoral Research Associate >> Oak Ridge National Laboratory >> http://users.nccs.gov/~jjhursey > > > > -- > Joshua Hursey > Postdoctoral Research Associate > Oak Ridge National Laboratory > http://users.nccs.gov/~jjhursey -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey
