Hi Jeff, Thanks for your help. It seems a good point to start :)
And what about PMIX messages (I think it's called OOB) during the process spawn? Where can I look the data structure? I will probably ask similar questions in the near future. I'm going to start a proof of concept project (only for fun) to monitor OMPI apps from different point of views using eBPF (not only network). Hopefully you and other active members in the list can help me during the process. Thanks again. BR, Víctor. El lunes, 16 de mayo de 2022, Jeff Squyres (jsquyres) <jsquy...@cisco.com> escribió: > Open MPI is generally structured in layers, but adjacent layers don't > necessarily have any knowledge of each other. > > For example, the PML (point-to-point messaging layer) is the first layer > behind MPI point-to-point functions such as MPI_SEND and MPI_RECV. > Different PMLs do not have the same packet layouts, and may, themselves, be > layered. For example, the OB1 PML is mostly a high-level protocol engine > that uses BTLs for low-level sending to and receiving from peers. The > BTLs, therefore, are responsible for their outermost packet formats, and > encapsulate the OB1 payload (which, itself, has a header and a payload). > > If you want to look at the TCP BTL (over the OB1 PML), you can look at the > various structs in opal/mca/btl/tcp. Additionally, you'll need to look at > the OB1 PML structs in ompi/mca/pml/ob1. It's conceivable that you could > make a Wireshark plugin for this use case, for example. The TCP BTL and > OB1 PML structs are subject to change at any time -- it's not like they're > published standards -- but they have been pretty stable for years. > > For other use cases -- e.g., OS-bypass networks -- you'll need to sniff > the packets from the network itself (because, by definition, the OS won't > have visibility of the packets). Regardless, all of those structs are > defined in their BTL / MTL / PML / etc. components. We don't have formal > documentation of any of them, sorry! > > -- > Jeff Squyres > jsquy...@cisco.com > > ________________________________________ > From: victor sv <victo...@gmail.com> > Sent: Monday, May 16, 2022 1:17 PM > To: Jeff Squyres (jsquyres) > Cc: users@lists.open-mpi.org > Subject: Re: [OMPI users] Network traffic packets documentation > > Hi Jeff, > > Ok, maybe "packet headers" are not the right words. What I would like to > know is how MPI application data is structured inside each packet in order > to dissect and caracterize the messages. > > As a first step I would like to start with TCP over Ethernet (MCA BTL TCP, > I think). How can I figure out how the application data structure looks > like inside network packets? > > In the future I would like to extend it to other network and transport > combinations. > > What do you think? Has it sense? > > Thanks in advance. > Víctor > > El lun, 16 may 2022 a las 15:45, Jeff Squyres (jsquyres) (< > jsquy...@cisco.com<mailto:jsquy...@cisco.com>>) escribió: > Open MPI doesn't proscribe a specific network protocol for anything. > Indeed, each network transport uses their own protocols, headers, etc. > It's basically a "each Open MPI plugin needs to be able to talk to itself", > and therefore no commonality is needed (or desired). > > Which network and Open MPI transport are you looking to sniff? > > -- > Jeff Squyres > jsquy...@cisco.com<mailto:jsquy...@cisco.com> > > ________________________________________ > From: users <users-boun...@lists.open-mpi.org<mailto:users-bounces@lists > .open-mpi.org>> on behalf of victor sv via users <users@lists.open-mpi.org > <mailto:users@lists.open-mpi.org>> > Sent: Sunday, May 15, 2022 3:55 PM > To: users@lists.open-mpi.org<mailto:users@lists.open-mpi.org> > Cc: victor sv > Subject: [OMPI users] Network traffic packets documentation > > Hello, > > I would like to sniff the OMPI network traffic from outside the MPI > application. > > I was traversing the OpenMPI code and documentation, but I have not found > any central point explaining MPI communications from the network point of > view. > > Please, is there any official documentation, or paper, or presentation or > picture about MPI packet headers? > > Sorry if this a basic question or if it was already answered. > > Thanks in advance for your help! > BR, > Víctor. >
