On Mar 4, 2010, at 12:10 PM, Yonggang Liu wrote: > Thank you so much, Sam. I found out the definition with your help. The paper > you gave is also very helpful, meanwhile, I still want to know if you have a > manual about the BMI protocol, mainly describing every field in each kind of > packet. As you know, that would save us a lot of work when we are > implementing the interposed scheduler. (We found Wireshark actually did some > work on PVFS2 packet dissection, but it seems they didn't do it perfect.) > Appreciate your help. Thank you very much! >
The wireshark packet dissection module is woefully out of date. Fortunately, the PVFS protocol is fairly simple, and you can pretty much just look at pvfs2-req-proto.h to understand the protocol. Note that BMI is just an interface for sending messages, and the header of the BMI message looks different for each method used. In the case of TCP, the BMI header looks like this: magic_number [4] -- always 51903, avoids parsing of unknown messages mode [4] -- 2 (unexpected message), 4 (eager message), 8 (rendezvous message) tag [8] -- uniquely identifies this message, used for matching size [8] -- the size of the payload The size field holds the size of the BMI message payload (not including the 24 byte header). The payload can be either a request, response, or data message. The data message is just a stream of bytes, PVFS doesn't decode it further, and it doesn't contain a header. The request and response messages contain headers (immediately following the BMI tcp header). They look like this: PVFS request header: version [4] -- PVFS protocol version, e.g. 6000 encoding [4] -- type of encoding used, always 2 operation [4] -- specifies the PVFS operation, one of the values in the enum PVFS_server_op. padding [4] -- ensure 8 byte boundary credentials [8]: user id [4] and group id [4] hint_count [4] -- count of hints sent, always 0 padding [4] PVFS response header: operation [4] -- PVFS operation status [4] -- error from operation The rest of the message for request and response messages contains the fields specified for that operation in pvfs2-req-proto.h. The fields are always padded to an 8 byte boundary. For example, the getattr operation looks like this: handle [8] fs_id [4] attrmask [4] Hope this helps! -sam > > Regards, > > On Thu, Mar 4, 2010 at 9:09 AM, Sam Lang <[email protected]> wrote: > > On Mar 3, 2010, at 8:46 PM, Yonggang Liu wrote: > > > Hello all, > > > > I'm pretty new to PVFS2, and now I'm doing a project about implementing > > interposed scheduler between the PVFS2 clients and PVFS2 servers to provide > > QoS. So I have read some code and doc about the PVFS2 protocols. But for a > > couple of days I get confused about one part of the code, > > In src/proto/PINT-le-bytefield.c, function lebf_encode_req(), we have: > > #define CASE(tag,var) \ > > case tag: encode_PVFS_servreq_##var(p,&req->u.var); break > > > > switch (req->op) { > > > > /* call standard function defined in headers */ > > CASE(PVFS_SERV_LOOKUP_PATH, lookup_path); > > CASE(PVFS_SERV_CREATE, create); > > CASE(PVFS_SERV_UNSTUFF, unstuff); > > CASE(PVFS_SERV_BATCH_CREATE, batch_create); > > CASE(PVFS_SERV_BATCH_REMOVE, batch_remove); > > CASE(PVFS_SERV_REMOVE, remove); > > ...... > > This means these functions exist to be called: > > encode_PVFS_servreq_lookup_path(p, &req->u.lookup_path); > > encode_PVFS_servreq_create(p, &req->u.create); > > ...... > > But I was confused that I couldn't find the definition of the above > > functions anywhere in the code. So can anyone help me to figure out the way > > they are defined? (Sorry, this is really a silly question caused by my poor > > C language skills ...) Thanks. > > Look in src/proto/pvfs2-req-proto.h. The macro: endecode_fields_5_struct > defines encode_PVFS_servreq_create. > > > > > Also, I am wondering if some one has a good documentation describing the > > PVFS2 network protocols. I would really appreciate it if you can share it > > with me. > > BMI is described in this paper: > > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.60.2472 > > -sam > > > > > > > Thank you very much, > > > > -- > > Yonggang Liu > > Advanced Computing and Information Systems Laboratory > > University of Florida > > _______________________________________________ > > Pvfs2-developers mailing list > > [email protected] > > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers > > > > > -- > Yonggang Liu > Advanced Computing and Information Systems Laboratory > University of Florida _______________________________________________ Pvfs2-developers mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
