Then let me provide a more elaborate answer. In the original design of the btl_sendi operation we do not provide an upper limit for the sendi (in the same sense as the eager protocol). Thus, an upper layer (PML in this instance) cannot know if the sendi will succeed or not before the call itself. Thus, in order to avoid several ping-pong between software layers, we force the sendi to either succeed or return a descriptor (up to the BTL eager size), identical to what the btl_alloc would have returned.
At this point the PML is forced to pack itself the data in the retuned descriptor, without knowing if the BTL is elf can do better (some kind of IN_PLACE). Therefore, this approach make sense in the case where a copy of the data is to be done, aka. the start_copy function. In the case of start_prepare there is no copy of the data as the PML entrust the BTL with the preparation of the optimal descriptor. That being said, I would not be against changing the btl_sendi rules slightly. Let's imagine that upon failure to immediately send the data we require from btl_sendi to return a descriptor that instead of being similar to what btl_alloc would return is similar to what btl_prepare_src would return. Such a scenario is possible as both calls have access to the convertor, which is the most critical piece of the data. This will cover the case of the start_prepare and of the start_copy, as will allow us to expand the usages of the btl_sendi capability. George. On Jun 18, 2013, at 22:52 , "Jeff Squyres (jsquyres)" <jsquy...@cisco.com> wrote: > George replied to me in IM -- posting here for completeness: > >> Yes, there is a reason. if sendi succeeds, it sends a very small data (at >> least on the devices that supports it), otherwise it returns a descriptor >> similar to btl_alloc() >> thus you will have to pack the data yourself, and the PML doesn't know if >> IN_PLACE should be used or not >> thus the resulting solution is slower than the default in the spart_prepare >> case (which is prepare_src + send) > > > On Jun 14, 2013, at 3:46 PM, Jeff Squyres (jsquyres) <jsquy...@cisco.com> > wrote: > >> In working on the upcoming Cisco USNIC BTL, we noticed that btl.sendi is >> invoked by OB1 in the non-MCA_BTL_FLAGS_SEND_INPLACE case. >> >> Is there a reason for this? Or is it only because no one who uses INPLACE >> has cared about sendi? >> >> -- >> Jeff Squyres >> jsquy...@cisco.com >> For corporate legal information go to: >> http://www.cisco.com/web/about/doing_business/legal/cri/ >> >> >> _______________________________________________ >> devel mailing list >> de...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/devel > > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > _______________________________________________ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel
