Re: [OMPI devel] BTL sendi

[George Bosilca]

Then let me provide a more elaborate answer.

In the original design of the btl_sendi operation we do not provide an upper 
limit for the sendi (in the same sense as the eager protocol). Thus, an upper 
layer (PML in this instance) cannot know if the sendi will succeed or not 
before the call itself. Thus, in order to avoid several ping-pong between 
software layers, we force the sendi to either succeed or return a descriptor 
(up to the BTL eager size), identical to what the btl_alloc would have returned.

At this point the PML is forced to pack itself the data in the retuned 
descriptor, without knowing if the BTL is elf can do better (some kind of 
IN_PLACE). Therefore, this approach make sense in the case where a copy of the 
data is to be done, aka. the start_copy function. In the case of start_prepare 
there is no copy of the data as the PML entrust the BTL with the preparation of 
the optimal descriptor.

That being said, I would not be against changing the btl_sendi rules slightly. 
Let's imagine that upon failure to immediately send the data we require from 
btl_sendi to return a descriptor that instead of being similar to what 
btl_alloc would return is similar to what btl_prepare_src would return. Such a 
scenario is possible as both calls have access to the convertor, which is the 
most critical piece of the data. This will cover the case of the start_prepare 
and of the start_copy, as will allow us to expand the usages of the btl_sendi 
capability.

  George.

On Jun 18, 2013, at 22:52 , "Jeff Squyres (jsquyres)"  
wrote:

> George replied to me in IM -- posting here for completeness:
> 
>> Yes, there is a reason. if sendi succeeds, it sends a very small data (at 
>> least on the devices that supports it), otherwise it returns a descriptor 
>> similar to btl_alloc()
>> thus you will have to pack the data yourself, and the PML doesn't know if 
>> IN_PLACE should be used or not
>> thus the resulting solution is slower than the default in the spart_prepare 
>> case (which is prepare_src + send)
> 
> 
> On Jun 14, 2013, at 3:46 PM, Jeff Squyres (jsquyres)  
> wrote:
> 
>> In working on the upcoming Cisco USNIC BTL, we noticed that btl.sendi is 
>> invoked by OB1 in the non-MCA_BTL_FLAGS_SEND_INPLACE case.
>> 
>> Is there a reason for this?  Or is it only because no one who uses INPLACE 
>> has cared about sendi?
>> 
>> -- 
>> Jeff Squyres
>> jsquy...@cisco.com
>> For corporate legal information go to: 
>> http://www.cisco.com/web/about/doing_business/legal/cri/
>> 
>> 
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
> 
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] BTL sendi

[Jeff Squyres (jsquyres)]

George replied to me in IM -- posting here for completeness:

> Yes, there is a reason. if sendi succeeds, it sends a very small data (at 
> least on the devices that supports it), otherwise it returns a descriptor 
> similar to btl_alloc()
> thus you will have to pack the data yourself, and the PML doesn't know if 
> IN_PLACE should be used or not
> thus the resulting solution is slower than the default in the spart_prepare 
> case (which is prepare_src + send)


On Jun 14, 2013, at 3:46 PM, Jeff Squyres (jsquyres)  wrote:

> In working on the upcoming Cisco USNIC BTL, we noticed that btl.sendi is 
> invoked by OB1 in the non-MCA_BTL_FLAGS_SEND_INPLACE case.
> 
> Is there a reason for this?  Or is it only because no one who uses INPLACE 
> has cared about sendi?
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

[OMPI devel] BTL sendi

[Jeff Squyres (jsquyres)]

In working on the upcoming Cisco USNIC BTL, we noticed that btl.sendi is 
invoked by OB1 in the non-MCA_BTL_FLAGS_SEND_INPLACE case.

Is there a reason for this?  Or is it only because no one who uses INPLACE has 
cared about sendi?

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/