Re: [lng-odp] [API-NEXT PATCHv4 1/4] api: packet reference count support

[Ola Liljedahl]

On 21 October 2015 at 14:46, Maxim Uvarov  wrote:

> On 10/21/2015 15:27, Ola Liljedahl wrote:
>
>> On 20 October 2015 at 17:41, Maxim Uvarov > > wrote:
>>
>> On 10/15/2015 23:49, Ola Liljedahl wrote:
>>
>> On 9 October 2015 at 08:59, Maxim Uvarov
>> 
>> > >> wrote:
>>
>> Add api for packet reference count support. Which is
>> useful in case:
>>  - multicast support
>>
>> Different packets will probably need different headers
>> prepended to the shared packet. How is this handled?
>>
>>
>> That has to be handled with reference counts for segments support.
>> For now you have packet, you can copy body of packet to new packet
>> and put your own header.
>>
>> So your proposal does not support multicasting (i.e. making it more
>> efficient, e.g. by avoiding copying the whole packet)?
>>
>
> yes, that is true.
>
>
>
>>
>>  - TCP retransmission
>>
>> When a TCP packet is retransmitted, the IP id should be
>> changed (similar for QUIC retransmissions where the SN is
>> updated). So you might have to modify the packet (header)
>> before you know that the previous transmit has actually
>> completed. How is this handled?
>>
>>
>> Same as above. Without segmentation you can only copy to new packet.
>>
>> So this use case is also not supported?
>>
>> In the normal case, the packet has already been sent when the
>> retransmission timer expires and you need to retransmit the packet. How can
>> the application see that it is OK to reuse (modify) the original packet and
>> not have to create a copy?
>> Without having to expose the reference counter, you could have a call
>> which conditionally creates a new packet (as a clone of the specified
>> packet) if the specified packet cannot be modified (because it has other
>> references). When transmission is complete, the system automatically frees
>> the packet (i.e. decreases the refcount). When the retransmission timer
>> expires, the application can call this function that either returns the
>> original packet (because it is no longer being "in transmission") or it
>> clones the packet and return the new handle. Safe and optimised for that
>> normal case of retransmission.
>>
>>
>>
>>  - traffic generator
>>
>> A traffic generator might want to include an increasing SN or
>> timestamp in each packet (packet payload) which otherwise are
>> identical. How is this handled?
>>
>> Same question. If reference count for hole packet is more then 1
>> you can not modify that packet.
>>
>> You can modify packet after send:
>>
>> I am not so sure. Packet transmission is not immediate, it is essentially
>> an asynchronous operation without any notification that it has completed.
>> pktio_send() does not make a copy of the packet so that you can immediately
>> modify the packet whose handle you kept. There is no specific ODP call that
>> indicates that you want to modify the packet and thus want a "clean" copy
>> that is safe to modify.
>>
>>
>> pkt = odp_packet_create_ref(pkt);
>> pktio_send(pkt)
>> modify (pkt)
>> pkt = odp_packet_create_ref(pkt);
>> pktio_send(pkt)
>> modify (pkt)
>> 
>>
>>
>> I don't see how this proposal actually solves any interesting problems. I
>> see reference counting as being related to packet segments (an mbuf or
>> skbuf represents a segment so they have a refcount per mbuf/skbuf (actually
>> skbufs are more complicated than this, the data has its own refcount,
>> different skbufs can refer to the same data but with separate start and
>> lengths)) but for ODP packets, the segmentation is not directly visible (we
>> don't link packets together), it happens under the hood with some limited
>> support so that you can access packet data directly.
>>
>> ODP reference counting should support the relevant use cases (e.g.
>> multicasting, retransmission, pktgen can be seen as similar to multicasting
>> I think), making them more performant (and less memory consuming) than
>> creating new copies of every packet (which is what you need to do today). A
>> solution should be able to reuse common segments (which normally
>> constitutes the larger part of the packet data) for different packets that
>> may use different headers (a smaller part of the packet data). The API
>> might not need a specific function for creating a new packet reference.
>> Instead you specify a packet handle and the area (starting offset, size)
>> you want to be able to modify in a safe way (so not to disturb other users
>> of the packet). If there is only one reference to this part of the packet,
>> the implementation can return the original packet handle. If 

Re: [lng-odp] [API-NEXT PATCHv4 1/4] api: packet reference count support

[Ola Liljedahl]

On 20 October 2015 at 17:41, Maxim Uvarov  wrote:

> On 10/15/2015 23:49, Ola Liljedahl wrote:
>
>> On 9 October 2015 at 08:59, Maxim Uvarov > > wrote:
>>
>> Add api for packet reference count support. Which is useful in case:
>>  - multicast support
>>
>> Different packets will probably need different headers prepended to the
>> shared packet. How is this handled?
>>
>>
> That has to be handled with reference counts for segments support. For now
> you have packet, you can copy body of packet to new packet and put your own
> header.

So your proposal does not support multicasting (i.e. making it more
efficient, e.g. by avoiding copying the whole packet)?


>
>
>  - TCP retransmission
>>
>> When a TCP packet is retransmitted, the IP id should be changed (similar
>> for QUIC retransmissions where the SN is updated). So you might have to
>> modify the packet (header) before you know that the previous transmit has
>> actually completed. How is this handled?
>>
>>
> Same as above. Without segmentation you can only copy to new packet.

So this use case is also not supported?

In the normal case, the packet has already been sent when the
retransmission timer expires and you need to retransmit the packet. How can
the application see that it is OK to reuse (modify) the original packet and
not have to create a copy?
Without having to expose the reference counter, you could have a call which
conditionally creates a new packet (as a clone of the specified packet) if
the specified packet cannot be modified (because it has other references).
When transmission is complete, the system automatically frees the packet
(i.e. decreases the refcount). When the retransmission timer expires, the
application can call this function that either returns the original packet
(because it is no longer being "in transmission") or it clones the packet
and return the new handle. Safe and optimised for that normal case of
retransmission.


>
>  - traffic generator
>>
>> A traffic generator might want to include an increasing SN or timestamp
>> in each packet (packet payload) which otherwise are identical. How is this
>> handled?
>>
> Same question. If reference count for hole packet is more then 1 you can
> not modify that packet.
>
> You can modify packet after send:
>
I am not so sure. Packet transmission is not immediate, it is essentially
an asynchronous operation without any notification that it has completed.
pktio_send() does not make a copy of the packet so that you can immediately
modify the packet whose handle you kept. There is no specific ODP call that
indicates that you want to modify the packet and thus want a "clean" copy
that is safe to modify.


> pkt = odp_packet_create_ref(pkt);
> pktio_send(pkt)
> modify (pkt)
> pkt = odp_packet_create_ref(pkt);
> pktio_send(pkt)
> modify (pkt)
> 
>

I don't see how this proposal actually solves any interesting problems. I
see reference counting as being related to packet segments (an mbuf or
skbuf represents a segment so they have a refcount per mbuf/skbuf (actually
skbufs are more complicated than this, the data has its own refcount,
different skbufs can refer to the same data but with separate start and
lengths)) but for ODP packets, the segmentation is not directly visible (we
don't link packets together), it happens under the hood with some limited
support so that you can access packet data directly.

ODP reference counting should support the relevant use cases (e.g.
multicasting, retransmission, pktgen can be seen as similar to multicasting
I think), making them more performant (and less memory consuming) than
creating new copies of every packet (which is what you need to do today). A
solution should be able to reuse common segments (which normally
constitutes the larger part of the packet data) for different packets that
may use different headers (a smaller part of the packet data). The API
might not need a specific function for creating a new packet reference.
Instead you specify a packet handle and the area (starting offset, size)
you want to be able to modify in a safe way (so not to disturb other users
of the packet). If there is only one reference to this part of the packet,
the implementation can return the original packet handle. If there are more
than one reference, a new packet needs to be allocated and the relevant
segments cloned (refcount==1) while (trailing) untouched segments can be
reused (refcount++).



>
> Maxim.
>
>>
>>
>> Introduced new call: newpkt = odp_packet_create_ref(pkt) which creates
>> reference to original packet, but handles for reference packet and
>> original
>> are different.
>>
>> I think something like Bill's "copy-on-write" proposal (from Connect) is
>> better than explicit reference counting (of the whole packet). Increase the
>> abstraction level and allow for innovation in the underlying implementation.
>>
>> Any API 

Re: [lng-odp] [API-NEXT PATCHv4 1/4] api: packet reference count support

[Maxim Uvarov]

On 10/21/2015 15:27, Ola Liljedahl wrote:
On 20 October 2015 at 17:41, Maxim Uvarov > wrote:


On 10/15/2015 23:49, Ola Liljedahl wrote:

On 9 October 2015 at 08:59, Maxim Uvarov

>> wrote:

Add api for packet reference count support. Which is
useful in case:
 - multicast support

Different packets will probably need different headers
prepended to the shared packet. How is this handled?


That has to be handled with reference counts for segments support.
For now you have packet, you can copy body of packet to new packet
and put your own header.

So your proposal does not support multicasting (i.e. making it more 
efficient, e.g. by avoiding copying the whole packet)?


yes, that is true.




 - TCP retransmission

When a TCP packet is retransmitted, the IP id should be
changed (similar for QUIC retransmissions where the SN is
updated). So you might have to modify the packet (header)
before you know that the previous transmit has actually
completed. How is this handled?


Same as above. Without segmentation you can only copy to new packet.

So this use case is also not supported?

In the normal case, the packet has already been sent when the 
retransmission timer expires and you need to retransmit the packet. 
How can the application see that it is OK to reuse (modify) the 
original packet and not have to create a copy?
Without having to expose the reference counter, you could have a call 
which conditionally creates a new packet (as a clone of the specified 
packet) if the specified packet cannot be modified (because it has 
other references). When transmission is complete, the system 
automatically frees the packet (i.e. decreases the refcount). When the 
retransmission timer expires, the application can call this function 
that either returns the original packet (because it is no longer being 
"in transmission") or it clones the packet and return the new handle. 
Safe and optimised for that normal case of retransmission.




 - traffic generator

A traffic generator might want to include an increasing SN or
timestamp in each packet (packet payload) which otherwise are
identical. How is this handled?

Same question. If reference count for hole packet is more then 1
you can not modify that packet.

You can modify packet after send:

I am not so sure. Packet transmission is not immediate, it is 
essentially an asynchronous operation without any notification that it 
has completed. pktio_send() does not make a copy of the packet so that 
you can immediately modify the packet whose handle you kept. There is 
no specific ODP call that indicates that you want to modify the packet 
and thus want a "clean" copy that is safe to modify.



pkt = odp_packet_create_ref(pkt);
pktio_send(pkt)
modify (pkt)
pkt = odp_packet_create_ref(pkt);
pktio_send(pkt)
modify (pkt)



I don't see how this proposal actually solves any interesting 
problems. I see reference counting as being related to packet segments 
(an mbuf or skbuf represents a segment so they have a refcount per 
mbuf/skbuf (actually skbufs are more complicated than this, the data 
has its own refcount, different skbufs can refer to the same data but 
with separate start and lengths)) but for ODP packets, the 
segmentation is not directly visible (we don't link packets together), 
it happens under the hood with some limited support so that you can 
access packet data directly.


ODP reference counting should support the relevant use cases (e.g. 
multicasting, retransmission, pktgen can be seen as similar to 
multicasting I think), making them more performant (and less memory 
consuming) than creating new copies of every packet (which is what you 
need to do today). A solution should be able to reuse common segments 
(which normally constitutes the larger part of the packet data) for 
different packets that may use different headers (a smaller part of 
the packet data). The API might not need a specific function for 
creating a new packet reference. Instead you specify a packet handle 
and the area (starting offset, size) you want to be able to modify in 
a safe way (so not to disturb other users of the packet). If there is 
only one reference to this part of the packet, the implementation can 
return the original packet handle. If there are more than one 
reference, a new packet needs to be allocated and the relevant 
segments cloned (refcount==1) while (trailing) untouched segments can 
be reused (refcount++).


Ola, I understand and agree with your point.

In my understanding api can be about the same. Like:

newpkt = odp_packet_create_ref(oldpkt);

Re: [lng-odp] [API-NEXT PATCHv4 1/4] api: packet reference count support

[Maxim Uvarov]

On 10/15/2015 23:49, Ola Liljedahl wrote:
On 9 October 2015 at 08:59, Maxim Uvarov > wrote:


Add api for packet reference count support. Which is useful in case:
 - multicast support

Different packets will probably need different headers prepended to 
the shared packet. How is this handled?




That has to be handled with reference counts for segments support. For 
now you have packet, you can copy body of packet to new packet and put 
your own header.



 - TCP retransmission

When a TCP packet is retransmitted, the IP id should be changed 
(similar for QUIC retransmissions where the SN is updated). So you 
might have to modify the packet (header) before you know that the 
previous transmit has actually completed. How is this handled?




Same as above. Without segmentation you can only copy to new packet.


 - traffic generator

A traffic generator might want to include an increasing SN or 
timestamp in each packet (packet payload) which otherwise are 
identical. How is this handled?
Same question. If reference count for hole packet is more then 1 you can 
not modify that packet.


You can modify packet after send:

pkt = odp_packet_create_ref(pkt);
pktio_send(pkt)
modify (pkt)
pkt = odp_packet_create_ref(pkt);
pktio_send(pkt)
modify (pkt)



Maxim.



Introduced new call: newpkt = odp_packet_create_ref(pkt) which creates
reference to original packet, but handles for reference packet and
original
are different.

I think something like Bill's "copy-on-write" proposal (from Connect) 
is better than explicit reference counting (of the whole packet). 
Increase the abstraction level and allow for innovation in the 
underlying implementation.


Any API support should be based on use case analysis and code examples 
of how the use cases are implemented using the suggested API's. 
Otherwise the risk of missing the target is significant.



Signed-off-by: Maxim Uvarov >
---
 include/odp/api/config.h | 5 +
 include/odp/api/packet.h | 9 +
 2 files changed, 14 insertions(+)

diff --git a/include/odp/api/config.h b/include/odp/api/config.h
index 295b10d..985290a 100644
--- a/include/odp/api/config.h
+++ b/include/odp/api/config.h
@@ -126,6 +126,11 @@ extern "C" {
  */
 #define ODP_CONFIG_PACKET_BUF_LEN_MAX
(ODP_CONFIG_PACKET_SEG_LEN_MIN*6)

+/**
+ * Maximum packet references.
+ */
+#define ODP_CONFIG_PACKET_REFS 2

???
Can a packet only have two references? Why? Why is there a limit at all?

+
 /** Maximum number of shared memory blocks.
  *
  * This the the number of separate SHM areas that can be reserved
concurrently
diff --git a/include/odp/api/packet.h b/include/odp/api/packet.h
index 5d46b7b..f9745fb 100644
--- a/include/odp/api/packet.h
+++ b/include/odp/api/packet.h
@@ -125,6 +125,15 @@ odp_packet_t
odp_packet_from_event(odp_event_t ev);
  */
 odp_event_t odp_packet_to_event(odp_packet_t pkt);

+/**
+ * Create reference for packet handle
+ *
+ * @param pkt  Packet handle
+ *
+ * @return New packet handle
+ */
+odp_packet_t odp_packet_create_ref(odp_packet_t pkt);
+
 /*
  *
  * Pointers and lengths
--
1.9.1

___
lng-odp mailing list
lng-odp@lists.linaro.org 
https://lists.linaro.org/mailman/listinfo/lng-odp




___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCHv4 1/4] api: packet reference count support

[Ola Liljedahl]

On 9 October 2015 at 08:59, Maxim Uvarov  wrote:

> Add api for packet reference count support. Which is useful in case:
>  - multicast support
>
Different packets will probably need different headers prepended to the
shared packet. How is this handled?

 - TCP retransmission
>
When a TCP packet is retransmitted, the IP id should be changed (similar
for QUIC retransmissions where the SN is updated). So you might have to
modify the packet (header) before you know that the previous transmit has
actually completed. How is this handled?

 - traffic generator
>
A traffic generator might want to include an increasing SN or timestamp in
each packet (packet payload) which otherwise are identical. How is this
handled?


>
> Introduced new call: newpkt = odp_packet_create_ref(pkt) which creates
> reference to original packet, but handles for reference packet and original
> are different.
>
I think something like Bill's "copy-on-write" proposal (from Connect) is
better than explicit reference counting (of the whole packet). Increase the
abstraction level and allow for innovation in the underlying implementation.

Any API support should be based on use case analysis and code examples of
how the use cases are implemented using the suggested API's. Otherwise the
risk of missing the target is significant.


> Signed-off-by: Maxim Uvarov 
> ---
>  include/odp/api/config.h | 5 +
>  include/odp/api/packet.h | 9 +
>  2 files changed, 14 insertions(+)
>
> diff --git a/include/odp/api/config.h b/include/odp/api/config.h
> index 295b10d..985290a 100644
> --- a/include/odp/api/config.h
> +++ b/include/odp/api/config.h
> @@ -126,6 +126,11 @@ extern "C" {
>   */
>  #define ODP_CONFIG_PACKET_BUF_LEN_MAX (ODP_CONFIG_PACKET_SEG_LEN_MIN*6)
>
> +/**
> + * Maximum packet references.
> + */
> +#define ODP_CONFIG_PACKET_REFS 2
>
???
Can a packet only have two references? Why? Why is there a limit at all?



> +
>  /** Maximum number of shared memory blocks.
>   *
>   * This the the number of separate SHM areas that can be reserved
> concurrently
> diff --git a/include/odp/api/packet.h b/include/odp/api/packet.h
> index 5d46b7b..f9745fb 100644
> --- a/include/odp/api/packet.h
> +++ b/include/odp/api/packet.h
> @@ -125,6 +125,15 @@ odp_packet_t odp_packet_from_event(odp_event_t ev);
>   */
>  odp_event_t odp_packet_to_event(odp_packet_t pkt);
>
> +/**
> + * Create reference for packet handle
> + *
> + * @param pkt  Packet handle
> + *
> + * @return New packet handle
> + */
> +odp_packet_t odp_packet_create_ref(odp_packet_t pkt);
> +
>  /*
>   *
>   * Pointers and lengths
> --
> 1.9.1
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [API-NEXT PATCHv4 1/4] api: packet reference count support

[Maxim Uvarov]

Add api for packet reference count support. Which is useful in case:
 - multicast support
 - TCP retransmission
 - traffic generator

Introduced new call: newpkt = odp_packet_create_ref(pkt) which creates
reference to original packet, but handles for reference packet and original
are different.

Signed-off-by: Maxim Uvarov 
---
 include/odp/api/config.h | 5 +
 include/odp/api/packet.h | 9 +
 2 files changed, 14 insertions(+)

diff --git a/include/odp/api/config.h b/include/odp/api/config.h
index 295b10d..985290a 100644
--- a/include/odp/api/config.h
+++ b/include/odp/api/config.h
@@ -126,6 +126,11 @@ extern "C" {
  */
 #define ODP_CONFIG_PACKET_BUF_LEN_MAX (ODP_CONFIG_PACKET_SEG_LEN_MIN*6)
 
+/**
+ * Maximum packet references.
+ */
+#define ODP_CONFIG_PACKET_REFS 2
+
 /** Maximum number of shared memory blocks.
  *
  * This the the number of separate SHM areas that can be reserved concurrently
diff --git a/include/odp/api/packet.h b/include/odp/api/packet.h
index 5d46b7b..f9745fb 100644
--- a/include/odp/api/packet.h
+++ b/include/odp/api/packet.h
@@ -125,6 +125,15 @@ odp_packet_t odp_packet_from_event(odp_event_t ev);
  */
 odp_event_t odp_packet_to_event(odp_packet_t pkt);
 
+/**
+ * Create reference for packet handle
+ *
+ * @param pkt  Packet handle
+ *
+ * @return New packet handle
+ */
+odp_packet_t odp_packet_create_ref(odp_packet_t pkt);
+
 /*
  *
  * Pointers and lengths
-- 
1.9.1

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCHv4 1/4] api: packet reference count support

[Nicolas Morey-Chaisemartin]

On 10/09/2015 02:59 PM, Maxim Uvarov wrote:
> Add api for packet reference count support. Which is useful in case:
>  - multicast support
>  - TCP retransmission
>  - traffic generator
>
> Introduced new call: newpkt = odp_packet_create_ref(pkt) which creates
> reference to original packet, but handles for reference packet and original
> are different.
"May" be different  more than "are". But not a big deal as only in the commit 
message.

>
> Signed-off-by: Maxim Uvarov 
> ---
>  include/odp/api/config.h | 5 +
>  include/odp/api/packet.h | 9 +
>  2 files changed, 14 insertions(+)
>
> diff --git a/include/odp/api/config.h b/include/odp/api/config.h
> index 295b10d..985290a 100644
> --- a/include/odp/api/config.h
> +++ b/include/odp/api/config.h
> @@ -126,6 +126,11 @@ extern "C" {
>   */
>  #define ODP_CONFIG_PACKET_BUF_LEN_MAX (ODP_CONFIG_PACKET_SEG_LEN_MIN*6)
>  
> +/**
> + * Maximum packet references.
> + */
> +#define ODP_CONFIG_PACKET_REFS 2
> +
>  /** Maximum number of shared memory blocks.
>   *
>   * This the the number of separate SHM areas that can be reserved 
> concurrently
> diff --git a/include/odp/api/packet.h b/include/odp/api/packet.h
> index 5d46b7b..f9745fb 100644
> --- a/include/odp/api/packet.h
> +++ b/include/odp/api/packet.h
> @@ -125,6 +125,15 @@ odp_packet_t odp_packet_from_event(odp_event_t ev);
>   */
>  odp_event_t odp_packet_to_event(odp_packet_t pkt);
>  
> +/**
> + * Create reference for packet handle
> + *
> + * @param pkt  Packet handle
> + *
> + * @return New packet handle
> + */
> +odp_packet_t odp_packet_create_ref(odp_packet_t pkt);
> +
>  /*
>   *
>   * Pointers and lengths

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp