Re: [lng-odp] Supporting ODP_PKTIO_OP_MT_SAFE
On Mon, Sep 11, 2017 at 12:27 PM, Honnappa Nagarahalli
wrote:
> On 10 September 2017 at 22:26, Brian Brooks wrote:
>> Honnappa,
>>
>> Could your proposal be simplified to: MT-safe pktio should be
>> deprecated because it is not a common use case. Applications will
>> either use MT-unsafe pktio or the MT-safe scheduler.
>
> Not sure about deprecation. Looks like there is some history to this
> feature, we need to consider all of that.
Agree. We can update usage notes to indicate preferred forms but any
actual deprecations needs to be managed carefully.
>
>>
>>> 1) Polling method - in which one pkt I/O will be created for each receive
>>> worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE is not
>>> required.
>>
>> Absence of MT-safe does not require 1:1 mapping of thread to pktio. It
>> just means that it is the application's responsibility to ensure
>> exclusive access to a pktio.
>>
>>> for high throughput packet I/Os [..] we do not need to support
>>> ODP_PKTIO_OP_MT_SAFE
>>> We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
>>
>> This would introduce an undesirable leaky abstraction.
>
>>
>> BB
>>
>> On Sun, Sep 10, 2017 at 12:40 PM, Bill Fischofer
>> wrote:
>>> We can discuss this during tomorrow's ARCH call, and probably further
>>> at Connect. MT Safe is the default behavior and it's opposite ("MT
>>> Unsafe") was added as a potential optimization when applications
>>> assure implementations that only a single thread will be polling a
>>> PktIn queue or adding to a Pktout queue.
>>>
>>> Ideally, we'd like to retire all application I/O polling and use the
>>> scheduler exclusively, but that's that's a longer-term goal. For now
>>> we have both.
>>>
>>> On Sun, Sep 10, 2017 at 8:11 AM, Honnappa Nagarahalli
>>> wrote:
Hi,
I think there are 2 ways in which pkt I/O will be used:
1) Polling method - in which one pkt I/O will be created for each
receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE
is not required.
2) Event method - the scheduler is used to receive packets. In this
case the scheduler will provide the exclusive access to a pkt I/O.
Again in this case support for ODP_PKTIO_OP_MT_SAFE is not required.
I am thinking, for high throughput packet I/Os such as dpdk or ODP
native drivers (in the future), we do not need to support
ODP_PKTIO_OP_MT_SAFE. The odp_pktio_open API call can return an error
if ODP_PKTIO_OP_MT_SAFE is asked for.
We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
This will save space in cache for the locks as well as instruction cycles.
Thank you,
Honnappa
Re: [lng-odp] Supporting ODP_PKTIO_OP_MT_SAFE
On 10 September 2017 at 22:26, Brian Brooks wrote:
> Honnappa,
>
> Could your proposal be simplified to: MT-safe pktio should be
> deprecated because it is not a common use case. Applications will
> either use MT-unsafe pktio or the MT-safe scheduler.
Not sure about deprecation. Looks like there is some history to this
feature, we need to consider all of that.
>
>> 1) Polling method - in which one pkt I/O will be created for each receive
>> worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE is not
>> required.
>
> Absence of MT-safe does not require 1:1 mapping of thread to pktio. It
> just means that it is the application's responsibility to ensure
> exclusive access to a pktio.
>
>> for high throughput packet I/Os [..] we do not need to support
>> ODP_PKTIO_OP_MT_SAFE
>> We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
>
> This would introduce an undesirable leaky abstraction.
>
> BB
>
> On Sun, Sep 10, 2017 at 12:40 PM, Bill Fischofer
> wrote:
>> We can discuss this during tomorrow's ARCH call, and probably further
>> at Connect. MT Safe is the default behavior and it's opposite ("MT
>> Unsafe") was added as a potential optimization when applications
>> assure implementations that only a single thread will be polling a
>> PktIn queue or adding to a Pktout queue.
>>
>> Ideally, we'd like to retire all application I/O polling and use the
>> scheduler exclusively, but that's that's a longer-term goal. For now
>> we have both.
>>
>> On Sun, Sep 10, 2017 at 8:11 AM, Honnappa Nagarahalli
>> wrote:
>>> Hi,
>>> I think there are 2 ways in which pkt I/O will be used:
>>>
>>> 1) Polling method - in which one pkt I/O will be created for each
>>> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE
>>> is not required.
>>> 2) Event method - the scheduler is used to receive packets. In this
>>> case the scheduler will provide the exclusive access to a pkt I/O.
>>> Again in this case support for ODP_PKTIO_OP_MT_SAFE is not required.
>>>
>>> I am thinking, for high throughput packet I/Os such as dpdk or ODP
>>> native drivers (in the future), we do not need to support
>>> ODP_PKTIO_OP_MT_SAFE. The odp_pktio_open API call can return an error
>>> if ODP_PKTIO_OP_MT_SAFE is asked for.
>>>
>>> We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
>>>
>>> This will save space in cache for the locks as well as instruction cycles.
>>>
>>> Thank you,
>>> Honnappa
Re: [lng-odp] Supporting ODP_PKTIO_OP_MT_SAFE
On Mon, Sep 11, 2017 at 12:22 PM, Honnappa Nagarahalli
wrote:
> We can come up with use cases. My question is what do we expect to get
> deployed? What is required for quick development might be different
> from what is required for deployment.
>
> With NICs supporting multiple queues, will we have a case of lesser
> pkt ins than the number of cores?
Hopefully not as the number of cores needed to achieve line rate
should be fairly low. Once you're at line rate on a pktio, you're
done. Use another pktio if you want to do more.
>
> Thanks,
> Honnappa
>
>
> On 11 September 2017 at 04:55, Maxim Uvarov wrote:
>> On 11 September 2017 at 12:11, Bogdan Pricope
>> wrote:
>>
>>> Hi,
>>>
>>> There is the case where a a pktio has less pktins than available
>>> cores. It is a valid case? We want to support it?
>>> For example: 4 pktins and 8 cores... or default (socket) pktio with
>>> one pktin/one pktout?
>>>
>>> Best regards,
>>> Bogdan
>>>
>>
>> I think - yes, there should not be limitation.
>>
>> Maxim.
>>
>>
>>>
>>> On 11 September 2017 at 11:33, Maxim Uvarov
>>> wrote:
>>> > On 11 September 2017 at 06:26, Brian Brooks
>>> wrote:
>>> >
>>> >> Honnappa,
>>> >>
>>> >> Could your proposal be simplified to: MT-safe pktio should be
>>> >> deprecated because it is not a common use case. Applications will
>>> >> either use MT-unsafe pktio or the MT-safe scheduler.
>>> >>
>>> >> > 1) Polling method - in which one pkt I/O will be created for each
>>> >> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE is
>>> >> not required.
>>> >>
>>> >
>>> >
>>> > That is not always a case. One pktio can be used in several working
>>> > threads. For that case safe is needed.
>>> >
>>> > All modes are:
>>> > /**
>>> > * Packet input mode
>>> > */
>>> > typedef enum odp_pktin_mode_t {
>>> > /** Direct packet input from the interface */
>>> > ODP_PKTIN_MODE_DIRECT = 0,
>>> > /** Packet input through scheduler and scheduled event queues */
>>> > ODP_PKTIN_MODE_SCHED,
>>> > /** Packet input through plain event queues */
>>> > ODP_PKTIN_MODE_QUEUE,
>>> > /** Application will never receive from this interface */
>>> > ODP_PKTIN_MODE_DISABLED
>>> > } odp_pktin_mode_t;
>>> >
>>> > /**
>>> > * Packet output mode
>>> > */
>>> > typedef enum odp_pktout_mode_t {
>>> > /** Direct packet output on the interface */
>>> > ODP_PKTOUT_MODE_DIRECT = 0,
>>> > /** Packet output through event queues */
>>> > ODP_PKTOUT_MODE_QUEUE,
>>> > /** Packet output through traffic manager API */
>>> > ODP_PKTOUT_MODE_TM,
>>> > /** Application will never send to this interface */
>>> > ODP_PKTOUT_MODE_DISABLED
>>> > } odp_pktout_mode_t;
>>> >
>>> >
>>> > For DIRECT, QUEUE and TM implementation can have different optimization
>>> > regrades is the same pktio or in/out queue connected to that pktio used
>>> in
>>> > single thread or shared between threads. Application can not provide
>>> > synchronization in that case because locking should be done on low level
>>> > for some short period of time. Locking ODP calls will significantly slow
>>> > down data path functions.
>>> >
>>> > Best regards,
>>> > Maxim.
>>> >
>>> >
>>> >
>>> >
>>> >>
>>> >> Absence of MT-safe does not require 1:1 mapping of thread to pktio. It
>>> >> just means that it is the application's responsibility to ensure
>>> >> exclusive access to a pktio.
>>> >>
>>> >> > for high throughput packet I/Os [..] we do not need to support
>>> >> ODP_PKTIO_OP_MT_SAFE
>>> >> > We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
>>> >>
>>> >> This would introduce an undesirable leaky abstraction.
>>> >>
>>> >> BB
>>> >>
>>> >> On Sun, Sep 10, 2017 at 12:40 PM, Bill Fischofer
>>> >> wrote:
>>> >> > We can discuss this during tomorrow's ARCH call, and probably further
>>> >> > at Connect. MT Safe is the default behavior and it's opposite ("MT
>>> >> > Unsafe") was added as a potential optimization when applications
>>> >> > assure implementations that only a single thread will be polling a
>>> >> > PktIn queue or adding to a Pktout queue.
>>> >> >
>>> >> > Ideally, we'd like to retire all application I/O polling and use the
>>> >> > scheduler exclusively, but that's that's a longer-term goal. For now
>>> >> > we have both.
>>> >> >
>>> >> > On Sun, Sep 10, 2017 at 8:11 AM, Honnappa Nagarahalli
>>> >> > wrote:
>>> >> >> Hi,
>>> >> >> I think there are 2 ways in which pkt I/O will be used:
>>> >> >>
>>> >> >> 1) Polling method - in which one pkt I/O will be created for each
>>> >> >> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE
>>> >> >> is not required.
>>> >> >> 2) Event method - the scheduler is used to receive packets. In this
>>> >> >> case the scheduler will provide the exclusive access to a pkt I/O.
>>> >> >> Again in this case support for ODP_PKTIO_OP_MT_SAFE is not required.
>>> >> >>
>>> >> >> I am think
Re: [lng-odp] Supporting ODP_PKTIO_OP_MT_SAFE
We can come up with use cases. My question is what do we expect to get
deployed? What is required for quick development might be different
from what is required for deployment.
With NICs supporting multiple queues, will we have a case of lesser
pkt ins than the number of cores?
Thanks,
Honnappa
On 11 September 2017 at 04:55, Maxim Uvarov wrote:
> On 11 September 2017 at 12:11, Bogdan Pricope
> wrote:
>
>> Hi,
>>
>> There is the case where a a pktio has less pktins than available
>> cores. It is a valid case? We want to support it?
>> For example: 4 pktins and 8 cores... or default (socket) pktio with
>> one pktin/one pktout?
>>
>> Best regards,
>> Bogdan
>>
>
> I think - yes, there should not be limitation.
>
> Maxim.
>
>
>>
>> On 11 September 2017 at 11:33, Maxim Uvarov
>> wrote:
>> > On 11 September 2017 at 06:26, Brian Brooks
>> wrote:
>> >
>> >> Honnappa,
>> >>
>> >> Could your proposal be simplified to: MT-safe pktio should be
>> >> deprecated because it is not a common use case. Applications will
>> >> either use MT-unsafe pktio or the MT-safe scheduler.
>> >>
>> >> > 1) Polling method - in which one pkt I/O will be created for each
>> >> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE is
>> >> not required.
>> >>
>> >
>> >
>> > That is not always a case. One pktio can be used in several working
>> > threads. For that case safe is needed.
>> >
>> > All modes are:
>> > /**
>> > * Packet input mode
>> > */
>> > typedef enum odp_pktin_mode_t {
>> > /** Direct packet input from the interface */
>> > ODP_PKTIN_MODE_DIRECT = 0,
>> > /** Packet input through scheduler and scheduled event queues */
>> > ODP_PKTIN_MODE_SCHED,
>> > /** Packet input through plain event queues */
>> > ODP_PKTIN_MODE_QUEUE,
>> > /** Application will never receive from this interface */
>> > ODP_PKTIN_MODE_DISABLED
>> > } odp_pktin_mode_t;
>> >
>> > /**
>> > * Packet output mode
>> > */
>> > typedef enum odp_pktout_mode_t {
>> > /** Direct packet output on the interface */
>> > ODP_PKTOUT_MODE_DIRECT = 0,
>> > /** Packet output through event queues */
>> > ODP_PKTOUT_MODE_QUEUE,
>> > /** Packet output through traffic manager API */
>> > ODP_PKTOUT_MODE_TM,
>> > /** Application will never send to this interface */
>> > ODP_PKTOUT_MODE_DISABLED
>> > } odp_pktout_mode_t;
>> >
>> >
>> > For DIRECT, QUEUE and TM implementation can have different optimization
>> > regrades is the same pktio or in/out queue connected to that pktio used
>> in
>> > single thread or shared between threads. Application can not provide
>> > synchronization in that case because locking should be done on low level
>> > for some short period of time. Locking ODP calls will significantly slow
>> > down data path functions.
>> >
>> > Best regards,
>> > Maxim.
>> >
>> >
>> >
>> >
>> >>
>> >> Absence of MT-safe does not require 1:1 mapping of thread to pktio. It
>> >> just means that it is the application's responsibility to ensure
>> >> exclusive access to a pktio.
>> >>
>> >> > for high throughput packet I/Os [..] we do not need to support
>> >> ODP_PKTIO_OP_MT_SAFE
>> >> > We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
>> >>
>> >> This would introduce an undesirable leaky abstraction.
>> >>
>> >> BB
>> >>
>> >> On Sun, Sep 10, 2017 at 12:40 PM, Bill Fischofer
>> >> wrote:
>> >> > We can discuss this during tomorrow's ARCH call, and probably further
>> >> > at Connect. MT Safe is the default behavior and it's opposite ("MT
>> >> > Unsafe") was added as a potential optimization when applications
>> >> > assure implementations that only a single thread will be polling a
>> >> > PktIn queue or adding to a Pktout queue.
>> >> >
>> >> > Ideally, we'd like to retire all application I/O polling and use the
>> >> > scheduler exclusively, but that's that's a longer-term goal. For now
>> >> > we have both.
>> >> >
>> >> > On Sun, Sep 10, 2017 at 8:11 AM, Honnappa Nagarahalli
>> >> > wrote:
>> >> >> Hi,
>> >> >> I think there are 2 ways in which pkt I/O will be used:
>> >> >>
>> >> >> 1) Polling method - in which one pkt I/O will be created for each
>> >> >> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE
>> >> >> is not required.
>> >> >> 2) Event method - the scheduler is used to receive packets. In this
>> >> >> case the scheduler will provide the exclusive access to a pkt I/O.
>> >> >> Again in this case support for ODP_PKTIO_OP_MT_SAFE is not required.
>> >> >>
>> >> >> I am thinking, for high throughput packet I/Os such as dpdk or ODP
>> >> >> native drivers (in the future), we do not need to support
>> >> >> ODP_PKTIO_OP_MT_SAFE. The odp_pktio_open API call can return an error
>> >> >> if ODP_PKTIO_OP_MT_SAFE is asked for.
>> >> >>
>> >> >> We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt
>> I/Os.
>> >> >>
>> >> >> This will save space in ca
Re: [lng-odp] Supporting ODP_PKTIO_OP_MT_SAFE
On 11 September 2017 at 12:11, Bogdan Pricope
wrote:
> Hi,
>
> There is the case where a a pktio has less pktins than available
> cores. It is a valid case? We want to support it?
> For example: 4 pktins and 8 cores... or default (socket) pktio with
> one pktin/one pktout?
>
> Best regards,
> Bogdan
>
I think - yes, there should not be limitation.
Maxim.
>
> On 11 September 2017 at 11:33, Maxim Uvarov
> wrote:
> > On 11 September 2017 at 06:26, Brian Brooks
> wrote:
> >
> >> Honnappa,
> >>
> >> Could your proposal be simplified to: MT-safe pktio should be
> >> deprecated because it is not a common use case. Applications will
> >> either use MT-unsafe pktio or the MT-safe scheduler.
> >>
> >> > 1) Polling method - in which one pkt I/O will be created for each
> >> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE is
> >> not required.
> >>
> >
> >
> > That is not always a case. One pktio can be used in several working
> > threads. For that case safe is needed.
> >
> > All modes are:
> > /**
> > * Packet input mode
> > */
> > typedef enum odp_pktin_mode_t {
> > /** Direct packet input from the interface */
> > ODP_PKTIN_MODE_DIRECT = 0,
> > /** Packet input through scheduler and scheduled event queues */
> > ODP_PKTIN_MODE_SCHED,
> > /** Packet input through plain event queues */
> > ODP_PKTIN_MODE_QUEUE,
> > /** Application will never receive from this interface */
> > ODP_PKTIN_MODE_DISABLED
> > } odp_pktin_mode_t;
> >
> > /**
> > * Packet output mode
> > */
> > typedef enum odp_pktout_mode_t {
> > /** Direct packet output on the interface */
> > ODP_PKTOUT_MODE_DIRECT = 0,
> > /** Packet output through event queues */
> > ODP_PKTOUT_MODE_QUEUE,
> > /** Packet output through traffic manager API */
> > ODP_PKTOUT_MODE_TM,
> > /** Application will never send to this interface */
> > ODP_PKTOUT_MODE_DISABLED
> > } odp_pktout_mode_t;
> >
> >
> > For DIRECT, QUEUE and TM implementation can have different optimization
> > regrades is the same pktio or in/out queue connected to that pktio used
> in
> > single thread or shared between threads. Application can not provide
> > synchronization in that case because locking should be done on low level
> > for some short period of time. Locking ODP calls will significantly slow
> > down data path functions.
> >
> > Best regards,
> > Maxim.
> >
> >
> >
> >
> >>
> >> Absence of MT-safe does not require 1:1 mapping of thread to pktio. It
> >> just means that it is the application's responsibility to ensure
> >> exclusive access to a pktio.
> >>
> >> > for high throughput packet I/Os [..] we do not need to support
> >> ODP_PKTIO_OP_MT_SAFE
> >> > We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
> >>
> >> This would introduce an undesirable leaky abstraction.
> >>
> >> BB
> >>
> >> On Sun, Sep 10, 2017 at 12:40 PM, Bill Fischofer
> >> wrote:
> >> > We can discuss this during tomorrow's ARCH call, and probably further
> >> > at Connect. MT Safe is the default behavior and it's opposite ("MT
> >> > Unsafe") was added as a potential optimization when applications
> >> > assure implementations that only a single thread will be polling a
> >> > PktIn queue or adding to a Pktout queue.
> >> >
> >> > Ideally, we'd like to retire all application I/O polling and use the
> >> > scheduler exclusively, but that's that's a longer-term goal. For now
> >> > we have both.
> >> >
> >> > On Sun, Sep 10, 2017 at 8:11 AM, Honnappa Nagarahalli
> >> > wrote:
> >> >> Hi,
> >> >> I think there are 2 ways in which pkt I/O will be used:
> >> >>
> >> >> 1) Polling method - in which one pkt I/O will be created for each
> >> >> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE
> >> >> is not required.
> >> >> 2) Event method - the scheduler is used to receive packets. In this
> >> >> case the scheduler will provide the exclusive access to a pkt I/O.
> >> >> Again in this case support for ODP_PKTIO_OP_MT_SAFE is not required.
> >> >>
> >> >> I am thinking, for high throughput packet I/Os such as dpdk or ODP
> >> >> native drivers (in the future), we do not need to support
> >> >> ODP_PKTIO_OP_MT_SAFE. The odp_pktio_open API call can return an error
> >> >> if ODP_PKTIO_OP_MT_SAFE is asked for.
> >> >>
> >> >> We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt
> I/Os.
> >> >>
> >> >> This will save space in cache for the locks as well as instruction
> >> cycles.
> >> >>
> >> >> Thank you,
> >> >> Honnappa
> >>
>
Re: [lng-odp] Supporting ODP_PKTIO_OP_MT_SAFE
Hi,
There is the case where a a pktio has less pktins than available
cores. It is a valid case? We want to support it?
For example: 4 pktins and 8 cores... or default (socket) pktio with
one pktin/one pktout?
Best regards,
Bogdan
On 11 September 2017 at 11:33, Maxim Uvarov wrote:
> On 11 September 2017 at 06:26, Brian Brooks wrote:
>
>> Honnappa,
>>
>> Could your proposal be simplified to: MT-safe pktio should be
>> deprecated because it is not a common use case. Applications will
>> either use MT-unsafe pktio or the MT-safe scheduler.
>>
>> > 1) Polling method - in which one pkt I/O will be created for each
>> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE is
>> not required.
>>
>
>
> That is not always a case. One pktio can be used in several working
> threads. For that case safe is needed.
>
> All modes are:
> /**
> * Packet input mode
> */
> typedef enum odp_pktin_mode_t {
> /** Direct packet input from the interface */
> ODP_PKTIN_MODE_DIRECT = 0,
> /** Packet input through scheduler and scheduled event queues */
> ODP_PKTIN_MODE_SCHED,
> /** Packet input through plain event queues */
> ODP_PKTIN_MODE_QUEUE,
> /** Application will never receive from this interface */
> ODP_PKTIN_MODE_DISABLED
> } odp_pktin_mode_t;
>
> /**
> * Packet output mode
> */
> typedef enum odp_pktout_mode_t {
> /** Direct packet output on the interface */
> ODP_PKTOUT_MODE_DIRECT = 0,
> /** Packet output through event queues */
> ODP_PKTOUT_MODE_QUEUE,
> /** Packet output through traffic manager API */
> ODP_PKTOUT_MODE_TM,
> /** Application will never send to this interface */
> ODP_PKTOUT_MODE_DISABLED
> } odp_pktout_mode_t;
>
>
> For DIRECT, QUEUE and TM implementation can have different optimization
> regrades is the same pktio or in/out queue connected to that pktio used in
> single thread or shared between threads. Application can not provide
> synchronization in that case because locking should be done on low level
> for some short period of time. Locking ODP calls will significantly slow
> down data path functions.
>
> Best regards,
> Maxim.
>
>
>
>
>>
>> Absence of MT-safe does not require 1:1 mapping of thread to pktio. It
>> just means that it is the application's responsibility to ensure
>> exclusive access to a pktio.
>>
>> > for high throughput packet I/Os [..] we do not need to support
>> ODP_PKTIO_OP_MT_SAFE
>> > We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
>>
>> This would introduce an undesirable leaky abstraction.
>>
>> BB
>>
>> On Sun, Sep 10, 2017 at 12:40 PM, Bill Fischofer
>> wrote:
>> > We can discuss this during tomorrow's ARCH call, and probably further
>> > at Connect. MT Safe is the default behavior and it's opposite ("MT
>> > Unsafe") was added as a potential optimization when applications
>> > assure implementations that only a single thread will be polling a
>> > PktIn queue or adding to a Pktout queue.
>> >
>> > Ideally, we'd like to retire all application I/O polling and use the
>> > scheduler exclusively, but that's that's a longer-term goal. For now
>> > we have both.
>> >
>> > On Sun, Sep 10, 2017 at 8:11 AM, Honnappa Nagarahalli
>> > wrote:
>> >> Hi,
>> >> I think there are 2 ways in which pkt I/O will be used:
>> >>
>> >> 1) Polling method - in which one pkt I/O will be created for each
>> >> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE
>> >> is not required.
>> >> 2) Event method - the scheduler is used to receive packets. In this
>> >> case the scheduler will provide the exclusive access to a pkt I/O.
>> >> Again in this case support for ODP_PKTIO_OP_MT_SAFE is not required.
>> >>
>> >> I am thinking, for high throughput packet I/Os such as dpdk or ODP
>> >> native drivers (in the future), we do not need to support
>> >> ODP_PKTIO_OP_MT_SAFE. The odp_pktio_open API call can return an error
>> >> if ODP_PKTIO_OP_MT_SAFE is asked for.
>> >>
>> >> We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
>> >>
>> >> This will save space in cache for the locks as well as instruction
>> cycles.
>> >>
>> >> Thank you,
>> >> Honnappa
>>
Re: [lng-odp] Supporting ODP_PKTIO_OP_MT_SAFE
On 11 September 2017 at 06:26, Brian Brooks wrote:
> Honnappa,
>
> Could your proposal be simplified to: MT-safe pktio should be
> deprecated because it is not a common use case. Applications will
> either use MT-unsafe pktio or the MT-safe scheduler.
>
> > 1) Polling method - in which one pkt I/O will be created for each
> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE is
> not required.
>
That is not always a case. One pktio can be used in several working
threads. For that case safe is needed.
All modes are:
/**
* Packet input mode
*/
typedef enum odp_pktin_mode_t {
/** Direct packet input from the interface */
ODP_PKTIN_MODE_DIRECT = 0,
/** Packet input through scheduler and scheduled event queues */
ODP_PKTIN_MODE_SCHED,
/** Packet input through plain event queues */
ODP_PKTIN_MODE_QUEUE,
/** Application will never receive from this interface */
ODP_PKTIN_MODE_DISABLED
} odp_pktin_mode_t;
/**
* Packet output mode
*/
typedef enum odp_pktout_mode_t {
/** Direct packet output on the interface */
ODP_PKTOUT_MODE_DIRECT = 0,
/** Packet output through event queues */
ODP_PKTOUT_MODE_QUEUE,
/** Packet output through traffic manager API */
ODP_PKTOUT_MODE_TM,
/** Application will never send to this interface */
ODP_PKTOUT_MODE_DISABLED
} odp_pktout_mode_t;
For DIRECT, QUEUE and TM implementation can have different optimization
regrades is the same pktio or in/out queue connected to that pktio used in
single thread or shared between threads. Application can not provide
synchronization in that case because locking should be done on low level
for some short period of time. Locking ODP calls will significantly slow
down data path functions.
Best regards,
Maxim.
>
> Absence of MT-safe does not require 1:1 mapping of thread to pktio. It
> just means that it is the application's responsibility to ensure
> exclusive access to a pktio.
>
> > for high throughput packet I/Os [..] we do not need to support
> ODP_PKTIO_OP_MT_SAFE
> > We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
>
> This would introduce an undesirable leaky abstraction.
>
> BB
>
> On Sun, Sep 10, 2017 at 12:40 PM, Bill Fischofer
> wrote:
> > We can discuss this during tomorrow's ARCH call, and probably further
> > at Connect. MT Safe is the default behavior and it's opposite ("MT
> > Unsafe") was added as a potential optimization when applications
> > assure implementations that only a single thread will be polling a
> > PktIn queue or adding to a Pktout queue.
> >
> > Ideally, we'd like to retire all application I/O polling and use the
> > scheduler exclusively, but that's that's a longer-term goal. For now
> > we have both.
> >
> > On Sun, Sep 10, 2017 at 8:11 AM, Honnappa Nagarahalli
> > wrote:
> >> Hi,
> >> I think there are 2 ways in which pkt I/O will be used:
> >>
> >> 1) Polling method - in which one pkt I/O will be created for each
> >> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE
> >> is not required.
> >> 2) Event method - the scheduler is used to receive packets. In this
> >> case the scheduler will provide the exclusive access to a pkt I/O.
> >> Again in this case support for ODP_PKTIO_OP_MT_SAFE is not required.
> >>
> >> I am thinking, for high throughput packet I/Os such as dpdk or ODP
> >> native drivers (in the future), we do not need to support
> >> ODP_PKTIO_OP_MT_SAFE. The odp_pktio_open API call can return an error
> >> if ODP_PKTIO_OP_MT_SAFE is asked for.
> >>
> >> We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
> >>
> >> This will save space in cache for the locks as well as instruction
> cycles.
> >>
> >> Thank you,
> >> Honnappa
>
Re: [lng-odp] Supporting ODP_PKTIO_OP_MT_SAFE
Honnappa,
Could your proposal be simplified to: MT-safe pktio should be
deprecated because it is not a common use case. Applications will
either use MT-unsafe pktio or the MT-safe scheduler.
> 1) Polling method - in which one pkt I/O will be created for each receive
> worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE is not required.
Absence of MT-safe does not require 1:1 mapping of thread to pktio. It
just means that it is the application's responsibility to ensure
exclusive access to a pktio.
> for high throughput packet I/Os [..] we do not need to support
> ODP_PKTIO_OP_MT_SAFE
> We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
This would introduce an undesirable leaky abstraction.
BB
On Sun, Sep 10, 2017 at 12:40 PM, Bill Fischofer
wrote:
> We can discuss this during tomorrow's ARCH call, and probably further
> at Connect. MT Safe is the default behavior and it's opposite ("MT
> Unsafe") was added as a potential optimization when applications
> assure implementations that only a single thread will be polling a
> PktIn queue or adding to a Pktout queue.
>
> Ideally, we'd like to retire all application I/O polling and use the
> scheduler exclusively, but that's that's a longer-term goal. For now
> we have both.
>
> On Sun, Sep 10, 2017 at 8:11 AM, Honnappa Nagarahalli
> wrote:
>> Hi,
>> I think there are 2 ways in which pkt I/O will be used:
>>
>> 1) Polling method - in which one pkt I/O will be created for each
>> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE
>> is not required.
>> 2) Event method - the scheduler is used to receive packets. In this
>> case the scheduler will provide the exclusive access to a pkt I/O.
>> Again in this case support for ODP_PKTIO_OP_MT_SAFE is not required.
>>
>> I am thinking, for high throughput packet I/Os such as dpdk or ODP
>> native drivers (in the future), we do not need to support
>> ODP_PKTIO_OP_MT_SAFE. The odp_pktio_open API call can return an error
>> if ODP_PKTIO_OP_MT_SAFE is asked for.
>>
>> We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
>>
>> This will save space in cache for the locks as well as instruction cycles.
>>
>> Thank you,
>> Honnappa
Re: [lng-odp] Supporting ODP_PKTIO_OP_MT_SAFE
We can discuss this during tomorrow's ARCH call, and probably further
at Connect. MT Safe is the default behavior and it's opposite ("MT
Unsafe") was added as a potential optimization when applications
assure implementations that only a single thread will be polling a
PktIn queue or adding to a Pktout queue.
Ideally, we'd like to retire all application I/O polling and use the
scheduler exclusively, but that's that's a longer-term goal. For now
we have both.
On Sun, Sep 10, 2017 at 8:11 AM, Honnappa Nagarahalli
wrote:
> Hi,
> I think there are 2 ways in which pkt I/O will be used:
>
> 1) Polling method - in which one pkt I/O will be created for each
> receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE
> is not required.
> 2) Event method - the scheduler is used to receive packets. In this
> case the scheduler will provide the exclusive access to a pkt I/O.
> Again in this case support for ODP_PKTIO_OP_MT_SAFE is not required.
>
> I am thinking, for high throughput packet I/Os such as dpdk or ODP
> native drivers (in the future), we do not need to support
> ODP_PKTIO_OP_MT_SAFE. The odp_pktio_open API call can return an error
> if ODP_PKTIO_OP_MT_SAFE is asked for.
>
> We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os.
>
> This will save space in cache for the locks as well as instruction cycles.
>
> Thank you,
> Honnappa
[lng-odp] Supporting ODP_PKTIO_OP_MT_SAFE
Hi, I think there are 2 ways in which pkt I/O will be used: 1) Polling method - in which one pkt I/O will be created for each receive worker thread. In this case, support for ODP_PKTIO_OP_MT_SAFE is not required. 2) Event method - the scheduler is used to receive packets. In this case the scheduler will provide the exclusive access to a pkt I/O. Again in this case support for ODP_PKTIO_OP_MT_SAFE is not required. I am thinking, for high throughput packet I/Os such as dpdk or ODP native drivers (in the future), we do not need to support ODP_PKTIO_OP_MT_SAFE. The odp_pktio_open API call can return an error if ODP_PKTIO_OP_MT_SAFE is asked for. We could keep the support for ODP_PKTIO_OP_MT_SAFE for other pkt I/Os. This will save space in cache for the locks as well as instruction cycles. Thank you, Honnappa
