Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-12-08 Thread Pavel Begunkov
On 11/6/23 22:55, Willem de Bruijn wrote: On Mon, Nov 6, 2023 at 2:34 PM Stanislav Fomichev wrote: On 11/06, Willem de Bruijn wrote: IMHO, we need a better UAPI to receive the tokens and give them back to the kernel. CMSG + setsockopt(SO_DEVMEM_DONTNEED) get the job done, but look dated and

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-12-08 Thread Pavel Begunkov
On 11/9/23 16:07, Edward Cree wrote: On 09/11/2023 02:39, Mina Almasry wrote: On Wed, Nov 8, 2023 at 7:36 AM Edward Cree wrote: If not then surely the way to return a memory area in an io_uring idiom is just to post a new read sqe ('RX descriptor') pointing into it, rather than

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-12-08 Thread Pavel Begunkov
On 11/6/23 22:34, Stanislav Fomichev wrote: On 11/06, Willem de Bruijn wrote: IMHO, we need a better UAPI to receive the tokens and give them back to the kernel. CMSG + setsockopt(SO_DEVMEM_DONTNEED) get the job done, but look dated and hacky :-( We should either do some kind of user/kernel

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-10 Thread Jakub Kicinski
On Sun, 5 Nov 2023 18:44:09 -0800 Mina Almasry wrote: > + if (!skb_frags_not_readable(skb)) { nit: maybe call the helper skb_frags_readable() and flip the polarity, avoid double negatives.

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-10 Thread Jakub Kicinski
On Thu, 09 Nov 2023 12:05:37 +0100 Paolo Abeni wrote: > > I suppose we just disagree on the elegance of the API. > > FWIW, I think sockopt +cmsg is the right API. FWIW it's fine by me as well.

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-09 Thread Edward Cree
On 09/11/2023 02:39, Mina Almasry wrote: > On Wed, Nov 8, 2023 at 7:36 AM Edward Cree wrote: >> If not then surely the way to return a memory area >> in an io_uring idiom is just to post a new read sqe ('RX descriptor') >> pointing into it, rather than explicitly returning it with setsockopt.

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-09 Thread Paolo Abeni
On Mon, 2023-11-06 at 14:55 -0800, Willem de Bruijn wrote: > On Mon, Nov 6, 2023 at 2:34 PM Stanislav Fomichev wrote: > > > > On 11/06, Willem de Bruijn wrote: > > > > > IMHO, we need a better UAPI to receive the tokens and give them back > > > > > to > > > > > the kernel. CMSG +

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-09 Thread Paolo Abeni
On Sun, 2023-11-05 at 18:44 -0800, Mina Almasry wrote: [...] > +/* On error, returns the -errno. On success, returns number of bytes sent to > the > + * user. May not consume all of @remaining_len. > + */ > +static int tcp_recvmsg_devmem(const struct sock *sk, const struct sk_buff > *skb, > +

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-08 Thread Mina Almasry
On Tue, Nov 7, 2023 at 4:01 PM David Ahern wrote: > > On 11/7/23 4:55 PM, Mina Almasry wrote: > > On Mon, Nov 6, 2023 at 4:03 PM Willem de Bruijn > > wrote: > >> > >> On Mon, Nov 6, 2023 at 3:55 PM David Ahern wrote: > >>> > >>> On 11/6/23 4:32 PM, Stanislav Fomichev wrote: > > The concise

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-08 Thread Edward Cree
On 06/11/2023 21:17, Stanislav Fomichev wrote: > I guess I'm just wondering whether other people have any suggestions > here. Not sure Jonathan's way was better, but we fundamentally > have two queues between the kernel and the userspace: > - userspace receiving tokens (recvmsg + magical flag) > -

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-07 Thread David Ahern
On 11/7/23 4:55 PM, Mina Almasry wrote: > On Mon, Nov 6, 2023 at 4:03 PM Willem de Bruijn > wrote: >> >> On Mon, Nov 6, 2023 at 3:55 PM David Ahern wrote: >>> >>> On 11/6/23 4:32 PM, Stanislav Fomichev wrote: > The concise notification API returns tokens as a range for > compression,

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-07 Thread Mina Almasry
On Mon, Nov 6, 2023 at 4:03 PM Willem de Bruijn wrote: > > On Mon, Nov 6, 2023 at 3:55 PM David Ahern wrote: > > > > On 11/6/23 4:32 PM, Stanislav Fomichev wrote: > > >> The concise notification API returns tokens as a range for > > >> compression, encoding as two 32-bit unsigned integers start

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-06 Thread Willem de Bruijn
On Mon, Nov 6, 2023 at 3:55 PM David Ahern wrote: > > On 11/6/23 4:32 PM, Stanislav Fomichev wrote: > >> The concise notification API returns tokens as a range for > >> compression, encoding as two 32-bit unsigned integers start + length. > >> It allows for even further batching by returning

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-06 Thread David Ahern
On 11/6/23 4:32 PM, Stanislav Fomichev wrote: >> The concise notification API returns tokens as a range for >> compression, encoding as two 32-bit unsigned integers start + length. >> It allows for even further batching by returning multiple such ranges >> in a single call. > > Tangential: should

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-06 Thread Stanislav Fomichev
On Mon, Nov 6, 2023 at 2:56 PM Willem de Bruijn wrote: > > On Mon, Nov 6, 2023 at 2:34 PM Stanislav Fomichev wrote: > > > > On 11/06, Willem de Bruijn wrote: > > > > > IMHO, we need a better UAPI to receive the tokens and give them back > > > > > to > > > > > the kernel. CMSG +

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-06 Thread Willem de Bruijn
On Mon, Nov 6, 2023 at 2:34 PM Stanislav Fomichev wrote: > > On 11/06, Willem de Bruijn wrote: > > > > IMHO, we need a better UAPI to receive the tokens and give them back to > > > > the kernel. CMSG + setsockopt(SO_DEVMEM_DONTNEED) get the job done, > > > > but look dated and hacky :-( > > > > >

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-06 Thread Stanislav Fomichev
On 11/06, Willem de Bruijn wrote: > > > IMHO, we need a better UAPI to receive the tokens and give them back to > > > the kernel. CMSG + setsockopt(SO_DEVMEM_DONTNEED) get the job done, > > > but look dated and hacky :-( > > > > > > We should either do some kind of user/kernel shared memory queue

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-06 Thread Stanislav Fomichev
On 11/06, Mina Almasry wrote: > On Mon, Nov 6, 2023 at 10:44 AM Stanislav Fomichev wrote: > > > > On 11/05, Mina Almasry wrote: > > > In tcp_recvmsg_locked(), detect if the skb being received by the user > > > is a devmem skb. In this case - if the user provided the MSG_SOCK_DEVMEM > > > flag -

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-06 Thread Willem de Bruijn
> > IMHO, we need a better UAPI to receive the tokens and give them back to > > the kernel. CMSG + setsockopt(SO_DEVMEM_DONTNEED) get the job done, > > but look dated and hacky :-( > > > > We should either do some kind of user/kernel shared memory queue to > > receive/return the tokens (similar to

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-06 Thread Mina Almasry
On Mon, Nov 6, 2023 at 10:44 AM Stanislav Fomichev wrote: > > On 11/05, Mina Almasry wrote: > > In tcp_recvmsg_locked(), detect if the skb being received by the user > > is a devmem skb. In this case - if the user provided the MSG_SOCK_DEVMEM > > flag - pass it to tcp_recvmsg_devmem() for custom

Re: [RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-06 Thread Stanislav Fomichev
On 11/05, Mina Almasry wrote: > In tcp_recvmsg_locked(), detect if the skb being received by the user > is a devmem skb. In this case - if the user provided the MSG_SOCK_DEVMEM > flag - pass it to tcp_recvmsg_devmem() for custom handling. > > tcp_recvmsg_devmem() copies any data in the skb header

[RFC PATCH v3 10/12] tcp: RX path for devmem TCP

2023-11-05 Thread Mina Almasry
In tcp_recvmsg_locked(), detect if the skb being received by the user is a devmem skb. In this case - if the user provided the MSG_SOCK_DEVMEM flag - pass it to tcp_recvmsg_devmem() for custom handling. tcp_recvmsg_devmem() copies any data in the skb header to the linear buffer, and returns a