Re: [PATCH V5 net-next 06/14] net/tls: Add generic NIC offload infrastructure
Hi Kirill, On 3/28/2018 6:11 PM, Kirill Tkhai wrote: On 28.03.2018 02:56, Saeed Mahameed wrote: From: Ilya LesokhinThis patch adds a generic infrastructure to offload TLS crypto to a network device. It enables the kernel TLS socket to skip encryption and authentication operations on the transmit side of the data path. Leaving those computationally expensive operations to the NIC. The NIC offload infrastructure builds TLS records and pushes them to the TCP layer just like the SW KTLS implementation and using the same API. TCP segmentation is mostly unaffected. Currently the only exception is that we prevent mixed SKBs where only part of the payload requires offload. In the future we are likely to add a similar restriction following a change cipher spec record. The notable differences between SW KTLS and NIC offloaded TLS implementations are as follows: 1. The offloaded implementation builds "plaintext TLS record", those records contain plaintext instead of ciphertext and place holder bytes instead of authentication tags. 2. The offloaded implementation maintains a mapping from TCP sequence number to TLS records. Thus given a TCP SKB sent from a NIC offloaded TLS socket, we can use the tls NIC offload infrastructure to obtain enough context to encrypt the payload of the SKB. A TLS record is released when the last byte of the record is ack'ed, this is done through the new icsk_clean_acked callback. The infrastructure should be extendable to support various NIC offload implementations. However it is currently written with the implementation below in mind: The NIC assumes that packets from each offloaded stream are sent as plaintext and in-order. It keeps track of the TLS records in the TCP stream. When a packet marked for offload is transmitted, the NIC encrypts the payload in-place and puts authentication tags in the relevant place holders. The responsibility for handling out-of-order packets (i.e. TCP retransmission, qdisc drops) falls on the netdev driver. The netdev driver keeps track of the expected TCP SN from the NIC's perspective. If the next packet to transmit matches the expected TCP SN, the driver advances the expected TCP SN, and transmits the packet with TLS offload indication. If the next packet to transmit does not match the expected TCP SN. The driver calls the TLS layer to obtain the TLS record that includes the TCP of the packet for transmission. Using this TLS record, the driver posts a work entry on the transmit queue to reconstruct the NIC TLS state required for the offload of the out-of-order packet. It updates the expected TCP SN accordingly and transmits the now in-order packet. The same queue is used for packet transmission and TLS context reconstruction to avoid the need for flushing the transmit queue before issuing the context reconstruction request. Signed-off-by: Ilya Lesokhin Signed-off-by: Boris Pismenny Signed-off-by: Aviad Yehezkel Signed-off-by: Saeed Mahameed --- include/net/tls.h | 120 +-- net/tls/Kconfig | 10 + net/tls/Makefile | 2 + net/tls/tls_device.c | 759 ++ net/tls/tls_device_fallback.c | 454 + net/tls/tls_main.c| 120 --- net/tls/tls_sw.c | 132 7 files changed, 1476 insertions(+), 121 deletions(-) create mode 100644 net/tls/tls_device.c create mode 100644 net/tls/tls_device_fallback.c diff --git a/include/net/tls.h b/include/net/tls.h index 437a746300bf..0a8529e9ec21 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -57,21 +57,10 @@ #define TLS_AAD_SPACE_SIZE 13 -struct tls_sw_context { +struct tls_sw_context_tx { What the reason splitting this into tx + rx does not go in separate patch? Added a separate patch for this in V6. struct crypto_aead *aead_send; - struct crypto_aead *aead_recv; struct crypto_wait async_wait; - /* Receive context */ - struct strparser strp; - void (*saved_data_ready)(struct sock *sk); - unsigned int (*sk_poll)(struct file *file, struct socket *sock, - struct poll_table_struct *wait); - struct sk_buff *recv_pkt; - u8 control; - bool decrypted; - - /* Sending context */ char aad_space[TLS_AAD_SPACE_SIZE]; unsigned int sg_plaintext_size; @@ -88,6 +77,50 @@ struct tls_sw_context { struct scatterlist sg_aead_out[2]; }; +struct tls_sw_context_rx { + struct crypto_aead *aead_recv; + struct crypto_wait async_wait; + + struct strparser strp; + void (*saved_data_ready)(struct sock *sk); + unsigned int (*sk_poll)(struct file *file, struct socket *sock, + struct poll_table_struct *wait); + struct sk_buff *recv_pkt; + u8 control; +
Re: [PATCH V5 net-next 06/14] net/tls: Add generic NIC offload infrastructure
On 28.03.2018 02:56, Saeed Mahameed wrote: > From: Ilya Lesokhin> > This patch adds a generic infrastructure to offload TLS crypto to a > network device. It enables the kernel TLS socket to skip encryption > and authentication operations on the transmit side of the data path. > Leaving those computationally expensive operations to the NIC. > > The NIC offload infrastructure builds TLS records and pushes them to > the TCP layer just like the SW KTLS implementation and using the same API. > TCP segmentation is mostly unaffected. Currently the only exception is > that we prevent mixed SKBs where only part of the payload requires > offload. In the future we are likely to add a similar restriction > following a change cipher spec record. > > The notable differences between SW KTLS and NIC offloaded TLS > implementations are as follows: > 1. The offloaded implementation builds "plaintext TLS record", those > records contain plaintext instead of ciphertext and place holder bytes > instead of authentication tags. > 2. The offloaded implementation maintains a mapping from TCP sequence > number to TLS records. Thus given a TCP SKB sent from a NIC offloaded > TLS socket, we can use the tls NIC offload infrastructure to obtain > enough context to encrypt the payload of the SKB. > A TLS record is released when the last byte of the record is ack'ed, > this is done through the new icsk_clean_acked callback. > > The infrastructure should be extendable to support various NIC offload > implementations. However it is currently written with the > implementation below in mind: > The NIC assumes that packets from each offloaded stream are sent as > plaintext and in-order. It keeps track of the TLS records in the TCP > stream. When a packet marked for offload is transmitted, the NIC > encrypts the payload in-place and puts authentication tags in the > relevant place holders. > > The responsibility for handling out-of-order packets (i.e. TCP > retransmission, qdisc drops) falls on the netdev driver. > > The netdev driver keeps track of the expected TCP SN from the NIC's > perspective. If the next packet to transmit matches the expected TCP > SN, the driver advances the expected TCP SN, and transmits the packet > with TLS offload indication. > > If the next packet to transmit does not match the expected TCP SN. The > driver calls the TLS layer to obtain the TLS record that includes the > TCP of the packet for transmission. Using this TLS record, the driver > posts a work entry on the transmit queue to reconstruct the NIC TLS > state required for the offload of the out-of-order packet. It updates > the expected TCP SN accordingly and transmits the now in-order packet. > The same queue is used for packet transmission and TLS context > reconstruction to avoid the need for flushing the transmit queue before > issuing the context reconstruction request. > > Signed-off-by: Ilya Lesokhin > Signed-off-by: Boris Pismenny > Signed-off-by: Aviad Yehezkel > Signed-off-by: Saeed Mahameed > --- > include/net/tls.h | 120 +-- > net/tls/Kconfig | 10 + > net/tls/Makefile | 2 + > net/tls/tls_device.c | 759 > ++ > net/tls/tls_device_fallback.c | 454 + > net/tls/tls_main.c| 120 --- > net/tls/tls_sw.c | 132 > 7 files changed, 1476 insertions(+), 121 deletions(-) > create mode 100644 net/tls/tls_device.c > create mode 100644 net/tls/tls_device_fallback.c > > diff --git a/include/net/tls.h b/include/net/tls.h > index 437a746300bf..0a8529e9ec21 100644 > --- a/include/net/tls.h > +++ b/include/net/tls.h > @@ -57,21 +57,10 @@ > > #define TLS_AAD_SPACE_SIZE 13 > > -struct tls_sw_context { > +struct tls_sw_context_tx { What the reason splitting this into tx + rx does not go in separate patch? > struct crypto_aead *aead_send; > - struct crypto_aead *aead_recv; > struct crypto_wait async_wait; > > - /* Receive context */ > - struct strparser strp; > - void (*saved_data_ready)(struct sock *sk); > - unsigned int (*sk_poll)(struct file *file, struct socket *sock, > - struct poll_table_struct *wait); > - struct sk_buff *recv_pkt; > - u8 control; > - bool decrypted; > - > - /* Sending context */ > char aad_space[TLS_AAD_SPACE_SIZE]; > > unsigned int sg_plaintext_size; > @@ -88,6 +77,50 @@ struct tls_sw_context { > struct scatterlist sg_aead_out[2]; > }; > > +struct tls_sw_context_rx { > + struct crypto_aead *aead_recv; > + struct crypto_wait async_wait; > + > + struct strparser strp; > + void (*saved_data_ready)(struct sock *sk); > + unsigned int (*sk_poll)(struct file *file, struct socket *sock, > + struct
Re: [PATCH V5 net-next 06/14] net/tls: Add generic NIC offload infrastructure
On 3/27/2018 4:56 PM, Saeed Mahameed wrote: From: Ilya LesokhinThis patch adds a generic infrastructure to offload TLS crypto to a network device. It enables the kernel TLS socket to skip encryption and authentication operations on the transmit side of the data path. Leaving those computationally expensive operations to the NIC. The NIC offload infrastructure builds TLS records and pushes them to the TCP layer just like the SW KTLS implementation and using the same API. TCP segmentation is mostly unaffected. Currently the only exception is that we prevent mixed SKBs where only part of the payload requires offload. In the future we are likely to add a similar restriction following a change cipher spec record. The notable differences between SW KTLS and NIC offloaded TLS implementations are as follows: 1. The offloaded implementation builds "plaintext TLS record", those records contain plaintext instead of ciphertext and place holder bytes instead of authentication tags. 2. The offloaded implementation maintains a mapping from TCP sequence number to TLS records. Thus given a TCP SKB sent from a NIC offloaded TLS socket, we can use the tls NIC offload infrastructure to obtain enough context to encrypt the payload of the SKB. A TLS record is released when the last byte of the record is ack'ed, this is done through the new icsk_clean_acked callback. The infrastructure should be extendable to support various NIC offload implementations. However it is currently written with the implementation below in mind: The NIC assumes that packets from each offloaded stream are sent as plaintext and in-order. It keeps track of the TLS records in the TCP stream. When a packet marked for offload is transmitted, the NIC encrypts the payload in-place and puts authentication tags in the relevant place holders. The responsibility for handling out-of-order packets (i.e. TCP retransmission, qdisc drops) falls on the netdev driver. The netdev driver keeps track of the expected TCP SN from the NIC's perspective. If the next packet to transmit matches the expected TCP SN, the driver advances the expected TCP SN, and transmits the packet with TLS offload indication. If the next packet to transmit does not match the expected TCP SN. The driver calls the TLS layer to obtain the TLS record that includes the TCP of the packet for transmission. Using this TLS record, the driver posts a work entry on the transmit queue to reconstruct the NIC TLS state required for the offload of the out-of-order packet. It updates the expected TCP SN accordingly and transmits the now in-order packet. The same queue is used for packet transmission and TLS context reconstruction to avoid the need for flushing the transmit queue before issuing the context reconstruction request. Signed-off-by: Ilya Lesokhin Signed-off-by: Boris Pismenny Signed-off-by: Aviad Yehezkel Signed-off-by: Saeed Mahameed Acked-by: Shannon Nelson --- include/net/tls.h | 120 +-- net/tls/Kconfig | 10 + net/tls/Makefile | 2 + net/tls/tls_device.c | 759 ++ net/tls/tls_device_fallback.c | 454 + net/tls/tls_main.c| 120 --- net/tls/tls_sw.c | 132 7 files changed, 1476 insertions(+), 121 deletions(-) create mode 100644 net/tls/tls_device.c create mode 100644 net/tls/tls_device_fallback.c diff --git a/include/net/tls.h b/include/net/tls.h index 437a746300bf..0a8529e9ec21 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -57,21 +57,10 @@ #define TLS_AAD_SPACE_SIZE 13 -struct tls_sw_context { +struct tls_sw_context_tx { struct crypto_aead *aead_send; - struct crypto_aead *aead_recv; struct crypto_wait async_wait; - /* Receive context */ - struct strparser strp; - void (*saved_data_ready)(struct sock *sk); - unsigned int (*sk_poll)(struct file *file, struct socket *sock, - struct poll_table_struct *wait); - struct sk_buff *recv_pkt; - u8 control; - bool decrypted; - - /* Sending context */ char aad_space[TLS_AAD_SPACE_SIZE]; unsigned int sg_plaintext_size; @@ -88,6 +77,50 @@ struct tls_sw_context { struct scatterlist sg_aead_out[2]; }; +struct tls_sw_context_rx { + struct crypto_aead *aead_recv; + struct crypto_wait async_wait; + + struct strparser strp; + void (*saved_data_ready)(struct sock *sk); + unsigned int (*sk_poll)(struct file *file, struct socket *sock, + struct poll_table_struct *wait); + struct sk_buff *recv_pkt; + u8 control; + bool decrypted; +}; + +struct tls_record_info { + struct list_head list; + u32 end_seq; + int len;
[PATCH V5 net-next 06/14] net/tls: Add generic NIC offload infrastructure
From: Ilya LesokhinThis patch adds a generic infrastructure to offload TLS crypto to a network device. It enables the kernel TLS socket to skip encryption and authentication operations on the transmit side of the data path. Leaving those computationally expensive operations to the NIC. The NIC offload infrastructure builds TLS records and pushes them to the TCP layer just like the SW KTLS implementation and using the same API. TCP segmentation is mostly unaffected. Currently the only exception is that we prevent mixed SKBs where only part of the payload requires offload. In the future we are likely to add a similar restriction following a change cipher spec record. The notable differences between SW KTLS and NIC offloaded TLS implementations are as follows: 1. The offloaded implementation builds "plaintext TLS record", those records contain plaintext instead of ciphertext and place holder bytes instead of authentication tags. 2. The offloaded implementation maintains a mapping from TCP sequence number to TLS records. Thus given a TCP SKB sent from a NIC offloaded TLS socket, we can use the tls NIC offload infrastructure to obtain enough context to encrypt the payload of the SKB. A TLS record is released when the last byte of the record is ack'ed, this is done through the new icsk_clean_acked callback. The infrastructure should be extendable to support various NIC offload implementations. However it is currently written with the implementation below in mind: The NIC assumes that packets from each offloaded stream are sent as plaintext and in-order. It keeps track of the TLS records in the TCP stream. When a packet marked for offload is transmitted, the NIC encrypts the payload in-place and puts authentication tags in the relevant place holders. The responsibility for handling out-of-order packets (i.e. TCP retransmission, qdisc drops) falls on the netdev driver. The netdev driver keeps track of the expected TCP SN from the NIC's perspective. If the next packet to transmit matches the expected TCP SN, the driver advances the expected TCP SN, and transmits the packet with TLS offload indication. If the next packet to transmit does not match the expected TCP SN. The driver calls the TLS layer to obtain the TLS record that includes the TCP of the packet for transmission. Using this TLS record, the driver posts a work entry on the transmit queue to reconstruct the NIC TLS state required for the offload of the out-of-order packet. It updates the expected TCP SN accordingly and transmits the now in-order packet. The same queue is used for packet transmission and TLS context reconstruction to avoid the need for flushing the transmit queue before issuing the context reconstruction request. Signed-off-by: Ilya Lesokhin Signed-off-by: Boris Pismenny Signed-off-by: Aviad Yehezkel Signed-off-by: Saeed Mahameed --- include/net/tls.h | 120 +-- net/tls/Kconfig | 10 + net/tls/Makefile | 2 + net/tls/tls_device.c | 759 ++ net/tls/tls_device_fallback.c | 454 + net/tls/tls_main.c| 120 --- net/tls/tls_sw.c | 132 7 files changed, 1476 insertions(+), 121 deletions(-) create mode 100644 net/tls/tls_device.c create mode 100644 net/tls/tls_device_fallback.c diff --git a/include/net/tls.h b/include/net/tls.h index 437a746300bf..0a8529e9ec21 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -57,21 +57,10 @@ #define TLS_AAD_SPACE_SIZE 13 -struct tls_sw_context { +struct tls_sw_context_tx { struct crypto_aead *aead_send; - struct crypto_aead *aead_recv; struct crypto_wait async_wait; - /* Receive context */ - struct strparser strp; - void (*saved_data_ready)(struct sock *sk); - unsigned int (*sk_poll)(struct file *file, struct socket *sock, - struct poll_table_struct *wait); - struct sk_buff *recv_pkt; - u8 control; - bool decrypted; - - /* Sending context */ char aad_space[TLS_AAD_SPACE_SIZE]; unsigned int sg_plaintext_size; @@ -88,6 +77,50 @@ struct tls_sw_context { struct scatterlist sg_aead_out[2]; }; +struct tls_sw_context_rx { + struct crypto_aead *aead_recv; + struct crypto_wait async_wait; + + struct strparser strp; + void (*saved_data_ready)(struct sock *sk); + unsigned int (*sk_poll)(struct file *file, struct socket *sock, + struct poll_table_struct *wait); + struct sk_buff *recv_pkt; + u8 control; + bool decrypted; +}; + +struct tls_record_info { + struct list_head list; + u32 end_seq; + int len; + int num_frags; + skb_frag_t frags[MAX_SKB_FRAGS]; +}; + +struct tls_offload_context { +