Re: svn commit: r330884 - in head/sys: dev/cxgbe dev/cxgbe/firmware dev/cxgbe/tom modules/cxgbe/tom
On Tuesday, March 13, 2018 11:05:51 PM John Baldwin wrote: > Author: jhb > Date: Tue Mar 13 23:05:51 2018 > New Revision: 330884 > URL: https://svnweb.freebsd.org/changeset/base/330884 > > Log: > Support for TLS offload of TOE connections on T6 adapters. > > The TOE engine in Chelsio T6 adapters supports offloading of TLS > encryption and TCP segmentation for offloaded connections. Sockets > using TLS are required to use a set of custom socket options to upload > RX and TX keys to the NIC and to enable RX processing. Currently > these socket options are implemented as TCP options in the vendor > specific range. A patched OpenSSL library will be made available in a > port / package for use with the TLS TOE support. Note that making use of this requires a patched SSL library. There is not yet a port for one, but I'm working with brnrd@ to make a package of a patched OpenSSL. -- John Baldwin ___ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
Re: svn commit: r330884 - in head/sys: dev/cxgbe dev/cxgbe/firmware dev/cxgbe/tom modules/cxgbe/tom
On Wed, 14 Mar 2018 06:25:10 +0100 "O. Hartmann"wrote: > On Tue, 13 Mar 2018 23:05:51 + (UTC) > John Baldwin wrote: > > > Author: jhb > > Date: Tue Mar 13 23:05:51 2018 > > New Revision: 330884 > > URL: https://svnweb.freebsd.org/changeset/base/330884 > > > > Log: > > Support for TLS offload of TOE connections on T6 adapters. > > > > The TOE engine in Chelsio T6 adapters supports offloading of TLS > > encryption and TCP segmentation for offloaded connections. Sockets > > using TLS are required to use a set of custom socket options to upload > > RX and TX keys to the NIC and to enable RX processing. Currently > > these socket options are implemented as TCP options in the vendor > > specific range. A patched OpenSSL library will be made available in a > > port / package for use with the TLS TOE support. > > > > TOE sockets can either offload both transmit and reception of TLS > > records or just transmit. TLS offload (both RX and TX) is enabled by > > setting the dev.t6nex..tls sysctl to 1 and requires TOE to be > > enabled on the relevant interface. Transmit offload can be used on > > any "normal" or TLS TOE socket by using the custom socket option to > > program a transmit key. This permits most TOE sockets to > > transparently offload TLS when applications use a patched SSL library > > (e.g. using LD_LIBRARY_PATH to request use of a patched OpenSSL > > library). Receive offload can only be used with TOE sockets using the > > TLS mode. The dev.t6nex.0.toe.tls_rx_ports sysctl can be set to a > > list of TCP port numbers. Any connection with either a local or > > remote port number in that list will be created as a TLS socket rather > > than a plain TOE socket. Note that although this sysctl accepts an > > arbitrary list of port numbers, the sysctl(8) tool is only able to set > > sysctl nodes to a single value. A TLS socket will hang without > > receiving data if used by an application that is not using a patched > > SSL library. Thus, the tls_rx_ports node should be used with care. > > For a server mostly concerned with offloading TLS transmit, this node > > is not needed as plain TOE sockets will fall back to software crypto > > when using an unpatched SSL library. > > > > New per-interface statistics nodes are added giving counts of TLS > > packets and payload bytes (payload bytes do not include TLS headers or > > authentication tags/MACs) offloaded via the TOE engine, e.g.: > > > > dev.cc.0.stats.rx_tls_octets: 149 > > dev.cc.0.stats.rx_tls_records: 13 > > dev.cc.0.stats.tx_tls_octets: 26501823 > > dev.cc.0.stats.tx_tls_records: 1620 > > > > TLS transmit work requests are constructed by a new variant of > > t4_push_frames() called t4_push_tls_records() in tom/t4_tls.c. > > > > TLS transmit work requests require a buffer containing IVs. If the > > IVs are too large to fit into the work request, a separate buffer is > > allocated when constructing a work request. This buffer is associated > > with the transmit descriptor and freed when the descriptor is ACKed by > > the adapter. > > > > Received TLS frames use two new CPL messages. The first message is a > > CPL_TLS_DATA containing the decryped payload of a single TLS record. > > The handler places the mbuf containing the received payload on an > > mbufq in the TOE pcb. The second message is a CPL_RX_TLS_CMP message > > which includes a copy of the TLS header and indicates if there were > > any errors. The handler for this message places the TLS header into > > the socket buffer followed by the saved mbuf with the payload data. > > Both of these handlers are contained in tom/t4_tls.c. > > > > A few routines were exposed from t4_cpl_io.c for use by t4_tls.c > > including send_rx_credits(), a new send_rx_modulate(), and > > t4_close_conn(). > > > > TLS keys for both transmit and receive are stored in onboard memory > > in the NIC in the "TLS keys" memory region. > > > > In some cases a TLS socket can hang with pending data available in the > > NIC that is not delivered to the host. As a workaround, TLS sockets > > are more aggressive about sending CPL_RX_DATA_ACK messages anytime that > > any data is read from a TLS socket. In addition, a fallback timer will > > periodically send CPL_RX_DATA_ACK messages to the NIC for connections > > that are still in the handshake phase. Once the connection has > > finished the handshake and programmed RX keys via the socket option, > > the timer is stopped. > > > > A new function select_ulp_mode() is used to determine what sub-mode a > > given TOE socket should use (plain TOE, DDP, or TLS). The existing > > set_tcpddp_ulp_mode() function has been renamed to set_ulp_mode() and > > handles initialization of TLS-specific state when necessary in > > addition to DDP-specific state. > >
Re: svn commit: r330884 - in head/sys: dev/cxgbe dev/cxgbe/firmware dev/cxgbe/tom modules/cxgbe/tom
On Tue, 13 Mar 2018 23:05:51 + (UTC) John Baldwinwrote: > Author: jhb > Date: Tue Mar 13 23:05:51 2018 > New Revision: 330884 > URL: https://svnweb.freebsd.org/changeset/base/330884 > > Log: > Support for TLS offload of TOE connections on T6 adapters. > > The TOE engine in Chelsio T6 adapters supports offloading of TLS > encryption and TCP segmentation for offloaded connections. Sockets > using TLS are required to use a set of custom socket options to upload > RX and TX keys to the NIC and to enable RX processing. Currently > these socket options are implemented as TCP options in the vendor > specific range. A patched OpenSSL library will be made available in a > port / package for use with the TLS TOE support. > > TOE sockets can either offload both transmit and reception of TLS > records or just transmit. TLS offload (both RX and TX) is enabled by > setting the dev.t6nex..tls sysctl to 1 and requires TOE to be > enabled on the relevant interface. Transmit offload can be used on > any "normal" or TLS TOE socket by using the custom socket option to > program a transmit key. This permits most TOE sockets to > transparently offload TLS when applications use a patched SSL library > (e.g. using LD_LIBRARY_PATH to request use of a patched OpenSSL > library). Receive offload can only be used with TOE sockets using the > TLS mode. The dev.t6nex.0.toe.tls_rx_ports sysctl can be set to a > list of TCP port numbers. Any connection with either a local or > remote port number in that list will be created as a TLS socket rather > than a plain TOE socket. Note that although this sysctl accepts an > arbitrary list of port numbers, the sysctl(8) tool is only able to set > sysctl nodes to a single value. A TLS socket will hang without > receiving data if used by an application that is not using a patched > SSL library. Thus, the tls_rx_ports node should be used with care. > For a server mostly concerned with offloading TLS transmit, this node > is not needed as plain TOE sockets will fall back to software crypto > when using an unpatched SSL library. > > New per-interface statistics nodes are added giving counts of TLS > packets and payload bytes (payload bytes do not include TLS headers or > authentication tags/MACs) offloaded via the TOE engine, e.g.: > > dev.cc.0.stats.rx_tls_octets: 149 > dev.cc.0.stats.rx_tls_records: 13 > dev.cc.0.stats.tx_tls_octets: 26501823 > dev.cc.0.stats.tx_tls_records: 1620 > > TLS transmit work requests are constructed by a new variant of > t4_push_frames() called t4_push_tls_records() in tom/t4_tls.c. > > TLS transmit work requests require a buffer containing IVs. If the > IVs are too large to fit into the work request, a separate buffer is > allocated when constructing a work request. This buffer is associated > with the transmit descriptor and freed when the descriptor is ACKed by > the adapter. > > Received TLS frames use two new CPL messages. The first message is a > CPL_TLS_DATA containing the decryped payload of a single TLS record. > The handler places the mbuf containing the received payload on an > mbufq in the TOE pcb. The second message is a CPL_RX_TLS_CMP message > which includes a copy of the TLS header and indicates if there were > any errors. The handler for this message places the TLS header into > the socket buffer followed by the saved mbuf with the payload data. > Both of these handlers are contained in tom/t4_tls.c. > > A few routines were exposed from t4_cpl_io.c for use by t4_tls.c > including send_rx_credits(), a new send_rx_modulate(), and > t4_close_conn(). > > TLS keys for both transmit and receive are stored in onboard memory > in the NIC in the "TLS keys" memory region. > > In some cases a TLS socket can hang with pending data available in the > NIC that is not delivered to the host. As a workaround, TLS sockets > are more aggressive about sending CPL_RX_DATA_ACK messages anytime that > any data is read from a TLS socket. In addition, a fallback timer will > periodically send CPL_RX_DATA_ACK messages to the NIC for connections > that are still in the handshake phase. Once the connection has > finished the handshake and programmed RX keys via the socket option, > the timer is stopped. > > A new function select_ulp_mode() is used to determine what sub-mode a > given TOE socket should use (plain TOE, DDP, or TLS). The existing > set_tcpddp_ulp_mode() function has been renamed to set_ulp_mode() and > handles initialization of TLS-specific state when necessary in > addition to DDP-specific state. > > Since TLS sockets do not receive individual TCP segments but always > receive full TLS records, they can receive more data than is available > in the current window (e.g. if a 16k TLS record is received but the > socket buffer is itself 16k). To