Overview
========
This work consists of a set of kernel interfaces that can be used by
applications that require (time-based) Scheduled Tx of packets.
It is comprised by 3 new components to the kernel:
- SO_TXTIME: socket option + cmsg programming interfaces.
- etf: the "earliest txtime first" qdisc, that provides per-queue
TxTime-based scheduling. This has been renamed from 'tbs' to
'etf' to better describe its functionality.
- taprio: the "time-aware priority scheduler" qdisc, that provides
per-port Time-Aware scheduling;
This patchset is providing the first 2 components, which have been
developed for longer. The taprio qdisc will be shared as an RFC separately
(shortly).
Note that this series is a follow up of the "Time based packet
transmission" RFCv3 [1].
etf (formerly known as 'tbs')
=============================
Changes since the RFC v3:
- removed patch adding CLOCKID_INVALID;
- now we report packet drops through the socket's error queue;
- the usage of CLOCK_TAI is enforced by the qdisc;
- fixed bug on igb driver to avoid timestamps from being overwritten;
- simplified queueing modes by making 'sorting' mandatory;
- renamed qdisc from 'tbs' to 'etf'.
For applications/systems that the concept of time slices isn't precise
enough, the etf qdisc allows applications to control the instant when
a packet should leave the network controller. When used in conjunction
with taprio, it can also be used in case the application needs to
control with greater guarantee the offset into each time slice a packet
will be sent. Another use case of etf, is when only a small number of
applications on a system are time sensitive, so it can then be used
with a more traditional root qdisc (like mqprio).
The etf qdisc is designed so it buffers packets until a configurable
time before their deadline (Tx time). The qdisc uses a rbtree internally
so the buffered packets are always 'ordered' by their txtime (deadline)
and will be dequeued following the earliest txtime first.
The qdisc will drop any packets with a Tx time in the past, or if a
packet expires while waiting for being dequeued. Drops can be reported
as errors back to userspace through the socket's error queue.
Example configuration:
$ tc qdisc add dev enp2s0 parent 100:1 etf offload delta 200000 \
clockid CLOCK_TAI
Here, the Qdisc will use HW offload for the txtime control.
Packets will be dequeued by the qdisc "delta" (200000) nanoseconds before
their transmission time. Because this will be using HW offload and
since dynamic clocks are not supported by hrtimers, the system clock
and the PHC clock must be synchronized for this mode to behave as expected.
A more complete example can be found here, with instructions of how to
test it:
https://gist.github.com/jeez/bd3afeff081ba64a695008dd8215866f [2]
Note that we haven't modified the qdisc so it uses a timerqueue because
the modification needed was increasing the number of cachelines of a sk_buff.
SO_TXTIME
=========
Changes since the RFC v3:
- skb->tstamp is now cleared in skb_scrub_packet();
- transmit time is now set for other send paths, and not only the
"fast" ones as before;
- removed the per-packet parameters (clockid and drop_if_late).
Now just the skb->tstamp is used;
- flags and clockid_t are now set per-socket as a parameter of
SO_TXTIME.
This series is also hosted on github and can be found at [3].
The companion iproute2 patches can be found at [4].
[1] https://patchwork.ozlabs.org/cover/882342/
[2] github doesn't make it clear, but the gist can be cloned like this:
$ git clone https://gist.github.com/jeez/bd3afeff081ba64a695008dd8215866f
scheduled-tx-tests
[3] https://github.com/jeez/linux/tree/etf-v1
[4] https://github.com/jeez/iproute2/tree/etf-v1
Jesus Sanchez-Palencia (10):
net: Clear skb->tstamp only on the forwarding path
net: ipv4: Hook into time based transmission
net/sched: Add HW offloading capability to ETF
igb: Refactor igb_configure_cbs()
igb: Only change Tx arbitration when CBS is on
igb: Refactor igb_offload_cbs()
igb: Add support for ETF offload
igb: Only call skb_tx_timestamp after descriptors are ready
net/sched: Enforce usage of CLOCK_TAI for sch_etf
net/sched: Make etf report drops on error_queue
Richard Cochran (2):
net: Add a new socket option for a future transmit time.
net: packet: Hook into time based transmission.
Vinicius Costa Gomes (2):
net/sched: Allow creating a Qdisc watchdog with other clocks
net/sched: Introduce the ETF Qdisc
arch/alpha/include/uapi/asm/socket.h | 3 +
arch/ia64/include/uapi/asm/socket.h | 3 +
arch/mips/include/uapi/asm/socket.h | 3 +
arch/parisc/include/uapi/asm/socket.h | 3 +
arch/s390/include/uapi/asm/socket.h | 3 +
arch/sparc/include/uapi/asm/socket.h | 3 +
arch/xtensa/include/uapi/asm/socket.h | 3 +
.../net/ethernet/intel/igb/e1000_defines.h | 16 +
drivers/net/ethernet/intel/igb/igb.h | 1 +
drivers/net/ethernet/intel/igb/igb_main.c | 257 ++++++---
include/linux/netdevice.h | 1 +
include/linux/socket.h | 7 +
include/net/inet_sock.h | 1 +
include/net/pkt_sched.h | 7 +
include/net/sock.h | 9 +
include/uapi/asm-generic/socket.h | 3 +
include/uapi/linux/errqueue.h | 2 +
include/uapi/linux/pkt_sched.h | 18 +
net/core/skbuff.c | 2 +-
net/core/sock.c | 32 ++
net/ipv4/ip_output.c | 3 +
net/ipv4/raw.c | 2 +
net/ipv4/udp.c | 1 +
net/packet/af_packet.c | 6 +
net/sched/Kconfig | 11 +
net/sched/Makefile | 1 +
net/sched/sch_api.c | 11 +-
net/sched/sch_etf.c | 487 ++++++++++++++++++
28 files changed, 829 insertions(+), 70 deletions(-)
create mode 100644 net/sched/sch_etf.c
--
2.17.1
