As the packet traverses through OVS, offloading Tx flags must be carefully
evaluated and updated which results in a bit of complexity because of a
separate "outer" Tx offloading flag coming from DPDK API,
and a "normal"/"inner" Tx offloading flag.
On the other hand, the DPDK mbuf API specifies 4 status when it comes to
IP checksums:
- RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN: no information about the RX IP checksum
- RTE_MBUF_F_RX_IP_CKSUM_BAD: the IP checksum in the packet is wrong
- RTE_MBUF_F_RX_IP_CKSUM_GOOD: the IP checksum in the packet is valid
- RTE_MBUF_F_RX_IP_CKSUM_NONE: the IP checksum is not correct in the
packet data, but the integrity of the IP header is verified.
This patch changes OVS API so that OVS code only tracks the status of
the checksum of the "current" L3 header and let the Tx flags aspect to
the netdev-* implementations.
With this API, the flow extraction can be cleaned up.
During packet processing, OVS can simply look for the IP checksum validity
(either good, or partial) before changing some IP header, and then mark
the checksum as partial.
In the conntrack case, when natting packets, the checksum status of the
inner part (ICMP error case) must be forced temporarily as unknown
to force checksum resolution.
When tunneling comes into play, IP checksums status are bit-shifted
for future considerations in the processing if, for example, the tunnel
header gets decapsulated again, or in the netdev-* implementations that
support tunnel offloading.
Finally, thet netdev-* implementations only need to care about packets
in partial status: a good checksum does not need touching, a bad
checksum has been updated by kept as bad by OVS, an unknown checksum is
either a IPv6 or if it was a IPv4, OVS updated it too (keeping it good
or bad accordingly).
Rename current API for consistency with dp_packet_(inner_)?ip_checksum_.
Signed-off-by: David Marchand <david.march...@redhat.com>
---
Changes since v1:
- fixed packet_expand IP checksum,
- fixed tunnel decapsulation checksum validation,
- fixed ipf checksum validation,
- added "partial" handling in netdev-dummy and updated unit tests,
---
.../topics/userspace-checksum-offloading.rst | 27 ++-
lib/conntrack.c | 10 +-
lib/dp-packet-gso.c | 2 +
lib/dp-packet.c | 35 ++--
lib/dp-packet.h | 194 +++++++-----------
lib/dpif-netdev-extract-avx512.c | 13 --
lib/flow.c | 20 +-
lib/ipf.c | 10 +-
lib/netdev-dpdk.c | 34 ++-
lib/netdev-dummy.c | 37 ++--
lib/netdev-native-tnl.c | 60 +++---
lib/odp-execute-avx512.c | 4 +-
lib/odp-execute.c | 8 +-
lib/packets.c | 16 +-
tests/dpif-netdev.at | 73 +++++++
tests/tunnel-push-pop.at | 21 +-
16 files changed, 287 insertions(+), 277 deletions(-)
diff --git a/Documentation/topics/userspace-checksum-offloading.rst
b/Documentation/topics/userspace-checksum-offloading.rst
index e5f87ff1fe..12c0d491cd 100644
--- a/Documentation/topics/userspace-checksum-offloading.rst
+++ b/Documentation/topics/userspace-checksum-offloading.rst
@@ -71,22 +71,21 @@ Rules
2) OVS must not correct a known bad packet checksum.
-3) Packet with flag ``DP_PACKET_OL_RX_IP_CKSUM_GOOD`` means that the IP
+3) Packet with only flag ``DP_PACKET_OL_RX_IP_CKSUM_GOOD`` means that the IP
checksum is present in the packet and it is good.
-4) Packet with flag ``DP_PACKET_OL_RX_IP_CKSUM_BAD`` means that the IP
+4) Packet with only flag ``DP_PACKET_OL_RX_IP_CKSUM_BAD`` means that the IP
checksum is present in the packet and it is bad. Extra care should be taken
to not fix the packet during data path processing.
-5) The ingress packet parser can only set ``DP_PACKET_OL_TX_IP_CKSUM`` if the
- packet has ``DP_PACKET_OL_RX_IP_CKSUM_GOOD`` to not violate rule #2.
-
-6) Packet with flag ``DP_PACKET_OL_TX_IP_CKSUM`` tells the datapath to skip
- updating the IP checksum if the packet is modified. The IP checksum will be
- calculated by the egress interface if that supports IP checksum offload,
- otherwise the IP checksum will be performed in software before handing over
- the packet to the interface.
-
-7) When there are modifications to the packet that requires a checksum update,
- the datapath needs to remove the ``DP_PACKET_OL_RX_IP_CKSUM_GOOD`` flag,
- otherwise the checksum is assumed to be good in the packet.
+5) Packet with both ``DP_PACKET_OL_RX_IP_CKSUM_GOOD`` and
+ ``DP_PACKET_OL_RX_IP_CKSUM_BAD`` means that the IP header is valid, but the
+ checksum may not be set in the packet data. This is basically encountered
+ with some virtual drivers, or after OVS modified the IPv4 header content
+ of a not bad ("bad" as defined in 4)) packet.
+
+6) Packet with neither ``DP_PACKET_OL_RX_IP_CKSUM_GOOD`` nor
+ ``DP_PACKET_OL_RX_IP_CKSUM_BAD`` means that the IP header status is unknown.
+ It may be a IPv4 packet, or not. It may have a valid IPv4 cheksum, or not.
+ This situation is encountered with virtual drivers that provide no
+ information about the IP header, and for IPv6 packets.
diff --git a/lib/conntrack.c b/lib/conntrack.c
index a25893e0fb..20dcf5eaa2 100644
--- a/lib/conntrack.c
+++ b/lib/conntrack.c
@@ -914,6 +914,7 @@ nat_inner_packet(struct dp_packet *pkt, struct conn_key
*key,
uint16_t orig_l3_ofs = pkt->l3_ofs;
uint16_t orig_l4_ofs = pkt->l4_ofs;
uint64_t orig_ol_flags = *dp_packet_ol_flags_ptr(pkt);
+ uint32_t orig_offloads = pkt->offloads;
void *l3 = dp_packet_l3(pkt);
void *l4 = dp_packet_l4(pkt);
@@ -935,6 +936,7 @@ nat_inner_packet(struct dp_packet *pkt, struct conn_key
*key,
/* Drop any offloads to force below helpers to calculate checksums
* if needed. */
*dp_packet_ol_flags_ptr(pkt) &= ~DP_PACKET_OL_TX_ANY_CKSUM;
+ dp_packet_ip_checksum_set_unknown(pkt);
/* Reverse the key for inner packet. */
struct conn_key rev_key = *key;
@@ -961,6 +963,7 @@ nat_inner_packet(struct dp_packet *pkt, struct conn_key
*key,
pkt->l3_ofs = orig_l3_ofs;
pkt->l4_ofs = orig_l4_ofs;
*dp_packet_ol_flags_ptr(pkt) = orig_ol_flags;
+ pkt->offloads = orig_offloads;
}
static void
@@ -2236,8 +2239,7 @@ conn_key_extract(struct conntrack *ct, struct dp_packet
*pkt, ovs_be16 dl_type,
/* Validate the checksum only when hwol is not supported and the
* packet's checksum status is not known. */
ok = extract_l3_ipv4(&ctx->key, l3, dp_packet_l3_size(pkt), NULL,
- !dp_packet_hwol_l3_csum_ipv4_ol(pkt)
- && !dp_packet_ip_checksum_good(pkt));
+ dp_packet_ip_checksum_unknown(pkt));
}
} else if (ctx->key.dl_type == htons(ETH_TYPE_IPV6)) {
ok = extract_l3_ipv6(&ctx->key, l3, dp_packet_l3_size(pkt), NULL);
@@ -3667,8 +3669,8 @@ handle_ftp_ctl(struct conntrack *ct, const struct
conn_lookup_ctx *ctx,
}
if (seq_skew) {
ip_len = ntohs(l3_hdr->ip_tot_len) + seq_skew;
- if (dp_packet_hwol_tx_ip_csum(pkt)) {
- dp_packet_ol_reset_ip_csum_good(pkt);
+ if (dp_packet_ip_checksum_valid(pkt)) {
+ dp_packet_ip_checksum_set_partial(pkt);
} else {
l3_hdr->ip_csum = recalc_csum16(l3_hdr->ip_csum,
l3_hdr->ip_tot_len,
diff --git a/lib/dp-packet-gso.c b/lib/dp-packet-gso.c
index da600a3220..f49a296cd4 100644
--- a/lib/dp-packet-gso.c
+++ b/lib/dp-packet-gso.c
@@ -169,6 +169,7 @@ dp_packet_gso(struct dp_packet *p, struct dp_packet_batch
**batches)
ip_hdr->ip_tot_len = htons(dp_packet_inner_l3_size(seg));
ip_hdr->ip_id = htons(inner_ip_id);
ip_hdr->ip_csum = 0;
+ dp_packet_inner_ip_checksum_set_partial(seg);
inner_ip_id++;
} else {
struct ovs_16aligned_ip6_hdr *ip6_hdr;
@@ -185,6 +186,7 @@ dp_packet_gso(struct dp_packet *p, struct dp_packet_batch
**batches)
ip_hdr->ip_tot_len = htons(dp_packet_l3_size(seg));
ip_hdr->ip_id = htons(outer_ip_id);
ip_hdr->ip_csum = 0;
+ dp_packet_ip_checksum_set_partial(seg);
outer_ip_id++;
} else {
struct ovs_16aligned_ip6_hdr *ip6_hdr = dp_packet_l3(seg);
diff --git a/lib/dp-packet.c b/lib/dp-packet.c
index 9d55ffaad9..adcd5e52f1 100644
--- a/lib/dp-packet.c
+++ b/lib/dp-packet.c
@@ -556,20 +556,17 @@ dp_packet_compare_offsets(struct dp_packet *b1, struct
dp_packet *b2,
void
dp_packet_ol_send_prepare(struct dp_packet *p, uint64_t flags)
{
- if (!dp_packet_hwol_tx_is_any_csum(p)) {
+ if (!dp_packet_ip_checksum_partial(p)
+ && !dp_packet_inner_ip_checksum_partial(p)
+ && !dp_packet_hwol_tx_is_any_csum(p)) {
/* Only checksumming needs actions. */
return;
}
if (!dp_packet_tunnel(p)) {
- if (dp_packet_hwol_tx_ip_csum(p)) {
- if (dp_packet_ip_checksum_good(p)) {
- dp_packet_hwol_reset_tx_ip_csum(p);
- } else if (!(flags & NETDEV_TX_OFFLOAD_IPV4_CKSUM)) {
- dp_packet_ip_set_header_csum(p, false);
- dp_packet_ol_set_ip_csum_good(p);
- dp_packet_hwol_reset_tx_ip_csum(p);
- }
+ if (dp_packet_ip_checksum_partial(p)
+ && !(flags & NETDEV_TX_OFFLOAD_IPV4_CKSUM)) {
+ dp_packet_ip_set_header_csum(p, false);
}
if (dp_packet_hwol_tx_l4_checksum(p)) {
@@ -615,14 +612,9 @@ dp_packet_ol_send_prepare(struct dp_packet *p, uint64_t
flags)
}
}
- if (dp_packet_hwol_tx_ip_csum(p)) {
- if (dp_packet_ip_checksum_good(p)) {
- dp_packet_hwol_reset_tx_ip_csum(p);
- } else if (!(flags & NETDEV_TX_OFFLOAD_IPV4_CKSUM)) {
- dp_packet_ip_set_header_csum(p, true);
- dp_packet_ol_set_ip_csum_good(p);
- dp_packet_hwol_reset_tx_ip_csum(p);
- }
+ if (dp_packet_inner_ip_checksum_partial(p)
+ && !(flags & NETDEV_TX_OFFLOAD_IPV4_CKSUM)) {
+ dp_packet_ip_set_header_csum(p, true);
}
if (dp_packet_hwol_tx_l4_checksum(p)) {
@@ -644,12 +636,9 @@ dp_packet_ol_send_prepare(struct dp_packet *p, uint64_t
flags)
}
}
- if (dp_packet_hwol_is_outer_ipv4_cksum(p)) {
- if (!(flags & NETDEV_TX_OFFLOAD_OUTER_IP_CKSUM)) {
- dp_packet_ip_set_header_csum(p, false);
- dp_packet_ol_set_ip_csum_good(p);
- dp_packet_hwol_reset_outer_ipv4_csum(p);
- }
+ if (dp_packet_ip_checksum_partial(p)
+ && !(flags & NETDEV_TX_OFFLOAD_OUTER_IP_CKSUM)) {
+ dp_packet_ip_set_header_csum(p, false);
}
if (!dp_packet_hwol_is_outer_udp_cksum(p)) {
diff --git a/lib/dp-packet.h b/lib/dp-packet.h
index b6a45ca944..42da5c09a6 100644
--- a/lib/dp-packet.h
+++ b/lib/dp-packet.h
@@ -66,11 +66,6 @@ enum {
DEF_OL_FLAG(DP_PACKET_OL_TX_UDP_CKSUM, RTE_MBUF_F_TX_UDP_CKSUM, 0x400),
/* Offload SCTP checksum. */
DEF_OL_FLAG(DP_PACKET_OL_TX_SCTP_CKSUM, RTE_MBUF_F_TX_SCTP_CKSUM, 0x800),
- /* Offload IP checksum. */
- DEF_OL_FLAG(DP_PACKET_OL_TX_IP_CKSUM, RTE_MBUF_F_TX_IP_CKSUM, 0x1000),
- /* Offload tunnel outer IPv4 checksum. */
- DEF_OL_FLAG(DP_PACKET_OL_TX_OUTER_IP_CKSUM,
- RTE_MBUF_F_TX_OUTER_IP_CKSUM, 0x10000),
/* Offload tunnel outer UDP checksum. */
DEF_OL_FLAG(DP_PACKET_OL_TX_OUTER_UDP_CKSUM,
RTE_MBUF_F_TX_OUTER_UDP_CKSUM, 0x20000),
@@ -82,16 +77,12 @@ enum {
DP_PACKET_OL_TX_TCP_CKSUM | \
DP_PACKET_OL_TX_UDP_CKSUM | \
DP_PACKET_OL_TX_SCTP_CKSUM | \
- DP_PACKET_OL_TX_IP_CKSUM | \
- DP_PACKET_OL_TX_OUTER_IP_CKSUM | \
DP_PACKET_OL_TX_OUTER_UDP_CKSUM)
#define DP_PACKET_OL_TX_L4_MASK (DP_PACKET_OL_TX_TCP_CKSUM | \
DP_PACKET_OL_TX_UDP_CKSUM | \
DP_PACKET_OL_TX_SCTP_CKSUM)
#define DP_PACKET_OL_TX_ANY_CKSUM (DP_PACKET_OL_TX_L4_MASK | \
- DP_PACKET_OL_TX_IP_CKSUM | \
- DP_PACKET_OL_TX_OUTER_IP_CKSUM | \
DP_PACKET_OL_TX_OUTER_UDP_CKSUM)
/* Bit masks for the 'offloads' member of the 'dp_packet' structure. */
@@ -109,6 +100,14 @@ enum OVS_PACKED_ENUM dp_packet_offload_mask {
/* Bits for marking a packet as tunneled. */
DP_PACKET_OL_TUNNEL_GENEVE = UINT16_C(1) << 11,
DP_PACKET_OL_TUNNEL_VXLAN = UINT16_C(1) << 12,
+
+#define DP_PACKET_OL_SHIFT_COUNT 16
+
+ /* Inner offloads. */
+ DP_PACKET_OL_INNER_IP_CKSUM_BAD =
+ DP_PACKET_OL_IP_CKSUM_BAD << DP_PACKET_OL_SHIFT_COUNT,
+ DP_PACKET_OL_INNER_IP_CKSUM_GOOD =
+ DP_PACKET_OL_IP_CKSUM_GOOD << DP_PACKET_OL_SHIFT_COUNT,
};
#ifdef DPDK_NETDEV
@@ -126,6 +125,9 @@ BUILD_ASSERT_DECL(DP_PACKET_OL_L4_CKSUM_GOOD ==
RTE_MBUF_F_RX_L4_CKSUM_GOOD);
#define DP_PACKET_OL_TUNNEL_MASK (DP_PACKET_OL_TUNNEL_GENEVE \
| DP_PACKET_OL_TUNNEL_VXLAN)
+#define DP_PACKET_OL_INNER_IP_CKSUM_MASK (DP_PACKET_OL_INNER_IP_CKSUM_GOOD \
+ | DP_PACKET_OL_INNER_IP_CKSUM_BAD)
+
/* Buffer for holding packet data. A dp_packet is automatically reallocated
* as necessary if it grows too large for the available memory.
* By default the packet type is set to Ethernet (PT_ETH).
@@ -171,7 +173,7 @@ struct dp_packet {
};
BUILD_ASSERT_DECL(MEMBER_SIZEOF(struct dp_packet, offloads)
- <= sizeof(uint32_t));
+ == sizeof(uint32_t));
#if HAVE_AF_XDP
struct dp_packet_afxdp {
@@ -1163,13 +1165,6 @@ dp_packet_hwol_l4_is_sctp(struct dp_packet *b)
DP_PACKET_OL_TX_SCTP_CKSUM;
}
-/* Returns 'true' if packet 'b' is marked for outer IPv4 checksum offload. */
-static inline bool
-dp_packet_hwol_is_outer_ipv4_cksum(const struct dp_packet *b)
-{
- return !!(*dp_packet_ol_flags_ptr(b) & DP_PACKET_OL_TX_OUTER_IP_CKSUM);
-}
-
/* Returns 'true' if packet 'b' is marked for outer UDP checksum offload. */
static inline bool
dp_packet_hwol_is_outer_udp_cksum(struct dp_packet *b)
@@ -1190,26 +1185,6 @@ dp_packet_hwol_reset_tx_l4_csum(struct dp_packet *p)
*dp_packet_ol_flags_ptr(p) &= ~DP_PACKET_OL_TX_L4_MASK;
}
-/* Returns 'true' if packet 'p' is marked for IPv4 checksum offloading. */
-static inline bool
-dp_packet_hwol_tx_ip_csum(const struct dp_packet *p)
-{
- return !!(*dp_packet_ol_flags_ptr(p) & DP_PACKET_OL_TX_IP_CKSUM);
-}
-
-/* Marks packet 'p' for IPv4 checksum offloading. */
-static inline void
-dp_packet_hwol_set_tx_ip_csum(struct dp_packet *p)
-{
- *dp_packet_ol_flags_ptr(p) |= DP_PACKET_OL_TX_IP_CKSUM;
-}
-
-static inline void
-dp_packet_hwol_reset_tx_ip_csum(struct dp_packet *p)
-{
- *dp_packet_ol_flags_ptr(p) &= ~DP_PACKET_OL_TX_IP_CKSUM;
-}
-
/* Mark packet 'b' for TCP checksum offloading. It implies that either
* the packet 'b' is marked for IPv4 or IPv6 checksum offloading. */
static inline void
@@ -1243,19 +1218,6 @@ dp_packet_hwol_set_tcp_seg(struct dp_packet *b)
*dp_packet_ol_flags_ptr(b) |= DP_PACKET_OL_TX_TCP_SEG;
}
-/* Mark packet 'b' for csum offloading in outer IPv4 header. */
-static inline void
-dp_packet_hwol_set_tx_outer_ipv4_csum(struct dp_packet *b)
-{
- *dp_packet_ol_flags_ptr(b) |= DP_PACKET_OL_TX_OUTER_IP_CKSUM;
-}
-
-static inline void
-dp_packet_hwol_reset_outer_ipv4_csum(struct dp_packet *p)
-{
- *dp_packet_ol_flags_ptr(p) &= ~DP_PACKET_OL_TX_OUTER_IP_CKSUM;
-}
-
static inline void
dp_packet_hwol_reset_outer_udp_csum(struct dp_packet *p)
{
@@ -1276,110 +1238,96 @@ dp_packet_hwol_reset_tcp_seg(struct dp_packet *p)
{
uint64_t ol_flags = *dp_packet_ol_flags_ptr(p)
| DP_PACKET_OL_TX_TCP_CKSUM;
- const struct ip_header *ip_hdr;
ol_flags &= ~DP_PACKET_OL_TX_TCP_SEG;
- p->offloads &= ~(DP_PACKET_OL_L4_CKSUM_GOOD | DP_PACKET_OL_IP_CKSUM_GOOD);
-
- if (dp_packet_tunnel(p)) {
- ip_hdr = dp_packet_inner_l3(p);
- } else {
- ip_hdr = dp_packet_l3(p);
- }
- if (IP_VER(ip_hdr->ip_ihl_ver) == 4) {
- ol_flags |= DP_PACKET_OL_TX_IP_CKSUM;
- }
-
- if (dp_packet_tunnel(p)) {
- ip_hdr = dp_packet_l3(p);
- if (IP_VER(ip_hdr->ip_ihl_ver) == 4) {
- ol_flags |= DP_PACKET_OL_TX_OUTER_IP_CKSUM;
- }
+ p->offloads &= ~DP_PACKET_OL_L4_CKSUM_GOOD;
- if (dp_packet_tunnel_geneve(p)
- || dp_packet_tunnel_vxlan(p)) {
- ol_flags |= DP_PACKET_OL_TX_OUTER_UDP_CKSUM;
- }
+ if (dp_packet_tunnel_geneve(p)
+ || dp_packet_tunnel_vxlan(p)) {
+ ol_flags |= DP_PACKET_OL_TX_OUTER_UDP_CKSUM;
}
*dp_packet_ol_flags_ptr(p) = ol_flags;
}
-/* Returns 'true' if the IP header has good integrity and the
- * checksum in it is complete. */
-static inline bool
-dp_packet_ip_checksum_good(const struct dp_packet *p)
-{
- return (p->offloads & DP_PACKET_OL_IP_CKSUM_MASK)
- == DP_PACKET_OL_IP_CKSUM_GOOD;
-}
-
/* Marks packet 'p' with good IPv4 checksum. */
static inline void
-dp_packet_ol_set_ip_csum_good(struct dp_packet *p)
+dp_packet_ip_checksum_set_good(struct dp_packet *p)
{
p->offloads &= ~DP_PACKET_OL_IP_CKSUM_BAD;
p->offloads |= DP_PACKET_OL_IP_CKSUM_GOOD;
}
-/* Resets IP good checksum flag in packet 'p'. */
+static inline bool OVS_WARN_UNUSED_RESULT
+dp_packet_ip_checksum_bad(const struct dp_packet *p)
+{
+ return (p->offloads & DP_PACKET_OL_IP_CKSUM_MASK)
+ == DP_PACKET_OL_IP_CKSUM_BAD;
+}
+
static inline void
-dp_packet_ol_reset_ip_csum_good(struct dp_packet *p)
+dp_packet_ip_checksum_set_bad(struct dp_packet *p)
{
p->offloads &= ~DP_PACKET_OL_IP_CKSUM_GOOD;
+ p->offloads |= DP_PACKET_OL_IP_CKSUM_BAD;
}
-static inline bool
-dp_packet_ip_checksum_bad(const struct dp_packet *p)
+/* Returns 'true' if the IPv4 header has good integrity but the
+ * checksum in it is incomplete. */
+static inline bool OVS_WARN_UNUSED_RESULT
+dp_packet_ip_checksum_partial(const struct dp_packet *p)
{
return (p->offloads & DP_PACKET_OL_IP_CKSUM_MASK)
- == DP_PACKET_OL_IP_CKSUM_BAD;
+ == DP_PACKET_OL_IP_CKSUM_MASK;
}
+/* Marks packet 'p' as having a valid IPv4 header, but no checksum. */
static inline void
-dp_packet_ol_set_ip_csum_bad(struct dp_packet *p)
+dp_packet_ip_checksum_set_partial(struct dp_packet *p)
{
- p->offloads &= ~DP_PACKET_OL_IP_CKSUM_GOOD;
- p->offloads |= DP_PACKET_OL_IP_CKSUM_BAD;
+ p->offloads |= DP_PACKET_OL_IP_CKSUM_MASK;
}
-/* Return 'true' is packet 'b' is not encapsulated and is marked for IPv4
- * checksum offload, or if 'b' is encapsulated and the outer layer is marked
- * for IPv4 checksum offload. IPv6 packets, non offloaded packets, and IPv4
- * packets that are marked as good return 'false'. */
-static inline bool
-dp_packet_hwol_l3_csum_ipv4_ol(const struct dp_packet *b)
+static inline bool OVS_WARN_UNUSED_RESULT
+dp_packet_ip_checksum_unknown(const struct dp_packet *p)
{
- const struct ip_header *ip_hdr;
+ return !(p->offloads & DP_PACKET_OL_IP_CKSUM_MASK);
+}
- if (dp_packet_tunnel(b)) {
- ip_hdr = dp_packet_l3(b);
- if (IP_VER(ip_hdr->ip_ihl_ver) == 4) {
- return dp_packet_hwol_is_outer_ipv4_cksum(b);
- }
- } else {
- return dp_packet_hwol_tx_ip_csum(b) &&
- !dp_packet_ip_checksum_good(b);
- }
- return false;
+static inline void
+dp_packet_ip_checksum_set_unknown(struct dp_packet *p)
+{
+ p->offloads &= ~DP_PACKET_OL_IP_CKSUM_MASK;
}
-/* Return 'true' is packet 'b' is not encapsulated and is marked for IPv4
- * checksum offload, or if 'b' is encapsulated and the outer layer is marked
- * for IPv4 checksum offload. IPv6 packets and non offloaded packets return
- * 'false'. */
-static inline bool
-dp_packet_hwol_l3_ipv4(const struct dp_packet *b)
+static inline bool OVS_WARN_UNUSED_RESULT
+dp_packet_ip_checksum_valid(const struct dp_packet *p)
{
- const struct ip_header *ip_hdr;
+ return !!(p->offloads & DP_PACKET_OL_IP_CKSUM_GOOD);
+}
- if (dp_packet_tunnel(b)) {
- ip_hdr = dp_packet_l3(b);
- return IP_VER(ip_hdr->ip_ihl_ver) == 4;
- } else {
- return dp_packet_hwol_tx_ip_csum(b);
- }
- return false;
+/* Marks packet 'p' with good inner IPv4 checksum. */
+static inline void
+dp_packet_inner_ip_checksum_set_good(struct dp_packet *p)
+{
+ p->offloads &= ~DP_PACKET_OL_INNER_IP_CKSUM_BAD;
+ p->offloads |= DP_PACKET_OL_INNER_IP_CKSUM_GOOD;
+}
+
+/* Returns 'true' if the inner IPv4 header has good integrity but the
+ * checksum in it is incomplete. */
+static inline bool OVS_WARN_UNUSED_RESULT
+dp_packet_inner_ip_checksum_partial(const struct dp_packet *p)
+{
+ return (p->offloads & DP_PACKET_OL_INNER_IP_CKSUM_MASK)
+ == DP_PACKET_OL_INNER_IP_CKSUM_MASK;
+}
+
+/* Marks packet 'p' as having a valid inner IPv4 header, but no checksum. */
+static inline void
+dp_packet_inner_ip_checksum_set_partial(struct dp_packet *p)
+{
+ p->offloads |= DP_PACKET_OL_INNER_IP_CKSUM_MASK;
}
/* Calculate and set the IPv4 header checksum in packet 'p'. */
@@ -1406,6 +1354,12 @@ dp_packet_ip_set_header_csum(struct dp_packet *p, bool
inner)
ip->ip_csum = 0;
ip->ip_csum = csum(ip, ip_len);
}
+
+ if (inner) {
+ dp_packet_inner_ip_checksum_set_good(p);
+ } else {
+ dp_packet_ip_checksum_set_good(p);
+ }
}
/* Returns 'true' if the packet 'p' has good integrity and the
diff --git a/lib/dpif-netdev-extract-avx512.c b/lib/dpif-netdev-extract-avx512.c
index 20e1b5d691..3ad892d375 100644
--- a/lib/dpif-netdev-extract-avx512.c
+++ b/lib/dpif-netdev-extract-avx512.c
@@ -758,14 +758,6 @@ mfex_check_tcp_data_offset(const struct tcp_header *tcp)
return ret;
}
-static void
-mfex_ipv4_set_hwol(struct dp_packet *pkt)
-{
- if (dp_packet_ip_checksum_good(pkt)) {
- dp_packet_hwol_set_tx_ip_csum(pkt);
- }
-}
-
static void
mfex_tcp_set_hwol(struct dp_packet *pkt)
{
@@ -885,7 +877,6 @@ mfex_avx512_process(struct dp_packet_batch *packets,
const struct tcp_header *tcp = (void *)&pkt[38];
mfex_handle_tcp_flags(tcp, &blocks[7]);
dp_packet_update_rss_hash_ipv4_tcp_udp(packet);
- mfex_ipv4_set_hwol(packet);
mfex_tcp_set_hwol(packet);
} break;
@@ -899,7 +890,6 @@ mfex_avx512_process(struct dp_packet_batch *packets,
continue;
}
dp_packet_update_rss_hash_ipv4_tcp_udp(packet);
- mfex_ipv4_set_hwol(packet);
mfex_udp_set_hwol(packet);
} break;
@@ -916,7 +906,6 @@ mfex_avx512_process(struct dp_packet_batch *packets,
continue;
}
dp_packet_update_rss_hash_ipv4_tcp_udp(packet);
- mfex_ipv4_set_hwol(packet);
mfex_tcp_set_hwol(packet);
} break;
@@ -929,7 +918,6 @@ mfex_avx512_process(struct dp_packet_batch *packets,
continue;
}
dp_packet_update_rss_hash_ipv4_tcp_udp(packet);
- mfex_ipv4_set_hwol(packet);
mfex_udp_set_hwol(packet);
} break;
@@ -1033,7 +1021,6 @@ mfex_avx512_process(struct dp_packet_batch *packets,
continue;
}
dp_packet_update_rss_hash_ipv4(packet);
- mfex_ipv4_set_hwol(packet);
mfex_udp_set_hwol(packet);
} break;
diff --git a/lib/flow.c b/lib/flow.c
index c515acde31..e400b42669 100644
--- a/lib/flow.c
+++ b/lib/flow.c
@@ -949,15 +949,6 @@ miniflow_extract(struct dp_packet *packet, struct miniflow
*dst)
nw_proto = nh->ip_proto;
nw_frag = ipv4_get_nw_frag(nh);
data_pull(&data, &size, ip_len);
- if (tunneling) {
- if (dp_packet_ip_checksum_good(packet)) {
- dp_packet_hwol_set_tx_outer_ipv4_csum(packet);
- }
- } else {
- if (dp_packet_ip_checksum_good(packet)) {
- dp_packet_hwol_set_tx_ip_csum(packet);
- }
- }
} else if (dl_type == htons(ETH_TYPE_IPV6)) {
const struct ovs_16aligned_ip6_hdr *nh = data;
ovs_be32 tc_flow;
@@ -3297,12 +3288,7 @@ packet_expand(struct dp_packet *p, const struct flow
*flow, size_t size)
struct ip_header *ip = dp_packet_l3(p);
ip->ip_tot_len = htons(p->l4_ofs - p->l3_ofs + l4_len);
- if (dp_packet_hwol_tx_ip_csum(p)) {
- dp_packet_ol_reset_ip_csum_good(p);
- } else {
- dp_packet_ip_set_header_csum(p, false);
- dp_packet_ol_set_ip_csum_good(p);
- }
+ dp_packet_ip_set_header_csum(p, false);
pseudo_hdr_csum = packet_csum_pseudoheader(ip);
} else { /* ETH_TYPE_IPV6 */
struct ovs_16aligned_ip6_hdr *nh = dp_packet_l3(p);
@@ -3401,9 +3387,9 @@ flow_compose(struct dp_packet *p, const struct flow *flow,
* bit.
*/
ip->ip_csum ^= (OVS_FORCE ovs_be16) 0x1;
- dp_packet_ol_set_ip_csum_bad(p);
+ dp_packet_ip_checksum_set_bad(p);
} else {
- dp_packet_ol_set_ip_csum_good(p);
+ dp_packet_ip_checksum_set_good(p);
}
pseudo_hdr_csum = packet_csum_pseudoheader(ip);
diff --git a/lib/ipf.c b/lib/ipf.c
index 0066aeb50b..cbbf2f56fd 100644
--- a/lib/ipf.c
+++ b/lib/ipf.c
@@ -435,8 +435,8 @@ ipf_reassemble_v4_frags(struct ipf_list *ipf_list)
len += rest_len;
l3 = dp_packet_l3(pkt);
ovs_be16 new_ip_frag_off = l3->ip_frag_off & ~htons(IP_MORE_FRAGMENTS);
- if (dp_packet_hwol_tx_ip_csum(pkt)) {
- dp_packet_ol_reset_ip_csum_good(pkt);
+ if (dp_packet_ip_checksum_valid(pkt)) {
+ dp_packet_ip_checksum_set_partial(pkt);
} else {
l3->ip_csum = recalc_csum16(l3->ip_csum, l3->ip_frag_off,
new_ip_frag_off);
@@ -614,7 +614,7 @@ ipf_is_valid_v4_frag(struct ipf *ipf, struct dp_packet *pkt)
}
bool bad_csum = dp_packet_ip_checksum_bad(pkt);
- if (OVS_UNLIKELY(!bad_csum && !dp_packet_ip_checksum_good(pkt))) {
+ if (OVS_UNLIKELY(!bad_csum && dp_packet_ip_checksum_unknown(pkt))) {
COVERAGE_INC(ipf_l3csum_checked);
bad_csum = csum(l3, ip_hdr_len);
}
@@ -1209,8 +1209,8 @@ ipf_post_execute_reass_pkts(struct ipf *ipf,
} else {
struct ip_header *l3_frag = dp_packet_l3(frag_i->pkt);
struct ip_header *l3_reass = dp_packet_l3(pkt);
- if (dp_packet_hwol_tx_ip_csum(frag_i->pkt)) {
- dp_packet_ol_reset_ip_csum_good(frag_i->pkt);
+ if (dp_packet_ip_checksum_valid(frag_i->pkt)) {
+ dp_packet_ip_checksum_set_partial(frag_i->pkt);
} else {
ovs_be32 reass_ip =
get_16aligned_be32(&l3_reass->ip_src);
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
index 2658021816..ee5124622d 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -2648,18 +2648,20 @@ netdev_dpdk_prep_hwol_packet(struct netdev_dpdk *dev,
struct rte_mbuf *mbuf)
{
struct dp_packet *pkt = CONTAINER_OF(mbuf, struct dp_packet, mbuf);
const struct ip_header *ip;
+ bool l3_csum;
void *l2;
void *l3;
void *l4;
- const uint64_t all_inner_requests = (RTE_MBUF_F_TX_IP_CKSUM |
- RTE_MBUF_F_TX_L4_MASK |
- RTE_MBUF_F_TX_TCP_SEG);
- const uint64_t all_outer_requests = (RTE_MBUF_F_TX_OUTER_IP_CKSUM |
- RTE_MBUF_F_TX_OUTER_UDP_CKSUM);
+ const uint64_t all_inner_requests = (RTE_MBUF_F_TX_L4_MASK
+ | RTE_MBUF_F_TX_TCP_SEG);
+ const uint64_t all_outer_requests = RTE_MBUF_F_TX_OUTER_UDP_CKSUM;
const uint64_t all_requests = all_inner_requests | all_outer_requests;
- if (!(mbuf->ol_flags & all_requests)) {
+ if (!dp_packet_ip_checksum_partial(pkt)
+ && !dp_packet_inner_ip_checksum_partial(pkt)
+ && !(mbuf->ol_flags & all_requests)) {
+
uint64_t unexpected = mbuf->ol_flags & RTE_MBUF_F_TX_OFFLOAD_MASK;
if (OVS_UNLIKELY(unexpected)) {
VLOG_WARN_RL(&rl, "%s: Unexpected Tx offload flags: %#"PRIx64,
@@ -2672,8 +2674,10 @@ netdev_dpdk_prep_hwol_packet(struct netdev_dpdk *dev,
struct rte_mbuf *mbuf)
}
if (dp_packet_tunnel(pkt)
- && (mbuf->ol_flags & all_inner_requests)) {
- if (mbuf->ol_flags & all_outer_requests) {
+ && (dp_packet_inner_ip_checksum_partial(pkt)
+ || (mbuf->ol_flags & all_inner_requests))) {
+ if (dp_packet_ip_checksum_partial(pkt)
+ || (mbuf->ol_flags & all_outer_requests)) {
mbuf->outer_l2_len = (char *) dp_packet_l3(pkt) -
(char *) dp_packet_eth(pkt);
mbuf->outer_l3_len = (char *) dp_packet_l4(pkt) -
@@ -2688,6 +2692,10 @@ netdev_dpdk_prep_hwol_packet(struct netdev_dpdk *dev,
struct rte_mbuf *mbuf)
mbuf->ol_flags |= RTE_MBUF_F_TX_TUNNEL_GRE;
}
+ if (dp_packet_ip_checksum_partial(pkt)) {
+ mbuf->ol_flags |= RTE_MBUF_F_TX_OUTER_IP_CKSUM;
+ }
+
ip = dp_packet_l3(pkt);
mbuf->ol_flags |= IP_VER(ip->ip_ihl_ver) == 4
? RTE_MBUF_F_TX_OUTER_IPV4
@@ -2696,6 +2704,7 @@ netdev_dpdk_prep_hwol_packet(struct netdev_dpdk *dev,
struct rte_mbuf *mbuf)
/* Inner L2 length must account for the tunnel header length. */
l2 = dp_packet_l4(pkt);
l3 = dp_packet_inner_l3(pkt);
+ l3_csum = dp_packet_inner_ip_checksum_partial(pkt);
l4 = dp_packet_inner_l4(pkt);
} else {
mbuf->outer_l2_len = 0;
@@ -2704,15 +2713,13 @@ netdev_dpdk_prep_hwol_packet(struct netdev_dpdk *dev,
struct rte_mbuf *mbuf)
/* Skip outer headers. */
l2 = dp_packet_eth(pkt);
l3 = dp_packet_inner_l3(pkt);
+ l3_csum = dp_packet_inner_ip_checksum_partial(pkt);
l4 = dp_packet_inner_l4(pkt);
}
} else {
if (dp_packet_tunnel(pkt)) {
/* No inner offload is requested, fallback to non tunnel
* checksum offloads. */
- if (mbuf->ol_flags & RTE_MBUF_F_TX_OUTER_IP_CKSUM) {
- mbuf->ol_flags |= RTE_MBUF_F_TX_IP_CKSUM;
- }
if (mbuf->ol_flags & RTE_MBUF_F_TX_OUTER_UDP_CKSUM) {
mbuf->ol_flags |= RTE_MBUF_F_TX_UDP_CKSUM;
}
@@ -2723,6 +2730,7 @@ netdev_dpdk_prep_hwol_packet(struct netdev_dpdk *dev,
struct rte_mbuf *mbuf)
l2 = dp_packet_eth(pkt);
l3 = dp_packet_l3(pkt);
+ l3_csum = dp_packet_ip_checksum_partial(pkt);
l4 = dp_packet_l4(pkt);
}
@@ -2732,6 +2740,10 @@ netdev_dpdk_prep_hwol_packet(struct netdev_dpdk *dev,
struct rte_mbuf *mbuf)
mbuf->ol_flags |= IP_VER(ip->ip_ihl_ver) == 4
? RTE_MBUF_F_TX_IPV4 : RTE_MBUF_F_TX_IPV6;
+ if (l3_csum) {
+ mbuf->ol_flags |= RTE_MBUF_F_TX_IP_CKSUM;
+ }
+
mbuf->l2_len = (char *) l3 - (char *) l2;
mbuf->l3_len = (char *) l4 - (char *) l3;
diff --git a/lib/netdev-dummy.c b/lib/netdev-dummy.c
index f7a1988dea..42dfddd3a1 100644
--- a/lib/netdev-dummy.c
+++ b/lib/netdev-dummy.c
@@ -154,6 +154,8 @@ struct netdev_dummy {
bool ol_ip_rx_csum_set_good OVS_GUARDED;
/* Force IP Rx csum bad. */
bool ol_ip_rx_csum_set_bad OVS_GUARDED;
+ /* Force IP Rx csum partial. */
+ bool ol_ip_rx_csum_set_partial OVS_GUARDED;
/* Announce netdev IP Tx csum offload. */
bool ol_ip_tx_csum OVS_GUARDED;
/* Disable IP Tx csum offload. */
@@ -822,6 +824,9 @@ netdev_dummy_get_config(const struct netdev *dev, struct
smap *args)
if (netdev->ol_ip_rx_csum_set_bad) {
smap_add_format(args, "ol_ip_rx_csum_set_bad", "%s", "true");
}
+ if (netdev->ol_ip_rx_csum_set_partial) {
+ smap_add_format(args, "ol_ip_rx_csum_set_partial", "%s", "true");
+ }
if (netdev->ol_ip_tx_csum) {
smap_add_format(args, "ol_ip_tx_csum", "%s", "true");
if (netdev->ol_ip_tx_csum_disabled) {
@@ -974,6 +979,8 @@ netdev_dummy_set_config(struct netdev *netdev_, const
struct smap *args,
smap_get_bool(args, "ol_ip_rx_csum_set_good", false);
netdev->ol_ip_rx_csum_set_bad =
smap_get_bool(args, "ol_ip_rx_csum_set_bad", false);
+ netdev->ol_ip_rx_csum_set_partial =
+ smap_get_bool(args, "ol_ip_rx_csum_set_partial", false);
netdev->ol_ip_tx_csum = smap_get_bool(args, "ol_ip_tx_csum", false);
if (netdev->ol_ip_tx_csum) {
netdev_->ol_flags |= NETDEV_TX_OFFLOAD_IPV4_CKSUM;
@@ -1188,11 +1195,13 @@ netdev_dummy_rxq_recv(struct netdev_rxq *rxq_, struct
dp_packet_batch *batch,
netdev->custom_stats[1].value++;
if (netdev->ol_ip_rx_csum_set_good) {
- dp_packet_ol_set_ip_csum_good(packet);
+ dp_packet_ip_checksum_set_good(packet);
} else if (netdev->ol_ip_rx_csum_set_bad) {
- dp_packet_ol_set_ip_csum_bad(packet);
+ dp_packet_ip_checksum_set_bad(packet);
+ } else if (netdev->ol_ip_rx_csum_set_partial) {
+ dp_packet_ip_checksum_set_partial(packet);
} else {
- dp_packet_ol_reset_ip_csum_good(packet);
+ dp_packet_ip_checksum_set_unknown(packet);
}
if (netdev->ol_l4_rx_csum_set_good) {
@@ -1215,16 +1224,14 @@ netdev_dummy_rxq_recv(struct netdev_rxq *rxq_, struct
dp_packet_batch *batch,
bool ip_csum_bad;
bool l4_csum_bad;
- ip_csum_good = !!(*dp_packet_ol_flags_ptr(packet)
- & DP_PACKET_OL_IP_CKSUM_GOOD);
- ip_csum_bad = !!(*dp_packet_ol_flags_ptr(packet)
- & DP_PACKET_OL_IP_CKSUM_BAD);
+ ip_csum_good = !!(packet->offloads & DP_PACKET_OL_IP_CKSUM_GOOD);
+ ip_csum_bad = !!(packet->offloads & DP_PACKET_OL_IP_CKSUM_BAD);
l4_csum_good = !!(*dp_packet_ol_flags_ptr(packet)
& DP_PACKET_OL_L4_CKSUM_GOOD);
l4_csum_bad = !!(*dp_packet_ol_flags_ptr(packet)
& DP_PACKET_OL_L4_CKSUM_BAD);
VLOG_DBG("Rx: packet with csum IP %s, L4 %s, segsz %"PRIu16,
- ip_csum_good ? (ip_csum_bad ? "good+bad" : "good")
+ ip_csum_good ? (ip_csum_bad ? "partial" : "good")
: (ip_csum_bad ? "bad" : "unknown"),
l4_csum_good ? (l4_csum_bad ? "good+bad" : "good")
: (l4_csum_bad ? "bad" : "unknown"),
@@ -1333,29 +1340,25 @@ netdev_dummy_send(struct netdev *netdev, int qid,
bool ip_csum_bad;
bool l4_csum_bad;
- ip_csum_good = !!(*dp_packet_ol_flags_ptr(packet)
- & DP_PACKET_OL_IP_CKSUM_GOOD);
- ip_csum_bad = !!(*dp_packet_ol_flags_ptr(packet)
- & DP_PACKET_OL_IP_CKSUM_BAD);
+ ip_csum_good = !!(packet->offloads & DP_PACKET_OL_IP_CKSUM_GOOD);
+ ip_csum_bad = !!(packet->offloads & DP_PACKET_OL_IP_CKSUM_BAD);
l4_csum_good = !!(*dp_packet_ol_flags_ptr(packet)
& DP_PACKET_OL_L4_CKSUM_GOOD);
l4_csum_bad = !!(*dp_packet_ol_flags_ptr(packet)
& DP_PACKET_OL_L4_CKSUM_BAD);
VLOG_DBG("Tx: packet with csum IP %s, L4 %s, segsz %"PRIu16
- ", Tx flags %s, %s, %s",
- ip_csum_good ? (ip_csum_bad ? "good+bad" : "good")
+ ", Tx flags %s, %s",
+ ip_csum_good ? (ip_csum_bad ? "partial" : "good")
: (ip_csum_bad ? "bad" : "unknown"),
l4_csum_good ? (l4_csum_bad ? "good+bad" : "good")
: (l4_csum_bad ? "bad" : "unknown"),
dp_packet_get_tso_segsz(packet),
- dp_packet_hwol_tx_ip_csum(packet) ? "ip_csum" : "none",
dp_packet_hwol_tx_l4_checksum(packet) ? "l4_csum"
: "none",
dp_packet_hwol_is_tso(packet) ? "tso" : "none");
}
- if (dp_packet_hwol_tx_ip_csum(packet)
- && !dp_packet_ip_checksum_good(packet)) {
+ if (dp_packet_ip_checksum_partial(packet)) {
dp_packet_ol_send_prepare(packet, flags);
}
if (dp_packet_hwol_l4_is_tcp(packet)
diff --git a/lib/netdev-native-tnl.c b/lib/netdev-native-tnl.c
index 85784f1525..d5023f0ac5 100644
--- a/lib/netdev-native-tnl.c
+++ b/lib/netdev-native-tnl.c
@@ -118,8 +118,7 @@ ip_extract_tnl_md(struct dp_packet *packet, struct flow_tnl
*tnl,
/* A packet coming from a network device might have the
* csum already checked. In this case, skip the check. */
- if (OVS_UNLIKELY(!bad_csum
- && !dp_packet_hwol_l3_csum_ipv4_ol(packet))) {
+ if (OVS_UNLIKELY(!bad_csum && dp_packet_ip_checksum_unknown(packet))) {
COVERAGE_INC(netdev_native_tnl_l3csum_checked);
bad_csum = csum(ip, IP_IHL(ip->ip_ihl_ver) * 4);
}
@@ -202,23 +201,20 @@ netdev_tnl_push_ip_header(struct dp_packet *packet, const
void *header,
*ip_tot_size -= IPV6_HEADER_LEN;
ip6->ip6_plen = htons(*ip_tot_size);
packet_set_ipv6_flow_label(&ip6->ip6_flow, ipv6_label);
+ dp_packet_ip_checksum_set_unknown(packet);
+
packet->l4_ofs = dp_packet_size(packet) - *ip_tot_size;
- dp_packet_ol_reset_ip_csum_good(packet);
return ip6 + 1;
} else {
ip = netdev_tnl_ip_hdr(eth);
ip->ip_tot_len = htons(*ip_tot_size);
+ *ip_tot_size -= IP_HEADER_LEN;
/* Postpone checksum to when the packet is pushed to the port. */
- if (dp_packet_tunnel(packet)) {
- dp_packet_hwol_set_tx_outer_ipv4_csum(packet);
- } else {
- dp_packet_hwol_set_tx_ip_csum(packet);
- }
+ dp_packet_ip_checksum_set_partial(packet);
- dp_packet_ol_reset_ip_csum_good(packet);
- *ip_tot_size -= IP_HEADER_LEN;
packet->l4_ofs = dp_packet_size(packet) - *ip_tot_size;
+
return ip + 1;
}
}
@@ -268,24 +264,10 @@ udp_extract_tnl_md(struct dp_packet *packet, struct
flow_tnl *tnl,
}
static void
-dp_packet_tnl_ol_process(struct dp_packet *packet,
- const struct ovs_action_push_tnl *data)
+tnl_ol_push(struct dp_packet *packet,
+ const struct ovs_action_push_tnl *data)
{
- struct ip_header *ip = NULL;
-
- if (dp_packet_hwol_l4_mask(packet)) {
- ip = dp_packet_l3(packet);
-
- if (data->tnl_type == OVS_VPORT_TYPE_GENEVE ||
- data->tnl_type == OVS_VPORT_TYPE_VXLAN ||
- data->tnl_type == OVS_VPORT_TYPE_GRE ||
- data->tnl_type == OVS_VPORT_TYPE_IP6GRE) {
-
- if (IP_VER(ip->ip_ihl_ver) == 4) {
- dp_packet_hwol_set_tx_ip_csum(packet);
- }
- }
- }
+ packet->offloads <<= DP_PACKET_OL_SHIFT_COUNT;
if (data->tnl_type == OVS_VPORT_TYPE_GENEVE) {
dp_packet_tunnel_set_geneve(packet);
@@ -297,6 +279,14 @@ dp_packet_tnl_ol_process(struct dp_packet *packet,
}
}
+static void
+tnl_ol_pop(struct dp_packet *packet, int off)
+{
+ packet->offloads >>= DP_PACKET_OL_SHIFT_COUNT;
+
+ dp_packet_reset_packet(packet, off);
+}
+
void
netdev_tnl_push_udp_header(const struct netdev *netdev OVS_UNUSED,
struct dp_packet *packet,
@@ -312,7 +302,7 @@ netdev_tnl_push_udp_header(const struct netdev *netdev
OVS_UNUSED,
* modifying the packet. */
udp_src = netdev_tnl_get_src_port(packet);
- dp_packet_tnl_ol_process(packet, data);
+ tnl_ol_push(packet, data);
udp = netdev_tnl_push_ip_header(packet, data->header, data->header_len,
&ip_tot_size, 0);
@@ -525,7 +515,7 @@ netdev_gre_pop_header(struct dp_packet *packet)
goto err;
}
- dp_packet_reset_packet(packet, hlen);
+ tnl_ol_pop(packet, hlen);
return packet;
err:
@@ -544,7 +534,7 @@ netdev_gre_push_header(const struct netdev *netdev,
struct gre_base_hdr *greh;
int ip_tot_size;
- dp_packet_tnl_ol_process(packet, data);
+ tnl_ol_push(packet, data);
greh = netdev_tnl_push_ip_header(packet, data->header, data->header_len,
&ip_tot_size, 0);
@@ -688,7 +678,7 @@ netdev_erspan_pop_header(struct dp_packet *packet)
goto err;
}
- dp_packet_reset_packet(packet, hlen);
+ tnl_ol_pop(packet, hlen);
return packet;
err:
@@ -850,7 +840,7 @@ netdev_gtpu_pop_header(struct dp_packet *packet)
} else {
VLOG_WARN_RL(&err_rl, "GTP-U: Receive non-IP packet.");
}
- dp_packet_reset_packet(packet, hlen + gtpu_hlen);
+ tnl_ol_pop(packet, hlen + gtpu_hlen);
} else {
/* non-GPDU GTP-U messages, ex: echo request, end marker.
* Users should redirect these packets to controller, or.
@@ -1088,7 +1078,7 @@ netdev_srv6_pop_header(struct dp_packet *packet)
goto err;
}
- dp_packet_reset_packet(packet, hlen);
+ tnl_ol_pop(packet, hlen);
return packet;
err:
@@ -1155,7 +1145,7 @@ netdev_vxlan_pop_header(struct dp_packet *packet)
tnl->flags |= FLOW_TNL_F_KEY;
packet->packet_type = htonl(next_pt);
- dp_packet_reset_packet(packet, hlen + VXLAN_HLEN);
+ tnl_ol_pop(packet, hlen + VXLAN_HLEN);
if (next_pt != PT_ETH) {
packet->l3_ofs = 0;
}
@@ -1263,7 +1253,7 @@ netdev_geneve_pop_header(struct dp_packet *packet)
tnl->flags |= FLOW_TNL_F_UDPIF;
packet->packet_type = htonl(PT_ETH);
- dp_packet_reset_packet(packet, hlen);
+ tnl_ol_pop(packet, hlen);
return packet;
err:
diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c
index 09eb685cba..9ed5dff356 100644
--- a/lib/odp-execute-avx512.c
+++ b/lib/odp-execute-avx512.c
@@ -473,8 +473,8 @@ action_avx512_ipv4_set_addrs(struct dp_packet_batch *batch,
* (v_pkt_masked). */
__m256i v_new_hdr = _mm256_or_si256(v_key_shuf, v_pkt_masked);
- if (dp_packet_hwol_l3_ipv4(packet)) {
- dp_packet_ol_reset_ip_csum_good(packet);
+ if (dp_packet_ip_checksum_valid(packet)) {
+ dp_packet_ip_checksum_set_partial(packet);
} else {
ovs_be16 old_csum = ~nh->ip_csum;
diff --git a/lib/odp-execute.c b/lib/odp-execute.c
index 15577d5394..55858f52e3 100644
--- a/lib/odp-execute.c
+++ b/lib/odp-execute.c
@@ -171,8 +171,8 @@ odp_set_ipv4(struct dp_packet *packet, const struct
ovs_key_ipv4 *key,
new_tos = key->ipv4_tos | (nh->ip_tos & ~mask->ipv4_tos);
if (nh->ip_tos != new_tos) {
- if (dp_packet_hwol_tx_ip_csum(packet)) {
- dp_packet_ol_reset_ip_csum_good(packet);
+ if (dp_packet_ip_checksum_valid(packet)) {
+ dp_packet_ip_checksum_set_partial(packet);
} else {
nh->ip_csum = recalc_csum16(nh->ip_csum,
htons((uint16_t) nh->ip_tos),
@@ -187,8 +187,8 @@ odp_set_ipv4(struct dp_packet *packet, const struct
ovs_key_ipv4 *key,
new_ttl = key->ipv4_ttl | (nh->ip_ttl & ~mask->ipv4_ttl);
if (OVS_LIKELY(nh->ip_ttl != new_ttl)) {
- if (dp_packet_hwol_tx_ip_csum(packet)) {
- dp_packet_ol_reset_ip_csum_good(packet);
+ if (dp_packet_ip_checksum_valid(packet)) {
+ dp_packet_ip_checksum_set_partial(packet);
} else {
nh->ip_csum = recalc_csum16(nh->ip_csum,
htons(nh->ip_ttl << 8),
diff --git a/lib/packets.c b/lib/packets.c
index 05aa5c7664..46e2fb323e 100644
--- a/lib/packets.c
+++ b/lib/packets.c
@@ -1149,8 +1149,8 @@ packet_set_ipv4_addr(struct dp_packet *packet,
}
}
- if (dp_packet_hwol_l3_ipv4(packet)) {
- dp_packet_ol_reset_ip_csum_good(packet);
+ if (dp_packet_ip_checksum_valid(packet)) {
+ dp_packet_ip_checksum_set_partial(packet);
} else {
nh->ip_csum = recalc_csum32(nh->ip_csum, old_addr, new_addr);
}
@@ -1328,8 +1328,8 @@ packet_set_ipv4(struct dp_packet *packet, ovs_be32 src,
ovs_be32 dst,
if (nh->ip_tos != tos) {
uint8_t *field = &nh->ip_tos;
- if (dp_packet_hwol_l3_ipv4(packet)) {
- dp_packet_ol_reset_ip_csum_good(packet);
+ if (dp_packet_ip_checksum_valid(packet)) {
+ dp_packet_ip_checksum_set_partial(packet);
} else {
nh->ip_csum = recalc_csum16(nh->ip_csum, htons((uint16_t) *field),
htons((uint16_t) tos));
@@ -1341,8 +1341,8 @@ packet_set_ipv4(struct dp_packet *packet, ovs_be32 src,
ovs_be32 dst,
if (nh->ip_ttl != ttl) {
uint8_t *field = &nh->ip_ttl;
- if (dp_packet_hwol_l3_ipv4(packet)) {
- dp_packet_ol_reset_ip_csum_good(packet);
+ if (dp_packet_ip_checksum_valid(packet)) {
+ dp_packet_ip_checksum_set_partial(packet);
} else {
nh->ip_csum = recalc_csum16(nh->ip_csum, htons(*field << 8),
htons(ttl << 8));
@@ -1979,8 +1979,8 @@ IP_ECN_set_ce(struct dp_packet *pkt, bool is_ipv6)
tos |= IP_ECN_CE;
if (nh->ip_tos != tos) {
- if (dp_packet_hwol_l3_ipv4(pkt)) {
- dp_packet_ol_reset_ip_csum_good(pkt);
+ if (dp_packet_ip_checksum_valid(pkt)) {
+ dp_packet_ip_checksum_set_partial(pkt);
} else {
nh->ip_csum = recalc_csum16(nh->ip_csum, htons(nh->ip_tos),
htons((uint16_t) tos));
diff --git a/tests/dpif-netdev.at b/tests/dpif-netdev.at
index ff226dee24..01cfcd56a9 100644
--- a/tests/dpif-netdev.at
+++ b/tests/dpif-netdev.at
@@ -783,6 +783,14 @@ AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0],
[${good_expected}
])
AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_bad=false])
+dnl Flag as Rx partial.
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=true])
+AT_CHECK([ovs-appctl netdev-dummy/receive p1 ${good_frame}])
+AT_CHECK([ovs-pcap p2.pcap > p2.pcap.txt 2>&1])
+AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0], [${good_expected}
+])
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=false])
+
dnl Checks for bad packet (Tx offloads disabled).
bad_frame=$(ovs-ofctl compose-packet --bare --bad-csum "${flow_s}")
bad_expected=$(ovs-ofctl compose-packet --bare --bad-csum "${flow_expected}")
@@ -810,6 +818,14 @@ AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0],
[${bad_expected}
])
AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_bad=false])
+dnl Flag as Rx partial.
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=true])
+AT_CHECK([ovs-appctl netdev-dummy/receive p1 ${bad_frame}])
+AT_CHECK([ovs-pcap p2.pcap > p2.pcap.txt 2>&1])
+AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0], [${good_expected}
+])
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=false])
+
dnl Checks for good packet (Tx offloads enabled).
AT_CHECK([ovs-vsctl set Interface p2 options:ol_ip_tx_csum=true])
@@ -835,6 +851,14 @@ AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0],
[${good_expected}
])
AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_bad=false])
+dnl Flag as Rx partial.
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=true])
+AT_CHECK([ovs-appctl netdev-dummy/receive p1 ${good_frame}])
+AT_CHECK([ovs-pcap p2.pcap > p2.pcap.txt 2>&1])
+AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0], [${good_expected}
+])
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=false])
+
dnl Checks for bad packet (Tx offloads enabled).
dnl No Rx flag.
@@ -860,6 +884,14 @@ AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0],
[${bad_expected}
])
AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_bad=false])
+dnl Flag as Rx partial.
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=true])
+AT_CHECK([ovs-appctl netdev-dummy/receive p1 ${bad_frame}])
+AT_CHECK([ovs-pcap p2.pcap > p2.pcap.txt 2>&1])
+AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0], [${good_expected}
+])
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=false])
+
dnl Special case, to check if Tx offload did happen in the driver.
AT_CHECK([ovs-vsctl set Interface p2 options:ol_ip_tx_csum_disabled=true])
AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_good=true])
@@ -1539,6 +1571,14 @@ AT_CHECK([ovs-appctl coverage/read-counter
conntrack_l3csum_err], [0], [1
])
AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_bad=false])
+dnl Flag as Rx partial.
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=true])
+AT_CHECK([ovs-appctl netdev-dummy/receive p1 ${good_frame}])
+AT_CHECK([ovs-pcap p2.pcap > p2.pcap.txt 2>&1])
+AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0], [${good_expected}
+])
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=false])
+
dnl Checks for bad packet (Tx offloads disabled).
bad_frame=$(ovs-ofctl compose-packet --bare --bad-csum "${flow_s}")
@@ -1567,6 +1607,14 @@ AT_CHECK([ovs-appctl coverage/read-counter
conntrack_l3csum_err], [0], [3
])
AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_bad=false])
+dnl Flag as Rx partial.
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=true])
+AT_CHECK([ovs-appctl netdev-dummy/receive p1 ${bad_frame}])
+AT_CHECK([ovs-pcap p2.pcap > p2.pcap.txt 2>&1])
+AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0], [${good_expected}
+])
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=false])
+
dnl Checks for good packet (Tx offloads enabled).
AT_CHECK([ovs-vsctl set Interface p2 options:ol_ip_tx_csum=true])
@@ -1601,6 +1649,18 @@ AT_CHECK([ovs-appctl coverage/read-counter
conntrack_l3csum_err], [0], [4
])
AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_bad=false])
+dnl Flag as Rx partial.
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=true])
+AT_CHECK([ovs-appctl netdev-dummy/receive p1 ${good_frame}])
+AT_CHECK([ovs-appctl coverage/read-counter conntrack_l3csum_checked], [0], [3
+])
+AT_CHECK([ovs-appctl coverage/read-counter conntrack_l3csum_err], [0], [4
+])
+AT_CHECK([ovs-pcap p2.pcap > p2.pcap.txt 2>&1])
+AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0], [${good_expected}
+])
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=false])
+
dnl Checks for bad packet (Tx offloads enabled).
dnl No Rx flag.
@@ -1632,6 +1692,19 @@ AT_CHECK([ovs-appctl coverage/read-counter
conntrack_l3csum_err], [0], [6
])
AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_bad=false])
+dnl Flag as Rx partial.
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=true])
+AT_CHECK([ovs-appctl netdev-dummy/receive p1 ${bad_frame}])
+AT_CHECK([ovs-appctl coverage/read-counter conntrack_l3csum_checked], [0], [4
+])
+AT_CHECK([ovs-appctl coverage/read-counter conntrack_l3csum_err], [0], [6
+])
+AT_CHECK([ovs-pcap p2.pcap > p2.pcap.txt 2>&1])
+dnl In this case, datapath will fix the csum as it trusts the Rx status.
+AT_CHECK_UNQUOTED([tail -n 1 p2.pcap.txt], [0], [${good_expected}
+])
+AT_CHECK([ovs-vsctl set Interface p1 options:ol_ip_rx_csum_set_partial=false])
+
dnl Special case, check natted ICMP (for traffic flagged good).
icmp_frame="0a8f393fe0738abf7e2f05840800450000440001000040010364c0a87b02c0a87b010303746c0000000045000028000100004006037bc0a87b01c0a87b021451d4780000000000000000500220002fc40000"
icmp_expected="0a8f393fe0738abf7e2f05840800450000440001000040017d64c0a87b02c0a801010303fa6b00000000450000280001000040067d7bc0a80101c0a87b021451d478000000000000000050022000a9c40000"
diff --git a/tests/tunnel-push-pop.at b/tests/tunnel-push-pop.at
index 35a9dc4944..7f9ec12066 100644
--- a/tests/tunnel-push-pop.at
+++ b/tests/tunnel-push-pop.at
@@ -538,6 +538,19 @@ AT_CHECK([ovs-ofctl dump-ports int-br | grep -E 'port
[[248]]:' | sort], [0], [
port 8: rx pkts=2, bytes=84, drop=?, errs=?, frame=?, over=?, crc=?
])
+dnl Idem, with Rx cksum partial
+AT_CHECK([ovs-vsctl set interface p0 options:ol_ip_rx_csum_set_partial=true])
+AT_CHECK([ovs-appctl netdev-dummy/receive p0
'aa55aa550000001b213cab6408004500004e00010000401173e90101025c0101025812b512b5003a00000800000000007b00fe71d883724fbeb6f4e1494a08004500001c0001000040013ede1e0000011e0000020000ffff00000000'])
+AT_CHECK([ovs-appctl netdev-dummy/receive p0
'aa55aa550000001b213cab6408004500004e00010000401173e90101025c0101025812b512b5003a02320800000000007b00fe71d883724fbeb6f4e1494a08004500001c0001000040013ede1e0000011e0000020000ffff00000000'])
+AT_CHECK([ovs-appctl netdev-dummy/receive p0
'aa55aa550000001b213cab6408004500004e00010000401173e90101025c0101025812b512b5003a202e0c00000300015900fe71d883724fbeb6f4e1494a08004500001c0001000040013ede1e0000011e0000020000ffff00000000'])
+AT_CHECK([ovs-vsctl set interface p0 options:ol_ip_rx_csum_set_partial=false])
+ovs-appctl time/warp 1000
+AT_CHECK([ovs-ofctl dump-ports int-br | grep -E 'port [[248]]:' | sort], [0],
[dnl
+ port 2: rx pkts=6, bytes=252, drop=?, errs=?, frame=?, over=?, crc=?
+ port 4: rx pkts=0, bytes=0, drop=?, errs=?, frame=?, over=?, crc=?
+ port 8: rx pkts=3, bytes=126, drop=?, errs=?, frame=?, over=?, crc=?
+])
+
dnl Check decapsulation of GRE packet
AT_CHECK([ovs-appctl netdev-dummy/receive p0
'aa55aa550000001b213cab6408004500007e79464000402fba550101025c0101025820006558000001c8fe71d883724fbeb6f4e1494a080045000054ba200000400184861e0000011e00000200004227e75400030af3195500000000f265010000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637'])
AT_CHECK([ovs-appctl netdev-dummy/receive p0
'aa55aa550000001b213cab6408004500007e79464000402fba550101025c0101025820006558000001c8fe71d883724fbeb6f4e1494a080045000054ba200000400184861e0000011e00000200004227e75400030af3195500000000f265010000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637'])
@@ -567,9 +580,9 @@ AT_CHECK([ovs-appctl coverage/read-counter
netdev_native_tnl_l3csum_err], [0], [
AT_CHECK([ovs-appctl coverage/read-counter netdev_native_tnl_l4csum_err], [0],
[0
])
dnl Yet csum validation happened on all previous packets.
-AT_CHECK([ovs-appctl coverage/read-counter netdev_native_tnl_l3csum_checked],
[0], [12
+AT_CHECK([ovs-appctl coverage/read-counter netdev_native_tnl_l3csum_checked],
[0], [9
])
-AT_CHECK([ovs-appctl coverage/read-counter netdev_native_tnl_l4csum_checked],
[0], [4
+AT_CHECK([ovs-appctl coverage/read-counter netdev_native_tnl_l4csum_checked],
[0], [6
])
dnl Send various incorrect bad IP checksum packets.
@@ -603,9 +616,9 @@ AT_CHECK([ovs-appctl coverage/read-counter
netdev_native_tnl_l3csum_err], [0], [
])
AT_CHECK([ovs-appctl coverage/read-counter netdev_native_tnl_l4csum_err], [0],
[3
])
-AT_CHECK([ovs-appctl coverage/read-counter netdev_native_tnl_l3csum_checked],
[0], [18
+AT_CHECK([ovs-appctl coverage/read-counter netdev_native_tnl_l3csum_checked],
[0], [15
])
-AT_CHECK([ovs-appctl coverage/read-counter netdev_native_tnl_l4csum_checked],
[0], [5
+AT_CHECK([ovs-appctl coverage/read-counter netdev_native_tnl_l4csum_checked],
[0], [7
])
AT_CHECK([ovs-appctl netdev-dummy/receive p0
'aa55aa550000001b213cab6408004503007079464000402fba600101025c0101025820000800000001c845000054ba200000400184861e0000011e00000200004227e75400030af3195500000000f265010000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637'])
--
2.48.1
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
