Re: Linux kernel error stack
Michal Kubecek wrote: > On Mon, Aug 06, 2018 at 01:15:37AM +0200, Florian Westphal wrote: > > Michal Kubecek wrote: > > > Oops, exactly this issue was already discussed almost a year ago: > > > > > > http://lkml.kernel.org/r/20170824104824.2c318a0...@unicorn.suse.cz > > > > > > But something more urgent came and I forgot to get back to it. :-( > > > > I did not even remeber, thanks for the pointer. > > So I think best course of action is to update man page to clearly > > say this only works in postrouting and with udp, and is ONLY > > intended for working around old dhcp software. > > As GSO for UDP is on its way to mainline, one might get into trouble > even with UDP if the rule is not specific enough. Yes, we still need a fix to ignore GSO too.
Re: Linux kernel error stack
On Mon, Aug 06, 2018 at 01:15:37AM +0200, Florian Westphal wrote: > Michal Kubecek wrote: > > Oops, exactly this issue was already discussed almost a year ago: > > > > http://lkml.kernel.org/r/20170824104824.2c318a0...@unicorn.suse.cz > > > > But something more urgent came and I forgot to get back to it. :-( > > I did not even remeber, thanks for the pointer. > So I think best course of action is to update man page to clearly > say this only works in postrouting and with udp, and is ONLY > intended for working around old dhcp software. As GSO for UDP is on its way to mainline, one might get into trouble even with UDP if the rule is not specific enough. > From kernel, have checkentry emit a warning > when this is used for protocols other than udp, > and make this a no-op for everything else. > > Same for postrouting: warn from checkentry if the hook is something > else, and have target function not do anything for !postrouting. > > Does that make sense? It sounds reasonable to me. > I can work on a patch later this week if needed. I'll be on vacation until Sunday so that I certainly won't get to it earlier than next week. Michal Kubecek
Re: Linux kernel error stack
Michal Kubecek wrote: > Oops, exactly this issue was already discussed almost a year ago: > > http://lkml.kernel.org/r/20170824104824.2c318a0...@unicorn.suse.cz > > But something more urgent came and I forgot to get back to it. :-( I did not even remeber, thanks for the pointer. So I think best course of action is to update man page to clearly say this only works in postrouting and with udp, and is ONLY intended for working around old dhcp software. >From kernel, have checkentry emit a warning when this is used for protocols other than udp, and make this a no-op for everything else. Same for postrouting: warn from checkentry if the hook is something else, and have target function not do anything for !postrouting. Does that make sense? I can work on a patch later this week if needed.
Re: Linux kernel error stack
On Sun, Aug 05, 2018 at 11:03:42PM +0200, Florian Westphal wrote: > Satish Patel wrote: > > Florian, > > > > It seems those rules coming from here > > https://github.com/openstack/openstack-ansible-os_neutron/blob/master/files/post-up-metadata-checksum > > This is crazy, and, as you found, it doesn't even do what they seem to > think it does. > > I see no reason for these rules at all. Oops, exactly this issue was already discussed almost a year ago: http://lkml.kernel.org/r/20170824104824.2c318a0...@unicorn.suse.cz But something more urgent came and I forgot to get back to it. :-( Michal Kubecek
Re: Linux kernel error stack
Satish Patel wrote: > Florian, > > It seems those rules coming from here > https://github.com/openstack/openstack-ansible-os_neutron/blob/master/files/post-up-metadata-checksum This is crazy, and, as you found, it doesn't even do what they seem to think it does. I see no reason for these rules at all.
Re: Linux kernel error stack
Satish Patel wrote: > After reading further related DHCP checksum issue, it seems we need > that rules when you running DHCP on same host machine where your guest > using host DHCP service, in that case virtual nic won't do checksum. > If your DHCP running on different host then your physical nic perform > checksum. Partially right, its only needed for older dhcp software that relies on packet-header checksum correctness (metadata contains info when checksum will be filled in by nic later, modern servers will use that and ignore the packet-header checksums in this case). If you remove the rule and your VMs still get leases via DHCP then the CHECKSUM workaround isn't needed.
Re: Linux kernel error stack
After reading further related DHCP checksum issue, it seems we need that rules when you running DHCP on same host machine where your guest using host DHCP service, in that case virtual nic won't do checksum. If your DHCP running on different host then your physical nic perform checksum. On Sun, Aug 5, 2018 at 4:39 PM, Satish Patel wrote: > Florian, > > I have removed those port 80 CHECKSUM rules and everything looks good > i didn't see kernel WARN mesg. > > Thank you so much! You just nailed it :) > > On Sun, Aug 5, 2018 at 4:15 PM, Satish Patel wrote: >> Florian, >> >> It seems those rules coming from here >> https://github.com/openstack/openstack-ansible-os_neutron/blob/master/files/post-up-metadata-checksum >> >> On Sun, Aug 5, 2018 at 4:09 PM, Satish Patel wrote: >>> Yes this is openstack-ansible deployment tool which set them up. I am >>> wondering where are these rules saved? I believe openstack-ansible use >>> LXC container to deploy services so must be part of LXC startup >>> scripts. >>> >>> I have checked there is no firewalld and iptables service running on >>> system.. >>> >>> You think i should get rid of all CHEKSUM option in iptables rules? Am i >>> right? >>> >>> >>> On Sun, Aug 5, 2018 at 4:02 PM, Florian Westphal wrote: Satish Patel wrote: > > [84166:59495417] -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM > > --checksum-fill > > [68739:5153476] -A POSTROUTING -p tcp -m tcp --sport 8000 -j CHECKSUM > > --checksum-fill These rules make no sense to me, and are also source of your backtrace. Who set this up? If this is coming from openstack, I suggest asking openstack developers WTH this is supposed to do. > > [755:275452] -A POSTROUTING -s 10.0.3.0/24 -o lxcbr0 -p udp -m udp > > --dport 68 -j CHECKSUM --checksum-fill This was needed to work around dhcpd issues w. checksum offloading but I guess that DCHCP will work fine without this rule too nowadays. So I suggest you simply get rid of these rules.
Re: Linux kernel error stack
Florian, I have removed those port 80 CHECKSUM rules and everything looks good i didn't see kernel WARN mesg. Thank you so much! You just nailed it :) On Sun, Aug 5, 2018 at 4:15 PM, Satish Patel wrote: > Florian, > > It seems those rules coming from here > https://github.com/openstack/openstack-ansible-os_neutron/blob/master/files/post-up-metadata-checksum > > On Sun, Aug 5, 2018 at 4:09 PM, Satish Patel wrote: >> Yes this is openstack-ansible deployment tool which set them up. I am >> wondering where are these rules saved? I believe openstack-ansible use >> LXC container to deploy services so must be part of LXC startup >> scripts. >> >> I have checked there is no firewalld and iptables service running on system.. >> >> You think i should get rid of all CHEKSUM option in iptables rules? Am i >> right? >> >> >> On Sun, Aug 5, 2018 at 4:02 PM, Florian Westphal wrote: >>> Satish Patel wrote: > [84166:59495417] -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM > --checksum-fill > [68739:5153476] -A POSTROUTING -p tcp -m tcp --sport 8000 -j CHECKSUM > --checksum-fill >>> >>> These rules make no sense to me, and are also source of your backtrace. >>> Who set this up? >>> >>> If this is coming from openstack, I suggest asking openstack developers >>> WTH this is supposed to do. >>> > [755:275452] -A POSTROUTING -s 10.0.3.0/24 -o lxcbr0 -p udp -m udp > --dport 68 -j CHECKSUM --checksum-fill >>> >>> This was needed to work around dhcpd issues w. checksum offloading but I >>> guess that DCHCP will work fine without this rule too nowadays. >>> >>> So I suggest you simply get rid of these rules.
Re: Linux kernel error stack
Florian, It seems those rules coming from here https://github.com/openstack/openstack-ansible-os_neutron/blob/master/files/post-up-metadata-checksum On Sun, Aug 5, 2018 at 4:09 PM, Satish Patel wrote: > Yes this is openstack-ansible deployment tool which set them up. I am > wondering where are these rules saved? I believe openstack-ansible use > LXC container to deploy services so must be part of LXC startup > scripts. > > I have checked there is no firewalld and iptables service running on system.. > > You think i should get rid of all CHEKSUM option in iptables rules? Am i > right? > > > On Sun, Aug 5, 2018 at 4:02 PM, Florian Westphal wrote: >> Satish Patel wrote: >>> > [84166:59495417] -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM >>> > --checksum-fill >>> > [68739:5153476] -A POSTROUTING -p tcp -m tcp --sport 8000 -j CHECKSUM >>> > --checksum-fill >> >> These rules make no sense to me, and are also source of your backtrace. >> Who set this up? >> >> If this is coming from openstack, I suggest asking openstack developers >> WTH this is supposed to do. >> >>> > [755:275452] -A POSTROUTING -s 10.0.3.0/24 -o lxcbr0 -p udp -m udp >>> > --dport 68 -j CHECKSUM --checksum-fill >> >> This was needed to work around dhcpd issues w. checksum offloading but I >> guess that DCHCP will work fine without this rule too nowadays. >> >> So I suggest you simply get rid of these rules.
Re: Linux kernel error stack
Yes this is openstack-ansible deployment tool which set them up. I am wondering where are these rules saved? I believe openstack-ansible use LXC container to deploy services so must be part of LXC startup scripts. I have checked there is no firewalld and iptables service running on system.. You think i should get rid of all CHEKSUM option in iptables rules? Am i right? On Sun, Aug 5, 2018 at 4:02 PM, Florian Westphal wrote: > Satish Patel wrote: >> > [84166:59495417] -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM >> > --checksum-fill >> > [68739:5153476] -A POSTROUTING -p tcp -m tcp --sport 8000 -j CHECKSUM >> > --checksum-fill > > These rules make no sense to me, and are also source of your backtrace. > Who set this up? > > If this is coming from openstack, I suggest asking openstack developers > WTH this is supposed to do. > >> > [755:275452] -A POSTROUTING -s 10.0.3.0/24 -o lxcbr0 -p udp -m udp >> > --dport 68 -j CHECKSUM --checksum-fill > > This was needed to work around dhcpd issues w. checksum offloading but I > guess that DCHCP will work fine without this rule too nowadays. > > So I suggest you simply get rid of these rules.
Re: Linux kernel error stack
Satish Patel wrote: > > [84166:59495417] -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM > > --checksum-fill > > [68739:5153476] -A POSTROUTING -p tcp -m tcp --sport 8000 -j CHECKSUM > > --checksum-fill These rules make no sense to me, and are also source of your backtrace. Who set this up? If this is coming from openstack, I suggest asking openstack developers WTH this is supposed to do. > > [755:275452] -A POSTROUTING -s 10.0.3.0/24 -o lxcbr0 -p udp -m udp > > --dport 68 -j CHECKSUM --checksum-fill This was needed to work around dhcpd issues w. checksum offloading but I guess that DCHCP will work fine without this rule too nowadays. So I suggest you simply get rid of these rules.
Re: Linux kernel error stack
I am configured where are those rules stored why iptables -L -n -t nat not showing them? On Sun, Aug 5, 2018 at 3:56 PM, Satish Patel wrote: > Oh wait!! I think you are right there i didn't notice that iptables > has following setting. > > "CHECKSUM --checksum-fill" > > On Sun, Aug 5, 2018 at 3:54 PM, Satish Patel wrote: >> I have following kernel logging set. >> >> [root@ostack-infra-02 tools]# cat /proc/sys/kernel/printk >> 3 4 1 3 >> >> >> iptables output >> >> [root@ostack-infra-02 tools]# iptables-save -c >> # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 >> *raw >> :PREROUTING ACCEPT [42284573:38782693391] >> :OUTPUT ACCEPT [219668:42675751] >> COMMIT >> # Completed on Sun Aug 5 15:52:56 2018 >> # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 >> *nat >> :PREROUTING ACCEPT [656413:38450886] >> :INPUT ACCEPT [5524:564390] >> :OUTPUT ACCEPT [3217:212903] >> :POSTROUTING ACCEPT [559058:34193539] >> [5197:311820] -A POSTROUTING -s 10.0.3.0/24 ! -d 10.0.3.0/24 -j MASQUERADE >> COMMIT >> # Completed on Sun Aug 5 15:52:56 2018 >> # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 >> *mangle >> :PREROUTING ACCEPT [42209568:38687818558] >> :INPUT ACCEPT [170634:197796840] >> :FORWARD ACCEPT [42939243:38526139954] >> :OUTPUT ACCEPT [198956:41288636] >> :POSTROUTING ACCEPT [43138199:38567428590] >> [84166:59495417] -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM >> --checksum-fill >> [68739:5153476] -A POSTROUTING -p tcp -m tcp --sport 8000 -j CHECKSUM >> --checksum-fill >> [755:275452] -A POSTROUTING -s 10.0.3.0/24 -o lxcbr0 -p udp -m udp >> --dport 68 -j CHECKSUM --checksum-fill >> COMMIT >> # Completed on Sun Aug 5 15:52:56 2018 >> # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 >> *filter >> :INPUT ACCEPT [164954:197217701] >> :FORWARD ACCEPT [42801701:38377075499] >> :OUTPUT ACCEPT [198963:41289860] >> [0:0] -A INPUT -i lxcbr0 -p tcp -m tcp --dport 53 -j ACCEPT >> [4932:328918] -A INPUT -i lxcbr0 -p udp -m udp --dport 53 -j ACCEPT >> [0:0] -A INPUT -i lxcbr0 -p tcp -m tcp --dport 67 -j ACCEPT >> [755:250585] -A INPUT -i lxcbr0 -p udp -m udp --dport 67 -j ACCEPT >> [80745:145594351] -A FORWARD -o lxcbr0 -j ACCEPT >> [56801:3471569] -A FORWARD -i lxcbr0 -j ACCEPT >> COMMIT >> # Completed on Sun Aug 5 15:52:56 2018 >> >> On Sun, Aug 5, 2018 at 3:42 PM, Florian Westphal wrote: >>> Satish Patel wrote: Thanks Florian, FYI, I don't have any CHECKSUM configure in my iptables, >>> >>> You have, according to WARN stacktrace you provided. >>> >>> iptables-save -c >>> ip6tables-save -c
Re: Linux kernel error stack
Oh wait!! I think you are right there i didn't notice that iptables has following setting. "CHECKSUM --checksum-fill" On Sun, Aug 5, 2018 at 3:54 PM, Satish Patel wrote: > I have following kernel logging set. > > [root@ostack-infra-02 tools]# cat /proc/sys/kernel/printk > 3 4 1 3 > > > iptables output > > [root@ostack-infra-02 tools]# iptables-save -c > # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 > *raw > :PREROUTING ACCEPT [42284573:38782693391] > :OUTPUT ACCEPT [219668:42675751] > COMMIT > # Completed on Sun Aug 5 15:52:56 2018 > # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 > *nat > :PREROUTING ACCEPT [656413:38450886] > :INPUT ACCEPT [5524:564390] > :OUTPUT ACCEPT [3217:212903] > :POSTROUTING ACCEPT [559058:34193539] > [5197:311820] -A POSTROUTING -s 10.0.3.0/24 ! -d 10.0.3.0/24 -j MASQUERADE > COMMIT > # Completed on Sun Aug 5 15:52:56 2018 > # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 > *mangle > :PREROUTING ACCEPT [42209568:38687818558] > :INPUT ACCEPT [170634:197796840] > :FORWARD ACCEPT [42939243:38526139954] > :OUTPUT ACCEPT [198956:41288636] > :POSTROUTING ACCEPT [43138199:38567428590] > [84166:59495417] -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM > --checksum-fill > [68739:5153476] -A POSTROUTING -p tcp -m tcp --sport 8000 -j CHECKSUM > --checksum-fill > [755:275452] -A POSTROUTING -s 10.0.3.0/24 -o lxcbr0 -p udp -m udp > --dport 68 -j CHECKSUM --checksum-fill > COMMIT > # Completed on Sun Aug 5 15:52:56 2018 > # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 > *filter > :INPUT ACCEPT [164954:197217701] > :FORWARD ACCEPT [42801701:38377075499] > :OUTPUT ACCEPT [198963:41289860] > [0:0] -A INPUT -i lxcbr0 -p tcp -m tcp --dport 53 -j ACCEPT > [4932:328918] -A INPUT -i lxcbr0 -p udp -m udp --dport 53 -j ACCEPT > [0:0] -A INPUT -i lxcbr0 -p tcp -m tcp --dport 67 -j ACCEPT > [755:250585] -A INPUT -i lxcbr0 -p udp -m udp --dport 67 -j ACCEPT > [80745:145594351] -A FORWARD -o lxcbr0 -j ACCEPT > [56801:3471569] -A FORWARD -i lxcbr0 -j ACCEPT > COMMIT > # Completed on Sun Aug 5 15:52:56 2018 > > On Sun, Aug 5, 2018 at 3:42 PM, Florian Westphal wrote: >> Satish Patel wrote: >>> Thanks Florian, >>> >>> FYI, I don't have any CHECKSUM configure in my iptables, >> >> You have, according to WARN stacktrace you provided. >> >> iptables-save -c >> ip6tables-save -c
Re: Linux kernel error stack
I have following kernel logging set. [root@ostack-infra-02 tools]# cat /proc/sys/kernel/printk 3 4 1 3 iptables output [root@ostack-infra-02 tools]# iptables-save -c # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 *raw :PREROUTING ACCEPT [42284573:38782693391] :OUTPUT ACCEPT [219668:42675751] COMMIT # Completed on Sun Aug 5 15:52:56 2018 # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 *nat :PREROUTING ACCEPT [656413:38450886] :INPUT ACCEPT [5524:564390] :OUTPUT ACCEPT [3217:212903] :POSTROUTING ACCEPT [559058:34193539] [5197:311820] -A POSTROUTING -s 10.0.3.0/24 ! -d 10.0.3.0/24 -j MASQUERADE COMMIT # Completed on Sun Aug 5 15:52:56 2018 # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 *mangle :PREROUTING ACCEPT [42209568:38687818558] :INPUT ACCEPT [170634:197796840] :FORWARD ACCEPT [42939243:38526139954] :OUTPUT ACCEPT [198956:41288636] :POSTROUTING ACCEPT [43138199:38567428590] [84166:59495417] -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill [68739:5153476] -A POSTROUTING -p tcp -m tcp --sport 8000 -j CHECKSUM --checksum-fill [755:275452] -A POSTROUTING -s 10.0.3.0/24 -o lxcbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill COMMIT # Completed on Sun Aug 5 15:52:56 2018 # Generated by iptables-save v1.4.21 on Sun Aug 5 15:52:56 2018 *filter :INPUT ACCEPT [164954:197217701] :FORWARD ACCEPT [42801701:38377075499] :OUTPUT ACCEPT [198963:41289860] [0:0] -A INPUT -i lxcbr0 -p tcp -m tcp --dport 53 -j ACCEPT [4932:328918] -A INPUT -i lxcbr0 -p udp -m udp --dport 53 -j ACCEPT [0:0] -A INPUT -i lxcbr0 -p tcp -m tcp --dport 67 -j ACCEPT [755:250585] -A INPUT -i lxcbr0 -p udp -m udp --dport 67 -j ACCEPT [80745:145594351] -A FORWARD -o lxcbr0 -j ACCEPT [56801:3471569] -A FORWARD -i lxcbr0 -j ACCEPT COMMIT # Completed on Sun Aug 5 15:52:56 2018 On Sun, Aug 5, 2018 at 3:42 PM, Florian Westphal wrote: > Satish Patel wrote: >> Thanks Florian, >> >> FYI, I don't have any CHECKSUM configure in my iptables, > > You have, according to WARN stacktrace you provided. > > iptables-save -c > ip6tables-save -c
Re: Linux kernel error stack
Satish Patel wrote: > Thanks Florian, > > FYI, I don't have any CHECKSUM configure in my iptables, You have, according to WARN stacktrace you provided. iptables-save -c ip6tables-save -c
Re: Linux kernel error stack
Thanks Florian,
FYI, I don't have any CHECKSUM configure in my iptables, i have
following rules, also do you think this kernel WARNNING is just
warning and not impacting my performance, based on that i have to
decided criticality of this issue.
[root@ostack-infra-02 ~]# iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- 0.0.0.0/00.0.0.0/0tcp dpt:53
ACCEPT udp -- 0.0.0.0/00.0.0.0/0udp dpt:53
ACCEPT tcp -- 0.0.0.0/00.0.0.0/0tcp dpt:67
ACCEPT udp -- 0.0.0.0/00.0.0.0/0udp dpt:67
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 0.0.0.0/00.0.0.0/0
ACCEPT all -- 0.0.0.0/00.0.0.0/0
On Sun, Aug 5, 2018 at 5:48 AM, Florian Westphal wrote:
> Satish Patel wrote:
>> I am installing openstack and as you know i have lots of bridges and
>> vlan interface on my Linux CentOS 7.5
>>
>> I was getting following error stack on 3.10 kernel and found this is
>> kernel bug which required kernel upgrade so now i have upgraded my
>> kernel to 4.17.12 but i am still seeing same kernel stack error on my
>> dmesg
>>
>> I have disable TSO, LRO, SG & GSO on my NIC but still getting error
>> just wanted to understand what is this and why it popping up
>
> Get rid of CHECKSUM target in the iptables rules.
> This thing was added 8 years ago to work around dhcp bugs, I don't
> think its use is needed anymore.
> Try removing it and see that all VMs can still retrieve IP address
> via DHCP.
>
> I'm curious as to the rules, normally CHECKSUM target should be
> limited to -p udp --dport bootp; its bad idea to feed it normal
> packets, its expensive to do this in software rather than have device
> do the checksumming.
>
> As for fix, I'm tempted to send patch to make checksum target
> eval a no-op & add deprecation warning on init...
>
> Other "fix" is to
>
> diff --git a/net/netfilter/xt_CHECKSUM.c b/net/netfilter/xt_CHECKSUM.c
> index 9f4151ec3e06..23a17dda604d 100644
> --- a/net/netfilter/xt_CHECKSUM.c
> +++ b/net/netfilter/xt_CHECKSUM.c
> @@ -25,8 +25,12 @@ MODULE_ALIAS("ip6t_CHECKSUM");
> static unsigned int
> checksum_tg(struct sk_buff *skb, const struct xt_action_param *par)
> {
> - if (skb->ip_summed == CHECKSUM_PARTIAL)
> - skb_checksum_help(skb);
> + if (skb->ip_summed == CHECKSUM_PARTIAL) {
> + if (skb_shinfo(skb)->gso_size)
> + skb->ip_summed = CHECKSUM_NONE;
> + else
> + skb_checksum_help(skb);
> + }
>
> return XT_CONTINUE;
> }
>
> (unfortunately, the target isn't restricted to POSTROUTING, sigh).
Re: Linux kernel error stack
Satish Patel wrote:
> I am installing openstack and as you know i have lots of bridges and
> vlan interface on my Linux CentOS 7.5
>
> I was getting following error stack on 3.10 kernel and found this is
> kernel bug which required kernel upgrade so now i have upgraded my
> kernel to 4.17.12 but i am still seeing same kernel stack error on my
> dmesg
>
> I have disable TSO, LRO, SG & GSO on my NIC but still getting error
> just wanted to understand what is this and why it popping up
Get rid of CHECKSUM target in the iptables rules.
This thing was added 8 years ago to work around dhcp bugs, I don't
think its use is needed anymore.
Try removing it and see that all VMs can still retrieve IP address
via DHCP.
I'm curious as to the rules, normally CHECKSUM target should be
limited to -p udp --dport bootp; its bad idea to feed it normal
packets, its expensive to do this in software rather than have device
do the checksumming.
As for fix, I'm tempted to send patch to make checksum target
eval a no-op & add deprecation warning on init...
Other "fix" is to
diff --git a/net/netfilter/xt_CHECKSUM.c b/net/netfilter/xt_CHECKSUM.c
index 9f4151ec3e06..23a17dda604d 100644
--- a/net/netfilter/xt_CHECKSUM.c
+++ b/net/netfilter/xt_CHECKSUM.c
@@ -25,8 +25,12 @@ MODULE_ALIAS("ip6t_CHECKSUM");
static unsigned int
checksum_tg(struct sk_buff *skb, const struct xt_action_param *par)
{
- if (skb->ip_summed == CHECKSUM_PARTIAL)
- skb_checksum_help(skb);
+ if (skb->ip_summed == CHECKSUM_PARTIAL) {
+ if (skb_shinfo(skb)->gso_size)
+ skb->ip_summed = CHECKSUM_NONE;
+ else
+ skb_checksum_help(skb);
+ }
return XT_CONTINUE;
}
(unfortunately, the target isn't restricted to POSTROUTING, sigh).
Linux kernel error stack
Folks, I am installing openstack and as you know i have lots of bridges and vlan interface on my Linux CentOS 7.5 I was getting following error stack on 3.10 kernel and found this is kernel bug which required kernel upgrade so now i have upgraded my kernel to 4.17.12 but i am still seeing same kernel stack error on my dmesg I have disable TSO, LRO, SG & GSO on my NIC but still getting error just wanted to understand what is this and why it popping up [ 8177.718745] [ cut here ] [ 8177.718748] lxcbr0: caps=(0x0801ffc070a8, 0x) len=7292 data_len=7240 gso_size=1448 gso_type=1 ip_summed=3 [ 8177.718756] WARNING: CPU: 21 PID: 139 at net/core/dev.c:2662 skb_warn_bad_offload+0xd8/0xef [ 8177.718757] Modules linked in: ebtable_filter ebtable_nat tcp_diag udp_diag inet_diag unix_diag btrfs xor zstd_decompress zstd_compress xxhash raid6_pq loop iptable_raw veth bonding ip6table_mangle xt_CHECKSUM rpcrdma sunrpc ib_isert iscsi_target_mod ib_iser ib_srpt target_core_mod ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel crypto_simd cryptd glue_helper intel_cstate intel_rapl_perf iTCO_wdt iTCO_vendor_support ocrdma ib_core joydev input_leds pcspkr ipmi_si hpwdt be2iscsi lpc_ich ipmi_devintf sg hpilo mfd_core ipmi_msghandler video acpi_power_meter iscsi_boot_sysfs ioatdma shpchp dca pcc_cpufreq xfs sd_mod mgag200 [ 8177.718807] i2c_algo_bit serio_raw drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm hpsa scsi_transport_sas be2net dm_mirror dm_region_hash dm_log vhost_net tun tap vhost iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_vs iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 iptable_mangle iptable_filter ipt_REJECT nf_reject_ipv4 ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_nat nf_conntrack libcrc32c crc32c_intel ip6table_filter ip6_tables ebtables dm_snapshot dm_bufio dm_multipath dm_mod 8021q garp mrp ip_tables [ 8177.718833] CPU: 21 PID: 139 Comm: ksoftirqd/21 Tainted: GW 4.17.12-1.el7.elrepo.x86_64 #1 [ 8177.718833] Hardware name: HP ProLiant BL460c Gen8, BIOS I31 06/01/2015 [ 8177.718835] RIP: 0010:skb_warn_bad_offload+0xd8/0xef [ 8177.718836] RSP: :c90003797470 EFLAGS: 00010282 [ 8177.718837] RAX: RBX: 8807aaa09c00 RCX: 0006 [ 8177.718838] RDX: RSI: 0096 RDI: 88042f8d69d0 [ 8177.718840] RBP: c900037974a0 R08: R09: 0d96 [ 8177.718840] R10: 0003 R11: 0d95 R12: 88082a035000 [ 8177.718841] R13: 0003 R14: 8807aaa09c00 R15: a0004618 [ 8177.718843] FS: () GS:88042f8c() knlGS: [ 8177.718844] CS: 0010 DS: ES: CR0: 80050033 [ 8177.718845] CR2: 55ac69aea158 CR3: 0220a001 CR4: 001606e0 [ 8177.718846] Call Trace: [ 8177.718849] skb_checksum_help+0x1a5/0x1c0 [ 8177.718851] checksum_tg+0x22/0x29 [xt_CHECKSUM] [ 8177.718854] ipt_do_table+0x2e7/0x630 [ip_tables] [ 8177.718858] ? br_dev_queue_push_xmit+0x170/0x170 [ 8177.718860] ? br_forward+0x92/0xa0 [ 8177.718862] ? br_dev_xmit+0x23f/0x3c0 [ 8177.718865] iptable_mangle_hook+0x37/0xf0 [iptable_mangle] [ 8177.718868] nf_hook_slow+0x42/0xc0 [ 8177.718870] ip_output+0xd0/0xe0 [ 8177.718873] ? ip_fragment.constprop.49+0x80/0x80 [ 8177.718875] ip_forward_finish+0x49/0x70 [ 8177.718877] ip_forward+0x388/0x470 [ 8177.718880] ? pskb_trim_rcsum.part.11+0x4c/0x4c [ 8177.718882] ip_rcv_finish+0x120/0x400 [ 8177.718885] ip_sabotage_in+0x35/0x40 [ 8177.71] nf_hook_slow+0x42/0xc0 [ 8177.718890] ip_rcv+0x32f/0x410 [ 8177.718892] ? inet_del_offload+0x40/0x40 [ 8177.718895] __netif_receive_skb_core+0x401/0xb10 [ 8177.718896] ? poll_idle+0x90/0x9a [ 8177.718898] ? ebt_do_table+0x663/0x73a [ebtables] [ 8177.718901] __netif_receive_skb+0x18/0x60 [ 8177.718903] netif_receive_skb_internal+0x45/0xf0 [ 8177.718904] netif_receive_skb+0x1c/0x70 [ 8177.718906] br_netif_receive_skb+0x28/0x60 [ 8177.718909] br_pass_frame_up+0xcd/0x160 [ 8177.718913] ? __br_handle_local_finish+0xa0/0xa0 [ 8177.718915] br_handle_frame_finish+0x23a/0x450 [ 8177.718917] ? br_pass_frame_up+0x160/0x160 [ 8177.718919] br_nf_hook_thresh+0xdf/0xf0 [ 8177.718922] ? br_pass_frame_up+0x160/0x160 [ 8177.718924] br_nf_pre_routing_finish+0x1a1/0x380 [ 8177.718926] ? br_pass_frame_up+0x160/0x160 [ 8177.718928] ? nf_nat_ipv4_in+0x2e/0x80 [nf_nat_ipv4] [ 8177.718930] ? iptable_nat_ipv4_in+0x15/0x20 [iptable_nat] [ 8177.718932] br_nf_pre_routing+0x2d6/0x3f0 [ 8177.718934] ? br_nf_forward_arp+0x270/0x270 [ 8177.718936] nf_hook_slow+0x42/0xc0 [ 8177.718939] br_handle_frame+0x1ef/0x300 [ 8177.718941] ? br_pass_frame_up+0x160/0x160 [ 8177.718945] __netif_receive_skb_core+0x20d/0xb10 [
