Re: Linux 4.2.4
On 08.11.2015 18:20, Greg KH wrote: On Sun, Nov 08, 2015 at 02:51:01PM +0100, Gerhard Wiesinger wrote: On 25.10.2015 17:29, Greg KH wrote: On Sun, Oct 25, 2015 at 11:48:54AM +0100, Gerhard Wiesinger wrote: On 25.10.2015 10:46, Willy Tarreau wrote: ipset *triggered* the problem. The whole stack dump would tell more. OK, find the stack traces in the bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1272645 Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands and IPv6, details in the bug report Kernel 4.2 seems to me not well tested in the netfilter parts at all (Bug with already known bugfix https://lists.debian.org/debian-kernel/2015/10/msg00034.html was triggered on 2 of 3 of my machines, the new bug on 1 of 1 tested machine). There's a reason why Greg maintains stable and LTS kernels :-) Stable kernels don't crash but definiton. :-) At least triggered 2 kernel panics in 5min, even with 4.1.10 and ipset commands ... Does this happen also with Linus's tree? I suggest you ask the networking developers about this on netdev@vger.kernel.org, there's nothing that I can do on my own about this, sorry. Patch is now available, see: [PATCH 0/3] ipset patches for nf https://marc.info/?l=netfilter-devel=144690007708041=2 https://marc.info/?l=netfilter-devel=144690007808042=2 https://marc.info/?l=netfilter-devel=144690008608043=2 https://marc.info/?l=netfilter-devel=144690007708039=2 [ANNOUNCE] ipset 6.27 released https://marc.info/?l=netfilter-devel=144690048308099=2 Requires also new userland ipset version. Please integrate it upstream. Thanx to Jozsef Kadlecsik for fixing it. That's great, can you let me know the git commits that end up in Linus's tree? That's what we need for the stable kernel. Find the commits here: https://git.kernel.org/cgit/linux/kernel/git/pablo/nf.git/ https://git.kernel.org/cgit/linux/kernel/git/pablo/nf.git/commit/?id=e75cb467df29a428612c162e6f1451c5c0717091 Don't know exactly the merging processes, so feel free to merge or contact Pablo. Ciao, Gerhard -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 4.2.4
On 25.10.2015 17:29, Greg KH wrote: On Sun, Oct 25, 2015 at 11:48:54AM +0100, Gerhard Wiesinger wrote: On 25.10.2015 10:46, Willy Tarreau wrote: ipset *triggered* the problem. The whole stack dump would tell more. OK, find the stack traces in the bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1272645 Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands and IPv6, details in the bug report Kernel 4.2 seems to me not well tested in the netfilter parts at all (Bug with already known bugfix https://lists.debian.org/debian-kernel/2015/10/msg00034.html was triggered on 2 of 3 of my machines, the new bug on 1 of 1 tested machine). There's a reason why Greg maintains stable and LTS kernels :-) Stable kernels don't crash but definiton. :-) At least triggered 2 kernel panics in 5min, even with 4.1.10 and ipset commands ... Does this happen also with Linus's tree? I suggest you ask the networking developers about this on netdev@vger.kernel.org, there's nothing that I can do on my own about this, sorry. Patch is now available, see: [PATCH 0/3] ipset patches for nf https://marc.info/?l=netfilter-devel=144690007708041=2 https://marc.info/?l=netfilter-devel=144690007808042=2 https://marc.info/?l=netfilter-devel=144690008608043=2 https://marc.info/?l=netfilter-devel=144690007708039=2 [ANNOUNCE] ipset 6.27 released https://marc.info/?l=netfilter-devel=144690048308099=2 Requires also new userland ipset version. Please integrate it upstream. Thanx to Jozsef Kadlecsik for fixing it. Ciao, Gerhard -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 4.2.4
On 25.10.2015 22:53, Jozsef Kadlecsik wrote: On Sun, 25 Oct 2015, Gerhard Wiesinger wrote: Any further ideas? Does it crash without counters? That could narrow down where to look for. Hello Jozsef, it doesn't crash i I don't use the counters so far. So there must be a bug with the counters. Any idea for the root cause? Thnx. Ciao, Gerhard -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 4.2.4
On 26.10.2015 09:58, Jozsef Kadlecsik wrote: On Sun, 25 Oct 2015, Gerhard Wiesinger wrote: Also any idea regarding the second isssue? Or do you think it has the same root cause? Looking at your RedHat bugzilla report, the "nf_conntrack: table full, dropping packet" and "Alignment trap: not handling instruction" are two unrelated issues and the second one is triggered by the unaligned counter extension acccess in ipset, I'm investigating. I can't think of any reason how those issues could be related to each other. Yes, they are unrelated. Issue 1: nf_conntrack: table full, dropping packet => Fixed with 4.2.4 Issue 2: Alignment trap: not handling instruction => Happens when ipset counters are enabled Please keep in mind it happens with IPv6 commands. Currently 4.2.4 without ipset counters runs well. Ciao, Gerhard -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: IPv6 and private net with masquerading not working correctly
Any update on this issue? Thank you. Ciao, Gerhard On 10.08.2015 19:39, Cong Wang wrote: (Cc'ing netdev and netfilter-devel) On Fri, Aug 7, 2015 at 6:00 AM, Gerhard Wiesinger <li...@wiesinger.com> wrote: On 06.08.2015 20:43, Gerhard Wiesinger wrote: Hello, I'm having the following problem with IPv6 and a private internal LAN which will be masqueraded to the public internet (I don't want to have public IPs in the LAN because of some static IPs and tracking) . Rules are generated by shorewall. Problem is that ICMP6 packets source address is not translated by the kernel on the reply when MTU has to be discovered because of too big packets and limited MTU capabilities on the path (happens also on tcp6 which works thereofore not correctly). # From an internal host on net fd00:1234:5678::/64 ping6 -s 2000 2a02:1234:5678:7::2 /etc/shorewall6/masq EXT_IF fc00::/7 ip6tables rule: MASQUERADE all * * fc00::/7 ::/0 # Internal interface IP6 fd00:1234:5678::9 > 2a02:1234:5678:7::2: frag (0|1432) ICMP6, echo request, seq 1, length 1432 IP6 fd00:1234:5678::9 > 2a02:1234:5678:7::2: frag (1432|576) IP6 2a02:1234:5678:9abc::115 > fd00:1234:5678::9: ICMP6, packet too big, mtu 1440, length 1240 # External interface IP6 2001:1234:5678:9abc::1 > 2a02:1234:5678:7::2: frag (0|1432) ICMP6, echo request, seq 1, length 1432 IP6 2001:1234:5678:9abc::1 > 2a02:1234:5678:7::2: frag (1432|576) IP6 2a02:1234:5678:9abc::115 > 2001:1234:5678:9abc::1: ICMP6, packet too big, mtu 1440, length 1240 Looks to me like a a major kernel bug. Kernel version is: 4.1.3-201.fc22.x86_64 from Fedora 22 Any ideas? Any comments? Ciao, Gerhard -- http://www.wiesinger.com/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 4.2.4
On 25.10.2015 10:46, Willy Tarreau wrote: ipset *triggered* the problem. The whole stack dump would tell more. OK, find the stack traces in the bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1272645 Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands and IPv6, details in the bug report Kernel 4.2 seems to me not well tested in the netfilter parts at all (Bug with already known bugfix https://lists.debian.org/debian-kernel/2015/10/msg00034.html was triggered on 2 of 3 of my machines, the new bug on 1 of 1 tested machine). There's a reason why Greg maintains stable and LTS kernels :-) Stable kernels don't crash but definiton. :-) At least triggered 2 kernel panics in 5min, even with 4.1.10 and ipset commands ... Ciao, Gerhard -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 4.2.4
On 25.10.2015 21:08, Gerhard Wiesinger wrote:
On 25.10.2015 20:46, Jozsef Kadlecsik wrote:
Hi,
On Sun, 25 Oct 2015, Gerhard Wiesinger wrote:
On 25.10.2015 10:46, Willy Tarreau wrote:
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset
commands
and IPv6, details in the bug report
It seems to me it is an architecture-specific alignment issue. I don't
have a Cortex-A7 ARM hardware and qemu doesn't seem to support it
either,
so I'm unable to reproduce it (ipset passes all my tests on my hardware,
including more complex ones than what breaks here). My first wild
guess is
that the dynamic array of the element structure is not aligned properly.
Could you give a try to the next patch?
diff --git a/net/netfilter/ipset/ip_set_hash_gen.h
b/net/netfilter/ipset/ip_set_hash_gen.h
index afe905c..1cf357d 100644
--- a/net/netfilter/ipset/ip_set_hash_gen.h
+++ b/net/netfilter/ipset/ip_set_hash_gen.h
@@ -1211,6 +1211,9 @@ static const struct ip_set_type_variant
mtype_variant = {
.same_set = mtype_same_set,
};
+#define IP_SET_BASE_ALIGN(dtype)\
+ALIGN(sizeof(struct dtype), __alignof__(struct dtype))
+
#ifdef IP_SET_EMIT_CREATE
static int
IPSET_TOKEN(HTYPE, _create)(struct net *net, struct ip_set *set,
@@ -1319,12 +1322,12 @@ IPSET_TOKEN(HTYPE, _create)(struct net *net,
struct ip_set *set,
#endif
set->variant = _TOKEN(HTYPE, 4_variant);
set->dsize = ip_set_elem_len(set, tb,
-sizeof(struct IPSET_TOKEN(HTYPE, 4_elem)));
+IP_SET_BASE_ALIGN(IPSET_TOKEN(HTYPE, 4_elem)));
#ifndef IP_SET_PROTO_UNDEF
} else {
set->variant = _TOKEN(HTYPE, 6_variant);
set->dsize = ip_set_elem_len(set, tb,
-sizeof(struct IPSET_TOKEN(HTYPE, 6_elem)));
+IP_SET_BASE_ALIGN(IPSET_TOKEN(HTYPE, 6_elem)));
}
#endif
if (tb[IPSET_ATTR_TIMEOUT]) {
If that does not solve it, then could you help to narrow down the issue?
Does the bug still appear if your remove the counter extension of the
set?
Hello Jozsef,
Patch applied well, compiling ...
Hello Jozsef,
Thank you for the patch it but still crashes, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Any further ideas?
Thank you.
Ciao,
Gerhard
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 4.2.4
On 25.10.2015 17:29, Greg KH wrote: On Sun, Oct 25, 2015 at 11:48:54AM +0100, Gerhard Wiesinger wrote: On 25.10.2015 10:46, Willy Tarreau wrote: ipset *triggered* the problem. The whole stack dump would tell more. OK, find the stack traces in the bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1272645 Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands and IPv6, details in the bug report Kernel 4.2 seems to me not well tested in the netfilter parts at all (Bug with already known bugfix https://lists.debian.org/debian-kernel/2015/10/msg00034.html was triggered on 2 of 3 of my machines, the new bug on 1 of 1 tested machine). There's a reason why Greg maintains stable and LTS kernels :-) Stable kernels don't crash but definiton. :-) At least triggered 2 kernel panics in 5min, even with 4.1.10 and ipset commands ... Does this happen also with Linus's tree? I suggest you ask the networking developers about this on netdev@vger.kernel.org, there's nothing that I can do on my own about this, sorry. Already CCed netdev and netfilter-devel mailinglist. Need patches for the switch driver of the banana Pi to get networking up but that patch is stable. Maybe also some patches from the Fedora SRPMS are needed. But I'm pretty sure that this also happens with plain vanilla kernel. Ciao, Gerhard -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 4.2.4
On 25.10.2015 20:46, Jozsef Kadlecsik wrote:
Hi,
On Sun, 25 Oct 2015, Gerhard Wiesinger wrote:
On 25.10.2015 10:46, Willy Tarreau wrote:
ipset *triggered* the problem. The whole stack dump would tell more.
OK, find the stack traces in the bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1272645
Kernel 4.1.10 triggered also a kernel dump when playing with ipset commands
and IPv6, details in the bug report
It seems to me it is an architecture-specific alignment issue. I don't
have a Cortex-A7 ARM hardware and qemu doesn't seem to support it either,
so I'm unable to reproduce it (ipset passes all my tests on my hardware,
including more complex ones than what breaks here). My first wild guess is
that the dynamic array of the element structure is not aligned properly.
Could you give a try to the next patch?
diff --git a/net/netfilter/ipset/ip_set_hash_gen.h
b/net/netfilter/ipset/ip_set_hash_gen.h
index afe905c..1cf357d 100644
--- a/net/netfilter/ipset/ip_set_hash_gen.h
+++ b/net/netfilter/ipset/ip_set_hash_gen.h
@@ -1211,6 +1211,9 @@ static const struct ip_set_type_variant mtype_variant = {
.same_set = mtype_same_set,
};
+#define IP_SET_BASE_ALIGN(dtype) \
+ ALIGN(sizeof(struct dtype), __alignof__(struct dtype))
+
#ifdef IP_SET_EMIT_CREATE
static int
IPSET_TOKEN(HTYPE, _create)(struct net *net, struct ip_set *set,
@@ -1319,12 +1322,12 @@ IPSET_TOKEN(HTYPE, _create)(struct net *net, struct
ip_set *set,
#endif
set->variant = _TOKEN(HTYPE, 4_variant);
set->dsize = ip_set_elem_len(set, tb,
- sizeof(struct IPSET_TOKEN(HTYPE, 4_elem)));
+ IP_SET_BASE_ALIGN(IPSET_TOKEN(HTYPE, 4_elem)));
#ifndef IP_SET_PROTO_UNDEF
} else {
set->variant = _TOKEN(HTYPE, 6_variant);
set->dsize = ip_set_elem_len(set, tb,
- sizeof(struct IPSET_TOKEN(HTYPE, 6_elem)));
+ IP_SET_BASE_ALIGN(IPSET_TOKEN(HTYPE, 6_elem)));
}
#endif
if (tb[IPSET_ATTR_TIMEOUT]) {
If that does not solve it, then could you help to narrow down the issue?
Does the bug still appear if your remove the counter extension of the set?
Hello Jozsef,
Patch applied well, compiling ...
Interesting, that it didn't happen before. Device is in production for
more than 2 month without any issue.
Also any idea regarding the second isssue? Or do you think it has the
same root cause?
Greetings from Vienna, Austria :-)
BTW: You can get the Banana Pi R1 for example at:
http://www.aliexpress.com/item/BPI-R1-Set-1-R1-Board-Clear-Case-5dB-Antenna-Power-Adapter-Banana-PI-R1-Smart/32362127917.html
I can really recommend it as a router. Power consumption is as less as
3W. Price is also IMHO very good.
Ciao,
Gerhard
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
