Fail #2 - stdout mismatch

#                             -*- compilation -*-
124. rhbz1855140.at:1: testing rich rule icmptypes with one family ...
./rhbz1855140.at:1: if ! cp "${FIREWALLD_DEFAULT_CONFIG}/firewalld.conf" 
./firewalld.conf; then exit 77; fi
./rhbz1855140.at:1: sed -i 's/^CleanupOnExit.*/CleanupOnExit=no/' 
./firewalld.conf
./rhbz1855140.at:1: sed -i 's/^FirewallBackend.*/FirewallBackend=nftables/' 
./firewalld.conf
./rhbz1855140.at:1: ip netns add fwd-test-${at_group_normalized}
./rhbz1855140.at:1:     env 
DBUS_SYSTEM_BUS_ADDRESS="unix:abstract=firewalld-testsuite-dbus-system-socket-${at_group_normalized}"
 ip netns exec fwd-test-${at_group_normalized} nft -f ./nft_rule_index.nft 
./rhbz1855140.at:1:     env 
DBUS_SYSTEM_BUS_ADDRESS="unix:abstract=firewalld-testsuite-dbus-system-socket-${at_group_normalized}"
 ip netns exec fwd-test-${at_group_normalized} rm ./nft_rule_index.nft 
./rhbz1855140.at:1:     env 
DBUS_SYSTEM_BUS_ADDRESS="unix:abstract=firewalld-testsuite-dbus-system-socket-${at_group_normalized}"
 ip netns exec fwd-test-${at_group_normalized} nft delete table inet 
firewalld_check_rule_index 
not running
running
./rhbz1855140.at:4:     env 
DBUS_SYSTEM_BUS_ADDRESS="unix:abstract=firewalld-testsuite-dbus-system-socket-${at_group_normalized}"
 ip netns exec fwd-test-${at_group_normalized} firewall-cmd --permanent --zone 
public --add-rich-rule='rule icmp-type name="echo-request" accept' 
stdout:
success
./rhbz1855140.at:5:     env 
DBUS_SYSTEM_BUS_ADDRESS="unix:abstract=firewalld-testsuite-dbus-system-socket-${at_group_normalized}"
 ip netns exec fwd-test-${at_group_normalized} firewall-cmd --permanent --zone 
public --add-rich-rule='rule icmp-type name="neighbour-advertisement" accept' 
stdout:
success
./rhbz1855140.at:6:     env 
DBUS_SYSTEM_BUS_ADDRESS="unix:abstract=firewalld-testsuite-dbus-system-socket-${at_group_normalized}"
 ip netns exec fwd-test-${at_group_normalized} firewall-cmd --permanent --zone 
public --add-rich-rule='rule icmp-type name="timestamp-request" accept' 
stdout:
success
./rhbz1855140.at:7:     env 
DBUS_SYSTEM_BUS_ADDRESS="unix:abstract=firewalld-testsuite-dbus-system-socket-${at_group_normalized}"
 ip netns exec fwd-test-${at_group_normalized} firewall-cmd --permanent --zone 
public --add-rich-rule 'rule icmp-type name=bad-header mark set=0x86/0x86' 
stdout:
success
./rhbz1855140.at:8:     env 
DBUS_SYSTEM_BUS_ADDRESS="unix:abstract=firewalld-testsuite-dbus-system-socket-${at_group_normalized}"
 ip netns exec fwd-test-${at_group_normalized} firewall-cmd -q --reload 
./rhbz1855140.at:8:     env 
DBUS_SYSTEM_BUS_ADDRESS="unix:abstract=firewalld-testsuite-dbus-system-socket-${at_group_normalized}"
 ip netns exec fwd-test-${at_group_normalized} firewall-cmd -q --state 
./rhbz1855140.at:9:     env 
DBUS_SYSTEM_BUS_ADDRESS="unix:abstract=firewalld-testsuite-dbus-system-socket-${at_group_normalized}"
 ip netns exec fwd-test-${at_group_normalized}     sh <<-"HERE"
    { { { { nft $NFT_NUMERIC_ARGS list chain inet firewalld 
mangle_PRE_public_allow; echo $? >&3; } |     sed -e 's/^[ \t]*//' -e 's/[ 
\t]*$//' | sed -e '/^[ \t]*$/d' | sed -e 's/[ \t]\+/ /g' | { printf "%s" "$(cat 
/dev/stdin)"; echo; } |             sed -e 's/meta mark/mark/g'        -e 
'/type.*hook.*priority.*policy.*/d'                -e '/ct 
\(state\|status\)/{s/\(ct \(state\|status\)\) {/\1/g; s/ }//; s/\([a-z]*\), 
/\1,/g;}'  >&4; } 3>&1; } | { read RC; exit $RC; } } 4>&1
HERE
 
--- -   2021-05-10 12:59:11.409337617 +0000
+++ 
/tmp/autopkgtest.b8ayAF/build.gyK/src/src/tests/testsuite.dir/at-groups/124/stdout
  2021-05-10 12:59:11.402697416 +0000
@@ -1,6 +1,6 @@
 table inet firewalld {
 chain mangle_PRE_public_allow {
-icmpv6 type parameter-problem icmpv6 code no-route mark set mark & 0x00000086 
^ 0x00000086
+icmpv6 code no-route mark set mark & 0x00000086 ^ 0x00000086
 }
 }
 
124. rhbz1855140.at:1: 124. rich rule icmptypes with one family 
(rhbz1855140.at:1): FAILED (rhbz1855140.at:9)

** Description changed:

  The new nftables
-   https://launchpad.net/ubuntu/+source/nftables/0.9.8-3
+   https://launchpad.net/ubuntu/+source/nftables/0.9.8-3
  is stuck in proposed since it fails autopkgtest of firewalld
-   https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/amd64
-   
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/amd64/f/firewalld/20210510_135128_36f9c@/log.gz
-   https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/s390x
-   
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/s390x/f/firewalld/20210510_131115_faeb7@/log.gz
+   https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/amd64
+   
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/amd64/f/firewalld/20210510_135128_36f9c@/log.gz
+   https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/s390x
+   
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/s390x/f/firewalld/20210510_131115_faeb7@/log.gz
  
  It fails the same way across architectures in:
  ## ------------------------ ##
  ## Summary of the failures. ##
  ## ------------------------ ##
  Failed tests:
  firewalld 0.9.3 test suite test groups:
  
-  NUM: FILE-NAME:LINE     TEST-GROUP-NAME
-       KEYWORDS
+  NUM: FILE-NAME:LINE     TEST-GROUP-NAME
+       KEYWORDS
  
-   97: icmp_block_in_forward_chain.at:1 ICMP block present FORWARD chain
-       nftables icmp
-  124: rhbz1855140.at:1   rich rule icmptypes with one family
-       nftables rich icmp rhbz1855140
+   97: icmp_block_in_forward_chain.at:1 ICMP block present FORWARD chain
+       nftables icmp
+  124: rhbz1855140.at:1   rich rule icmptypes with one family
+       nftables rich icmp rhbz1855140
  
  The upstream issue tracker
-   https://github.com/firewalld/firewalld/issues?q=is%3Aissue+is%3Aopen
+   https://github.com/firewalld/firewalld/issues?q=is%3Aissue+is%3Aopen
  does not list those cases, but there is a new v9.4.0 that we might try.
  
  In Debian this isn't showing up
-   https://ci.debian.net/packages/f/firewalld/
+   https://ci.debian.net/packages/f/firewalld/
  Because they are all Skipped for not having machine level isolation
-   
https://ci.debian.net/data/autopkgtest/testing/amd64/f/firewalld/13738304/log.gz
+   
https://ci.debian.net/data/autopkgtest/testing/amd64/f/firewalld/13738304/log.gz
+ 
+ In detail it seems there re two cases of expected-output-mismatch
+ in #97:
+ -icmp type destination-unreachable icmp code host-prohibited reject with 
icmpx type admin-prohibited
+ +icmp code host-prohibited reject with icmpx type admin-prohibited
+ in #124:
+ -icmpv6 type parameter-problem icmpv6 code no-route mark set mark & 
0x00000086 ^ 0x00000086
+ +icmpv6 code no-route mark set mark & 0x00000086 ^ 0x00000086
+ 
+ Those look like they might have the same root cause.
  
  Furthermore it is yet unclear if this is locally reproducible.

** Description changed:

  The new nftables
    https://launchpad.net/ubuntu/+source/nftables/0.9.8-3
  is stuck in proposed since it fails autopkgtest of firewalld
    https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/amd64
    
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/amd64/f/firewalld/20210510_135128_36f9c@/log.gz
    https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/s390x
    
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/s390x/f/firewalld/20210510_131115_faeb7@/log.gz
  
  It fails the same way across architectures in:
  ## ------------------------ ##
  ## Summary of the failures. ##
  ## ------------------------ ##
  Failed tests:
  firewalld 0.9.3 test suite test groups:
  
   NUM: FILE-NAME:LINE     TEST-GROUP-NAME
        KEYWORDS
  
    97: icmp_block_in_forward_chain.at:1 ICMP block present FORWARD chain
        nftables icmp
   124: rhbz1855140.at:1   rich rule icmptypes with one family
        nftables rich icmp rhbz1855140
  
  The upstream issue tracker
    https://github.com/firewalld/firewalld/issues?q=is%3Aissue+is%3Aopen
  does not list those cases, but there is a new v9.4.0 that we might try.
  
  In Debian this isn't showing up
    https://ci.debian.net/packages/f/firewalld/
  Because they are all Skipped for not having machine level isolation
    
https://ci.debian.net/data/autopkgtest/testing/amd64/f/firewalld/13738304/log.gz
  
+ 
  In detail it seems there re two cases of expected-output-mismatch
  in #97:
  -icmp type destination-unreachable icmp code host-prohibited reject with 
icmpx type admin-prohibited
  +icmp code host-prohibited reject with icmpx type admin-prohibited
  in #124:
  -icmpv6 type parameter-problem icmpv6 code no-route mark set mark & 
0x00000086 ^ 0x00000086
  +icmpv6 code no-route mark set mark & 0x00000086 ^ 0x00000086
  
  Those look like they might have the same root cause.
  
- Furthermore it is yet unclear if this is locally reproducible.
+ 
+ It seems that this is present for a while, this is nftables nftables/0.9.8-1 
in Hirsute half a year ago.
+   
https://autopkgtest.ubuntu.com/results/autopkgtest-hirsute/hirsute/amd64/f/firewalld/20210118_230221_66bea@/log.gz
+ 
+ before nftables 0.9.8 it worked on 0.9.7-1:
+ 
https://autopkgtest.ubuntu.com/results/autopkgtest-hirsute/hirsute/amd64/f/firewalld/20201101_064747_2b123@/log.gz
+ 
+ TODO:
+ - Furthermore it is yet unclear if this is locally reproducible.
+ - this is old enough someone else might have debugged this but missed to file 
bugs?

** Description changed:

  The new nftables
    https://launchpad.net/ubuntu/+source/nftables/0.9.8-3
  is stuck in proposed since it fails autopkgtest of firewalld
    https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/amd64
    
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/amd64/f/firewalld/20210510_135128_36f9c@/log.gz
    https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/s390x
    
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/s390x/f/firewalld/20210510_131115_faeb7@/log.gz
  
  It fails the same way across architectures in:
  ## ------------------------ ##
  ## Summary of the failures. ##
  ## ------------------------ ##
  Failed tests:
  firewalld 0.9.3 test suite test groups:
  
   NUM: FILE-NAME:LINE     TEST-GROUP-NAME
        KEYWORDS
  
    97: icmp_block_in_forward_chain.at:1 ICMP block present FORWARD chain
        nftables icmp
   124: rhbz1855140.at:1   rich rule icmptypes with one family
        nftables rich icmp rhbz1855140
  
  The upstream issue tracker
    https://github.com/firewalld/firewalld/issues?q=is%3Aissue+is%3Aopen
  does not list those cases, but there is a new v9.4.0 that we might try.
  
  In Debian this isn't showing up
    https://ci.debian.net/packages/f/firewalld/
  Because they are all Skipped for not having machine level isolation
    
https://ci.debian.net/data/autopkgtest/testing/amd64/f/firewalld/13738304/log.gz
  
- 
  In detail it seems there re two cases of expected-output-mismatch
  in #97:
  -icmp type destination-unreachable icmp code host-prohibited reject with 
icmpx type admin-prohibited
  +icmp code host-prohibited reject with icmpx type admin-prohibited
  in #124:
  -icmpv6 type parameter-problem icmpv6 code no-route mark set mark & 
0x00000086 ^ 0x00000086
  +icmpv6 code no-route mark set mark & 0x00000086 ^ 0x00000086
  
  Those look like they might have the same root cause.
  
- 
  It seems that this is present for a while, this is nftables nftables/0.9.8-1 
in Hirsute half a year ago.
-   
https://autopkgtest.ubuntu.com/results/autopkgtest-hirsute/hirsute/amd64/f/firewalld/20210118_230221_66bea@/log.gz
+   
https://autopkgtest.ubuntu.com/results/autopkgtest-hirsute/hirsute/amd64/f/firewalld/20210118_230221_66bea@/log.gz
  
  before nftables 0.9.8 it worked on 0.9.7-1:
  
https://autopkgtest.ubuntu.com/results/autopkgtest-hirsute/hirsute/amd64/f/firewalld/20201101_064747_2b123@/log.gz
  
+ With the right keywords I've found closed bugs in firewalld pointing to a 
nftables fix:
+ - https://github.com/firewalld/firewalld/issues/752 (thanks locutus for 
filing)
+ - https://marc.info/?l=netfilter-devel&m=161221629204555&w=2 <- supposed to 
be the fix
+ 
  TODO:
  - Furthermore it is yet unclear if this is locally reproducible.
- - this is old enough someone else might have debugged this but missed to file 
bugs?

** Description changed:

  The new nftables
    https://launchpad.net/ubuntu/+source/nftables/0.9.8-3
  is stuck in proposed since it fails autopkgtest of firewalld
    https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/amd64
    
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/amd64/f/firewalld/20210510_135128_36f9c@/log.gz
    https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/s390x
    
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/s390x/f/firewalld/20210510_131115_faeb7@/log.gz
  
  It fails the same way across architectures in:
  ## ------------------------ ##
  ## Summary of the failures. ##
  ## ------------------------ ##
  Failed tests:
  firewalld 0.9.3 test suite test groups:
  
   NUM: FILE-NAME:LINE     TEST-GROUP-NAME
        KEYWORDS
  
    97: icmp_block_in_forward_chain.at:1 ICMP block present FORWARD chain
        nftables icmp
   124: rhbz1855140.at:1   rich rule icmptypes with one family
        nftables rich icmp rhbz1855140
  
  The upstream issue tracker
    https://github.com/firewalld/firewalld/issues?q=is%3Aissue+is%3Aopen
  does not list those cases, but there is a new v9.4.0 that we might try.
  
  In Debian this isn't showing up
    https://ci.debian.net/packages/f/firewalld/
  Because they are all Skipped for not having machine level isolation
    
https://ci.debian.net/data/autopkgtest/testing/amd64/f/firewalld/13738304/log.gz
  
  In detail it seems there re two cases of expected-output-mismatch
  in #97:
  -icmp type destination-unreachable icmp code host-prohibited reject with 
icmpx type admin-prohibited
  +icmp code host-prohibited reject with icmpx type admin-prohibited
  in #124:
  -icmpv6 type parameter-problem icmpv6 code no-route mark set mark & 
0x00000086 ^ 0x00000086
  +icmpv6 code no-route mark set mark & 0x00000086 ^ 0x00000086
  
  Those look like they might have the same root cause.
  
  It seems that this is present for a while, this is nftables nftables/0.9.8-1 
in Hirsute half a year ago.
    
https://autopkgtest.ubuntu.com/results/autopkgtest-hirsute/hirsute/amd64/f/firewalld/20210118_230221_66bea@/log.gz
  
  before nftables 0.9.8 it worked on 0.9.7-1:
  
https://autopkgtest.ubuntu.com/results/autopkgtest-hirsute/hirsute/amd64/f/firewalld/20201101_064747_2b123@/log.gz
  
  With the right keywords I've found closed bugs in firewalld pointing to a 
nftables fix:
  - https://github.com/firewalld/firewalld/issues/752 (thanks locutus for 
filing)
  - https://marc.info/?l=netfilter-devel&m=161221629204555&w=2 <- supposed to 
be the fix
  
- TODO:
- - Furthermore it is yet unclear if this is locally reproducible.
+ The issue is locally reproducible in e.g. autopkgtest VM and thereby
+ fixes can be tested the same way.

** Description changed:

  The new nftables
    https://launchpad.net/ubuntu/+source/nftables/0.9.8-3
  is stuck in proposed since it fails autopkgtest of firewalld
    https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/amd64
    
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/amd64/f/firewalld/20210510_135128_36f9c@/log.gz
    https://autopkgtest.ubuntu.com/packages/f/firewalld/impish/s390x
    
https://autopkgtest.ubuntu.com/results/autopkgtest-impish/impish/s390x/f/firewalld/20210510_131115_faeb7@/log.gz
  
  It fails the same way across architectures in:
  ## ------------------------ ##
  ## Summary of the failures. ##
  ## ------------------------ ##
  Failed tests:
  firewalld 0.9.3 test suite test groups:
  
   NUM: FILE-NAME:LINE     TEST-GROUP-NAME
        KEYWORDS
  
    97: icmp_block_in_forward_chain.at:1 ICMP block present FORWARD chain
        nftables icmp
   124: rhbz1855140.at:1   rich rule icmptypes with one family
        nftables rich icmp rhbz1855140
  
  The upstream issue tracker
    https://github.com/firewalld/firewalld/issues?q=is%3Aissue+is%3Aopen
  does not list those cases, but there is a new v9.4.0 that we might try.
  
  In Debian this isn't showing up
    https://ci.debian.net/packages/f/firewalld/
  Because they are all Skipped for not having machine level isolation
    
https://ci.debian.net/data/autopkgtest/testing/amd64/f/firewalld/13738304/log.gz
  
  In detail it seems there re two cases of expected-output-mismatch
  in #97:
  -icmp type destination-unreachable icmp code host-prohibited reject with 
icmpx type admin-prohibited
  +icmp code host-prohibited reject with icmpx type admin-prohibited
  in #124:
  -icmpv6 type parameter-problem icmpv6 code no-route mark set mark & 
0x00000086 ^ 0x00000086
  +icmpv6 code no-route mark set mark & 0x00000086 ^ 0x00000086
  
  Those look like they might have the same root cause.
  
  It seems that this is present for a while, this is nftables nftables/0.9.8-1 
in Hirsute half a year ago.
    
https://autopkgtest.ubuntu.com/results/autopkgtest-hirsute/hirsute/amd64/f/firewalld/20210118_230221_66bea@/log.gz
  
  before nftables 0.9.8 it worked on 0.9.7-1:
  
https://autopkgtest.ubuntu.com/results/autopkgtest-hirsute/hirsute/amd64/f/firewalld/20201101_064747_2b123@/log.gz
  
  With the right keywords I've found closed bugs in firewalld pointing to a 
nftables fix:
- - https://github.com/firewalld/firewalld/issues/752 (thanks locutus for 
filing)
+ - https://github.com/firewalld/firewalld/issues/752 (thanks 
Costamagna/Michael for filing)
  - https://marc.info/?l=netfilter-devel&m=161221629204555&w=2 <- supposed to 
be the fix
  
  The issue is locally reproducible in e.g. autopkgtest VM and thereby
  fixes can be tested the same way.

** Changed in: nftables (Ubuntu)
       Status: New => Triaged

** Changed in: firewalld (Ubuntu)
       Status: New => Confirmed

** Changed in: firewalld (Ubuntu)
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1936902

Title:
  new nftables 0.9.8-3 breaks firewalld 0.9.3 autopkgtest

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/firewalld/+bug/1936902/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to