Looking at http://testing.libreswan.org/results/testing/ and excluding WIP we seem to consistently have 100 failing tests (you can eyeball this from the graph) so I did a little analysis using the script testing/web/never-passed.sh:
- 45 are 'original' and have never ever passed - 16 are more recent yet also show no sign of passing I'm going to mark the first 45 as wip. Andrew PS: here's the full list with colums <#passes> <#fails> <#times-tested> sorted by #passes 0 230 230 ikev2-21-mode-mismatch-01 0 230 230 ikev2-45-impair-gx-02 0 230 230 ikev2-allow-narrow-01 0 230 230 ikev2-allow-narrow-02 0 230 230 ikev2-allow-narrow-03 0 230 230 ikev2-ddns-02 0 230 230 interop-ikev2-racoon-02-psk-responder 0 230 230 interop-ikev2-strongswan-19-x509-res-certreq 0 230 230 interop-ikev2-strongswan-21-transport-02 0 230 230 interop-ikev2-strongswan-21-transport-03 0 230 230 interop-ikev2-strongswan-26-ke-mismatch-responder 0 230 230 interop-ikev2-strongswan-27-fragmentation 0 230 230 newoe-07-ike-replace-initiator-esp 0 230 230 newoe-15-portpass 0 230 230 newoe-18-poc-block 0 230 230 newoe-18-poc-blockall 0 230 230 newoe-18-private-clear 0 230 230 newoe-18-private-clearall 0 230 230 newoe-19-poc-poc-clear 0 230 230 newoe-20-ipv6 0 230 230 newoe-21-liveness-clear 0 230 230 newoe-25-cat-2 0 230 230 nss-cert-08-mismatch 0 230 230 nss-cert-nosecret 0 230 230 nss-cert-ocsp-01-strict 0 230 230 nss-cert-ocsp-02 0 230 230 nss-cert-ocsp-02-ikev2 0 230 230 nss-cert-ocsp-03-strict 0 230 230 nss-cert-ocsp-04 0 230 230 nss-cert-ocsp-05-strict 0 230 230 nss-cert-ocsp-06 0 230 230 nss-cert-ocsp-07-nourl 0 230 230 x509-pluto-frag-01 0 230 230 xauth-pluto-17 0 230 230 xauth-pluto-18 0 218 218 dnssec-pluto-01 0 218 218 dpd-01 0 218 218 dpd-04 0 218 218 dpd-06 0 218 218 l2tp-01 0 218 218 l2tp-02 0 218 218 nat-pluto-04 0 218 218 nflog-03-conns 0 177 177 seccomp-01-enabled 0 177 177 seccomp-02-tolerant 0 153 153 interop-ikev2-strongswan-21-transport-04 0 152 152 nss-cert-09-notyetvalid-initiator 0 152 152 nss-cert-09-notyetvalid-initiator-ikev2 0 152 152 nss-cert-10-notyetvalid-responder 0 152 152 nss-cert-10-notyetvalid-responder-ikev2 0 152 152 nss-cert-ocsp-08-post 0 75 75 addconn-01 0 53 53 certoe-01-whack 0 53 53 certoe-07-nat-2-clients 0 53 53 certoe-08-nat-packet-cop-restart 0 49 49 nss-cert-chain-01-ikev2 0 49 49 nss-cert-chain-03-ikev2 0 49 49 nss-cert-chain-04-ikev2 0 37 37 ikev1-aggr-sendcert-01 0 26 26 ikev2-child-ipseac-replace-ike 0 18 18 netkey-vti-06c 1 229 230 newoe-25-cat-3-4-way 2 228 230 ikev2-27-uniqueid 2 3 5 ikev2-child-ipsec-responder 4 11 15 ikev2-54-replace-leave-old 5 225 230 nflog-01-global 6 224 230 ipsec-hostkey-ckaid-02-fips 6 224 230 nss-cert-05 8 0 8 ikev2-algo-ike-dh-ecp-01 8 0 8 interop-ikev2-strongswan-33-ike-dh-ecp-responder 10 8 18 netkey-vti-05 11 28 39 interop-ikev2-strongswan-31-asymmetric-responder 12 139 151 nat-pluto-02-klips-klips 13 1 14 ikev2-initiate-template-01 17 1 18 netkey-vti-06 18 200 218 netkey-vti-01 19 132 151 nat-pluto-02-netkey-klips 24 194 218 ikev2-liveness-05 25 205 230 ikev2-frag-02-ipv6 25 70 95 nss-cert-badca-01 25 1 26 interop-ikev2-strongswan-15-child-sa 27 203 230 ikev2-31-nat-rw-no-rekey 31 199 230 newoe-25-cat-4 33 197 230 interop-ikev2-strongswan-23-initiator-cp 33 197 230 nss-cert-ocsp-01 33 197 230 nss-cert-ocsp-01-chain 33 197 230 nss-cert-ocsp-01-ikev2 33 197 230 nss-cert-ocsp-03 33 197 230 nss-cert-ocsp-03-ikev2 33 197 230 nss-cert-ocsp-05 33 197 230 nss-cert-ocsp-05-ikev2 33 185 218 compress-pluto-01 33 185 218 compress-pluto-02 34 2 36 ikev1-27-uniqueid 35 70 105 klips-passthrough-00 35 1 36 ikev1-aggr-replace-01 39 191 230 newoe-25-cat-1 39 14 53 certoe-06-nat-packet-cop 41 189 230 ikev2-12-x509-ikev1-rw 41 12 53 certoe-03-cop-whack 41 12 53 certoe-03-poc-whack 41 6 47 delete-sa-09-rhbz1311360-c32 42 11 53 certoe-04-poc-packet 45 8 53 ikev2-asymmetric-01-parsing 47 1 48 delete-sa-08-rhbz1311360 48 5 53 ikev2-asymmetric-03-null-rsacert 48 1 49 nss-cert-chain-02-ikev2 48 1 49 nss-cert-chain-04 49 4 53 ikev2-asymmetric-02-orienting 49 4 53 ikev2-asymmetric-10-rsaraw-psk 50 180 230 ikev2-09-rw-rsa 50 83 133 ikev2-16-alias-whack-up 50 3 53 ikev2-asymmetric-04-null-rsaraw 50 3 53 ikev2-asymmetric-07-psk-rsaraw 50 3 53 ikev2-asymmetric-08-psk-rsacert 50 3 53 ikev2-asymmetric-09-rsaraw-null 50 3 53 ikev2-asymmetric-12-rsacert-null 50 3 53 ikev2-asymmetric-13-rsacert-psk 50 3 53 ikev2-asymmetric-15-rsaraw-rsaraw 52 116 168 interop-ikev2-strongswan-09-dh22-initiator 52 1 53 certoe-02-whack-badca 52 1 53 certoe-10-symetric-cert-whack 52 1 53 certoe-11-symetric-cert-nat 52 1 53 ikev2-asymmetric-16-auth-mismatch 52 1 53 multinet-04 53 0 53 certoe-05-poc-reverse 57 152 209 delete-sa-04 63 167 230 ikev2-01-fallback-ikev1 66 39 105 klips-algo-twofish-01 67 163 230 nss-cert-crl-02 71 34 105 klips-algo-serpent-01 72 158 230 interop-ikev2-strongswan-03-psk-initiator 72 158 230 interop-ikev2-strongswan-28-reauth 72 158 230 nss-cert-chain-02 72 100 172 ikev1-impair-gx-02 72 100 172 psk-pluto-02 73 157 230 nss-cert-04 74 156 230 fips-08-ikev2-x509 81 149 230 interop-ikev2-strongswan-07-strongswan 83 147 230 interop-ikev1-strongswan-08-strongswan-cast 87 143 230 nss-cert-crl-03-strict 89 16 105 klips-algo-cast-01 90 140 230 nss-cert-crl-03 90 15 105 klips-basic-pluto-01 94 124 218 klips-netkey-pluto-06 94 124 218 netkey-klips-pluto-03 105 125 230 fips-03-ikev1-md5 105 26 131 ikev2-53-clean-pending 109 121 230 interop-ikev2-strongswan-04-x509-responder 110 41 151 nat-pluto-02-klips-netkey 113 117 230 interop-ikev2-strongswan-21-transport-01 113 105 218 netkey-audit-01 117 113 230 ikev2-liveness-03 117 101 218 ikev1-algo-ike-bogus-01 118 112 230 interop-ikev2-strongswan-22-cp-responder-psk 120 3 123 ikev2-48-nat-cp-start 121 47 168 interop-ikev1-strongswan-09-dh22-initiator 126 104 230 ikev2-algo-sha2-06 128 102 230 newoe-08-ike-replace-responder 132 98 230 nss-cert-crl-01-strict 140 78 218 nat-pluto-03 140 69 209 ikev2-10-2behind-nat 145 85 230 aggr-pluto-01 146 72 218 algo-pluto-08 149 81 230 newoe-07-ike-replace-initiator 150 80 230 ikev2-48-nat-cp 150 80 230 newoe-25-cat-5 151 79 230 nflog-02-conn 151 79 230 nss-cert-chain-01 153 77 230 nss-cert-02-ikev2 154 76 230 nss-cert-chain-03 155 75 230 nss-cert-07 159 59 218 algo-pluto-11 159 59 218 ikev1-algo-ike-aes-01 159 59 218 netkey-algo-twofish-01 160 58 218 netkey-algo-aes_ccm-01 160 58 218 netkey-algo-aes_gcm-01 160 58 218 netkey-algo-aes_gcm-02 160 58 218 netkey-algo-aes_gcm-03 161 57 218 ikev1-algo-ike-aes-02 167 63 230 dpd-05 167 63 230 ikev2-delete-02 167 63 230 netkey-tfc-01 167 63 230 netkey-tfc-02 168 62 230 ikev2-algo-ike-sha2-02 168 62 230 nss-cert-crl-01 169 61 230 aggr-pluto-02 170 48 218 nat-pluto-02 171 59 230 netkey-tfc-03 171 47 218 ah-pluto-07-klips-netkey 172 58 230 ikev2-invalid-ke-02-wrong-modp 174 56 230 ikev2-11-simple-psk 174 56 230 ikev2-algo-ike-sha2-03 174 56 230 interop-ikev2-strongswan-02-psk-responder 174 56 230 xauth-pluto-05 174 56 230 xauth-pluto-06 174 56 230 xauth-pluto-09 174 56 230 xauth-pluto-19 175 55 230 basic-pluto-06 175 55 230 ikev2-algo-07-aes_ctr 175 55 230 nss-cert-01-ikev2 175 55 230 xauth-pluto-05-netkey 177 53 230 fips-default-ikev2-01-nofips-east 177 53 230 ikev2-03-basic-rawrsa 177 53 230 ikev2-29-no-rekey 177 53 230 ikev2-47-priority 177 53 230 ikev2-algo-04-aes-gcm256 177 53 230 ikev2-algo-06-aes-aes_xcbc 177 53 230 ikev2-algo-sha2-08 177 53 230 ikev2-delete-01 177 53 230 interop-ikev2-strongswan-25-ke-mismatch-initiator 178 52 230 ikev2-41-rw-replace 178 52 230 ikev2-42-rw-replace-responder 178 52 230 ikev2-algo-03-aes-ccm 178 52 230 ikev2-algo-10-aes-gcm128 178 52 230 ikev2-algo-sha2-01 178 52 230 ikev2-algo-sha2-02 178 52 230 ikev2-algo-sha2-03 178 52 230 ikev2-algo-sha2-04 178 52 230 ikev2-allow-narrow-04 178 40 218 basic-pluto-03 179 51 230 basic-pluto-13-netkey-route 179 51 230 ikev2-04-basic-x509 179 51 230 ikev2-04-basic-x509-ckaid 179 51 230 ikev2-04-basic-x509-nhelpers0 179 51 230 ikev2-04-basic-x509-nsspw 179 51 230 ikev2-algo-11-gcm-prop2 179 51 230 ikev2-algo-ike-sha2-01 179 51 230 ikev2-allow-narrow-05 179 51 230 ikev2-allow-narrow-06 179 51 230 ikev2-allow-narrow-07 179 39 218 algo-pluto-10 179 39 218 basic-pluto-01 179 39 218 ikev1-algo-ike-sha2-01 179 39 218 ikev1-algo-ike-sha2-02 179 39 218 ikev1-algo-ike-sha2-03 179 39 218 ikev2-17-rekey-ipsec 179 39 218 netkey-algo-null-01 179 39 218 netkey-algo-serpent-01 180 50 230 ikev2-12-transport-psk 180 50 230 newoe-22-nat-poc-cop 181 49 230 x509-pluto-02 181 49 230 x509-pluto-03 181 37 218 algo-pluto-01 181 37 218 basic-pluto-02 181 37 218 ikev1-algo-esp-null-01 181 37 218 ikev1-cryptoload-00 181 37 218 ikev1-isakmp-reserved-flags-01 181 37 218 ikev1-isakmp-reserved-flags-02 182 48 230 nss-cert-03-ikev2 182 36 218 compress-pluto-netkey-03 182 36 218 delete-sa-03 183 47 230 fips-04-ikev2-md5 183 47 230 x509-pluto-01 183 35 218 compress-pluto-netkey-klips-04 183 35 218 ipv6-tunnel-mode-04-rw 184 46 230 algo-pluto-13-invalid-3des 184 46 230 ikev1-algo-esp-sha2-05 184 46 230 ikev2-12-x509-ikev1 184 46 230 ikev2-21-mode-mismatch-02 184 46 230 ikev2-algo-sha2-05 184 46 230 interop-ikev2-strongswan-05-psk-md5 184 46 230 newoe-10-expire-inactive-ike 184 46 230 nss-cert-01 184 46 230 nss-cert-02 184 46 230 nss-cert-02-eku 184 46 230 nss-cert-02-eku-combined 184 46 230 nss-cert-03 184 46 230 nss-cert-04-ikev2 184 46 230 nss-cert-05-ikev2 184 46 230 nss-cert-06 185 45 230 ikev2-liveness-01 186 44 230 ikev1-algo-esp-sha2-01-netkey-klips 187 43 230 fips-07-ikev2-3des-sha256 187 43 230 ikev2-50-propnum 188 42 230 interop-ikev1-strongswan-02-psk-responder 188 42 230 interop-ikev2-strongswan-06-aes192 188 42 230 newoe-10-expire-inactive 188 30 218 netkey-algo-null-02 188 28 216 ipv6-tunnel-mode-02-netkey-netkey 193 37 230 newoe-09-mutual 194 36 230 newoe-06-prio 195 35 230 ikev2-dcookie-01 197 33 230 ikev2-30-rw-no-rekey 197 33 230 interop-ikev2-strongswan-12-nat-responder 199 31 230 ikev1-algo-esp-sha2-02-netkey-klips 199 31 230 newoe-04-pass-pass 201 29 230 fips-09-ikev2-gcm 201 29 230 ikev2-invalid-ke-01-invalid-modp 201 29 230 ikev2-invalid-ke-03-default-initiator 201 29 230 ikev2-invalid-ke-04-default-responder 202 28 230 interop-ikev2-strongswan-20-psk-aes_xcbc 202 28 230 interop-ikev2-strongswan-24-aes_ctr 204 26 230 interop-ikev1-strongswan-15-twofish 204 14 218 xauth-pluto-20-pam 204 5 209 ikev2-dcookie-02 204 5 209 ikev2-dcookie-03 205 13 218 xauth-pluto-14 206 24 230 psk-pluto-06 206 12 218 dynamic-iface-01 207 11 218 fips-06-ikev1-3des-sha1 208 10 218 algo-pluto-12-aes-default 209 21 230 interop-ikev1-strongswan-16-serpent 209 9 218 ah-pluto-01 210 8 218 ah-pluto-02 210 8 218 ah-pluto-03 210 8 218 ah-pluto-04 210 8 218 ah-pluto-05 210 8 218 ah-pluto-11 210 8 218 nat-pluto-07 210 8 218 nat-transport-02 211 7 218 pluto-ipcmp-01 211 7 218 pluto-rekey-02 211 7 218 xauth-pluto-03 212 18 230 ikev2-03-basic-rawrsa-nhelpers0 212 18 230 ikev2-algo-05-aes-default 212 18 230 ikev2-algo-esn-04 212 18 230 ikev2-isakmp-reserved-flags-01 212 18 230 ikev2-isakmp-reserved-flags-02 212 18 230 ikev2-minor-version-initiator 212 18 230 ikev2-minor-version-responder 212 18 230 ikev2-no-nhelpers-01 212 18 230 ikev2-payload-reserved-flags-01 212 18 230 newoe-05-hold-pass 213 17 230 algo-pluto-07 213 17 230 fips-default-ikev2-02-nofips-west 213 17 230 ikev2-algo-esn-01 213 17 230 ikev2-algo-esn-03 213 17 230 ikev2-algo-esn-05 213 17 230 ikev2-algo-esn-06 213 5 218 dpd-03 213 5 218 ikev1-algo-ike-pfs-01 213 5 218 l2tp-02-netkey 214 16 230 ikev2-32-nat-rw-rekey 214 4 218 ikev2-02-responder-send-notify 214 4 218 netkey-algo-cast-01 214 4 218 netkey-vti-02 215 15 230 xauth-pluto-07 215 3 218 algo-pluto-14-aes_ctr 215 3 218 dpd-02 215 3 218 fips-default-ikev1-01-nofips-east 215 3 218 fips-default-ikev1-02-nofips-west 215 3 218 netkey-algo-camellia-01 215 3 218 netkey-passthrough-02 216 14 230 dpd-07 216 14 230 xauth-pluto-08 216 14 230 xauth-pluto-13 216 14 230 xauth-pluto-15 216 14 230 xauth-pluto-16 216 2 218 delete-sa-01 217 13 230 ikev2-algo-esn-02 217 1 218 ah-pluto-10 217 1 218 listen-change-01 217 1 218 netkey-passthrough-00 217 1 218 netkey-passthrough-01 217 1 218 netkey-passthrough-03 217 1 218 netkey-passthrough-ipxfrm 218 12 230 ikev2-25-rw-nat 218 12 230 ikev2-algo-09-camellia 218 12 230 ikev2-liveness-02 218 12 230 interop-ikev1-strongswan-14-camellia 219 11 230 ikev2-13-ah 219 11 230 nat-pluto-01 219 11 230 xauth-pluto-12 220 10 230 ikev2-46-basic-psk-netkey 221 9 230 ikev2-liveness-06 222 8 230 ikev2-liveness-07 222 8 230 ikev2-x509-01 223 7 230 nat-pluto-08 223 7 230 nat-pluto-09 223 7 230 xauth-pluto-04 224 6 230 ikev2-05-basic-psk 224 6 230 ikev2-05-basic-psk-nhelpers0 224 6 230 ikev2-08-delete-notify 224 6 230 ikev2-38-psk-65bytes 224 6 230 ikev2-39-psk-64bytes 224 6 230 ikev2-43-init-retransmit 224 6 230 ikev2-algo-01-modp2048-initiator 224 6 230 ikev2-algo-02-modp2048-responder 224 6 230 ikev2-frag-01 225 5 230 ikev2-algo-08-cast 225 5 230 ikev2-auth-null-01 225 5 230 interop-ikev2-strongswan-05-psk-aes 225 5 230 interop-ikev2-strongswan-09-psk-aes-ccm 225 5 230 interop-ikev2-strongswan-09-psk-aes-gcm 225 5 230 interop-ikev2-strongswan-17-delete-sa-responder 225 5 230 interop-ikev2-strongswan-18-psk-cast 225 5 230 interop-ikev2-strongswan-30-esn 225 5 230 psk-pluto-05 226 4 230 ikev1-algo-05-3des-sha2 226 4 230 interop-ikev1-strongswan-04-psk-aes-ccm 226 4 230 interop-ikev1-strongswan-04-psk-aes-gcm 226 4 230 interop-ikev2-strongswan-10-nat-initiator 226 4 230 interop-ikev2-strongswan-11-nat-initiator 226 4 230 psk-pluto-04 227 3 230 ikev2-14-missing-ke 227 3 230 ikev2-15-fuzzer 227 3 230 x509-pluto-frag-04 228 2 230 ikev1-01-fuzzer 228 2 230 ikev1-impair-gx-01 228 2 230 ikev2-45-impair-gx-01 228 2 230 ikev2-ddns-01 228 2 230 interop-ikev1-strongswan-12-esp-sha2_256 228 2 230 interop-ikev2-strongswan-13-ah-initiator 228 2 230 interop-ikev2-strongswan-16-ah-initiator-sha512 228 2 230 newoe-01-whack 228 2 230 newoe-18-poc-cop 228 2 230 psk-pluto-01 228 2 230 x509-pluto-frag-02 229 1 230 ah-pluto-09 229 1 230 ikev1-algo-04-mismatch 229 1 230 ikev1-cryptoload-01 229 1 230 ikev2-major-version-initiator 229 1 230 ikev2-major-version-responder 229 1 230 interop-ikev1-strongswan-03-psk-initiator 229 1 230 interop-ikev1-strongswan-07-cast128 229 1 230 interop-ikev1-strongswan-10-ah-initiator-sha256 229 1 230 interop-ikev1-strongswan-11-ah-initiator-sha512 229 1 230 interop-ikev1-strongswan-13-esp-sha2_512 229 1 230 interop-ikev2-strongswan-14-delete-sa 229 1 230 interop-ikev2-strongswan-17-ah-initiator-sha256 229 1 230 ipsec-hostkey-ckaid-03-nsspw 229 1 230 newoe-08-restart 229 1 230 newoe-12-clear 229 1 230 newoe-13-block 229 1 230 newoe-14-clear-in-block-clear 229 1 230 newoe-16-pass-hold 229 1 230 newoe-18-block-block 229 1 230 newoe-18-block-clear 229 1 230 newoe-18-block-cop 229 1 230 newoe-18-block-poc 229 1 230 newoe-18-block-private 229 1 230 newoe-18-clear-block 229 1 230 newoe-18-clear-clear 229 1 230 newoe-18-clear-cop 229 1 230 newoe-18-clear-poc 229 1 230 newoe-18-clear-private 229 1 230 newoe-18-cop-block 229 1 230 newoe-18-cop-clear 229 1 230 newoe-18-cop-cop 229 1 230 newoe-18-cop-poc 229 1 230 newoe-18-cop-private 229 1 230 newoe-18-poc-clear 229 1 230 newoe-18-poc-poc 229 1 230 newoe-18-poc-private 229 1 230 newoe-18-private-block 229 1 230 newoe-18-private-cop 229 1 230 newoe-18-private-copall 229 1 230 newoe-18-private-poc 229 1 230 newoe-18-private-pocall 229 1 230 newoe-18-private-private 229 1 230 newoe-23-reverse-nat 229 1 230 psk-pluto-03 229 1 230 replay-authip-01 229 1 230 x509-pluto-frag-00 229 1 230 x509-pluto-frag-03 230 0 230 ipsec-hostkey-ckaid-01 230 0 230 newoe-03-oeself 230 0 230 newoe-11-failike _______________________________________________ Swan-dev mailing list [email protected] https://lists.libreswan.org/mailman/listinfo/swan-dev
