Re: OpenSMTPd: Unable to use TLS/SSL over IPv6
Hey, On 14/01/2022 09:19, Stuart Henderson wrote: That hostname doesn't match the certificate, it should validate ok for storm-peaks.northrend.azeroth.wow-data.net (I also checked with -servername to send SNI). There's no difference between v4 and v6 for that though. thank you very much for spending time in testing this again. Sadly i cannot reproduce the issue. For me the certificate validates correctly for the hostname storm-peaks.northrend.azeroth.wow-data.net. I also used a couple of online certificate checking tools and they also report that it works fine. (https://www.hardenize.com/report/storm-peaks.northrend.azeroth.wow-data.net/1642159474#email and https://www.hardenize.com/report/storm-peaks.northrend.azeroth.wow-data.net/1642159474#email) I read the OpenSMTPd code again last night and i cannot reproduce the initial issue. There is basically no difference in IPv4 and IPv6 connections when they arrive at OpenSMTPd. Its just an open socket and then OpenSMTPd operates on that completely ignoring the IP version. I grepped the log files and in the last 7 days i had 263183 connections via IPv6 to OpenSMTPd. 82% of them used TLS (ciphers=TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256 beeing the most used) according to the log. So i think this should be fine. Thanks for everyone spending time looking into this, but i don't think its a configuration or OpenBSD issue at this point. Thanks so much and greetings Leo
Re: OpenSMTPd: Unable to use TLS/SSL over IPv6
Hey, On 14/01/2022 08:31, Crystal Kolipe wrote: Reading the manual page for openssl, specifically the section on s_client would be a very good idea. thank you for the hint. I did not know about this behavour. It does not explain the initial bug, but certenly my testing of it. For the archive, here is the important part from the manual. If a connection is established with an SSL server, any data received from the server is displayed and any key presses will be sent to the server. When used interactively (which means neither -quiet nor -ign_eof have been given), the session will be renegotiated if the line begins with an R; if the line begins with a Q or if end of file is reached, the connection will be closed down. Thanks for letting me know :)
Re: OpenSMTPd: Unable to use TLS/SSL over IPv6
On 2022-01-13, Crystal Kolipe wrote: > On Thu, Jan 13, 2022 at 05:25:41PM +, Stuart Henderson wrote: >> On 2022/01/13 18:05, Leo Unglaub wrote: >> > Hey, >> > >> > On 11/01/2022 21:28, Stuart Henderson wrote: >> > > I bet it is MTU related. Try lowering MTU on that interface (you >> > > cannot do it separately for IPv4 and IPv6 so it will change both, >> > > but that's not likely to be a problem) and get someone who has >> > > seen the problems to re-test. >> > >> > thank you so much for your answer. I would have never ever thought about >> > the >> > MTU in this case. I used the default 1500. I talked to the technical >> > support >> > from the datacenter (Hetzner Online) and they asured me that 1500 is >> > correct. >> > >> > However, i have set the value to 1400 and asked some people who had the >> > issue to re-test it. I will post the results of the test here so other >> > people can find them via a search engine. >> > >> > Thank you so much, very kind of you! >> >> The possible issue is that many people (especially people connecting >> over tunnels, but also those on pppoe) are on lower MTUs than this. >> Normally this is OK as fragmentation-needed messages will sort things >> out but sometimes firewalls are not be configured to pass these which >> will cause problems. If that _is_ what's happening then there are >> other ways to fix it but changing MTU is often the easiest one that >> you can do yourself. > > Well, I can connect to his server using: > > openssl s_client -starttls smtp -connect mail.unglaub.at:25 > > The handshake completes and I'm able to issue smtp commands. > > However smtpd always reports that opportunistic TLS failed, and > downgrades to plaintext. That hostname doesn't match the certificate, it should validate ok for storm-peaks.northrend.azeroth.wow-data.net (I also checked with -servername to send SNI). There's no difference between v4 and v6 for that though.
Re: OpenSMTPd: Unable to use TLS/SSL over IPv6
On Fri, Jan 14, 2022 at 01:17:47AM +0100, Leo Unglaub wrote: > >RCPT TO: RENEGOTIATING > >139809772520832:error:1420410A:SSL routines:SSL_renegotiate:wrong ssl > >version:../ssl/ssl_lib.c:2142: > > Are the last two lines expected behavour? I get then on IPv4 and IPv6. > Someone else beeing so kind trying to debug this send me something similar. Reading the manual page for openssl, specifically the section on s_client would be a very good idea.
Re: OpenSMTPd: Unable to use TLS/SSL over IPv6
Hey, On 1/13/22 19:18, Crystal Kolipe wrote: Well, I can connect to his server using: openssl s_client -starttls smtp -connect mail.unglaub.at:25 The handshake completes and I'm able to issue smtp commands. However smtpd always reports that opportunistic TLS failed, and downgrades to plaintext. when you connect to the server, can you do the SMTP dialog? I tried it on my server and other instances running OpenSMTPd and i get the following error: $ openssl s_client -starttls smtp -connect mail.unglaub.at:25 CONNECTED(0003) depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1 verify return:1 depth=1 C = US, O = Let's Encrypt, CN = R3 verify return:1 depth=0 CN = storm-peaks.northrend.azeroth.wow-data.net verify return:1 --- Certificate chain 0 s:CN = storm-peaks.northrend.azeroth.wow-data.net i:C = US, O = Let's Encrypt, CN = R3 1 s:C = US, O = Let's Encrypt, CN = R3 i:C = US, O = Internet Security Research Group, CN = ISRG Root X1 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1 i:O = Digital Signature Trust Co., CN = DST Root CA X3 --- Server certificate -BEGIN CERTIFICATE- XXX -END CERTIFICATE- subject=CN = storm-peaks.northrend.azeroth.wow-data.net issuer=C = US, O = Let's Encrypt, CN = R3 --- No client certificate CA names sent Peer signing digest: SHA256 Peer signature type: RSA-PSS Server Temp Key: X25519, 253 bits --- SSL handshake has read 5457 bytes and written 420 bytes Verification: OK --- New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384 Server public key is 4096 bit Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated Early data was not sent Verify return code: 0 (ok) --- 250 HELP EHLO unglaub.at 250-storm-peaks.northrend.azeroth.wow-data.net Hello unglaub.at [2001:871:210:554:6c50:40ef:c73c:d401], pleased to meet you 250-8BITMIME 250-ENHANCEDSTATUSCODES 250-SIZE 83886080 250-DSN 250 HELP MAIL FROM: 250 2.0.0 Ok RCPT TO: RENEGOTIATING 139809772520832:error:1420410A:SSL routines:SSL_renegotiate:wrong ssl version:../ssl/ssl_lib.c:2142: Are the last two lines expected behavour? I get then on IPv4 and IPv6. Someone else beeing so kind trying to debug this send me something similar. I am shorting it down to the error itself: RENEGOTIATING 139809772520832:error:1420410A:SSL routines:SSL_renegotiate:wrong ssl version:../ssl/ssl_lib.c:2142: Greetings Leo
Re: OpenSMTPd: Unable to use TLS/SSL over IPv6
On Thu, Jan 13, 2022 at 05:25:41PM +, Stuart Henderson wrote: > On 2022/01/13 18:05, Leo Unglaub wrote: > > Hey, > > > > On 11/01/2022 21:28, Stuart Henderson wrote: > > > I bet it is MTU related. Try lowering MTU on that interface (you > > > cannot do it separately for IPv4 and IPv6 so it will change both, > > > but that's not likely to be a problem) and get someone who has > > > seen the problems to re-test. > > > > thank you so much for your answer. I would have never ever thought about the > > MTU in this case. I used the default 1500. I talked to the technical support > > from the datacenter (Hetzner Online) and they asured me that 1500 is > > correct. > > > > However, i have set the value to 1400 and asked some people who had the > > issue to re-test it. I will post the results of the test here so other > > people can find them via a search engine. > > > > Thank you so much, very kind of you! > > The possible issue is that many people (especially people connecting > over tunnels, but also those on pppoe) are on lower MTUs than this. > Normally this is OK as fragmentation-needed messages will sort things > out but sometimes firewalls are not be configured to pass these which > will cause problems. If that _is_ what's happening then there are > other ways to fix it but changing MTU is often the easiest one that > you can do yourself. Well, I can connect to his server using: openssl s_client -starttls smtp -connect mail.unglaub.at:25 The handshake completes and I'm able to issue smtp commands. However smtpd always reports that opportunistic TLS failed, and downgrades to plaintext.
Re: OpenSMTPd: Unable to use TLS/SSL over IPv6
On 2022/01/13 18:05, Leo Unglaub wrote: > Hey, > > On 11/01/2022 21:28, Stuart Henderson wrote: > > I bet it is MTU related. Try lowering MTU on that interface (you > > cannot do it separately for IPv4 and IPv6 so it will change both, > > but that's not likely to be a problem) and get someone who has > > seen the problems to re-test. > > thank you so much for your answer. I would have never ever thought about the > MTU in this case. I used the default 1500. I talked to the technical support > from the datacenter (Hetzner Online) and they asured me that 1500 is > correct. > > However, i have set the value to 1400 and asked some people who had the > issue to re-test it. I will post the results of the test here so other > people can find them via a search engine. > > Thank you so much, very kind of you! The possible issue is that many people (especially people connecting over tunnels, but also those on pppoe) are on lower MTUs than this. Normally this is OK as fragmentation-needed messages will sort things out but sometimes firewalls are not be configured to pass these which will cause problems. If that _is_ what's happening then there are other ways to fix it but changing MTU is often the easiest one that you can do yourself.
Re: OpenSMTPd: Unable to use TLS/SSL over IPv6
Hey, On 11/01/2022 21:28, Stuart Henderson wrote: I bet it is MTU related. Try lowering MTU on that interface (you cannot do it separately for IPv4 and IPv6 so it will change both, but that's not likely to be a problem) and get someone who has seen the problems to re-test. thank you so much for your answer. I would have never ever thought about the MTU in this case. I used the default 1500. I talked to the technical support from the datacenter (Hetzner Online) and they asured me that 1500 is correct. However, i have set the value to 1400 and asked some people who had the issue to re-test it. I will post the results of the test here so other people can find them via a search engine. Thank you so much, very kind of you!
Re: OpenSMTPd: Unable to use TLS/SSL over IPv6
On 2022-01-11, Leo Unglaub wrote: > i am running OpenBSD 7.0 with all patches applied. Some weeks ago i > noticed a very strange issue with my OpenSMTPd instance. People are > unable to use TLS when connecting via IPv6. This is not just my > observation, some people on misc@ told me so as well. Works for me. I bet it is MTU related. Try lowering MTU on that interface (you cannot do it separately for IPv4 and IPv6 so it will change both, but that's not likely to be a problem) and get someone who has seen the problems to re-test.
Re: OpenSMTPd: Unable to use TLS/SSL over IPv6
Hi Leo, > Am 11.01.2022 um 19:10 schrieb Leo Unglaub : > > Hey friends, > i am running OpenBSD 7.0 with all patches applied. Some weeks ago i noticed a > very strange issue with my OpenSMTPd instance. People are unable to use TLS > when connecting via IPv6. This is not just my observation, some people on > misc@ told me so as well. > > I talked to gilles@ in private and he could confirm the issue, but he thinks > its not related to OpenSMTPd itsef and might be even an OpenBSD (LibreSSL) > issue itself. gilles@ told me to post this to the ML because it might be a > little bit more complicated. > > Here are some basics from the System. I am using the real hostname and IP > addresses so every one can look at the problem directly. > > The Server is configured to use both IPv4 and IPv6: > >> $ cat /etc/hostname.vio0 >>inet >> 116.202.103.165 255.255.255.255 >> inet6 2a01:4f8:c010:3301::dead:beef 64 -soii >> !route add -inet 172.31.1.1 -llinfo -link -static -iface vio0 >> !route add -inet default 172.31.1.1 > > I confimed it via ifconfig: > >> $ ifconfig vio0 vio0: >> flags=408843 mtu 1500 >>lladdr 96:00:00:31:1f:b5 >>index 1 priority 0 llprio 3 >>groups: egress >>media: Ethernet autoselect >>status: active >>inet 116.202.103.165 netmask 0x >>inet6 fe80::9400:ff:fe31:1fb5%vio0 prefixlen 64 scopeid 0x1 >>inet6 2a01:4f8:c010:3301::dead:beef prefixlen 64 > > I also can use ping and ping6 to reach other servers and the server can be > reached over IPv4 and IPv6. So this seams to work. You can test the SSL connection (from a host that supports OpenSSL and IPv6) as follows: $ openssl s_client -connect [2a01:4f8:c010:3301::dead:beef]:25 -starttls smtp https://www.hardenize.com/report/storm-peaks.northrend.azeroth.wow-data.net/1641931125 HTH Mike
OpenSMTPd: Unable to use TLS/SSL over IPv6
Hey friends,
i am running OpenBSD 7.0 with all patches applied. Some weeks ago i
noticed a very strange issue with my OpenSMTPd instance. People are
unable to use TLS when connecting via IPv6. This is not just my
observation, some people on misc@ told me so as well.
I talked to gilles@ in private and he could confirm the issue, but he
thinks its not related to OpenSMTPd itsef and might be even an OpenBSD
(LibreSSL) issue itself. gilles@ told me to post this to the ML because
it might be a little bit more complicated.
Here are some basics from the System. I am using the real hostname and
IP addresses so every one can look at the problem directly.
The Server is configured to use both IPv4 and IPv6:
$ cat /etc/hostname.vio0
inet 116.202.103.165 255.255.255.255
inet6 2a01:4f8:c010:3301::dead:beef 64 -soii
!route add -inet 172.31.1.1 -llinfo -link -static -iface vio0
!route add -inet default 172.31.1.1
I confimed it via ifconfig:
$ ifconfig vio0
vio0: flags=408843 mtu 1500
lladdr 96:00:00:31:1f:b5
index 1 priority 0 llprio 3
groups: egress
media: Ethernet autoselect
status: active
inet 116.202.103.165 netmask 0x
inet6 fe80::9400:ff:fe31:1fb5%vio0 prefixlen 64 scopeid 0x1
inet6 2a01:4f8:c010:3301::dead:beef prefixlen 64
I also can use ping and ping6 to reach other servers and the server can
be reached over IPv4 and IPv6. So this seams to work.
Here is my OpenSMTTPd config. The only thing i replaced is the
encryption key:
##
## Queue
##
queue compression
queue encryption x
##
## SMTP
##
smtp max-message-size 80M
smtp sub-addr-delim "+"
##
## Tables
##
table aliases file:/etc/mail/aliases
table vdomains file:/etc/mail/table-vdomains
table vaddr file:/etc/mail/table-vaddr
table credentials file:/etc/mail/table-credentials
table filter-dyndns file:/etc/mail/table-filter-dyndns
table vmailstub file:/etc/mail/table-vmailstub
##
## PKI
##
pki "*" cert "/etc/ssl/storm-peaks.northrend.azeroth.wow-data.net.fullchain.pem"
pki "*" key "/etc/ssl/private/storm-peaks.northrend.azeroth.wow-data.net.key"
##
## Filter
##
filter "check-dyndns" phase connect match rdns regex disconnect "550
no residential/dyndns connections"
filter "check-rdns" phase connect match !rdns disconnect "550 rDNS missmatch"
filter "check-fcrdns" phase connect match !fcrdns disconnect "550 FCrDNS
missmatch"
filter "dnsbl" proc-exec "filter-dnsbl -v ix.dnsbl.manitu.net dnsbl.dronebl.org
all.spamrats.com dnsbl.sorbs.net bl.spamcop.net"
##
## Listen
##
listen on lo0
listen on egress tls pki "*" filter { "check-dyndns" "check-rdns" "check-fcrdns"
"dnsbl" }
listen on egress port submission tls-require pki "*" auth
listen on egress port 25255 tls-require pki "*" auth
##
## Actions
##
action "outbound" relay
action "local-lmtp" lmtp "/var/dovecot/lmtp" rcpt-to virtual
##
## Matches
##
match from any for domain rcpt-to action "local-lmtp"
match auth from any for any action "outbound"
To me it looks like i am not doing anything different for IPv4 or IPv6.
I am just listening on egress and according to ifconfig is assigned to
vio0. But people cannot use SSL/TLS on IPv6, but it works fine when
using IPv4. This results in some emails getting delayed from IPv6
senders, until they downgrade or switch to IPv4.
Does someone of you have an idea why this might happen? To me the config
seams clean. Do you have this issue on other instances as well?
Thank you so much and greetings
Leo
OpenBSD 7.0 (GENERIC.MP) #3: Wed Dec 15 13:14:26 MST 2021
r...@syspatch-70-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4177379328 (3983MB)
avail mem = 4034760704 (3847MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5ad0 (10 entries)
bios0: vendor Hetzner version "2017" date 11/11/2017
bios0: Hetzner vServer
acpi0 at bios0: ACPI 1.0
acpi0: sleep states S5
acpi0: tables DSDT FACP APIC HPET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel Xeon Processor (Skylake, IBRS), 2100.37 MHz, 06-55-04
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,CLWB,AVX512CD,AVX512BW,AVX512VL,PKU,MD_CLEAR,IBRS,IBPB,SSBD,ARAT,XSAVEOPT,XSAVEC,XGETBV1,MELTDOWN
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line
16-way L2 cache
cpu0: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu0: DTLB 255 4KB entries
