Re: portable layer rework
On Tuesday, November 19, 2019 12:07:10 AM PST gil...@poolp.org wrote: > Ihor Antonov has setup a CI that lets us spot failures to build on Linux > glibc/musl, however: > > - we don't have CI for FreeBSD, NetBSD, DFlyBSD, OSX, Solaris As soon as I get some free time I will setup FreeBSD and OpenBSD builds on SourceHut [1][2] so at least these 2 should be covered soon. [1] https://man.sr.ht/builds.sr.ht/compatibility.md [2] https://sourcehut.org/blog/2019-09-12-sourcehut-makes-bsd-software-better/ -- Ihor Antonov https://useplaintext.email
Re: portable layer rework
I herewith report that the portable branch as distributed by Void Linux works on Void Linux. (I could not get the 6.6.1p1 that I myself downloaded to work, though it did compile and install. I have not looked at what, if any, difference(s) there may be between the two.) My use of Opensmtpd is simply for myself on my home machine, with a couple different email accounts on a smarthost. RPTN On Tue, Nov 19, 2019 at 08:07:10AM +, gil...@poolp.org wrote: > Hello, > > OpenSMTPD portable was first released many many years ago thanks to the work > of Charles Longeau. > IIRC this was his first OpenBSD portable project and he bootstrapped it from > OpenSSH. > It served us well and opened us to the Linux world but it also came with its > share of errors inherent to a new project. > > Unless some of you are interested, I won't enter the details, but the basic > idea is that the portable version has a library which contains the bits that > were detected as missing/broken on a target system and, because of it's done, > it may contain a bit more than we think it does. In most cases, this isn't a > problem because if OpenSMTPD uses the portable version of strwhatever() > instead of libc's, it isn't really a big deal. > > > There are two cases however that bit us: > > The first one was the "OpenSMTPD portable on OpenBSD" case. For years, I kept > repeating that running OpenSMTPD portable on OpenBSD was not supported and > that this would lead to unexpected issues. This became a reality at some > point when building portable on OpenBSD resulted in an executable that would > crash at startup despite being fundamentally the same code as the master > branch which would work just fine. It turned out that this was because, > despite being on OpenBSD, some bits from the compat layer were still built > and these bits not being the system ones, they didn't comply with our > pledge(). > > The second one came shortly before 6.6.0p1 as people started reporting > crashes on some systems and the issue was traced back to an incorrect > inclusion of arc4random() related functions. If a target system was using > LibreSSL, which brings arc4random(), we would somehow end up using the > LibreSSL one but use our own entropy stirring function and blow up in pieces. > > > There are other downsides to the way the portable layer works and while we > had work-arounds so people wouldn't notice on systems we were actively > supporting (Linux, FreeBSD, ...), it is less than perfect for developers > willing to bring new targets. Freddy Dissaux who worked on bringing OpenSMTPD > to Solaris hit several of these downsides. > > > Anyways, I have started reworking the layer so that it conditionally builds > what it needs. > The compat layer on OpenBSD is now empty as it should and the resulting > executable runs reliably as it should too. > There's still work required because code set aside, we also have headers > issues (for which we have a workaround). > > > I REALLY need feedback from portable users running the development branch. > > Ihor Antonov has setup a CI that lets us spot failures to build on Linux > glibc/musl, however: > > - we don't have CI for FreeBSD, NetBSD, DFlyBSD, OSX, Solaris > - CI won't spot either one of the two issues I solved above as they occur at > startup / runtime > > > You can help us make the portable branch really good in a very short time and > on the long term by testing and reporting regularly that the portable branch > still works for you. > > > Gilles > > --
portable layer rework
Hello, OpenSMTPD portable was first released many many years ago thanks to the work of Charles Longeau. IIRC this was his first OpenBSD portable project and he bootstrapped it from OpenSSH. It served us well and opened us to the Linux world but it also came with its share of errors inherent to a new project. Unless some of you are interested, I won't enter the details, but the basic idea is that the portable version has a library which contains the bits that were detected as missing/broken on a target system and, because of it's done, it may contain a bit more than we think it does. In most cases, this isn't a problem because if OpenSMTPD uses the portable version of strwhatever() instead of libc's, it isn't really a big deal. There are two cases however that bit us: The first one was the "OpenSMTPD portable on OpenBSD" case. For years, I kept repeating that running OpenSMTPD portable on OpenBSD was not supported and that this would lead to unexpected issues. This became a reality at some point when building portable on OpenBSD resulted in an executable that would crash at startup despite being fundamentally the same code as the master branch which would work just fine. It turned out that this was because, despite being on OpenBSD, some bits from the compat layer were still built and these bits not being the system ones, they didn't comply with our pledge(). The second one came shortly before 6.6.0p1 as people started reporting crashes on some systems and the issue was traced back to an incorrect inclusion of arc4random() related functions. If a target system was using LibreSSL, which brings arc4random(), we would somehow end up using the LibreSSL one but use our own entropy stirring function and blow up in pieces. There are other downsides to the way the portable layer works and while we had work-arounds so people wouldn't notice on systems we were actively supporting (Linux, FreeBSD, ...), it is less than perfect for developers willing to bring new targets. Freddy Dissaux who worked on bringing OpenSMTPD to Solaris hit several of these downsides. Anyways, I have started reworking the layer so that it conditionally builds what it needs. The compat layer on OpenBSD is now empty as it should and the resulting executable runs reliably as it should too. There's still work required because code set aside, we also have headers issues (for which we have a workaround). I REALLY need feedback from portable users running the development branch. Ihor Antonov has setup a CI that lets us spot failures to build on Linux glibc/musl, however: - we don't have CI for FreeBSD, NetBSD, DFlyBSD, OSX, Solaris - CI won't spot either one of the two issues I solved above as they occur at startup / runtime You can help us make the portable branch really good in a very short time and on the long term by testing and reporting regularly that the portable branch still works for you. Gilles
