Re: portable layer rework

2019-11-19 Thread Ihor Antonov
On Tuesday, November 19, 2019 12:07:10 AM PST gil...@poolp.org wrote:
> Ihor Antonov has setup a CI that lets us spot failures to build on Linux
> glibc/musl, however:
> 
> - we don't have CI for FreeBSD, NetBSD, DFlyBSD, OSX, Solaris

As soon as I get some free time I will setup FreeBSD and OpenBSD builds on 
SourceHut [1][2] so at least these 2 should be covered soon. 

[1] https://man.sr.ht/builds.sr.ht/compatibility.md
[2] https://sourcehut.org/blog/2019-09-12-sourcehut-makes-bsd-software-better/


-- 
Ihor Antonov
https://useplaintext.email





Re: portable layer rework

2019-11-19 Thread epektasis
I herewith report that the portable branch as distributed by Void Linux
works on Void Linux.  (I could not get the 6.6.1p1 that I myself
downloaded to work, though it did compile and install.  I have not
looked at what, if any, difference(s) there may be between the two.)  My
use of Opensmtpd is simply for myself on my home machine, with a couple
different email accounts on a smarthost.
RPTN

On Tue, Nov 19, 2019 at 08:07:10AM +, gil...@poolp.org wrote:
> Hello,
> 
> OpenSMTPD portable was first released many many years ago thanks to the work 
> of Charles Longeau.
> IIRC this was his first OpenBSD portable project and he bootstrapped it from 
> OpenSSH.
> It served us well and opened us to the Linux world but it also came with its 
> share of errors inherent to a new project.
> 
> Unless some of you are interested, I won't enter the details, but the basic 
> idea is that the portable version has a library which contains the bits that 
> were detected as missing/broken on a target system and, because of it's done, 
> it may contain a bit more than we think it does. In most cases, this isn't a 
> problem because if OpenSMTPD uses the portable version of strwhatever() 
> instead of libc's, it isn't really a big deal.
> 
> 
> There are two cases however that bit us:
> 
> The first one was the "OpenSMTPD portable on OpenBSD" case. For years, I kept 
> repeating that running OpenSMTPD portable on OpenBSD was not supported and 
> that this would lead to unexpected issues. This became a reality at some 
> point when building portable on OpenBSD resulted in an executable that would 
> crash at startup despite being fundamentally the same code as the master 
> branch which would work just fine. It turned out that this was because, 
> despite being on OpenBSD, some bits from the compat layer were still built 
> and these bits not being the system ones, they didn't comply with our 
> pledge().
> 
> The second one came shortly before 6.6.0p1 as people started reporting 
> crashes on some systems and the issue was traced back to an incorrect 
> inclusion of arc4random() related functions. If a target system was using 
> LibreSSL, which brings arc4random(), we would somehow end up using the 
> LibreSSL one but use our own entropy stirring function and blow up in pieces.
> 
> 
> There are other downsides to the way the portable layer works and while we 
> had work-arounds so people wouldn't notice on systems we were actively 
> supporting (Linux, FreeBSD, ...), it is less than perfect for developers 
> willing to bring new targets. Freddy Dissaux who worked on bringing OpenSMTPD 
> to Solaris hit several of these downsides.
> 
> 
> Anyways, I have started reworking the layer so that it conditionally builds 
> what it needs.
> The compat layer on OpenBSD is now empty as it should and the resulting 
> executable runs reliably as it should too.
> There's still work required because code set aside, we also have headers 
> issues (for which we have a workaround).
> 
> 
> I REALLY need feedback from portable users running the development branch.
> 
> Ihor Antonov has setup a CI that lets us spot failures to build on Linux 
> glibc/musl, however:
> 
> - we don't have CI for FreeBSD, NetBSD, DFlyBSD, OSX, Solaris
> - CI won't spot either one of the two issues I solved above as they occur at 
> startup / runtime
> 
> 
> You can help us make the portable branch really good in a very short time and 
> on the long term by testing and reporting regularly that the portable branch 
> still works for you.
> 
> 
> Gilles
> 
> 

--