On Sat, May 09, 2020 at 05:12:59PM +0200, Willy Tarreau wrote: (...) > > I also have the frustation to not be 100% > > sure I did not forget some includes which are naturally included by > > other dependencies, > > It's less of a problem. We consider that a file should include the files > that it needs. Often if you forget some it continues to work because most > of the low-level includes are used everywhere and already included where > the other ones are used. So you don't really risk failing in this case. > What sometimes happens is that someone cleans a few unneeded includes and > discovers an unrelated breakage, but that's quickly fixed then.
So I reached the end of my journey through this mess. At least it was very informative. I discovered that almost all include files are used by almost all files. This is due to the multiple occurrences of embedded structs. For example in server.h we have a struct ssl_ctx, which then requires the SSL dependencies to be loaded. And given that in a hop-by-hop fashion the struct server appears almost everywhere, we end up loading SSL for every file. And the funniest part is that the SSL stack is split into many small independent files which all load openssl_conf.h, resulting in that file being loaded 2400 times to build haproxy. Simply enabling SSL adds 42% more lines to the build process. Regardless, I've done what was possible at this stage in the development cycle, i.e. only move stuff between files but not doing any code change. Doing this I could still significantly shrink the amount of cross dependencies, and above all, make them almost insensitive to the load order (I'm saying "almost" because I found and fixed a counter-example late in the process). The files are easier to find, there's no more types+proto+common, everything coming from haproxy is in haproxy/ and what's imported from outside is in import/. The files are mostly referenced in alphabetical order (I'd do a final pass before releasing, but experience shows that it doesn't last long, in part because not everyone in the world uses the latin alphabet on a daily basis and mistakes happen). I purposely broke some long dependency chains using a few forward declarations, but not quite often because I don't like to do this as the caller must include the correct file to use the member and that's not always obvious (plus that's not possible with typedefs but fortunately they're absolutely not welcome here). A few files were split (e.g. checks -> check+mailers+tcpcheck+extcheck). I've built on a very old and stubborn AIX-5.1 and on FreeBSD 11.3, so far so good. Given that rebasing this work is not as hard as I'd have imagined thanks to Git's extremely powerful rename detection, I'm OK with merging it a bit later if needed but I'd rather have some feedback before the final release. I'm just asking here that the regular contributors have a glance at the branch named "20200605-rework-include-final" and honestly say how they feel about adopting this (even if they don't care, that's fine). Such work may only be done very late in the LTS cycle, and always has an impact on backports later, so I'd rather not do that again for the next few years! Some doc is still needed about this. I even though about placing a README in include/ I just have no idea whether any newcomer would read it. Advices welcome as well. Have a nice week-end, Willy
