Hi,
I have been looking into how to test some cases where integration tests are very difficult or even impossible to create for. Those are often found in networking related and system specific code code (network.c, streams, FPM and more). I was recently fixing one such bug and decided to give a try which resulted in this PR:
https://github.com/php/php-src/pull/16987 .
There was a suggestion of RFC but that might be a bit too much as it's just an internal change / addition. But certainly some overview on internals should be done so writing this instead.
I decided to use cmocka in that PR because I had some experience with that. It's quite small and still very powerful and allow vast mocking options. It's a bit manual but it gives a bigger control over the mock. It relies on --wrap linking option that replaces original functions with wraps. This is however available only on Linux (or maybe some other Unix variants) but doesn't work on MacOS or Windows. The developers that want to use it on those platforms would need to use some Linux Virtualisation option (e.g. Docker). It also requires static library which is supported by embed SAPI that can be compiled statically. That limits number of extensions to use but the main use cases don't really have deps so it should be fine.
I did also some research into the other mocking libraries in C. There is a Unity with CMock, FFF and some C++ libs like GUnit, Criterion and Trompeloeil that I looked into. I quickly discarded GUnit and Trompeloeil as they relay on C++ virtual methods and require wrapping C code to C++ which is very inconvenient. FFF seems too simple and maybe quite inflexible for our needs as well. Criterion also optionally uses wrap so I didn't see much advantages compare to cmocka. So it left Unity with CMock that allows generating custom mocks using a Ruby script. That seemed initially quite nice but after spending around two hours with trying to make it works for PHP codebase, I just gave up. It gets quite messy for complex scenarios and I just didn't figure out how to nicely mock libc functions without any modification to php-src.
In terms of CI. It has got its own build which is very simple and it tests just specific parts so we could just limit it to run only for changed files which might be quite convenient.
So the proposed PR is probably the only reasonable unit testing that I can come up with. I think it should be completely optional initially for people to use - more like an experiment. If it becomes used, then good of course. And if it becomes pain, we can just get rid of it. Has anyone got any objections to get this merged? If not I plan to merge it early in January.
Cheers
Jakub
