On Saturday, September 8, 2018 7:44:09 AM MDT Nicholas Wilson via Digitalmars-d wrote: > On Friday, 7 September 2018 at 19:15:21 UTC, Jonathan M Davis > > wrote: > > Honestly, I wouldn't rely on anything beyond dub build working > > in a consistent manner across projects. As far as I can tell, > > you can't actually do anything properly custom with dub test, > > and I'm inclined to think that how it approaches things is > > outright wrong. > > For Dcompute I define a unittest configuration that has a main > that runs tests, and the (default) library configuration excludes > source files in the test directory. It seems to work fine. > Admittedly it is a little different to you usual package, and > there are no test specific symbols and no unit tests, and I > haven't tested anything that has dcompute as a dependancy. > > Are you suggesting this will break users of Dcompute that run dub > test in their code? If so that truly does suck, because regular > unittests will not cut the mustard.
Okay. It's been several months since I dug into this, so hopefully I have the details straight. Unless no one using dcompute uses dub test on their own projects, you probably don't have the worst part of this problem, because if you did, they'd be complaining about linker errors. Honestly, unittest blocks in general are somewhat broken, but what really doesn't work correctly is code that's versioned for unit tests. e.g. if you've declared some types or functions to use specifically in your unit tests and which have no business being in the actual library (e.g. dxml has dxml/internal.d which includes stuff specifically to help the unit tests and is not part of the public API). The core problem is caused by the language, but the way dub handles things negates the workarounds for the problem. When you import a module that includes unittest blocks, and you're compiling your module with -unittest, the version identifier unittest is defined when importing the module, meaning that all unittest blocks are processed, and everything inside a version(unittest) block is considered to be part of the module. With regards to the unittest blocks themselves, this is nothing but negative. It means that they have to compile regardless of whether everything they use is available to the module importing them, and it's a pointless waste of compiler resources. This is even worse when a unittest block is inside a template, because than anyone using that template then also gets the unittest blocks compiled into their code (once for each instantation of the template), meaning that the unittest blocks in the libraries you use can actually be compiled into your program and get run when you run your own unit tests. IMHO, when importing a module, unittest blocks should be treated as version(none) rather than version(unittest), but that's not currently the case. The less straightforward problem is the helpers that are compiled with version(unittest). We can't treat everything that's version(unittest) as version(none) when importing, because that would make it impossible to share unit test helpers between modules. The cleanest way to deal with those is therefore to put them in a separate module that gets imported locally by the unittest blocks. That way, you don't accidentally mark anything in the module with the unittest blocks as version(unittest), and it allows you to hide the symbols from any modules that import your module. However, because the unittest blocks themselves are still processed when importing, it doesn't actually work to hide the symbols being imported like it should. The compiler still processes them when it imports the module. This causes problems when the modules being imported weren't actually compiled into the library that you're linking against (since normally, libraries aren't compiled with -unittest) - hence the linker errors. If you use unit test helpers that are version(unittest) in a library, you risk linker errors in any project that imports that library. Not marking them with version(unittest) can fix that problem, but it then pollutes projects using the library. While it's certainly ugly, the easiest way to solve the problem at present is to version all of your unittest blocks with a version identifier specific to your project. That way, any projects depending on your library won't actually end up compiling the unittest blocks, because while the unittest version identifier will have been declared, the version identifier specific to your project will not have been. There was actually a PR for Phobos a few months back that tried to do this to all of Phobos to fix this problem with Phobos, but it was rejected on the grounds that it was hideous and that we should solve it in the language - which I agree with, but in the interim, we still have to deal with the problem - hence why dxml has the dxmlTests version identifier. Without it, any projects using dxml would get linker errors when running their own tests. Where dub makes this whole situation worse is that for some reason, the build configuration for your library that's used when you run dub test is also used by anything that pulls in your library via dub. I had made it so that dub test declared dxmlTests so that dub test would work properly for dxml but with the idea that then any projects depending on dxml would not declare dxmlTests. However, because dub pulls in the dependencies' unit test configurations instead of just using the non-unit test one like it should, that results in any projects that depend on dxml having dxmlTests declared, so the problem isn't fixed, and all of those annoying version(dxmlTests) blocks were a waste of time. To workaround this stupidity of dub, I was forced to create a separate set of buildTypes for running the unit tests in debug, release, or with code coverage. Those buildTypes must be used, or there are no unit tests, meaning that dub test goes really fast and does nothing useful whatsoever. And because of this whole issue, I avoid dub test like the plague. So, dub needs to fix what it's doing (I can't remember what I reported at the time - I really should make sure that I have all of the facts straight and clearly explained in a bug report for dub), and once it is fixed, then dub test can be used again by projects that need to do custom stuff with their unit test configurations. But that still leaves the compiler problem, and a DIP needs to written to fix that. IIRC, Jonathan Marler was trying to fix it several months ago, but I think that he was mucking around with version(unittest) in general, and he ran into all kinds of weird problems. So, I don't know what it would really take to fix the problem in the compiler, but I think that it's pretty clear that unittest blocks should be utterly ignored when the module they're in is imported, and right now, they're not. - Jonathan M Davis
