I have just finished a first iteration of dubster, a test runner that runs `dub test` on each package for each dmd release.

see https://github.com/skoppe/dubster

Please provide feedback as it will determine the direction/life of this tester.

I am planning on adding a web ui/api next to look around in the data.

Today I gave it a spin and let it run on 488 packages on dub (about half). The component that runs `dub test` was done on a 2gb 2vcpu cloud instance. It compiled the packages with dmd 2.071.2-b2 and it took about 20 sec per package on average.

59 packages didn't build because of missing libraries. See the end for a total list of missing libaries. 13 packages caused dmd to use too much memory for the instance I was running it on. 112 packages had build errors, almost all of them with exitcode 1, no segfaults. 35 packages had unittests that retured non zero exitcodes (due to exceptions and failing assertions).
213 packages passed their unittests.

The remaining 56 packages I couldn't categorize automatically so easily. Would have to take a deeper look at them.

Some issues along the way:

- code.dlang.org has an api but doesn't provide an endpoint to retrieve all packages/version. Now I just scrape the site instead (thanks Adam for your dom implementation).

- Originally I was planning on running with nightlies, but the ones on the download section don't have a git commit hash associated with them. For now I just use digger to build the latest dmd releases on the worker nodes.

- Some packages when running `dub test` didn't terminate on their own.

- Linker errors (a lot of them windows)

aclui, advapi32, asound, blas, bzip2, comctl32, comdlg32, ev, fcgi, fdb_c, fmod, ftgl, gccjit, gdi32, git2, GL, gsl, gslcblas, gumbo, imm32, iup, iupcontrols, jack, Judy, kernel32, lapack, lapacke, leveldb, libco, libshp, lz32, mad, miniupnpc, mpr, mrss, mysqlclient, nanomsg, nanovg, netapi32, Netapi32, netcdf, nlopt, ole32, oleacc, oleaut32, OpenCL, powrprof, pq, rasapi32, rdkafka, rpcns4, Rpcrt4, rpcrt4, sapnwrfc, scrypt, secur32, setupapi, shell32, shlwapi, snappy, sodium, tarsnap, tcc, tcl, tcmalloc, tk, udis86, usb, user32, version, vfw32, wayland, webp, winhttp, wininet, winmm, winspool, Ws2_32, wtsapi32, X11, xcb, xkbcommon, zlib, zmq, zookeeper_mt

Reply via email to