Boring recap: jprint is patch to make that allows you to print out the make database in json format such that it's relatively easy to parse https://github.com/tnmurphy/gmake-experimental/tree/feature/jprint
2026 Updates: As a result of use in a real project of a reasonable size I'm at last getting some insights into how to make use of the json database-dump feature (make -P) to get information out of a makefile-based build. I find this useful with AI which normally goes down the regexp route when it is creating little tools that will explore or modify the build. The AI tends to burn off a lot of tokens trying various regexp matches and correcting them when they don't quite work. I found that parseresult files short circuit that quite well because of course they're JSON and all comments and escaped newlines and other potential variable formatting is removed. The insights are mainly about how one needs the results to get created, what information is in them and so on. As an example it's generally best at the moment to create result files in the same directory as the makefile they relate to - otherwise it can become difficult to naturally discover information about a particular component. You cd to the directory with some source component and want to look something up about one of the targets or sources and it's natural to look in the parseresult in the same dir. If one accepts this then there also needs to be a way to discover all of the parseresults generated by a particular build when one is doing a whole-build analysis. The configurable parseresult filename helps but there was an even easier to use index file that had the name of the parseresults appended to it as each makefile was being parsed. The problem was that if you spread the parseresults where their makefiles were then these index files would be spread out too because the index was put wherever the parseresult file was put. i.e. this was a usability bug that defeated the purpose of the index file feature. gmake-experimental is now to append a complete list of absolute paths for parseresult files in a recurseive build into a single index file specified by a new environment variable, MAKE_JSON_INDEX. Now it's easy to do a build with many recursive submakes and still get a list of all the parseresults without searching - it's theoretically possible to analyse them in parallel as they are being generated because if e.g. one sets MAKE_JSON_INDEX to indicate a FIFO. I've been using a lot of little commandline tools to do interesting things with parseresult files - analysis of builds, lists of dependencies and reverse dependencies, searching for circular dependencies, maps from the name of a binary to where the source code is located and so on - but at the moment they're very specific to a particular system (which is of course not open) and they wouldn't really be useful to anyone else. I'm thinking about what generic tools could be made that would start to extract the value of parseresults and that might be the next addition if I don't encounter too many new bugs. BR, Tim CHANGES: A few small updates on gmake-experimental relatting to dumping out the make database as json since last year: * A couple of crash bugs have been dealt with. * A couple of json bugs have been removed - situations where invalid json could be produced. * A crude testing mechanism has been added which doesn't fit very well with the existing gnu make tests but it's better than nothing and found one of the aforementioned bugs. * Improved a way of collecting all the json dump files without forcing them to be in the same directory. On Thu, 30 Oct 2025 at 10:21, Tim Murphy <[email protected]> wrote: > https://github.com/tnmurphy/gmake-experimental/tree/feature/jprint > > I've added the the hash table stats for files and the string cache and so > on. At this point I think I have done almost everything that > --print-database did. There's probably something missing but most of it is > there. > > One now gets stuff like this: > "strcachestats": { > "buffers": { > "count": 3, > "full": 2, > "total_strings": 1272, > "total_size": 23214, > "average_size": 18 > }, > .... > "hashtable": { > "fill": 1272, > "size": 8192, > "load_percent": 16, > "rehash": 0, > "lookups": 17653, > "collisions": 827, > "collision_percent": 5 > } > > I haven't got a good set of test makefiles - I'm just using make's own > makefile. This means that some features are not getting tried out properly. > > I've got a python module with classes that match the json so one can write > programs to traverse the data. I'm not happy with it yet so I haven't > included it. In theory one could use this to start doing "useful" things - > whatever your imagination comes up with. I keep thinking along the lines > of being able to read in an old makefile and write out some new makefile > using a strategy I prefer or even converting a makefile to some alternate > build system. > > Other uses could be to get at the line number and filename information to > help one index variables or targets in a large makefile - perhaps in some > editor. > > The stats also might help some people to see why they're getting poor > performance - if they're using really large makefiles perhaps. > > Anyhow we'll see. > > Best regards, > > Tim Murphy > > On Fri, 24 Oct 2025 at 19:16, Tim Murphy <[email protected]> wrote: > >> https://github.com/tnmurphy/gmake-experimental/tree/feature/jprint >> >> On Fri, 24 Oct 2025 at 12:30, Tim Murphy <[email protected]> wrote: >> >>> >>> What's still not done: >>> Directories are not dumped in json - this is a bit of a pain but I will >>> get to it when I'm desperate and there's absolutely nothing easy or fun >>> left to do :-). >>> >>> OK, Directories now get dumped as json like so: >> "directories": { >> "RCS": { >> "status": "stat_fail" >> }, >> "src": { >> "status": "ok", >> "device": 2100, >> "inode": 32708350, >> "files": 109, >> "impossibilities": 3 >> }, >> "": { >> "files": 299, >> "impossibilities": 87, >> "directories": 7 >> } >> }, >> >> The last one is fake. I don't like this much but it contains the stats >> about how many files and directories were found and is incorrect here >> becuase I've shortened the example. >> >> So the vpath info is next and then the string cache stats. After that I'm >> not sure there's anything left. It will then be time to write some >> utilities that can use all this stuff. >> >> Best regards, >> >> Tim Murphy >> >
