when you have a clean mkfile, doing mk clean; mk install is faster than all the dependency checking you'd want to do, specially is the project is a big bloat
take X11 for instance.... how long does it take to build it? On Mon, Jan 17, 2011 at 2:31 PM, Ciprian Dorin Craciun <[email protected]> wrote: > On Mon, Jan 17, 2011 at 17:53, Robert Raschke <[email protected]> wrote: >> >> On Mon, Jan 17, 2011 at 3:33 PM, Ciprian Dorin Craciun >> <[email protected]> wrote: >>> >>> On Mon, Jan 17, 2011 at 17:00, Robert Raschke <[email protected]> >>> wrote: >>> > Your email also doesn't explain why you cannot generate a "normal" >>> > mk file. >>> >>> I'm afraid I don't understand the question. What do you mean by >>> "generating a normal mk file"? >>> A) Do you mean why am I using a generator that writes the `mk` >>> script instead of writing the `mk` script myself by hand? The answer >>> to this is complexity: writing `mk` is Ok when you have a simple >>> application to build, but as the application grows larger so does the >>> make script. (And using meta rules is not always possible.) >>> B) Why isn't the output script a "normal" `mk` script? Actually is >>> a very simple script (no meta-rules, no shell expansion, etc.). It's >>> just big. :) >>> >> >> Sorry, I meant an idiomatic mk file, in the sense as they are used within >> the Plan 9 distribution. Have a look at "Plan 9 Mkfiles" >> (http://www.cs.bell-labs.com/sys/doc/mkfiles.html) and "Maintaining Files on >> Plan 9 with Mk" (http://www.cs.bell-labs.com/sys/doc/mk.html), if you >> haven't already done so. >> >> I think by listing all your dependencies one by one, step by step, you are >> bypassing a lot of the strengths of a make system. I would expect your >> generator to produce a mk include file with the meta rules plus the mk file >> itself which lists file dependencies in a concise manner. >> >> Robby > > On Mon, Jan 17, 2011 at 17:51, Federico G. Benavento > <[email protected]> wrote: >> >> a normal mkfile does have meta-rules and if you have so many targets >> wouldn't it make sense to have more mkfiles? >> >> -- >> Federico G. Benavento > > > I'll respond to both Robert and Federico in the same email, as > their observations an suggestions are on the same topic. > > So for starters I've read both mentioned papers "Plan9 Mkfiles" > and "Maintaining Files on Plan9 with Mk", and I have the following > observations: > * the second paper "Maintaining Files on ..." is more a general > guide that describes the semantic and syntax of `mk` files; > * the first paper "Plan9 Mkfiles" focuses entirely and exclusively > on writing `mk` files for building Plan9 native applications; that is > it describes the general rules on how to write short and simple `mk` > files that take advantage on the existing Plan9 build infrastructure; > * as a consequence both of them seem to suggest writing small `mk` > scripts that individually build each application or library; > * unfortunately what they don't deal with is inter-dependencies > between multiple projects or libraries each with it's own `mk` script; > * thus if looking into the `plan9port` source code, inside the > `src/mkfile` I see the following snippet (and I count about 50 other > similar instances): > ~~~~ > libs-%:V: > for i in $LIBDIRS > do > (cd $i; echo cd `pwd`';' mk $MKFLAGS $stem; mk $MKFLAGS $stem) > done > ~~~~ > > But after reading the paper "Recursive Make Considered Harmful", I > see that this approach poses at least the following problems: > * when developing and updating a single library, there is no way > in which I can instruct `mk` to rebuild all dependent applications -- > unless I know about them and enumerate them one by one; as a > consequence -- unless I know very well the real dependency graph -- > `mk` doesn't help me at all and I have to `mk clean all`; > * second, there is a performance bottleneck as now libraries can't > be built in parallel; (this is fine if a library is quite big, but if > I have libraries with a number of files on par with the number of > processors I have wasted time;) > * third, it's almost impossible to make a dependency between one > library to another; the dependency is encoded in their order in the > variables like `LIBDIRS`; > > I hope that these observations answer the first question of why > don't I just create a number of separate files one for each project or > library. > > (The cited paper is at http://aegis.sourceforge.net/auug97.pdf .) > > For the second question about why no "meta-rules", the answer is > somehow trickier, thus I'll give a few problems which arise when using > meta-rules: > * not all files of the same type are built by using the same rule; > (for example for some files I want to enable debugging, for others > not); thus I don't see how a meta-rule would solve this problem; > (unless I create separate `mk` files or I resort to filename tags -- > for example I would call `*.debug.c` all C files that I want to be > debugged, and `*.release.c` for those I don't); > * second, each file has a unique dependency graph -- for example > one `*.c` file might include other headers than another `*.c` file in > the same project; thus when updating a single header I want to be able > to build only those `*.c` files that actually depend on it; (this > observation is less important for C applications, but for other type > of programs -- like Java -- this fine grained dependency tracking > means a lot of saved computing power); > * third, in my brief experience with make files, meta-rules are > quite hard to get right... furthermore it is impossible to have two > patterns like: `%.%.x`; just imagine I have two types of images: > background and foreground and I want to superimpose them, then I would > like to be able to write `%{foreground}.%{background}.jpg : > %{foreground}.jpg %{background}.jpg ..."; (I know that I can resort > here to "<| generating command" but it seems just plain wrong...) > > Now I hope nobody was offended by my observations regarding the > way `mk` scripts are currently written. I am sure that for the purpose > of building Plan9 applications this way is the best trade-off. But > applying the same techniques to more complex programming languages or > other systems is quite hard... Thus I just wanted to use the good `mk` > tool as a backend for a build script generator that is more intimately > acquainted with what it tries to build. (I want to extend my generator > to Python -- as a replacement for `setup.py` -- and Java -- as a > replacement for all the awful tools that exist in that ecosystem, > especially Ant.) > > Ciprian. > > -- Federico G. Benavento
