On Mon, Jan 2, 2017 at 8:08 AM, <[email protected]> wrote: > Hello, > > I have some commands in my build system that do not take the input files > from the command line but have hardcoded input paths. My Tupfile looks > basically like this: > > : foreach src/*.in |> genx %f -o %o |> tmp/%B.out tmp/<out> > > : tmp/<out> |> geny -o %o |> build/mainy > : tmp/<out> |> genz -o %o |> build/mainz > > So some command generates files in tmp/, then geny and genz basically grab > tmp/*.out to build two main files. So far so good. > > But when I add a new *.in file in src/ tup will only execute the first line > and not the last two. As a work-around I've added ;: %<out> to the last two > commands, but that feels like a dirty hack, which I don't want in my build > system, otherwise I would still use make :-) > > Is there a more idiomatic way to have the group as a real dependency?
Hi Tobias, I think the underlying problem here is less about the groups and more about the fact that tup doesn't track dependencies on directories. I'm assuming that your geny / genz commands do something like: cat tmp/*.out > build/mainy In other words, the geny command is the one responsible for doing the wildcard expansion on tmp, which means its basically doing an opendir()/readdir() on the tmp directory. Tup doesn't track the opendir() like it tracks open(), though it definitely should if we could find a way to do so. I think the main problem I have is trying to figure out what to do in a case where you opendir() on a directory and also write a file to that same directory - in other words, the command both reads from and writes to the same directory, causing a circular dependency. (Though it looks like your case doesn't have this, due to the separate tmp/ and build/ directories, so maybe we should try adding it?) As a simpler example where tup fails here, consider this Tupfile: : *.c |> gcc %f -o %o |> prog : |> gcc *.c -o %o |> prog2 Both prog and prog2 are exactly the same, but in one case the wildcard is tracked by tup, and in the other case the *.c is expanded by the shell (meaning the second command should really have a dependency on the current directory, as well as all the .c files). The first command works properly when a new .c file is created, but the second one incorrectly does not re-execute because we aren't tracking the directory accesses. And this is an example of where it'd be a circular dependency, since creating prog2 in the current directory changes the contents of the directory, so then we'd have to re-execute the command. So tup tries to work around not supporting opendir() by making you do the wildcards in the Tupfile. In the prog/prog2 example, you would favor the first one over the second. In your case, maybe instead of geny doing: (find -o flag) cat tmp/*.out > $output You would do: (find -o flag and inputs) cat $inputs > $output and then using %<out> in your rule is a little more natural: : tmp/<out> |> geny %<out> -o %o |> build/mainy The way groups factor into your case is that group dependencies are "sticky" dependencies, meaning they only affect the ordering of commands that need to be executed. So if genx and geny both needed to be rebuilt, then it ensures geny is built after genx. However, "normal" dependencies are the ones that tup picks up automatically via its dependency checking, so the files actually read by genx & geny are the ones used to determine if they need to be rebuilt in the first place. The sticky vs. normal link distinction is elaborated a bit in the generated header example: http://gittup.org/tup/ex_generated_header.html When you use a %<group> inside the command string, it essentially upgrades the sticky link to a normal link, which "fixes" your build because it works around the fact that we ignore opendir() :/. If anyone has ideas on how to incorporate directory level dependencies without throwing in circular dependencies in places, please post! Thanks, -Mike -- -- tup-users mailing list email: [email protected] unsubscribe: [email protected] options: http://groups.google.com/group/tup-users?hl=en --- You received this message because you are subscribed to the Google Groups "tup-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
