I'm (ab)using tup to manage a pipeline that processes natural language data. I have a dependency issue that I'm not sure how to express.
The project root contains directories - bin/, which contains my Python scripts - training/, in which a Tupfile generates certain model files from data - workspace/, in which a Tupfile produces output files from input files. .tup is at the project root, so Tup should be able to track dependencies between any of these directories. Some of the scripts in bin/, which are used by the Tupfile in workspace/, need to load language model files which are currently located in training/ (e.g. bin/dostuff.py will load training/model.txt every time it is run from workspace/Tupfile). Tup quite rightly insists when I try to run my scripts in bin/ that the respective files in training/ should be listed as order dependencies in workspace/Tupfile, and at the moment that's what I'm doing. However, as the project grows this is not going to keep flying. workspace/Tupfile should not know what the model files are that the scripts in bin/ use; having workspace/Tupfile mention the model files specifically makes that tupfile more verbose, makes the project less DRY, and prevents good separation of concerns. I'd much prefer that Tup know that anything which invokes those scripts also depends on the model files. What I *think* I want is a way to attach the model files as dependencies to the python scripts themselves, but since the python scripts are not generated, I don't see how to tell Tup that. : ???? | training/model.txt |> ???? |> dostuff.py Another less precise but fairly reasonable solution would be a way to make all the rules in workspace/Tupfile depend on all the models in training: : example.en-fr.txt | training/** |> ../bin/dostuff.py %f > %o |> example.en-fr.stuff but Tup doesn't wildcard order-only dependencies (and even if it did, I would need an rsync-style ** wildcard to really get everything). I'm not really sure how I would fake up either of the above solutions, even with lua scripting. My question here is: is there an elegant way to express these dependencies? Am I just going about this backwards? Thanks, - Zack -- -- tup-users mailing list email: [email protected] unsubscribe: [email protected] options: http://groups.google.com/group/tup-users?hl=en --- You received this message because you are subscribed to the Google Groups "tup-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
