On Fri, Dec 11, 2015 at 2:31 PM, Erik <[email protected]> wrote:
> I probably use Tup a little differently than most people. Instead of using
> it to build software (which I do sometimes), I use it for repeated data
> analysis and graph generation. One my tup rules looks something like
>
> : input.json |> < %f simulator | jq -f extract_something.jq > %o |>
> output.json
>
> where the simulator produces a lot of output, but in this case I only care
> about one aspect. Without pipefail, the simulator might fail, jq will get
> empty input, which it's fine with, and the result is an empty output file.
> This then causes the next rule that uses output.json to fail, because it's
> empty, but the problem was the previous rule.
>
Another option here is to move the command bits into a bash script, and
execute that in the rule:
foo.bash:
#! /bin/bash -e -o pipefail (does this work?)
$1 simulator | jq -f ...
Tupfile:
: input.json |> ./foo.bash %f > %o |> output.json
That's not ideal if the majority of your commands are pipelines though, of
course.
>
> In general, I find it hard to believe that (especially with tup), there's
> ever a time when you wouldn't want pipefail to be active, but maybe I'm
> unique in that feeling.
>
I'm inclined to agree, and if /bin/sh supported pipefail I'd definitely
consider turning it on. Maybe it does and I haven't found where that is in
the manual?
I'm hesitant to change the shell wholesale to bash though - even though I
use it everyday, I'm not sure if there are places where it isn't installed
by default.
>
>
> I like your suggestion about the ^p^ flag, although it seems a little
> strange that a flag would also change execution to bash. However, it feels
> like a reasonable way to accomplish this without requiring pipefail to be
> active for all :-rules. After looking at the code though, another option
> that seems easy would be to have a tup config option that specifies the
> command that executes all of :-rules. This may be somewhat frowned upon due
> to "changing any of these options should not affect the end result of a
> successful build" (*), but having the default:
>
> updater.command = /bin/sh -e -c
>
> be there, but have the ability to change it to
>
> updater.command = /bin/bash -e -o pipefail -c
>
> would be very convenient, and would provide a lot of flexibility with how
> tup is run. E.g. a user could just specify bash to get access to the more
> advanced bash syntax. The downside is that it has the potential to violate
> the rules (*) of a tup config as stated above. An potentially absurd
> example would be setting
>
> update.command = /usr/bin/env python -c
>
I think if we did something like this, we would need to specify it in the
:-rule somehow rather than in the options. For example, if one of the rules
requires bash (or other shell) for something, you wouldn't want to require
every user of the software to overwrite their options file with the correct
update.command setting.
Potentially we could add this more easily in the lua parser, with something
like:
tup.definerule{command='false | echo foo', shell='/bin/bash -e -o pipefail
-c'}
I'm not sure what it would look like in the regular Tupfile parser without
something like the ^-flag suggestion, which is less flexible.
>
> I'm curious about everyone's thoughts about this. I have my hacky
> solution, which means I'm fine for the time being, but I'd be up for
> submitting a pull request of a more reasonable implementation of one of
> these ideas if it seemed worthwhile.
>
>
I'm curious if others have thoughts here too :)
Thanks,
-Mike
--
--
tup-users mailing list
email: [email protected]
unsubscribe: [email protected]
options: http://groups.google.com/group/tup-users?hl=en
---
You received this message because you are subscribed to the Google Groups
"tup-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
