RE: Catching errors in pipes

[Cook, Malcolm]

>On Tue, 2023-12-05 at 13:43 +0100, Sébastien Hinderer wrote:
>> Is there a recommended way to catch errors in pipelines of coommands
>> that are used in recipes, please?
>
>This isn't a GNU Make issue. It's a shell programming issue.
>
>Make just invokes the shell and waits for it to exit, then looks at the
>exit code. If you want the behavior of pipefail you have to get your
>recipe to implement that behavior: there's nothing make can do about
>it.

Sébastien,

If you like to write your recipes using bash, you might try putting this at the 
top of your makefile:

## If .ONESHELL is mentioned as a target, then when a target is built
## all lines of the recipe will be given to a single invocation of the
## shell rather than each line being invoked separately (see Recipe
## Execution).  This is typically how we want to use make in a data
## process pipeline.
.ONESHELL:

### Use bash as shell for interpreting make commands instead of the
### default SHELL, which is '/bin/sh'.  Do this even though /bin/sh is
### often symlink to '/bin/bash' since bash behaves differently when
### invoked as 'sh'.  In particular, it does not perform 'Command
### Substitution' (or FIFO processing) for $(command).
SHELL=/usr/bin/bash

# -e: quit on 1st error ; -u: unset vars are an error ; pipefail: error in 
middle of pipeline is error for the pipeline
.SHELLFLAGS = -eu -o pipefail -c

Good luck,

Malcolm Cook

>
>I recommend checking on StackOverflow or similar places looking for
>idioms that will support this for POSIX shells.
>
>Just to note, starting with the next POSIX version, set -o pipefail is
>part of the standard so you can expect it to start appearing in POSIX-
>conforming shells that don't already support it. Of course, that
>doesn't help if you must support older shells.
>
>-- 
>Paul D. Smith  Find some GNU Make tips at:
>https://www.gnu.org http://make.mad-scientist.net
>"Please remain calm...I may be mad, but I am a professional." --Mad
>Scientist
>
>
>

Re: Catching errors in pipes

[Paul Smith]

On Tue, 2023-12-05 at 13:43 +0100, Sébastien Hinderer wrote:
> Is there a recommended way to catch errors in pipelines of coommands
> that are used in recipes, please?

This isn't a GNU Make issue.  It's a shell programming issue.

Make just invokes the shell and waits for it to exit, then looks at the
exit code.  If you want the behavior of pipefail you have to get your
recipe to implement that behavior: there's nothing make can do about
it.

I recommend checking on StackOverflow or similar places looking for
idioms that will support this for POSIX shells.

Just to note, starting with the next POSIX version, set -o pipefail is
part of the standard so you can expect it to start appearing in POSIX-
conforming shells that don't already support it.  Of course, that
doesn't help if you must support older shells.

-- 
Paul D. Smith Find some GNU Make tips at:
https://www.gnu.org   http://make.mad-scientist.net
"Please remain calm...I may be mad, but I am a professional." --Mad
Scientist

Re: Catching errors in pipes

[Sébastien Hinderer]

Dear Malcolm,

So sorry for the late response!

I just wanted to say thank you for your repsone!

In what we distribute, at themoment we do not require bash for running
our GNU make commands.

That being said, I find all your suggestions very valuable and I think
even if we don't use them in our distribution, we can use them locally
and that will definitely help us debugging things.

Best wishes,

Seb.

Cook, Malcolm (2023/12/05 18:54 +):
> > Is there a recommended way to catch errors in pipelines of coommands that
> > are used in recipes, please?
>
> I use the following with good success:
>
> SHELL=/usr/bin/bash
> .SHELLFLAGS = -eu -o pipefail -c
>
> FWIW: I usually use the above in combination with:
>
> .ONESHELL:
> MAKEFLAGS += --no-builtin-rules
> MAKEFLAGS += --no-builtin-variables
> MAKEFLAGS += --output-sync
> MAKEFLAGS += --warn-undefined-variables
> .DELETE_ON_ERROR:
> .SUFFIXES:
> .SECONDEXPANSION:
>
> > If so, how backward-compatible are they?
>
> To my knowledge this has always had the same results.
>
> > I saw the .SHELLFLAGS variable but also found a bug report saying that its
> > content is passed to the shell as one single argument. Is it thus mandatory 
> > to
> > prefix every pipeline with set -o pipefail; or is there a more global way 
> > to get
> > the desired behaviour?
>
> If you reference this report you might get a reply that considers it.
>
>

RE: Catching errors in pipes

[Cook, Malcolm]

> Is there a recommended way to catch errors in pipelines of coommands that
> are used in recipes, please?

I use the following with good success:

SHELL=/usr/bin/bash
.SHELLFLAGS = -eu -o pipefail -c

FWIW: I usually use the above in combination with:

.ONESHELL:
MAKEFLAGS += --no-builtin-rules
MAKEFLAGS += --no-builtin-variables
MAKEFLAGS += --output-sync
MAKEFLAGS += --warn-undefined-variables
.DELETE_ON_ERROR:
.SUFFIXES:
.SECONDEXPANSION:

> If so, how backward-compatible are they?

To my knowledge this has always had the same results.

> I saw the .SHELLFLAGS variable but also found a bug report saying that its
> content is passed to the shell as one single argument. Is it thus mandatory to
> prefix every pipeline with set -o pipefail; or is there a more global way to 
> get
> the desired behaviour?

If you reference this report you might get a reply that considers it.