Hi Eric
On Tue, Aug 23, 2011 at 19:13, Eric Schulte <schulte.e...@gmail.com> wrote:
> Note that during interactive evaluation if the exit status is non-0 then
> STDERR will be dumped into a babel error buffer which will be poped up,
> so this information will not be silently discarded.
Thanks for emphasizing this. I didn't use code block evaluation much
for ob-sh and haven't yet noticed this rule with the last exit status
that seems very important to know.
Two things I would like to comment referring to the examples 1) to 5)
below:
a) _completeness_ of the process output:
- When the last exit status is 0, babel silently discards possible
important information from stderr, see mainly 1), also 2) to 3).
- When the last exit status is >0, babel pops up the "Org-Babel
Error Output" and silently discards possible important
information from stdout, see 4).
b) _decision_ whether the process succeeded or failed:
- This is made by discriminating the last exit status 0 or not. But
there are even standard processes with an exit status and stderr
not simple, see 4) to 5) ("simple": 0 => success and output only
on stdout, >0 => failure and output only on stderr).
My suggestions for changes (I have understood that this would not be
easy to do, babel grew differently):
a) _completeness_ of the process output:
- When the last exit status is 0: Include stderr in "#+results:" by
default. In the minority of use cases where the behavior changes
I think this is most of the time an improvement.
- When the last exit status is >0 and babel pops up the "Org-Babel
Error Output": Include stdout there by default. In the minority
of use cases where the behavior changes I think this is always an
improvement.
b) _decision_ whether the process succeeded or failed:
- My point of view is that babel code block evaluation is not
always able to make such a decision for two reasons, at least in
the case of the languages awk, sh and shell: Firstly not
universally because it would require to consider which process it
runs, e. g. the exit status 1 means success for diff or grep but
>0 means failure for most other processes. Secondly not reliably
because it would require to consider for which purpose this
process is used during this evaluation. When one wants to
document the output of a failure then "Org-Babel Error Output"
does not come in handy.
- An option to move this decision to the user would help. For this
I wish to have an additional and somehow more manual operation
mode that bypasses the interpretation of the last exit status and
omits "Org-Babel Error Output". Achim already phrased it in this
thread as:
- "I think that it would be generally useful (not just for shell
blocks) to be able to capture stderr, either together with
stdout or separately into a result target block and have the
return status available as a variable."
- I would only add: It would be nice to have both variants of
"either or" available to choose from. The first for better
human readability and the second to read stdout without stderr
into the next chained code block again.
----------------------------------------------------------------
Comments and questions for the examples and workarounds:
Examples 4) to 5) end up with an empty "Org-Babel Error Output" and
"#+results:" is emptied.
- Since they are both empty it is not necessarily clear what happened.
- The Messages buffer says "Babel evaluation exited with code 1" and
"Code block produced no output.". The second statement is not
necessarily clear:
- Is this an _info_ or was output expected and this is an _error_
that even stopped babel from doing more work?
- Is the message "Code block evaluation complete." from other
cases an indication whether babel finished all its work? If this
is an indication, isn't this message missing in the following
case?
#+begin_src sh :results output
true
#+end_src
#+results:
- Which "output" was meant to be missing?
- Output missing on stdout (example 5) or was stdout not
considered due to last exit status >0 (example 4)?
- Output missing on stderr?
At the end are the workarounds I will use for now for the examples 1)
to 5). Shown is the complete variant when I need to know the last exit
status.
- 2>&1 :: collect stderr
- $? :: last exit status
- true :: omit any "Org-Babel Error Output"
----------------------------------------------------------------
Examples:
1) command(s) before the last command have error
- non-existent file myfile, output mixed on stdout and stderr
- in a terminal (not babel code block evaluation):
- shell command:
#+begin_example
ls myfile
echo b
#+end_example
- output (stderr: ls, stdout: echo, exit status ls: 2, exit
status echo: 0):
#+begin_example
ls: myfile: No such file or directory
b
#+end_example
- with babel:
#+begin_src sh :results output
ls myfile
echo b
#+end_src
#+results:
: b
2) strace (Linux)
- output on stderr
- in a terminal (not babel code block evaluation):
- shell command:
#+begin_example
strace true
#+end_example
- output (stdout: empty, stderr: syscalls, exit status: 0):
#+begin_example
execve("/bin/true", ["true"], [/* 54 vars */]) = 0
[...]
exit_group(0) = ?
#+end_example
- with babel:
#+begin_src sh :results output
strace true
#+end_src
#+results:
3) truss (Solaris)
- output on stderr
- in a terminal (not babel code block evaluation):
- shell command:
#+begin_example
truss true
#+end_example
- output (stdout: empty, stderr: syscalls, exit status: 0):
#+begin_example
execve("/usr/bin/true", 0xFFBFF0FC, 0xFFBFF104) argc = 1
[...]
_exit(0)
#+end_example
- with babel:
#+begin_src sh :results output
truss true
#+end_src
#+results:
4) diff
- the exit status as standardized by POSIX/SUS
http://pubs.opengroup.org/onlinepubs/009695399/utilities/diff.html
- 0 :: No differences were found.
- 1 :: Differences were found.
- >1 :: An error occurred.
- example with two differing files
#+begin_src sh :results silent
echo a > /tmp/a
echo b > /tmp/b
#+end_src
- in a terminal (not babel code block evaluation):
- shell command:
#+begin_example
diff /tmp/a /tmp/b
#+end_example
- output (stdout: differences, stderr: empty, exit status: 1):
#+begin_example
1c1
< a
---
> b
#+end_example
- with babel:
#+begin_src sh :results output
diff /tmp/a /tmp/b
#+end_src
#+results:
5) grep
- the exit status as standardized by POSIX/SUS
http://pubs.opengroup.org/onlinepubs/009695399/utilities/grep.html
- 0 :: One or more lines were selected.
- 1 :: No lines were selected.
- >1 :: An error occurred.
- example:
- in a terminal (not babel code block evaluation):
- shell command:
#+begin_example
echo a | grep b
#+end_example
- output (stdout: empty, stderr: empty, exit status: 1):
#+begin_example
#+end_example
- with babel:
#+begin_src sh :results output
echo a | grep b
#+end_src
#+results:
----------------------------------------------------------------
Workarounds:
1) command(s) before the last command have error
#+begin_src sh :results output
ls myfile 2>&1
echo b 2>&1
echo "last exit status: $?" 2>&1
true 2>&1
#+end_src
#+results:
: ls: myfile: No such file or directory
: b
: last exit status: 0
2) strace (Linux)
#+begin_src sh :results output
strace true b 2>&1
echo "last exit status: $?" 2>&1
true 2>&1
#+end_src
#+results:
: execve("/bin/true", ["true", "b"], [/* 104 vars */]) = 0
: [...]
: exit_group(0) = ?
: last exit status: 0
3) truss (Solaris)
#+begin_src sh :results output
truss true b 2>&1
echo "last exit status: $?" 2>&1
true 2>&1
#+end_src
#+results:
: execve("/usr/bin/true", 0xFFBFF0FC, 0xFFBFF104) argc = 1
: [...]
: _exit(0)
: last exit status: 0
4) diff
#+begin_src sh :results output
diff /tmp/a /tmp/b 2>&1
echo "last exit status: $?" 2>&1
true 2>&1
#+end_src
#+results:
: 1c1
: < a
: ---
: > b
: last exit status: 1
5) grep
#+begin_src sh :results output
echo a | grep b 2>&1
echo "last exit status: $?" 2>&1
true 2>&1
#+end_src
#+results:
: last exit status: 1
----------------------------------------------------------------
Michael
