Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

2024-06-27 Thread Ihor Radchenko
Ihor Radchenko  writes:

>> #+header-arg: :results verbatim
>> #+begin_src sh :cmdline 1 ; touch /tmp/not-an-arg
>>printf '%s\n' "$@"
>> #+end_src
>>
>> #+RESULTS:
>> : 1
>>
>> "touch ..." *are not arguments of the script*. So users should be 
>> careful to get documented behavior.
>
> I do not see any way to address this concern without introducing feature
> regression. So, let's keep things as they are and maybe document that
> :cmdline ... is passed verbatim as shell command.

https://git.sr.ht/~bzg/worg/commit/1c6390fc

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 



Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

2024-05-21 Thread Ihor Radchenko
Max Nikulin  writes:

> Frankly speaking your plan is not clear for me. My special concern is 
> DWIM behavior
> https://list.orgmode.org/874jbkcmyg.fsf@localhost
> (Ihor Radchenko Mon, 29 Apr 2024 13:33:59 +)
> and
>
>   #+begin_src sh :script-args 1 ; touch /tmp/not-an-arg
>
> if you are going to pass it literally to "sh -c" then it is 
> :script-cmdline rather than :script-args.

Your pathological example is not how we encourage these header arguments
to be used. Their intended purpose is serving as arguments, not to
complete the cmdline. Anything else is implementation details we may or
may not change in future. It need not be reflected in the header
argument names. So, I stand on the :*-args names.

> I expect a way to explicitly specify if it is a single argument or 
> multiple ones
>
>  #+begin_src sh :script-args '("a b c")
>
> vs.
>
>  #+begin_src sh :script-args '("a" "b" "c")

Max, I believe that we discussed this. This problem has nothing to do
with introducing new header arguments. It is just a question of how we
pass these header arguments to scripts. Your concerns must not stop Matt
from working on the proposed patch.

> As to literal command line, taking into account stripped outer quotes 
> issue, I do not like requirement to quote characters for shells. Even 
> splitting string into arguments using `read' might be better, but there 
> are still enough issues.

I fail to see how this relates to new header arguments.

> Besides interpreters, there is may be a stack of "launchers" like 
> toolbox in the case of applications installed as isolated flatpak/snap 
> packages:
>
> Florin Boariu to emacs-orgmode. org-ditaa woes. Thu, 19 Oct 2023 
> 12:59:59 +0200.
> https://list.orgmode.org/ZTEML8zWrB6kQflk@toolbox

This is also out of scope of the discussed header arguments. Using
custom interpreter (flapak-spawn ... bash instead of default bash) has
nothing to do with script/interpreter arguments. We may (and should)
discuss that question separately.

Let's not raise all the possible related concerns at once, unless they
cannot be resolved in future because of the discussed patch. Otherwise,
it is almost impossible to make _incremental_ progress.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 



Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

2024-05-21 Thread Max Nikulin

On 21/05/2024 01:01, Matt wrote:

I'm okay with these.  I can start on a patch for :script-args and
:program-args within ob-shell.


Frankly speaking your plan is not clear for me. My special concern is 
DWIM behavior

https://list.orgmode.org/874jbkcmyg.fsf@localhost
(Ihor Radchenko Mon, 29 Apr 2024 13:33:59 +)
and

 #+begin_src sh :script-args 1 ; touch /tmp/not-an-arg

if you are going to pass it literally to "sh -c" then it is 
:script-cmdline rather than :script-args.


I expect a way to explicitly specify if it is a single argument or 
multiple ones


#+begin_src sh :script-args '("a b c")

vs.

#+begin_src sh :script-args '("a" "b" "c")

preferably passed as separate arguments of process-files or at least 
properly escaped.


As to literal command line, taking into account stripped outer quotes 
issue, I do not like requirement to quote characters for shells. Even 
splitting string into arguments using `read' might be better, but there 
are still enough issues.


Besides interpreters, there is may be a stack of "launchers" like 
toolbox in the case of applications installed as isolated flatpak/snap 
packages:


Florin Boariu to emacs-orgmode. org-ditaa woes. Thu, 19 Oct 2023 
12:59:59 +0200.

https://list.orgmode.org/ZTEML8zWrB6kQflk@toolbox




Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

2024-05-20 Thread Matt


  On Fri, 03 May 2024 14:11:56 +0200  Ihor Radchenko  wrote --- 
  > Maybe instead use two different extra arguments:
 > :interpreter-args / :compiler-args ?
 > 
 > Then, also :script-args / :program-args
 
I'm okay with these.  I can start on a patch for :script-args and :program-args 
within ob-shell.

I anticipate :interpreter-args will be more involved since it will interact 
with 'shell-command-switch' and it'd be nice to have it also work with sessions.
 
--
Matt Trzcinski
Emacs Org contributor (ob-shell)
Learn more about Org mode at https://orgmode.org
Support Org development at https://liberapay.com/org-mode





Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

2024-05-03 Thread Ihor Radchenko
Matt  writes:

> Since we're considering this for all babel backends, "interpreter-args" 
> wouldn't describe gcc or other compilers.  What about :command-args, 
> :command-args, :cmd-args?  "Command" is the only thing I can think of that 
> describes all of the languages, other than maybe "binary" or "executable".  
> Maybe we could simply use ":args"?  Maybe ":meta-args" since gcc or bash is 
> meta to the script/artifact?

I am not a big fan. It might be easily confused with :script-args.

Maybe instead use two different extra arguments:
:interpreter-args / :compiler-args ?

Then, also :script-args / :program-args

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 



Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

2024-05-02 Thread Matt


  On Wed, 01 May 2024 20:01:03 +0200  Ihor Radchenko  wrote --- 
 > Matt m...@excalamus.com> writes:
 > 
 > > I disagree with one aspect: we shouldn't use Worg as a source of
 > > truth. The argument holds based on historical behavior of :cmdline.
 > > AFAIU, Worg is a wiki which is open, more or less, to anyone. Worg
 > > contents, AFAIU, have not always undergone review. The manual should
 > > be the final authority. Fortunately, there's nothing in the manual
 > > about :cmdline.
 > 
 > For babel backends specifically, WORG is _the_ documentation for the
 > built-in backends. It is what we will eventually move to the official
 > manual and it is what we point users to from the manual for now.

Okay, I didn't know we made an explicit reference.

  > (2) nobody got around to actually move things to the manual.

I'd be happy to help with this.  Discussion for another thread, though.

 > > Are we thinking of implementing these for other languages, beyond
 > > ob-shell?
 > 
 > Yes. The title of this thread has "across babel backends" :)

:)

 > > If we're looking at these as general headers, then I don't think "arg"
 > > is the correct term here since a switch may not take a value. For
 > > example, the "-r" option for Bash (IIUC).
 > 
 > > Quick name ideas that aren't good yet may inspire better ones by
 > > inspiring disgust--:switches, :flags, :options, (using an "i" prefix
 > > for "interpreter") :iswitches, :iflags, :ioptions
 > 
 > Emm... but "command line arguments". No?
 
I was thinking it'd be strange to have an "argument to an argument."  However, 
the Bash man says things like "non-option arguments".  It wasn't my intent to 
have an argument argument argument.

Since we're considering this for all babel backends, "interpreter-args" 
wouldn't describe gcc or other compilers.  What about :command-args, 
:command-args, :cmd-args?  "Command" is the only thing I can think of that 
describes all of the languages, other than maybe "binary" or "executable".  
Maybe we could simply use ":args"?  Maybe ":meta-args" since gcc or bash is 
meta to the script/artifact?

--
Matt Trzcinski
Emacs Org contributor (ob-shell)
Learn more about Org mode at https://orgmode.org
Support Org development at https://liberapay.com/org-mode





Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

2024-05-01 Thread Ihor Radchenko
Matt  writes:

> I disagree with one aspect: we shouldn't use Worg as a source of
> truth. The argument holds based on historical behavior of :cmdline.
> AFAIU, Worg is a wiki which is open, more or less, to anyone. Worg
> contents, AFAIU, have not always undergone review. The manual should
> be the final authority. Fortunately, there's nothing in the manual
> about :cmdline.

For babel backends specifically, WORG is _the_ documentation for the
built-in backends. It is what we will eventually move to the official
manual and it is what we point users to from the manual for now.

The main reasons why relevant WORG pages are not yet in the manual are
(1) not all the backends yet fully support the common parameters we
introduce in the manual (see
https://orgmode.org/worg/org-contrib/babel/languages/lang-compat.html);
(2) nobody got around to actually move things to the manual.

>   On Mon, 29 Apr 2024 15:33:38 +0200  Ihor Radchenko  wrote --- 
>
>  > I like :script-args.
> ...
>  > The counterpart should then be :interpreter-args?
>
> Are we thinking of implementing these for other languages, beyond
> ob-shell?

Yes. The title of this thread has "across babel backends" :)

> If we're looking at these as general headers, then I don't think "arg"
> is the correct term here since a switch may not take a value. For
> example, the "-r" option for Bash (IIUC).

> Quick name ideas that aren't good yet may inspire better ones by
> inspiring disgust--:switches, :flags, :options, (using an "i" prefix
> for "interpreter") :iswitches, :iflags, :ioptions

Emm... but "command line arguments". No?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 



Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

2024-05-01 Thread Matt
  On Sat, 27 Apr 2024 12:53:25 +0200  Max Nikulin  wrote --- 
 > On 26/04/2024 20:09, Ihor Radchenko wrote:
 > > Max Nikulin writes:
 > > 
 > >> However looking wider, I do not like that :cmdline for ob-shell has
 > >> different meaning than for other languages, see e.g. ob-sql. Only for
 > >> shell this parameter is treated as arguments of a *script*. In other
 > >> cases :cmdline is used to specify arguments of *interpreter* and I think
 > >> ob-shell should follow this convention.
 > > 
 > > Alas, we already have the current state of affairs documented in
 > > https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-shell.html#orge70bc7b
 > > 
 > > So, no breaking changes.

Both points are valid.

I disagree with one aspect: we shouldn't use Worg as a source of truth.  The 
argument holds based on historical behavior of :cmdline.  AFAIU, Worg is a wiki 
which is open, more or less, to anyone.  Worg contents, AFAIU, have not always 
undergone review.  The manual should be the final authority.  Fortunately, 
there's nothing in the manual about :cmdline.

  On Mon, 29 Apr 2024 15:33:38 +0200  Ihor Radchenko  wrote --- 

 > I like :script-args.

+ 1

 > The counterpart should then be :interpreter-args?

Are we thinking of implementing these for other languages, beyond ob-shell?  If 
not, I believe we should consider it.   That may be a path to resolve the 
issues of consistency and backwards compatibility.

If we're looking at these as general headers, then I don't think "arg" is the 
correct term here since a switch may not take a value.   For example, the "-r" 
option for Bash (IIUC).

Quick name ideas that aren't good yet may inspire better ones by inspiring 
disgust--:switches, :flags, :options, (using an "i" prefix for "interpreter") 
:iswitches, :iflags, :ioptions

--
Matt Trzcinski
Emacs Org contributor (ob-shell)
Learn more about Org mode at https://orgmode.org
Support Org development at https://liberapay.com/org-mode





Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

2024-04-29 Thread Ihor Radchenko
Max Nikulin  writes:

> It is documented as
> " :cmdline  ... [arg_n]
>
> Use the :cmdline header arg to pass arguments to a shell command."
>
> However current implementation allows code injection through args, 
> including a trivial one
>
> #+header-arg: :results verbatim
> #+begin_src sh :cmdline 1 ; touch /tmp/not-an-arg
>printf '%s\n' "$@"
> #+end_src
>
> #+RESULTS:
> : 1
>
> "touch ..." *are not arguments of the script*. So users should be 
> careful to get documented behavior.

I do not see any way to address this concern without introducing feature
regression. So, let's keep things as they are and maybe document that
:cmdline ... is passed verbatim as shell command.

>> What might be done is introducing _two_ different header arguments - one
>> for interpreter switches, and another for script/program switches.
>> 
>> Say, :interpreter-cmdline and :script-cmdline.
>> Then, we can call the current :cmdline behaviour "dwim" and allow users
>> to be more explicit if necessary.
>
> It is too easy to confuse org-babel, so "dwim" works in simple cases 
> only. Independent header arguments make things more clear, I would 
> prefer :script-args. The question is whether they should be interpreted 
> by shell (flexibility and shooting feet) or more strict syntax `("hello 
> world" 1 a) should be used.

I like :script-args.
The counterpart should then be :interpreter-args?

The point of "dwim" is mostly to keep backwards-compatibility. We may
discourage :cmdline for non-trivial cases.

More strict syntax with '(   ...) is possible for the
new header arguments, not for the old :cmdline where the existing
backends may not be able to understand the list format.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 



Re: [DISCUSSION] The meaning of :cmdline header argument across babel backends

2024-04-27 Thread Max Nikulin

On 26/04/2024 20:09, Ihor Radchenko wrote:

Max Nikulin writes:


However looking wider, I do not like that :cmdline for ob-shell has
different meaning than for other languages, see e.g. ob-sql. Only for
shell this parameter is treated as arguments of a *script*. In other
cases :cmdline is used to specify arguments of *interpreter* and I think
ob-shell should follow this convention.


Alas, we already have the current state of affairs documented in
https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-shell.html#orge70bc7b

So, no breaking changes.


It is documented as
" :cmdline  ... [arg_n]

Use the :cmdline header arg to pass arguments to a shell command."

However current implementation allows code injection through args, 
including a trivial one


#+header-arg: :results verbatim
#+begin_src sh :cmdline 1 ; touch /tmp/not-an-arg
  printf '%s\n' "$@"
#+end_src

#+RESULTS:
: 1

"touch ..." *are not arguments of the script*. So users should be 
careful to get documented behavior.



And shell scripts are not like SQL queries - they often do need to check
arguments. So, the current behaviour is justified, IMHO.


stackoverflow is full of suggestion how to pass arguments to a SQL 
script executed by mysql. Unfortunately it is unsafe and allows 
injection of code. psql (PostgreSQL) allows to pass parameters, however 
it is more like :var than script arguments. So it is true that CLI 
clients for SQL databases do not implement positional parameters.


ARGV is treated in a quite specific way by awk. It may contain file 
names, variable assignments, and might be overwritten in BEGIN block. 
However a close ob-awk header argument is :cmd-line, not :cmdline, so 
inconsistency is even greater.



What might be done is introducing _two_ different header arguments - one
for interpreter switches, and another for script/program switches.

Say, :interpreter-cmdline and :script-cmdline.
Then, we can call the current :cmdline behaviour "dwim" and allow users
to be more explicit if necessary.


It is too easy to confuse org-babel, so "dwim" works in simple cases 
only. Independent header arguments make things more clear, I would 
prefer :script-args. The question is whether they should be interpreted 
by shell (flexibility and shooting feet) or more strict syntax `("hello 
world" 1 a) should be used.