Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Scott Kostyshak]

On Mon, Jul 31, 2017 at 11:57:55PM +0200, Christian Ridderström wrote:

> Do we have an overview somewhere (with patch reference) for the
> alternatives proposed for beta1, which is then what's likely to end up in
> 2.3?
> Note: I did just look at the wiki page but didn't see it there clearly.

No we don't have anything on the wiki.

Scott


signature.asc
Description: PGP signature

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Christian Ridderström]

On 31 July 2017 at 20:44, Guillaume MM  wrote:

> Le 31/07/2017 à 13:31, Jürgen Spitzmüller a écrit :
>
>> I meant it in this sense. If a vote only means "I did not have a
>> look at
>>
>> the patch but I am fed-up so let us go ahead" then it is not taking
>> responsibilities.
>>
>>
>> A vote is a vote. If the given voting will be Rates differently, this
>> will be have been the last voting I have participated on this list.
>>
>
I guess he wrote that from a iPhone, which I say because I (unwillingly)
have an iPhone and my girlfriend has remarked on the reduced quality of my
language when I use the phone.


> I am not sure I got your last sentence right :) but nobody proposed to
> discard any vote.
>

Do we have an overview somewhere (with patch reference) for the
alternatives proposed for beta1, which is then what's likely to end up in
2.3?
Note: I did just look at the wiki page but didn't see it there clearly.
/Christian

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Guillaume MM]

Le 31/07/2017 à 13:31, Jürgen Spitzmüller a écrit :

I meant it in this sense. If a vote only means "I did not have a
look at

the patch but I am fed-up so let us go ahead" then it is not taking
responsibilities.


A vote is a vote. If the given voting will be Rates differently, this 
will be have been the last voting I have participated on this list.




I am not sure I got your last sentence right :) but nobody proposed to
discard any vote.

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Jürgen Spitzmüller]

I meant it in this sense. If a vote only means "I did not have a look at
>
the patch but I am fed-up so let us go ahead" then it is not taking
responsibilities.


A vote is a vote. If the given voting will be Rates differently, this will
be have been the last voting I have participated on this list.

Jürgen

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Guillaume MM]

Le 29/07/2017 à 23:54, Scott Kostyshak a écrit :

On Thu, Jul 27, 2017 at 04:09:56PM +0200, Guillaume MM wrote:


* Having to use -shell-escape for running Pygments.


Yes, and if we go the way of the patch, I don't think any other
improvements (e.g. post-beta1) will be made to address this, until
perhaps 2.4.0 if the Github issues is addressed.


I take note.




I would also be more comfortable if somebody takes responsibility for
any patch that is to be committed, given that the author has said that
they do not endorse it.


Fair point. My goal with the vote was to collectively take
responsibility, since this is an important patch and involves security.
But I feel that most people are just tired of the debate and are hoping
too much to move forward that they have not taken a deep look.


I meant it in this sense. If a vote only means "I did not have a look at
the patch but I am fed-up so let us go ahead" then it is not taking
responsibilities.

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Scott Kostyshak]

On Sun, Jul 30, 2017 at 12:12:08AM +0200, Enrico Forestieri wrote:
> On Sat, Jul 29, 2017 at 05:54:33PM -0400, Scott Kostyshak wrote:
> > 
> > More important to me is that we interpret "take responsibility" in a
> > different way. Enrico, if we decide to go forward with something like
> > the latest patch, will you be around in the next couple of months and
> > willing to make potential updates and fixes?
> 
> Yes, of course.

OK good to know. 

> By not endorsing it I meant I was not pushing for its
> approval.

That's what I understood. My question was more because I wasn't sure if
e.g. you were going on vacation.

Scott


signature.asc
Description: PGP signature

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Enrico Forestieri]

On Sat, Jul 29, 2017 at 05:54:33PM -0400, Scott Kostyshak wrote:
> 
> More important to me is that we interpret "take responsibility" in a
> different way. Enrico, if we decide to go forward with something like
> the latest patch, will you be around in the next couple of months and
> willing to make potential updates and fixes?

Yes, of course. By not endorsing it I meant I was not pushing for its
approval.

-- 
Enrico

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Scott Kostyshak]

On Thu, Jul 27, 2017 at 04:09:56PM +0200, Guillaume MM wrote:

> * One has to decide which suggestions are needed for 2.3 and which ones
> can be implemented later.

Agreed. And the more immediate issue is which suggestions are needed
before beta1. Conditional on LyX devs supporting something like the
current patch, I'm fine with moving with the current state for beta.
However, I would like to see a stronger vote of support before I
conclude that LyX devs are indeed in favor of the approach (more on this
in a separate email).

> * Having to use -shell-escape for running Pygments.

Yes, and if we go the way of the patch, I don't think any other
improvements (e.g. post-beta1) will be made to address this, until
perhaps 2.4.0 if the Github issues is addressed.

> I would also be more comfortable if somebody takes responsibility for
> any patch that is to be committed, given that the author has said that
> they do not endorse it.

Fair point. My goal with the vote was to collectively take
responsibility, since this is an important patch and involves security.
But I feel that most people are just tired of the debate and are hoping
too much to move forward that they have not taken a deep look.

As for my personal opinion, I keep coming back to "I think this improves
security" (as I perceive the word, explained at [1]). I'm not *sure*
that it improves security, but all I can do is go with my best guess
(taking into account of course, that we are almost at beta stage). If I
am wrong and we end up shipping a LyX version that it turns out is less
secure, I will certainly blame myself.

More important to me is that we interpret "take responsibility" in a
different way. Enrico, if we decide to go forward with something like
the latest patch, will you be around in the next couple of months and
willing to make potential updates and fixes? If not, we will need to see
if anyone else can task responsibility for making potential fixes
post-beta pre-final.

Thanks to everyone for all of their time on this issue.

Scott


[1]
https://www.mail-archive.com/search?l=mid=20170721201254.hvh6jrbc3yrjxqr7%40steph


signature.asc
Description: PGP signature

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Guillaume MM]

Le 22/07/2017 à 00:47, Guenter Milde a écrit :

On 2017-07-19, Richard Heck wrote:

On 07/19/2017 01:48 AM, Christian Ridderström wrote:

On 18 July 2017 at 23:49, Jean-Marc Lasgouttes > wrote:
 Le 18/07/2017 à 23:42, Christian Ridderström a écrit :



 I think the default should be secure, and that the user should
 have to do something actively to go into a dangerous mode.




 Well, since you consider that turning off two options is not
 active enough, I am not sure what to propose :)




The problem I see with only unchecking two check boxes are e.g.:
- Users uncheck settings all the time, it doesn't seem very "scary"
- In the settings dialog, the real implications of unchecking these
options
   did not seem sufficiently clear to me.
   So calling it "Allow yourself to be shot in the foot by converters"
would help;-)
- The setting is persistent, and easily forgotten



This, I believe, was part of what was addressed by Enrico's patch. Or
the idea behind it.


Enrico's patch did not touch "needauth" but has some nice features for
"shell-escape":


+1


it addressed the "set and forget" issue by

a) adding a red icon to the status bar if a document has the "allow
shell-escape" flag.


+1

   
b) revoking the permission, if the document is moved/copied to another

location.


I like the principle, but I wonder whether this will cause annoyances
for files on removable filesystems.

   
I like the approach, especially b) seems a good compromise between user

comfort and security.

   
 From a user perspective, a common interface to "needauth" and "allow

shell escape" seems the best. "needauth" could actually take advantage of
Enrico's patch.


+1



Some ideas:

* Add "unsafe pdflatex" (== pdflatex --shell-escape) and "unsafe xelatex"
   as new converters requiring "needauth".

* Allow per-converter permission settings (instead of one generic: "I
   trust/don't trust all unsafe converters").


+1



* clicking the red icon should take you to the dialogue allowing to
   revoke the unsafe permission.


+1

   
* Give users the possibility to check scripts before allowing to run them

   with shell-escape or at least list all parts of the document that will be
   allowed to run in unsafe mode
   (e.g. all gnuplot scripts for "gnuplot allowed", all ERT, preamble,
   document classes and packages for latex with shell escape).



I like the idea, though for shell-escape this becomes more complicated.


* I also like the error dialog when -shell-escape has been configured
without needauth, for legacy configurations. (The specific wording can
be discussed later on.)

* Like Jean-Marc, I would prefer if the -shell-escape option was not
hardcoded, but integrated with needauth and the full command-line
visible in the converters dialog in some way. For instance a new token
$$unsafe together with a per-converter checkbox to allow its replacement
by whatever unsafe option.

* One has to decide which suggestions are needed for 2.3 and which ones
can be implemented later.


On the negative side, the patch does not address the original issues:
* The limitations of needauth in the context of adding new converters
such as gnuplot (the patch is only about -shell-escape),
* Having to use -shell-escape for running Pygments.


I would also be more comfortable if somebody takes responsibility for
any patch that is to be committed, given that the author has said that
they do not endorse it.


Guillaume

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Christian Ridderström]

On 24 July 2017 at 23:20, Tommaso Cucinotta  wrote:

> On 23/07/2017 20:55, Christian Ridderström wrote:
>
>> Regarding setting something in the preference file manually: The only
>> thing I mind is that it adds a global state to LyX, as opposed to
>> starting LyX with some parameters. The global state would likely
>> affect e.g. testing.
>>
>
> the good thing is that we already have the -userdir command-line option
> that allows testing from a predictable initial state, doesn't it :-) ?
>

Good point, thanks!
/Christian



> (AFAICR, extensively used in autotests for advanced F).
>
> T.
>

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Scott Kostyshak]

On Sat, Jul 22, 2017 at 01:09:09AM +0200, Jean-Marc Lasgouttes wrote:
> Le 21/07/2017 à 21:02, Scott Kostyshak a écrit :
> > > except if I disable needauth globally :(
> > 
> > What about editing the session file to add the paths of the .lyx files
> > that you want? If you're interested, I could write a Python/Bash script
> > that does it for you. I might end up using it also.
> 
> Well, I should not have to do that... Or it will be done automatically if I
> edit the file and try to typeset it, right?

Indeed. Rereading your question, I think I misunderstood it. I thought
you were looking for a way to get your scripts run without going through
the GUI for each one.

Scott

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Tommaso Cucinotta]

On 23/07/2017 20:55, Christian Ridderström wrote:

Regarding setting something in the preference file manually: The only
thing I mind is that it adds a global state to LyX, as opposed to
starting LyX with some parameters. The global state would likely
affect e.g. testing.


the good thing is that we already have the -userdir command-line optionthat 
allows testing from a predictable initial state, doesn't it :-) ?
(AFAICR, extensively used in autotests for advanced F).

T.

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Tommaso Cucinotta]

On 22/07/2017 00:47, Guenter Milde wrote:

Enrico's patch did not touch "needauth" but has some nice features for
"shell-escape": it addressed the "set and forget" issue by

a) adding a red icon to the status bar if a document has the "allow
shell-escape" flag.
   
b) revoking the permission, if the document is moved/copied to another

location.
   
I like the approach


+1, I like the idea of a visual feedback on the current security/trust status, 
e.g., it resembles the lock icon used in web browsers for https://.


From a user perspective, a common interface to "needauth" and "allow

shell escape" seems the best. "needauth" could actually take advantage of
Enrico's patch.


once I'll gain some spare time this summer, I'll try a merge :-)...


* Add "unsafe pdflatex" (== pdflatex --shell-escape) and "unsafe xelatex"
   as new converters requiring "needauth".


this sounds like something easy but already discussed and unliked/discarded.


* Allow per-converter permission settings (instead of one generic: "I
   trust/don't trust all unsafe converters").


the current system-wide setting is for all converters (disable any needauth,
allow them but warn me, allow them without constraints), whilst the memory
about trusted documents is per-document -- this makes sense because the
main source of untrust seems the document when coming from who knows where;
once the user acks that the doc is trusted, then we go without bugging the
user for each conversion. However, how would a per-converter settings
work, and how could I trust unconditionally, let's say, a R kneave/sweave
inset in a LyX doc coming from unknown sources, while at the same time
trust that an embedded gnuplot script or shell-escape command would not
delete my home folder ?


* Give users the possibility to check scripts before allowing to run them
   with shell-escape or at least list all parts of the document that will be
   allowed to run in unsafe mode
   (e.g. all gnuplot scripts for "gnuplot allowed", all ERT, preamble,
   document classes and packages for latex with shell escape).


that sounds like a feature enhancement deserving an entry on Trac ?

T.

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Christian Ridderström]

On 19 July 2017 at 12:00, Jean-Marc Lasgouttes  wrote:
> Le 19/07/2017 à 07:48, Christian Ridderström a écrit :
>>
>> If user does not want all these warnings, he could disable them by
>> launching LyX with some option like "--do-not-warn-me-about-unsafe-setting".
>> Instead of having a checkbox for "don't tell me these things again".
>
>
> It has the same issues as the checkbox when one considers batch command line
> processing. People add the switch to the command line and forget about it.

IMHO a user who does batch processing is pretty advanced...

Regarding setting something in the preference file manually: The only
thing I mind is that it adds a global state to LyX, as opposed to
starting LyX with some parameters. The global state would likely
affect e.g. testing.

/Christian

> The requested operation requires the use of a converter from sweave to
> pdflatex:
> Rscript --verbose --no-save --no-restore $$s/scripts/lyxsweave.R $$p$$i
> $$p$$o $$e $$r
> This external program can execute arbitrary commands on your system,
> including dangerous ones, if instructed to do so by a maliciously crafted
> .lyx document.
> Your current preference settings forbid its execution.
> (To change this setting, go to Preferences ▹ File Handling ▹ Converters and
> uncheck Security ▹ Forbid needauth converters.)

That's a good message in my opinion, especially as it points out the
"including dangerous ones".

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Jean-Marc Lasgouttes]

Le 21/07/2017 à 21:02, Scott Kostyshak a écrit :

except if I disable needauth globally :(


What about editing the session file to add the paths of the .lyx files
that you want? If you're interested, I could write a Python/Bash script
that does it for you. I might end up using it also.


Well, I should not have to do that... Or it will be done automatically 
if I edit the file and try to typeset it, right?


JMarc

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Guenter Milde]

On 2017-07-19, Richard Heck wrote:
> On 07/19/2017 01:48 AM, Christian Ridderström wrote:
>> On 18 July 2017 at 23:49, Jean-Marc Lasgouttes > > wrote:
>> Le 18/07/2017 à 23:42, Christian Ridderström a écrit :

>> I think the default should be secure, and that the user should
>> have to do something actively to go into a dangerous mode.


>> Well, since you consider that turning off two options is not
>> active enough, I am not sure what to propose :)


>> The problem I see with only unchecking two check boxes are e.g.:
>> - Users uncheck settings all the time, it doesn't seem very "scary"
>> - In the settings dialog, the real implications of unchecking these
>> options
>>   did not seem sufficiently clear to me.
>>   So calling it "Allow yourself to be shot in the foot by converters"
>> would help;-)
>> - The setting is persistent, and easily forgotten

> This, I believe, was part of what was addressed by Enrico's patch. Or
> the idea behind it.

Enrico's patch did not touch "needauth" but has some nice features for
"shell-escape": it addressed the "set and forget" issue by

a) adding a red icon to the status bar if a document has the "allow
   shell-escape" flag.
  
b) revoking the permission, if the document is moved/copied to another
   location.
  
I like the approach, especially b) seems a good compromise between user
comfort and security.

  
>From a user perspective, a common interface to "needauth" and "allow
shell escape" seems the best. "needauth" could actually take advantage of
Enrico's patch.

Some ideas:

* Add "unsafe pdflatex" (== pdflatex --shell-escape) and "unsafe xelatex"
  as new converters requiring "needauth".

* Allow per-converter permission settings (instead of one generic: "I
  trust/don't trust all unsafe converters").

* clicking the red icon should take you to the dialogue allowing to
  revoke the unsafe permission.
  
* Give users the possibility to check scripts before allowing to run them
  with shell-escape or at least list all parts of the document that will be
  allowed to run in unsafe mode
  (e.g. all gnuplot scripts for "gnuplot allowed", all ERT, preamble,
  document classes and packages for latex with shell escape).


Günter

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Scott Kostyshak]

On Wed, Jul 19, 2017 at 12:00:52PM +0200, Jean-Marc Lasgouttes wrote:

> Which make me think that I did not try to check whether my nice scripts to
> process Sweave lyx file still have a chance to work. Oops! they won't

This is good. It shows the needauth implementation works.

> except if I disable needauth globally :(

What about editing the session file to add the paths of the .lyx files
that you want? If you're interested, I could write a Python/Bash script
that does it for you. I might end up using it also.

Scott

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Jean-Marc Lasgouttes]

Le 19/07/2017 à 07:48, Christian Ridderström a écrit :
If user does not want all these warnings, he could disable them by 
launching LyX with some option like "--do-not-warn-me-about-unsafe-setting".

Instead of having a checkbox for "don't tell me these things again".


It has the same issues as the checkbox when one considers batch command 
line processing. People add the switch to the command line and forget 
about it.


Which make me think that I did not try to check whether my nice scripts 
to process Sweave lyx file still have a chance to work. Oops! they 
won't, except if I disable needauth globally :(


fantomas: LC_ALL=C ~/src/lyx/devbuild/src/lyx -e pdf2 cours-acp.lyx
Error: An external converter is disabled for security reasons

The requested operation requires the use of a converter from sweave to 
pdflatex:
Rscript --verbose --no-save --no-restore $$s/scripts/lyxsweave.R $$p$$i 
$$p$$o $$e $$r
This external program can execute arbitrary commands on your system, 
including dangerous ones, if instructed to do so by a maliciously 
crafted .lyx document.

Your current preference settings forbid its execution.
(To change this setting, go to Preferences ▹ File Handling ▹ Converters 
and uncheck Security ▹ Forbid needauth converters.)


JMarc

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Pavel Sanda]

Christian Ridderström wrote:
> - Users uncheck settings all the time, it doesn't seem very "scary"
> 
> Why does disabling something like needauth have to be done from within LyX?

... as I read through the list I see we come to similar conclusions ...
I don't have strong opinion about these.

Pavel

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Richard Heck]

On 07/19/2017 01:48 AM, Christian Ridderström wrote:
>
> On 18 July 2017 at 23:49, Jean-Marc Lasgouttes  > wrote:
>
> Le 18/07/2017 à 23:42, Christian Ridderström a écrit :
>
> I think the default should be secure, and that the user should
> have to do something actively to go into a dangerous mode.
>
>
> Well, since you consider that turning off two options is not
> active enough, I am not sure what to propose :)
>
>
> The problem I see with only unchecking two check boxes are e.g.:
> - Users uncheck settings all the time, it doesn't seem very "scary"
> - In the settings dialog, the real implications of unchecking these
> options
>   did not seem sufficiently clear to me.
>   So calling it "Allow yourself to be shot in the foot by converters"
> would help;-)
> - The setting is persistent, and easily forgotten

This, I believe, was part of what was addressed by Enrico's patch. Or
the idea behind it.

It would at least be possible to have a 'hidden' setting here: One you
could activate only by
editing the preferences file. That doesn't seem unreasonable to me. This
is definitely a feature
for power users. Of course, that would make it even more difficult to undo.

> If it has to be done from within LyX, then perhaps do some of the
> things below to make being in unsafe mode more difficult to forget:
> - When unchecking the boxes, display a dialog informing them that
> they're going into dangerous territory.
> - Show the warning each time LyX is started, forcing the user to
> acknowledge it.
>   And make it so that user with a single click can reenable needauth.
> - Possibly show the dialog each time before building a document

One or the other here is enough, I'd think. But this is otherwise
similar to a suggestion
I made elsewhere.

> - Enable a strong/annoying visual indication/reminder that you're
> unsafe mode

Also part of Enrico's patch idea, I believe.

The overall idea behind that patch was to make this setting per-document
and easy to
change, with a strong visual indication. Making it non-persistent, or at
least something
you have to acknowledge each session, would add security. Here again, if
that seems too
annoying, a power-user-only non-gui setting could be considered. Then
it's possible for
people to sidestep the security, but only by really getting their hands
dirty.

Richard