Re: Silent/automatic execution of converter and needauth, concrete questions to clarify my understanding

[Guillaume MM]

Le 17/07/2017 à 23:53, Christian Ridderström a écrit :

Hi,

I've gotten lots of information from Enrico and Guillaume related to the 
security "gap", but I'd like to boil it down to simpler questions to 
make the situation clear to me.


Assume that I've gotten a LyX document by e-mail. It was not created by 
me, but let's say that the sender of the e-mail appears to be from a 
colleague whom I trust, asking me to do him a favour and generate a PDF 
because his computer is acting up. It's urgent of course...


A) In LyX 2.2.x, if I open the document, no "converters" are executed. 
But when I attempt to generate the PDF, the document could via e.g. 'R' 
execute arbitrary code on my computer, as if it were my user account. 
And this would happen silently, with no warning etc.

Correct?

But what would happen if I used LyX 2.3.0alphaX and tried to build the 
document?

B) Would LyX still allow the document to run arbitrary code on my computer?


Depends on your needauth settings.
* Never (default for a new install): no, and an error message tells you
to change the needauth settings before you can proceed.
* Enable and ask: first you get a message asking to authorise the
converter (every time or only the first time depending on whether you
chose "allow" or "always allow for the document").
* Disabled: like in 2.2.

Note that currently all this and the below appears to hold as well for
gnuplot previews, so one does not need to compile to PDF, just to open a
file (this was not the case in 2.2).



C) Would the execution still happen "silently"?


In two cases:
* Enable and ask: if you previously clicked "always for the document".
* Disabled: it always happens silently.



D) Can the above happen with a document completely created by someone else?


In one case:
* Disabled

(Another one is if the path is ~/Download/new1.lyx and you happen to
have given permanent permissions for a file with the same path three
years earlier, deleted and forgotten about since...)


Guillaume

Silent/automatic execution of converter and needauth, concrete questions to clarify my understanding

[Christian Ridderström]

Hi,

I've gotten lots of information from Enrico and Guillaume related to the
security "gap", but I'd like to boil it down to simpler questions to make
the situation clear to me.

Assume that I've gotten a LyX document by e-mail. It was not created by me,
but let's say that the sender of the e-mail appears to be from a colleague
whom I trust, asking me to do him a favour and generate a PDF because his
computer is acting up. It's urgent of course...

A) In LyX 2.2.x, if I open the document, no "converters" are executed. But
when I attempt to generate the PDF, the document could via e.g. 'R' execute
arbitrary code on my computer, as if it were my user account. And this
would happen silently, with no warning etc.
Correct?

But what would happen if I used LyX 2.3.0alphaX and tried to build the
document?
B) Would LyX still allow the document to run arbitrary code on my computer?

C) Would the execution still happen "silently"?

D) Can the above happen with a document completely created by someone else?

Note that for all questions above I assume that the person who sent me the
document has attempted to configure it to allow the above, i.e. he's set
any flags etc he could within the document.

Regards,
Christian

Re: [LyX/master] Add some notes on forward/reverse search with evince.

[Jürgen Spitzmüller]

Am Montag, den 17.07.2017, 16:03 +0200 schrieb Pavel Sanda:
> FYI you have my full support to push those scripts into master and
> even ship
> it with tarball unless you fear copyright issues (3dparty dir could
> be used
> in such case). If evince worked with couple clicks only that would be
> useful.

No copyright issues. The scripts are GPL-licensed. And they are written
in Python ;-)

I'd like to test it a bit, though. I have stumbled upon it, and tweaked
it for the use with LyX, just yesterday.

Jürgen

> 
> Pavel

signature.asc
Description: This is a digitally signed message part

Re: Options for resolving the minted + shell-escape issue

[Richard Heck]

On 07/17/2017 06:30 AM, Enrico Forestieri wrote:
> On Mon, Jul 17, 2017 at 07:03:09AM +0200, Guillaume MM wrote:
>
>> Le 17/07/2017 à 00:49, Christian Ridderström a écrit :
>>> On 5 July 2017 at 06:59, Scott Kostyshak >> > wrote:
>>>
>>> Dear all,
>>>
>>> This is an important topic since it involves security. I'd appreciate it
>>> if you spent some time on understanding the issue.
>>>
>>> I see three options for what to do about the minted + shell-escape
>>> issue:
>>>
>>> 1. Revert the recently added minted support.
>>>
>>> 2. Keep the current state of master.
>>>
>>> 3. Apply the patch at [1] (also attached for convenience).
>>>
>>>
>>> Hi Scott,
>>>
>>> I think you're doing an excellent job as release manager and also when
>>> summarising the issue. I therefore have to apologise that even though I
>>> have read various postings/threads on this topic, I'm still confused.
>>> Perhaps I can expand on that:
>>>
>>> - What do we lose by doing 1) above?
>>> - Is 1) necessary for users in order to use minted, or is it a
>>> convenience feature?
>>> - How many users do we think need / want to use minted, i.e. how many
>>> users are affected?
>>>
>>> Opinions seem to vary regarding what's secure and I cannot say anything
>>> to that. My general inclination is to prefer that the user to go through
>>> a bunch of manual steps in order to use dangerous features. If they
>>> remain in place, so be it. But perhaps we could warn the user when he's
>>> configured converters to use shell-escape, or if LyX was started with
>>> some global option enabling shell-escape. Perhaps by making the LyX
>>> window "look dangerous" like private browser windows, or more
>>> realistically flash a warning before each build.
>>>
>>> As an aside, I've used org-mode documents in Emacs to invoke MATLAB on
>>> snippets from within the document and it's very nice, except the really
>>> annoying part where you for each snippet have to approve that it's run.
>>> Each time. A big bug in org-mode is that it's not properly showing the
>>> snippet that's about to be executed before you approve it to be run...
>>> Perhaps I'm a masochist...
>>>
>> Hi Christian,
>>
>>
>> The current needauth interface is not so much better in this respect. It
>> only shows the unsafe command line about to be run. Various degrees of
>> compromise between security and usability are proposed: always allow
>> needauth / always for this document / allow once / never allow. The
>> default is never allow, and after going to the preferences, the user is
>> suggested to always be asked.
>>
>> Various UI improvements that you and others have suggested are subsumed
>> by some articles I have sent to the list. I am convinced that if one
>> thinks enough about it, one can get to a better secure UI. This is
>> to me the best hope to get to a more reasonable balance between security
>> and usability in the future, because the AppArmor stuff will be harder
>> to put into place.
>>
>>> Enrico argued that there are other (equally) dangerous converters
>>> already in LyX. Then that's something to address. Does it have to be for
>>> this release?
>> In 2.2 and before, a document could contain R code which could cause
>> arbitrary code execution with user privilege during compilation. The
>> user was not warned that a document could contain such code. (R needed
>> to be installed, though.)
>>
>> I think I did not really measure the extent of the problem before
>> Tommaso's work to patch things up. This is because LyX has always been
>> quite strict with security, refusing for instance to directly open links
>> from documents (although a secure implementation of this would be quite
>> simple: one would just need a confirmation dialog with the URL clearly
>> displayed).
>>
>> Starting with 2.3, Tommaso's needauth feature asks for a confirmation
>> before running converters labelled as needauth, and R code is now
>> labelled as such. Users not expecting to see R code are now safe, but
>> ones who expect to run R code are still unsafe. According to Tommaso,
>> this was meant as a quick measure before a more elaborate containment is
>> in place (e.g. using AppArmor in Linux–this seems experimental for now
>> and it will be difficult to integrate it since a proper solution
>> requires one for each platform).
>>
>> Support for gnuplot has been rejected for a long time because of
>> security, but now taking advantage of needauth it has been introduced.
>> There are specific issues with gnuplot being also run for previews.
>> There is currently a separate discussion about gnuplot.
>>
>> Then there is the new minted implementation. The issue with minted is
>> that it cannot compile without -shell-escape. There seemed to be an
>> agreement that this would encourage users to configure -shell-escape in
>> whatever ways, and that this seemed bad enough to fast-track (between
>> feature-freeze and beta release) the inclusion 

Re: [LyX/master] Add some notes on forward/reverse search with evince.

[Pavel Sanda]

Juergen Spitzmueller wrote:
> commit 272cb6b4648c879d7a58b87071e14b3904e3b6f4
> Author: Juergen Spitzmueller 
> Date:   Mon Jul 17 10:26:43 2017 +0200
> 
> Add some notes on forward/reverse search with evince.
> 
> Evince is a special case, since it provides f/r search not via command
> line switches and pref settings, but via DBUS. On Linux at least, this
> can be used by LyX via some external scripts. The possibility is now
> mentioned here, the details (and the scripts) are provided on the wiki.

FYI you have my full support to push those scripts into master and even ship
it with tarball unless you fear copyright issues (3dparty dir could be used
in such case). If evince worked with couple clicks only that would be useful.

Pavel

Re: [LyX/master] Update fr.po for beta

[Pavel Sanda]

Jean-Marc Lasgouttes wrote:
> Le 16/07/2017 ? 20:53, Scott Kostyshak a écrit :
>> I don't have svn commit privileges. Also, I don't actually know how to
>> use svn.
>
> Do you want svn privileges? Do you want to learn some y2k technology that 
> is not needed anymore? ;)
>
> Seriously, it is less mind-damaging than git, but for this cheap price, you 
> obtain less, of course.

Might be worth it, there's no way Scott can update web news for beta/RCs. P

Re: Options for resolving the minted + shell-escape issue

[Enrico Forestieri]

On Mon, Jul 17, 2017 at 02:10:44AM +0200, Christian Ridderström wrote:
> 
> Regarding, Minted, which is an alternative to insert pretty program
> listings in your document.
> 
> At the moment it takes manual (typing) work to cause security issues in
> connection with minted.
> The "at the moment", likely refer to e.g. LyX 2.2.2 as [1] gives a nice
> illustration with screenshots on how to add the option '-shell-escape' to
> the converter 'pdflatex'.

No, "at the moment" also refers to 2.3.0 where the only difference with
respect to [1] would be that you don't have to use ERT and manually add
the minted package.

> The downside with adding this option is that now
> other LaTeX code in the document has the possibility of doing "bad" stuff
> to my system.

Yes, and this is in no way different than using a dangerous converter,
which might be doing "bad" stuff to your system.

> Further, my LyX is now configured with this -shell-escpae that will then be
> active for all other LyX documents that I build. Oops.

Yes, and this was the motivation that lead to the proposal of several
alternatives. I think the most effective was the one that allowed to
add -shell-escape to selected documents with the possibility of revoking
this permission. There was an icon indicating this fact and clicking it
one could revoke those rights.

This was a reasonable alternative, IMO, not specifically tied to minted,
but it was not agreed upon by everyone.

> Note: I'd probably deal with the security issue here by using two
> separately configured LyX instances that use different '-userdir':s.
> It would be nice with a strong visual warning that I'm using the "unsafe"
> LyX though, but i guess I could manually configure a different paper colour
> in LyX.

I think that it would suffice configuring two different converters rather
than using different user dirs.

> If I've understood the proposed patches correctly, they involve making it
> easier for the user to enable -shell-escape, and also easier to disable
> shell-escape.  I'm torn here. Some of the proposed UI-approaches weren't
> bad, but I'd probably still worry that we're making it to easy for the user
> to do dangerous things.

As you have discovered, people need to set -shell-escape for their purposes,
and it may happen that this setting remains there completely unnoticed in the
future. I think that making it easier enabling and disabling -shell-escape
simply increases security. In one of the patches I posted you could see an
icon that was indicating that that particular document would run with
increased privileges, and you were able to revoke it.

> For Minted I'd then prefer to keep the old behaviour for now and add better
> integration when/if minted doesn't need shell-escape.

So, there would be nothing to do, as the only difference with the old
behaviour would be not using ERT.

> As for the other dangerous features, presumably related to something called
> "needauth", I don't know anything...
> I googled and found this [2]:
> 
> 
> The converters definition syntax (LYX_HOME/lyxrc*) now supports a new
> option, 'needauth', to prevent completely automated execution of the
> converter, unless LyX acquired explicit consent by the user. This is a new
> security feature, useful for converters that are capable of executing
> arbitrary code, such as R scripts (used with sweave/knitr), embedded within
> LyX documents. The user needs to explicitly grant per-document permission
> on the first need for using the converter on each document, unless he/she
> checks the "Don't ask again for this document" checkbox in the permission
> dialog. The new behavior can be fine-tuned from two new options in the
> preferences dialog (see their description below). These also allow for
> disabling 'needauth' converters altogether, if desired (default behavior).
> 
> 
> I don't understand if the 'needauth' is new in LyX 2.3.0 or already existed.

This is new to 2.3.0 and is instead deemed a security improvement. Yes,
security is improved by allowing running dangerous converters.

> However, and here I'm probably offending people and stepping on mines in a
> single paragraph. This seems bad to me.
> The text indicates to me that it's possible for a document to store some
> kind of setting that allows a converter (here external program) to be run
> in an automated manner without my manual intervention or consent.
> Supposedly I first had to check "Don't ask again for this document" but
> consider the following example:
> I create a document with some embedded code to be run by converters. It's
> my document, I trust it. Then I e-mail it to a colleague or perhaps my
> customer for review. Time goes by and eventually I get the document back
> from review, but the review took longer than expected and my next deadline
> was yesterday, so I'm in a hurry and build "my" document.   Oops, some of
> the embedded code is now malicious, but the document still contains my
> setting that lets the converters 

Re: Can shell-escape take advantage of needauth framework?

[Enrico Forestieri]

On Mon, Jul 17, 2017 at 07:14:07AM +0200, Guillaume MM wrote:
> 
> But besides that I agree with your suggestions. Thanks again for
> spending your time looking into this issue with so much care.

Yes, it seems that Scott can be easily convinced by your constructed
arguments.

"There is a bomb under our table, but we cannot remove it because it has
been there since 2011."

-- 
Enrico

Re: Options for resolving the minted + shell-escape issue

[Enrico Forestieri]

On Mon, Jul 17, 2017 at 07:03:09AM +0200, Guillaume MM wrote:

> Le 17/07/2017 à 00:49, Christian Ridderström a écrit :
> > On 5 July 2017 at 06:59, Scott Kostyshak  > > wrote:
> > 
> > Dear all,
> > 
> > This is an important topic since it involves security. I'd appreciate it
> > if you spent some time on understanding the issue.
> > 
> > I see three options for what to do about the minted + shell-escape
> > issue:
> > 
> > 1. Revert the recently added minted support.
> > 
> > 2. Keep the current state of master.
> > 
> > 3. Apply the patch at [1] (also attached for convenience).
> > 
> > 
> > Hi Scott,
> > 
> > I think you're doing an excellent job as release manager and also when
> > summarising the issue. I therefore have to apologise that even though I
> > have read various postings/threads on this topic, I'm still confused.
> > Perhaps I can expand on that:
> > 
> > - What do we lose by doing 1) above?
> > - Is 1) necessary for users in order to use minted, or is it a
> > convenience feature?
> > - How many users do we think need / want to use minted, i.e. how many
> > users are affected?
> > 
> > Opinions seem to vary regarding what's secure and I cannot say anything
> > to that. My general inclination is to prefer that the user to go through
> > a bunch of manual steps in order to use dangerous features. If they
> > remain in place, so be it. But perhaps we could warn the user when he's
> > configured converters to use shell-escape, or if LyX was started with
> > some global option enabling shell-escape. Perhaps by making the LyX
> > window "look dangerous" like private browser windows, or more
> > realistically flash a warning before each build.
> > 
> > As an aside, I've used org-mode documents in Emacs to invoke MATLAB on
> > snippets from within the document and it's very nice, except the really
> > annoying part where you for each snippet have to approve that it's run.
> > Each time. A big bug in org-mode is that it's not properly showing the
> > snippet that's about to be executed before you approve it to be run...
> > Perhaps I'm a masochist...
> > 
> 
> Hi Christian,
> 
> 
> The current needauth interface is not so much better in this respect. It
> only shows the unsafe command line about to be run. Various degrees of
> compromise between security and usability are proposed: always allow
> needauth / always for this document / allow once / never allow. The
> default is never allow, and after going to the preferences, the user is
> suggested to always be asked.
> 
> Various UI improvements that you and others have suggested are subsumed
> by some articles I have sent to the list. I am convinced that if one
> thinks enough about it, one can get to a better secure UI. This is
> to me the best hope to get to a more reasonable balance between security
> and usability in the future, because the AppArmor stuff will be harder
> to put into place.
> 
> > Enrico argued that there are other (equally) dangerous converters
> > already in LyX. Then that's something to address. Does it have to be for
> > this release?
> 
> In 2.2 and before, a document could contain R code which could cause
> arbitrary code execution with user privilege during compilation. The
> user was not warned that a document could contain such code. (R needed
> to be installed, though.)
> 
> I think I did not really measure the extent of the problem before
> Tommaso's work to patch things up. This is because LyX has always been
> quite strict with security, refusing for instance to directly open links
> from documents (although a secure implementation of this would be quite
> simple: one would just need a confirmation dialog with the URL clearly
> displayed).
> 
> Starting with 2.3, Tommaso's needauth feature asks for a confirmation
> before running converters labelled as needauth, and R code is now
> labelled as such. Users not expecting to see R code are now safe, but
> ones who expect to run R code are still unsafe. According to Tommaso,
> this was meant as a quick measure before a more elaborate containment is
> in place (e.g. using AppArmor in Linux–this seems experimental for now
> and it will be difficult to integrate it since a proper solution
> requires one for each platform).
> 
> Support for gnuplot has been rejected for a long time because of
> security, but now taking advantage of needauth it has been introduced.
> There are specific issues with gnuplot being also run for previews.
> There is currently a separate discussion about gnuplot.
> 
> Then there is the new minted implementation. The issue with minted is
> that it cannot compile without -shell-escape. There seemed to be an
> agreement that this would encourage users to configure -shell-escape in
> whatever ways, and that this seemed bad enough to fast-track (between
> feature-freeze and beta release) the inclusion of a needauth-like
> interface for -shell-escape. Now it seems that there 

Re: [LyX/master] Add some notes on forward/reverse search with evince.

[Jean-Marc Lasgouttes]

Le 17/07/2017 à 10:29, Juergen Spitzmueller a écrit :

commit 272cb6b4648c879d7a58b87071e14b3904e3b6f4
Author: Juergen Spitzmueller 
Date:   Mon Jul 17 10:26:43 2017 +0200

 Add some notes on forward/reverse search with evince.
 
 Evince is a special case, since it provides f/r search not via command

 line switches and pref settings, but via DBUS. On Linux at least, this
 can be used by LyX via some external scripts. The possibility is now
 mentioned here, the details (and the scripts) are provided on the wiki.



It would be great to package this via configure.py...

JMarc