Re: Options for resolving the minted + shell-escape issue

[Jürgen Spitzmüller]

Am Mittwoch, den 19.07.2017, 16:37 +0200 schrieb Enrico Forestieri:
> The attached patch takes into account all of these ideas. As a
> disclaimer,
> note that I am providing it only because I am now familiar with this
> part
> of the code and can quickly come up with a patch. But I am not
> endorsing it.

I propose to apply this patch and return to productivity.

Jürgen

signature.asc
Description: This is a digitally signed message part

Re: Options for resolving the minted + shell-escape issue

[Richard Heck]

On 07/19/2017 02:04 PM, Enrico Forestieri wrote:
> On Wed, Jul 19, 2017 at 10:58:44AM -0400, Richard Heck wrote:
>> Thanks for this, Enrico. Let me just ask one question about it: Is the
>> mechanism here per-document
>> or per-document and also per-converter?
> This only addresses particular converters, i.e., the latex backends.
>
>> That is, suppose there was a
>> document that contained R
>> code and minted code.
> Please, note that this is not specifically aimed at minted. It only
> allows the latex backend to execute external commands. In case there
> is R code, it doesn't need this permission and would execute whether
> or not one allows shell-escape.
>
>> Would one be warned about both or would one only
>> be asked once?
> If there is a needauth converter involved in a latex run, one would
> be warned about it independently of whether shell-escape is allowed
> or not. So, one would be warned first to allow at all the latex run
> in case shell-escape is allowed, and then would be warned that there
> is also a dangerous converter to be run. Both warnings can be
> independently suppressed. So, even if you allowed the dangerous
> converter, you will be warned about shell-escape, and vice versa.
>
>> If the
>> the latter, how hard would it be to change that?
> I hope I answered your question.

Thanks for the clarifications. I've found all of this somewhat
confusing, since I don't use
any of these mechanisms myself.

Richard

Re: [LyX/master] Fixup the fixup d0acc3e57044: use editable()/isActive()

[Scott Kostyshak]

On Tue, Jun 27, 2017 at 04:47:10PM +0200, Jean-Marc Lasgouttes wrote:
> commit 13c3c1485b68980c51658cef8fadf804982d75ee
> Author: Jean-Marc Lasgouttes 
> Date:   Fri Jun 23 20:32:32 2017 +0200
> 
> Fixup the fixup d0acc3e57044: use editable()/isActive()
> 
> While 522516d9 was too strong and broke mathed, d0acc3e57044 is too
> lenient and can accept insets (mathed/CommandInset, InsetInfo) that
> have a positive nargs() but are not editable (because they encapsulate
> something).
> 
> Therefore the best solution for now is to use editable() in text and
> isActive() in mathed, until those two things are merged.
> 
> Part of #10667.

Git bisect suggests this broke the following:

Open the attached .lyx file. Make sure that the cursor is at the
beginning and that the footnote inset is not visible on the screen. Then
do a find for the word "find".

Expected: that Lyx opens the inset and highlights the word "find".
Actual: LyX places the cursor before the inset and leaves the inset
closed.

Note that if the footnote inset is on the screen, then for some reason
it works as expected.

Scott


p.s. I am currently traveling so have not yet looked at the other
discussions.
#LyX 2.2 created this file. For more info see http://www.lyx.org/
\lyxformat 508
\begin_document
\begin_header
\save_transient_properties true
\origin unavailable
\textclass article
\use_default_options true
\maintain_unincluded_children false
\language english
\language_package default
\inputencoding auto
\fontencoding global
\font_roman "default" "default"
\font_sans "default" "default"
\font_typewriter "default" "default"
\font_math "auto" "auto"
\font_default_family default
\use_non_tex_fonts false
\font_sc false
\font_osf false
\font_sf_scale 100 100
\font_tt_scale 100 100
\graphics default
\default_output_format default
\output_sync 1
\bibtex_command default
\index_command default
\paperfontsize default
\spacing single
\use_hyperref false
\papersize default
\use_geometry false
\use_package amsmath 1
\use_package amssymb 1
\use_package cancel 1
\use_package esint 1
\use_package mathdots 1
\use_package mathtools 1
\use_package mhchem 1
\use_package stackrel 1
\use_package stmaryrd 1
\use_package undertilde 1
\cite_engine basic
\cite_engine_type default
\biblio_style plain
\use_bibtopic false
\use_indices false
\paperorientation portrait
\suppress_date false
\justification true
\use_refstyle 1
\index Index
\shortcut idx
\color #008000
\end_index
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\paragraph_indentation default
\quotes_language english
\papercolumns 1
\papersides 1
\paperpagestyle default
\tracking_changes false
\output_changes false
\html_math_output 0
\html_css_as_file 0
\html_be_strict false
\end_header

\begin_body

\begin_layout Standard
Hello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to reproduce the regressionHello there.
 This text before the footnote must push the footnote off the screen to
 be able to 

Jenkins build is back to normal : Build branch "master" » ubuntu-xenial-qt4-autotools-extended #317

[ci-lyx]

https://ci.inria.fr/lyx/job/build-master-head/job/ubuntu-xenial-qt4-autotools-extended/317/

Re: Errors with vref on de/Additional.lyx with lualatex

[Kornel Benko]

Am Mittwoch, 19. Juli 2017 um 19:48:49, schrieb Guenter Milde 

> On 2017-07-19, Kornel Benko wrote:
> 
> > [-- Type: text/plain, Encoding: 7bit --]
> 
> > Am Mittwoch, 19. Juli 2017 um 15:00:16, schrieb Kornel Benko 
> > 
> >> So, maybe better we could omit \textcompwordmark in mono fonts.
> 
> > This patch works for me. It uses vphantom{}, but only between '<<' and '>>'.
> 
> See my other post for alternatives solving the "missing character" error
> with Unicode fonts for \textcompwordmark.

I have read it. I understand the usage and I don’t insist on removing 
textcompwordmark any longer.

> If you insist on changing away from textcompwordmark, this should be limited
> to TU, i.e. done in
> bool Paragraph::Private::latexSpecialTU(char_type const c, otexstream & os,
> 
> 
> Günter

I don't insist. I would insist on removing textcompwordmark in mono fonts, but 
did not see an easy way.
Instead, if you look into the lines around src/Paragraph.cpp:1378 you see the 
only reason to use
textcompwordmark there is to break the ligatures for '>>' and '<<'. But that 
can be done by vphantom{} too.

Kornel


signature.asc
Description: This is a digitally signed message part.

Re: Errors with vref on de/Additional.lyx with lualatex

[Guenter Milde]

On 2017-07-19, Kornel Benko wrote:

> [-- Type: text/plain, Encoding: 7bit --]

> Am Mittwoch, 19. Juli 2017 um 15:00:16, schrieb Kornel Benko 
>> So, maybe better we could omit \textcompwordmark in mono fonts.

> This patch works for me. It uses vphantom{}, but only between '<<' and '>>'.

See my other post for alternatives solving the "missing character" error
with Unicode fonts for \textcompwordmark.

If you insist on changing away from textcompwordmark, this should be limited
to TU, i.e. done in
bool Paragraph::Private::latexSpecialTU(char_type const c, otexstream & os,


Günter

Re: Any descriptions of the security aspects (related to needauth and shell-escape)?

[Guenter Milde]

On 2017-07-19, Christian Ridderström wrote:

...
> ... I would like to ask (not being
> optimistic), if there's some design description anywhere?

> I wonder because IMHO security requires a system wide approach and that
> it's very easy to screw up if only looking at isolated pieces. Further, it
> requires continuity so you know what you initially intended to achieve and
> what you consider good enough. Otherwise you might later introduce a new
> feature that inadvertently opens up a security whole. Without a system
> design, it's also easy to get caught in discussions trying to bandaid a
> small hole while missing entire walls missing.

> I think this kind of information would be good to gather and store in some
> kind of design document, which could just be a text file in the repo.  Then
> we could add knowledge to this document, and let if include the rationale
> behind our choices, as well as letting developers review the system design.

I support the suggestion to create such a document and suppose to make it
a section in "Development.lyx":

+ bundled with other project policies and developer documentation
+ write access for all developers
+ we can use LyX's version control for to-be-reviewed parts and diverging
  opinions/comments
  
Günter  

Re: Options for resolving the minted + shell-escape issue

[Enrico Forestieri]

On Wed, Jul 19, 2017 at 10:58:44AM -0400, Richard Heck wrote:
> 
> Thanks for this, Enrico. Let me just ask one question about it: Is the
> mechanism here per-document
> or per-document and also per-converter?

This only addresses particular converters, i.e., the latex backends.

> That is, suppose there was a
> document that contained R
> code and minted code.

Please, note that this is not specifically aimed at minted. It only
allows the latex backend to execute external commands. In case there
is R code, it doesn't need this permission and would execute whether
or not one allows shell-escape.

> Would one be warned about both or would one only
> be asked once?

If there is a needauth converter involved in a latex run, one would
be warned about it independently of whether shell-escape is allowed
or not. So, one would be warned first to allow at all the latex run
in case shell-escape is allowed, and then would be warned that there
is also a dangerous converter to be run. Both warnings can be
independently suppressed. So, even if you allowed the dangerous
converter, you will be warned about shell-escape, and vice versa.

> If the
> the latter, how hard would it be to change that?

I hope I answered your question.

> Oh, maybe another thought: Could clicking the 'warning' icon easily be
> used to disable execution?
> Just a UI thought there.

I don't know how much easily. But this is simply because of my ignorance
about Qt. It could be very easy to do that.

-- 
Enrico

Re: Any descriptions of the security aspects (related to needauth and shell-escape)?

[Richard Heck]

On 07/19/2017 02:22 AM, Christian Ridderström wrote:
> Hi,
>
> When having tried to contribute to the discussion on needauth and
> shell-escape I've felt that it's quite difficult to get a good picture
> of things like:
> - Goals of design, what are we trying to achieve
> - Principle of design and system
> - Assumed threat models, and perhaps list threat scenarios we _don't_
> try to protect against
>
> The e-mail threads are ... long, sometimes confusing and I suspect
> contains at least a few misunderstandings.  So I would like to ask
> (not being optimistic), if there's some design description anywhere?

No, as usual, there is not. The needauth mechanism was developed by
Tommaso in response
to security worries about certain sorts of converters, e.g., the ones
for R and related worries
about the use of gnuplot. (It may have been the latter that got him
interested.) Once that was on
board, Enrico decided to employ at least a somewhat similar mechanism to
support minted.sty,
and for whatever reason, that set off alarm bells which, in retrospect,
should have gone off
earlier. So we find ourselves in the middle of things.

Richard

Re: Options for resolving the minted + shell-escape issue

[Richard Heck]

On 07/19/2017 10:37 AM, Enrico Forestieri wrote:
> On Tue, Jul 18, 2017 at 07:26:23PM -0400, Richard Heck wrote:
>> On 07/18/2017 09:56 AM, Jürgen Spitzmüller wrote:
>>> Am Dienstag, den 18.07.2017, 15:39 +0200 schrieb Jean-Marc Lasgouttes:
 Whi, not, maybe along with the names of the converters (features) 
 Sweave/gnuplot/minted present in current document and accepted by the
 user.
>>> I would add a verbose tooltip when hovering the icon, something like
>>>
>>> '''
>>> NOTE: Shell escape access granted.
>>>
>>> For this document, access to the -shell-escape feature has been granted
>>> for the following converters: ...
>>>
>>> Note that this is a potential security risk. Use only if you trust the
>>> source of this document. Please refer to sec. xx of the User Guide for
>>> details.
>>>
>>> To withdraw shell escape access, press this icon.
>>> '''
>> This seems a reasonable solution to me. It is not perfect, but nothing is.
>>
>> As I see it, the issue is that there are actually a wide variety of
>> reasons that users might want to
>> enable -shell-escape for various converters. As LyX currently functions,
>> the only way to do this
>> is to add this to the converter itself. This is dangerous from our point
>> of view NOT so much (or
>> only) because it is intrinsically dangerous, but rather because it it is
>> the kind of thing that is too
>> easy to "do and forget". Or, to put it differently: It is a serious
>> hassle to enable -shell-escape as
>> things are, and that invites people to do it and leave it. And that
>> really is a security risk.
> The attached patch takes into account all of these ideas. As a disclaimer,
> note that I am providing it only because I am now familiar with this part
> of the code and can quickly come up with a patch. But I am not endorsing it.
>
> The user can choose to allow execution of external programs for a given
> document through the document settings. However, this is a property that
> only holds on the computer used to edit the document. There is no way
> to send out a document with the shell-escape thing activated.
>
> Once activated, the user is prompted for confirmation each time a
> latex backend is used, unless he decides to always allow execution for
> a particular document. A document marked as requiring shell escape
> privileges is characterized by a red icon on the status bar.
>
> The shell-escape privilege can be revoked through the document settings.
> Given the peculiarity that this cannot be a mere document property but
> rather a property tied to both document and computer, the check box
> works  differently from all other check boxes. Indeed, checking or
> unchecking it should not make dirty the document. So, the privilege
> is given or revoked instantly when checking or unchecking, without the
> need of confirming the change.
>
> The patch also nags the user when -shell-escape is added to a latex
> backend, suggesting to use the support provided by LyX instead of
> allowing this privilege to any document. This is all we can do in
> this case to try to increase security, because we should't change
> users' choices.
>
> When a document with shell-escape privileges is moved to a new location
> or removed, it loses the privilege. So, if a new file with same name is
> later placed there, it doesn't inherit the privilege. 

Thanks for this, Enrico. Let me just ask one question about it: Is the
mechanism here per-document
or per-document and also per-converter? That is, suppose there was a
document that contained R
code and minted code. Would one be warned about both or would one only
be asked once? If the
the latter, how hard would it be to change that?

Oh, maybe another thought: Could clicking the 'warning' icon easily be
used to disable execution?
Just a UI thought there.

Richard

Re: Can shell-escape take advantage of needauth framework?

[Richard Heck]

On 07/19/2017 05:06 AM, Pavel Sanda wrote:
> Christian Ridderström wrote:
>> I just did a test with gnuplot. In the LyX settings I had unchecked 'Forbid
>> of use of needauth converters' and unchecked 'Use needauth option'. Then I
>> opened a LyX doc with a gnuplot script. Result: LyX tried to run the script
>> due to the preview, without asking or alerting me.
>>
>> In my opinion this demonstrates a case where the security is _not_ good
>> enough, as I don't think it'd very difficult to trick someone into
>> unchecking these boxes.
> I think at the end it boils down to the question whether we rather want
> LyX for unaware users who can't handle any responsibility or we want
> to allow advanced features for more hackish crowd of people.
>
> I obviously stay in the hackish campground ground but understand your
> fear for the poor. 
>
> I would offer two quick options here:
> 1. Rename 'Forbid of use of needauth converters' to something scary
>so users have red flag.
> 2. Let the machinery alive, but move the flags from UI to RC files,
>and forcing people to edit them, so they have time to think
>what they are doing instead of randomly clicking.

I've suggested this, too. Just to be clear, you just have to remove the
UI for this setting. It
can stay in the same file, which can just be edited.

Rihcar

Re: Options for resolving the minted + shell-escape issue

[Enrico Forestieri]

On Tue, Jul 18, 2017 at 07:26:23PM -0400, Richard Heck wrote:
> On 07/18/2017 09:56 AM, Jürgen Spitzmüller wrote:
> > Am Dienstag, den 18.07.2017, 15:39 +0200 schrieb Jean-Marc Lasgouttes:
> >> Whi, not, maybe along with the names of the converters (features) 
> >> Sweave/gnuplot/minted present in current document and accepted by the
> >> user.
> > I would add a verbose tooltip when hovering the icon, something like
> >
> > '''
> > NOTE: Shell escape access granted.
> >
> > For this document, access to the -shell-escape feature has been granted
> > for the following converters: ...
> >
> > Note that this is a potential security risk. Use only if you trust the
> > source of this document. Please refer to sec. xx of the User Guide for
> > details.
> >
> > To withdraw shell escape access, press this icon.
> > '''
> 
> This seems a reasonable solution to me. It is not perfect, but nothing is.
> 
> As I see it, the issue is that there are actually a wide variety of
> reasons that users might want to
> enable -shell-escape for various converters. As LyX currently functions,
> the only way to do this
> is to add this to the converter itself. This is dangerous from our point
> of view NOT so much (or
> only) because it is intrinsically dangerous, but rather because it it is
> the kind of thing that is too
> easy to "do and forget". Or, to put it differently: It is a serious
> hassle to enable -shell-escape as
> things are, and that invites people to do it and leave it. And that
> really is a security risk.

The attached patch takes into account all of these ideas. As a disclaimer,
note that I am providing it only because I am now familiar with this part
of the code and can quickly come up with a patch. But I am not endorsing it.

The user can choose to allow execution of external programs for a given
document through the document settings. However, this is a property that
only holds on the computer used to edit the document. There is no way
to send out a document with the shell-escape thing activated.

Once activated, the user is prompted for confirmation each time a
latex backend is used, unless he decides to always allow execution for
a particular document. A document marked as requiring shell escape
privileges is characterized by a red icon on the status bar.

The shell-escape privilege can be revoked through the document settings.
Given the peculiarity that this cannot be a mere document property but
rather a property tied to both document and computer, the check box
works  differently from all other check boxes. Indeed, checking or
unchecking it should not make dirty the document. So, the privilege
is given or revoked instantly when checking or unchecking, without the
need of confirming the change.

The patch also nags the user when -shell-escape is added to a latex
backend, suggesting to use the support provided by LyX instead of
allowing this privilege to any document. This is all we can do in
this case to try to increase security, because we should't change
users' choices.

When a document with shell-escape privileges is moved to a new location
or removed, it loses the privilege. So, if a new file with same name is
later placed there, it doesn't inherit the privilege. 

To try the patch, the emblem-shellescape.svgz icon should be copied to
lib/images.

-- 
Enrico
diff --git a/src/Buffer.cpp b/src/Buffer.cpp
index df7efa8f0c..3f3f706251 100644
--- a/src/Buffer.cpp
+++ b/src/Buffer.cpp
@@ -55,6 +55,7 @@
 #include "ParagraphParameters.h"
 #include "ParIterator.h"
 #include "PDFOptions.h"
+#include "Session.h"
 #include "SpellChecker.h"
 #include "sgml.h"
 #include "texstream.h"
@@ -980,6 +981,8 @@ int Buffer::readHeader(Lexer & lex)
errorList.push_back(ErrorItem(_("Document header error"), s));
}
 
+   params().shell_escape = 
theSession().shellescapeFiles().find(absFileName());
+
params().makeDocumentClass();
 
return unknown_tokens;
diff --git a/src/BufferParams.cpp b/src/BufferParams.cpp
index f4890eaeec..d69c9f1ecf 100644
--- a/src/BufferParams.cpp
+++ b/src/BufferParams.cpp
@@ -38,6 +38,7 @@
 #include "Lexer.h"
 #include "LyXRC.h"
 #include "OutputParams.h"
+#include "Session.h"
 #include "Spacing.h"
 #include "texstream.h"
 #include "TexRow.h"
@@ -459,6 +460,7 @@ BufferParams::BufferParams()
html_css_as_file = false;
display_pixel_ratio = 1.0;
 
+   shell_escape = false;
output_sync = false;
use_refstyle = true;
use_minted = false;
@@ -1441,6 +1443,21 @@ void BufferParams::writeFile(ostream & os, Buffer const 
* buf) const
 os << "\\html_latex_end " << html_latex_end << '\n';
 
os << pimpl_->authorlist;
+
+   if (!shell_escape) {
+   std::set & shellescape_files =
+   theSession().shellescapeFiles().shellescapeFiles();
+   char const * ext[2] = { ",0", ",1" };
+   for (int i = 0; i < 2; ++i) {
+   

Re: Errors with vref on de/Additional.lyx with lualatex

[Kornel Benko]

Am Mittwoch, 19. Juli 2017 um 15:00:16, schrieb Kornel Benko 
> So, maybe better we could omit \textcompwordmark in mono fonts.

This patch works for me. It uses vphantom{}, but only between '<<' and '>>'.

Korneldiff --git a/src/Paragraph.cpp b/src/Paragraph.cpp
index 73157e6..622477b 100644
--- a/src/Paragraph.cpp
+++ b/src/Paragraph.cpp
@@ -1378,8 +1378,8 @@ bool Paragraph::Private::latexSpecialT1(char_type const c, otexstream & os,
 		// but we should avoid ligatures
 		if (i + 1 >= int(text_.size()) || text_[i + 1] != c)
 			return true;
-		os << "\\textcompwordmark" << termcmd;
-		column += 19;
+		os << "\\vphantom{}";
+		column += 16;
 		return true;
 	case '|':
 		os.put(c);


signature.asc
Description: This is a digitally signed message part.

Re: Errors with vref on de/Additional.lyx with lualatex

[Kornel Benko]

Am Mittwoch, 19. Juli 2017 um 10:56:21, schrieb Guenter Milde 

> On 2017-07-18, Kornel Benko wrote:
> 
> ...
> 
> > there is also error with the system font 'FreeSerif,FreeSans,FreeMono'
> > when exporting to PDF(luatex)
> 
> >  !Package varioref Error: \vref at page boundary 14-15 (may loop).
> 
> > See the varioref package documentation for explanation.
> > Type  H   for immediate help.
> >  ...  
> 
> > l.810 ...t\ \vref{Literaturverzeichnisse-mit-BibTeX}
> 
> > Please check the pages in question. You might need to replace the \vref
> > or \vpageref by a normal \(page)ref to stop LaTeX running forever.
> 
> > Changing the font to say Dejavu, the error disappears. Also inserting a line
> > for instance before paragraph 3.1.2 cures the compilation.
> 
> This is a rare running-condition that depends on the selected font (because
> this influences page-breaks).
> 
> > 1.) Should we do anything here? Or is the user on its own if he/she
> > gets such a message?
> 
> Generally, the user is on its own when seeing such a message - we cannot
> predict it from within LyX.
> 
> In the given case, it does not appear with the original fonts as defined
> in the documentation file, only when replacing them with FreeSerif
> (either in a script or in the document).
> 
> We could define DejaVu in the document (to solve the \textcompwordmark error)
> or try a different wording in the sentence containing the offending \vref.
> 
> Günter

I did as you proposed (Dejavu font), but maybe you are unaware that
With TL15 everything compiles here
With TL16 I get missing glyphs if font is Dejavu, but not with Free*
Mark, that changing 'Dejavu Sans Mono' to e.g. 'Dejavu Sans' cures the 
compilation though
With TL17 I get missing glyphs for any font. I have to replace the mono font to
get it to compile.
So, maybe better we could omit \textcompwordmark in mono fonts.

Kornel


signature.asc
Description: This is a digitally signed message part.

Re: [LyX/master] We have new translation which slipped through the cracks.

[Pavel Sanda]

Jürgen Spitzmüller wrote:
> OTOH I see now that this is how they are defined in listings and minted,
> respectively. So the (differing) strings are OK.

Yes, when I looked through the logs it seems Enrico was well aware of the issue.

> For de, I've just committed a small correction. Shall I modify the
> layouttranslations file for this, or do you re-generate?

Ich werde mich darum kummern :) P

Re: [LyX/master] We have new translation which slipped through the cracks.

[Jürgen Spitzmüller]

2017-07-19 14:17 GMT+02:00 Jürgen Spitzmüller :

>
> But we have the situation now that we output different strings for the
> list of listings heading, depending on whether we use minted or listings
> ("Listings" vs. "List of Listings").
>

OTOH I see now that this is how they are defined in listings and minted,
respectively. So the (differing) strings are OK.

For de, I've just committed a small correction. Shall I modify the
layouttranslations file for this, or do you re-generate?

Jürgen

Re: [LyX/master] We have new translation which slipped through the cracks.

[Jürgen Spitzmüller]

2017-07-19 13:51 GMT+02:00 Pavel Sanda :

> Pavel Sanda wrote:
> > commit cd7b1dad6713e0dba2b90e7757fce5b0ca8e
> > Author: Pavel Sanda 
> > Date:   Wed Jul 19 13:36:06 2017 +0200
> >
> > We have new translation which slipped through the cracks.
>
> Scott, I am sorry I did not catch that before.
>
> If/when you will be sending the announcement to translators for
> RC, please add paragraph requesting ack for particular string
> in layouttranslation table.
>
> I guess we can get quick ack for de, de_alt, fr.
>

de_alt is OK.

But we have the situation now that we output different strings for the list
of listings heading, depending on whether we use minted or listings
("Listings" vs. "List of Listings").

I'd opt to use the established "Listings" for both context. Otherwise we
change output of existing documents.

Jürgen


>
> Pavel
>

Re: [LyX/master] We have new translation which slipped through the cracks.

[Pavel Sanda]

Pavel Sanda wrote:
> commit cd7b1dad6713e0dba2b90e7757fce5b0ca8e
> Author: Pavel Sanda 
> Date:   Wed Jul 19 13:36:06 2017 +0200
> 
> We have new translation which slipped through the cracks.

Scott, I am sorry I did not catch that before.

If/when you will be sending the announcement to translators for
RC, please add paragraph requesting ack for particular string
in layouttranslation table.

I guess we can get quick ack for de, de_alt, fr.

Pavel

Re: [LyX/master] Overtake layout translations from fi.po, ja.po, zh_CN.po

[Pavel Sanda]

Pavel Sanda wrote:
> Kornel Benko wrote:
> > Am Mittwoch, 19. Juli 2017 um 05:58:29, schrieb Jari-Matti Mäkelä 
> > 
> > >  Fri, 09 Jun 2017 20:30:07 +0200
> > > Kornel Benko  wrote:
> > > 
> > > > At least 'make translations1' works. No, I did not so far. The changes
> > > > are because of the fi.po.patch from Jari-Matti Mäkelä.
> > > > Again, he should be asked if the new translation are OK for pdf.
> > > > But, as I see it, I would not touch lib/layouttranslations in the fi
> > > > part ATM. There are still about 2600 untranslated/fuzzy messages in
> > > > fi.po. Jari-Mati seems to be willing to work on this if he finds some
> > > > time.
> > > > 
> > > > Jari, what about the following in lib/layouttranslations?
> > > > 
> > > > -   "List of Algorithms" "Algoritmien luettelo"
> > > > -   "List of Charts" "Kaavioiden luettelo"
> > > > -   "List of Graphs[[mathematical]]" "Kuvaajien luettelo"
> > > > -   "List of Schemes" "Kuvausten luettelo"
> > > > -   "List of Tableaux" "Taulujen luettelo"
> > > > +   "List of Algorithms" "Algoritmit"
> > > > +   "List of Charts" "Kaaviot"
> > > > +   "List of Graphs[[mathematical]]" "Kaaviot"
> > > > +   "List of Schemes" "Kuvaukset"
> > > > +   "List of Tableaux" "Taulut"qgit show
> > > > 
> > > > The following is wrong (according to Jari, maybe also wrong in fi.po)
> > > > -   "Listing" "Ohjelmalistaus"
> > > > +   "Listing" "Listaus"
> > > > 
> > > 
> > > Sorry, only had time now to study this further.
> 
> Hi,
> thanks, for the clarifications!
> 
> I see you removed the "luettelo" parts which is supposedly "List" in your 
> language.
> Are you sure about that change?
> 
> If you have book with bunch of algorithm codes and you put somewhere into 
> appendix
> list of those, they will be introduced by Algoritmit instead of Algoritmien 
> luettelo.
> Was that intended?

I just checked that all the "List" strings were recently checked by Martin 
Vermeer,
so I think it was actually meant to be with "luettelo". (?)

Pavel

> 
> > > "List of Graphs[[mathematical]]"
> > > 
> > > I have to admit I didn't understand the difference between charts and
> > > graphs and couldn't find this yet from the GUI. Chart is e.g. a bar/pie
> > > chart? Graph is some other graphical notation for expressing data? This
> > > is confusing without knowing the context. If there is no other
> > > difference, the same translation works:
> > 
> > I don't know either. Anyway, there has to be different translation, 
> > otherwise
> > the two menu entries in outliner are not distinguishable.
> 
> Graphs/Chart/Scheme are for papers written in layout for American Chemical 
> Society
> and apparently they don't even care to show Graph example in their demo
> ( http://mirrors.ctan.org/macros/latex/contrib/achemso/achemso-demo.pdf )
> so I think this particular tranaslation is not that important. 
> 
> > > Apparently this is related to chemistry? Is the 'scheme' like 'Chemical
> > > equation', 'structural formula' or something else? It's hard to
> > > translate this without knowing the context.
> 
> Most likely they are your 'structural formula'. 
> 
> > > "List of Tableaux"
> > > 
> > > -> Taulukot
> > 
> > 
> > Done
> > 
> > > Tableaux is just a synonym for tables in french/latin? I tried to google
> > > this but couldn't find any special meaning in LaTeX/LyX other than the
> > > plural form of 'table'.
> 
> That's from linguistics and my understanding something similar to some logic 
> table.
> Likely so specific that using Tableaux in your language is just fine.
> 
> > > Basically all 'listings' in LyX use the lstlisting package? From a
> > > translators POV, it's really hard to identify the context, whether we
> > > are listing some generic items or program code.
> > > 
> > > 
> > > "List of Listings"
> > > "List of Program Listing"
> > > 
> > > -> Ohjelmalistaukset
> 
> They are program code.
> 
> Pavel

Re: [LyX/master] Overtake layout translations from fi.po, ja.po, zh_CN.po

[Pavel Sanda]

Kornel Benko wrote:
> Am Mittwoch, 19. Juli 2017 um 05:58:29, schrieb Jari-Matti Mäkelä 
> 
> >  Fri, 09 Jun 2017 20:30:07 +0200
> > Kornel Benko  wrote:
> > 
> > > At least 'make translations1' works. No, I did not so far. The changes
> > > are because of the fi.po.patch from Jari-Matti Mäkelä.
> > > Again, he should be asked if the new translation are OK for pdf.
> > > But, as I see it, I would not touch lib/layouttranslations in the fi
> > > part ATM. There are still about 2600 untranslated/fuzzy messages in
> > > fi.po. Jari-Mati seems to be willing to work on this if he finds some
> > > time.
> > > 
> > > Jari, what about the following in lib/layouttranslations?
> > > 
> > > -   "List of Algorithms" "Algoritmien luettelo"
> > > -   "List of Charts" "Kaavioiden luettelo"
> > > -   "List of Graphs[[mathematical]]" "Kuvaajien luettelo"
> > > -   "List of Schemes" "Kuvausten luettelo"
> > > -   "List of Tableaux" "Taulujen luettelo"
> > > +   "List of Algorithms" "Algoritmit"
> > > +   "List of Charts" "Kaaviot"
> > > +   "List of Graphs[[mathematical]]" "Kaaviot"
> > > +   "List of Schemes" "Kuvaukset"
> > > +   "List of Tableaux" "Taulut"qgit show
> > > 
> > > The following is wrong (according to Jari, maybe also wrong in fi.po)
> > > -   "Listing" "Ohjelmalistaus"
> > > +   "Listing" "Listaus"
> > > 
> > 
> > Sorry, only had time now to study this further.

Hi,
thanks, for the clarifications!

I see you removed the "luettelo" parts which is supposedly "List" in your 
language.
Are you sure about that change?

If you have book with bunch of algorithm codes and you put somewhere into 
appendix
list of those, they will be introduced by Algoritmit instead of Algoritmien 
luettelo.
Was that intended?

> > "List of Graphs[[mathematical]]"
> > 
> > I have to admit I didn't understand the difference between charts and
> > graphs and couldn't find this yet from the GUI. Chart is e.g. a bar/pie
> > chart? Graph is some other graphical notation for expressing data? This
> > is confusing without knowing the context. If there is no other
> > difference, the same translation works:
> 
> I don't know either. Anyway, there has to be different translation, otherwise
> the two menu entries in outliner are not distinguishable.

Graphs/Chart/Scheme are for papers written in layout for American Chemical 
Society
and apparently they don't even care to show Graph example in their demo
( http://mirrors.ctan.org/macros/latex/contrib/achemso/achemso-demo.pdf )
so I think this particular tranaslation is not that important. 

> > Apparently this is related to chemistry? Is the 'scheme' like 'Chemical
> > equation', 'structural formula' or something else? It's hard to
> > translate this without knowing the context.

Most likely they are your 'structural formula'. 

> > "List of Tableaux"
> > 
> > -> Taulukot
> 
> 
> Done
> 
> > Tableaux is just a synonym for tables in french/latin? I tried to google
> > this but couldn't find any special meaning in LaTeX/LyX other than the
> > plural form of 'table'.

That's from linguistics and my understanding something similar to some logic 
table.
Likely so specific that using Tableaux in your language is just fine.

> > Basically all 'listings' in LyX use the lstlisting package? From a
> > translators POV, it's really hard to identify the context, whether we
> > are listing some generic items or program code.
> > 
> > 
> > "List of Listings"
> > "List of Program Listing"
> > 
> > -> Ohjelmalistaukset

They are program code.

Pavel

Re: Errors with vref on de/Additional.lyx with lualatex

[Guenter Milde]

On 2017-07-18, Kornel Benko wrote:

...

> there is also error with the system font 'FreeSerif,FreeSans,FreeMono'
> when exporting to PDF(luatex)

>  !Package varioref Error: \vref at page boundary 14-15 (may loop).

>   See the varioref package documentation for explanation.
>   Type  H   for immediate help.
>...  

>   l.810 ...t\ \vref{Literaturverzeichnisse-mit-BibTeX}

>   Please check the pages in question. You might need to replace the \vref
>   or \vpageref by a normal \(page)ref to stop LaTeX running forever.

> Changing the font to say Dejavu, the error disappears. Also inserting a line
> for instance before paragraph 3.1.2 cures the compilation.

This is a rare running-condition that depends on the selected font (because
this influences page-breaks).

> 1.) Should we do anything here? Or is the user on its own if he/she
> gets such a message?

Generally, the user is on its own when seeing such a message - we cannot
predict it from within LyX.

In the given case, it does not appear with the original fonts as defined
in the documentation file, only when replacing them with FreeSerif
(either in a script or in the document).

We could define DejaVu in the document (to solve the \textcompwordmark error)
or try a different wording in the sentence containing the offending \vref.

Günter

Re: Errors with vref on de/Additional.lyx with lualatex

[Guenter Milde]

On 2017-07-18, Kornel Benko wrote:

...

> 2.) Can we replace outputting \textcompwordmark with \vphantom{} in
> exported tex file?
> I could not see any difference in the pdflatex output. Also
> lualatex and xelatex displayed correctly.

However, lib/unicodesymbols says:

  Do only add commands that give correct output, no hacks that look "similar".

but \textcompwordmark is defined as the LaTeX equivalent to
200C ZERO WIDTH NON-JOINER (ZWNJ) in the LaTeX base:

  t1enc.dfu:261:\DeclareUnicodeCharacter{200C}{\textcompwordmark}

  utf8.def:247:\DeclareUnicodeCharacter{200C}{\textcompwordmark}


The macro itself is defined in the LaTeX base as

  
latex.ltx:2036:\DeclareTextCommandDefault{\textcompwordmark}{\leavevmode\kern\z@}

For T1-encoded fonts, it is mapped to character 23:

  latex.ltx:8275:\lccode 23 =23% textcompwordmark in T1
  t1enc.def:112:\DeclareTextSymbol{\textcompwordmark}{T1}{23}

Since the recent additon of the standard Unicode text font encoding "TU",
there is a conversion to ZWNJ if Unicode fonts are used with fontspec:

  tuenc.def:169:\DeclareTextSymbol{\textcompwordmark}
\UnicodeEncodingName{"200C}

The problem is, that this character is missing in the default (LatinModern
Unicode) font. It works fine with, e.g., DejaVu.


> Moreover, using
> \textcompwordmark with xelatex displays wrong.

I cannot reproduce.  In my tests, the output is correct: \textcompwordmark
suppresses the ff ligature without other side-effects in the PDF.

However, there is a difference if you copy-paste from the PDF output:

With evince, I get

* for pdflatex and T1-encoded lmodern fonts:

test with textcompwordmark: ff
test with vphantom: ff

  \textcompwordmark translates to the unprintable character u0017 because
  this is the position of the ZWNJ character in T1 encoding

  (This is a LaTeX or Evince bug.)

* for xetex or luatax with fontspec and LatinModern

test with textcompwordmark: ff
test with vphantom: ff

  Output and copy-paste OK despite the "missing character" warning.
  The invisible ZWNJ is simply dropped but prevents the ligature.

* for xetex with DejaVu:

test with textcompwordmark: f f
test with vphantom: ff

  The ZWNJ is converted to an ordinary space when pasting into LyX

* for luatex with DejaVu:

   test with textcompwordmark: f‌f
   test with vphantom: ff

  The ZWNJ is kept (and correctly suppresses the ligature in GUI and output)
  when pasting into LyX

Options:

a) Do not turn the "missing character" warning into an error for
   characters where this is no data loss, e.g. 200C (ZWNJ)

   + helps also, if there are literal ZWNJ characters in the source,
   - side-effects for copy-paste with pdflatex and xetex

b) Re-set \textcompwordmark to the Default in the preamble if TU is
   used:

 \DeclareTextCommand{\textcompwordmark}{TU}{\leavevmode\kern\z@}

As immediate action, maybe also

c) Invert the test(s) with a comment declaring the reason.

d) Use DejaVu or FreeSerif in the affected tests
   (preferably added to the documents instead of the script-based
   replacement).


Günter

Re: [LyX/master] Overtake layout translations from fi.po, ja.po, zh_CN.po

[Kornel Benko]

Am Mittwoch, 19. Juli 2017 um 05:58:29, schrieb Jari-Matti Mäkelä 

>  Fri, 09 Jun 2017 20:30:07 +0200
> Kornel Benko  wrote:
> 
> > At least 'make translations1' works. No, I did not so far. The changes
> > are because of the fi.po.patch from Jari-Matti Mäkelä.
> > Again, he should be asked if the new translation are OK for pdf.
> > But, as I see it, I would not touch lib/layouttranslations in the fi
> > part ATM. There are still about 2600 untranslated/fuzzy messages in
> > fi.po. Jari-Mati seems to be willing to work on this if he finds some
> > time.
> > 
> > Jari, what about the following in lib/layouttranslations?
> > 
> > -   "List of Algorithms" "Algoritmien luettelo"
> > -   "List of Charts" "Kaavioiden luettelo"
> > -   "List of Graphs[[mathematical]]" "Kuvaajien luettelo"
> > -   "List of Schemes" "Kuvausten luettelo"
> > -   "List of Tableaux" "Taulujen luettelo"
> > +   "List of Algorithms" "Algoritmit"
> > +   "List of Charts" "Kaaviot"
> > +   "List of Graphs[[mathematical]]" "Kaaviot"
> > +   "List of Schemes" "Kuvaukset"
> > +   "List of Tableaux" "Taulut"qgit show
> > 
> > The following is wrong (according to Jari, maybe also wrong in fi.po)
> > -   "Listing" "Ohjelmalistaus"
> > +   "Listing" "Listaus"
> > 
> 
> Sorry, only had time now to study this further.
> 
> 
> Here are the correct translations.
>  
> "List of Algorithms"
> 
> -> Algoritmit
> 

Done

> "List of Charts"
> 
> -> Kaaviot
> 

Done

> "List of Graphs[[mathematical]]"
> 
> I have to admit I didn't understand the difference between charts and
> graphs and couldn't find this yet from the GUI. Chart is e.g. a bar/pie
> chart? Graph is some other graphical notation for expressing data? This
> is confusing without knowing the context. If there is no other
> difference, the same translation works:

I don't know either. Anyway, there has to be different translation, otherwise
the two menu entries in outliner are not distinguishable.

> -> Kaaviot
> 
> "List of Schemes"
> 
> Apparently this is related to chemistry? Is the 'scheme' like 'Chemical
> equation', 'structural formula' or something else? It's hard to
> translate this without knowing the context.
> 

I too found only one place in achemso.layout ...

> "List of Tableaux"
> 
> -> Taulukot


Done

> Tableaux is just a synonym for tables in french/latin? I tried to google
> this but couldn't find any special meaning in LaTeX/LyX other than the
> plural form of 'table'.
> 
> 
> "Listing"
> "Program Listing"
> 
> -> Ohjelmalistaus

Already there. (But no "Program Listing")

> Basically all 'listings' in LyX use the lstlisting package? From a
> translators POV, it's really hard to identify the context, whether we
> are listing some generic items or program code.
> 
> 
> "List of Listings"
> "List of Program Listing"
> 
> -> Ohjelmalistaukset
> 

Already there.

>  - Jari-Matti


Thanks Jari.

Committed together with changes to fi.po posted privately.

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: citeengines/ break make layouttranslation

[Jürgen Spitzmüller]

2017-07-19 12:38 GMT+02:00 Pavel Sanda :

> Fixed it. P
>

Thanks!

Jürgen

Re: citeengines/ break make layouttranslation

[Pavel Sanda]

Pavel Sanda wrote:
> Hi Juergen,
> 
> make ../lib/layouttranslations from within po/ dir
> used to update layout translations, but it breaks now on
> make: *** No rule to make target '../lib/citeengines/*.citeengines', needed 
> by '../lib/layouttranslations'.  Stop.
> Not sure what's going on.

Fixed it. P

citeengines/ break make layouttranslation

[Pavel Sanda]

Hi Juergen,

make ../lib/layouttranslations from within po/ dir
used to update layout translations, but it breaks now on
make: *** No rule to make target '../lib/citeengines/*.citeengines', needed by 
'../lib/layouttranslations'.  Stop.
Not sure what's going on.

Pavel

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Jean-Marc Lasgouttes]

Le 19/07/2017 à 07:48, Christian Ridderström a écrit :
If user does not want all these warnings, he could disable them by 
launching LyX with some option like "--do-not-warn-me-about-unsafe-setting".

Instead of having a checkbox for "don't tell me these things again".


It has the same issues as the checkbox when one considers batch command 
line processing. People add the switch to the command line and forget 
about it.


Which make me think that I did not try to check whether my nice scripts 
to process Sweave lyx file still have a chance to work. Oops! they 
won't, except if I disable needauth globally :(


fantomas: LC_ALL=C ~/src/lyx/devbuild/src/lyx -e pdf2 cours-acp.lyx
Error: An external converter is disabled for security reasons

The requested operation requires the use of a converter from sweave to 
pdflatex:
Rscript --verbose --no-save --no-restore $$s/scripts/lyxsweave.R $$p$$i 
$$p$$o $$e $$r
This external program can execute arbitrary commands on your system, 
including dangerous ones, if instructed to do so by a maliciously 
crafted .lyx document.

Your current preference settings forbid its execution.
(To change this setting, go to Preferences ▹ File Handling ▹ Converters 
and uncheck Security ▹ Forbid needauth converters.)


JMarc

Re: Can shell-escape take advantage of needauth framework?

[Pavel Sanda]

Guillaume MM wrote:
> Le 18/07/2017 ?? 23:27, Jean-Marc Lasgouttes a écrit :
>> Le 18/07/2017 ?? 23:24, Christian Ridderström a écrit :
>>> The threat model is one important aspect, but it's difficult for us to 
>>> know who uses LyX and in which industries. Or how many users there are at 
>>> all. And how many of them that use converters.  If we can achieve good 
>>> security we don't need to care about user / usage statistics though.
>>>
>>> If we talk principles, I think we should aim for really good security and 
>>> then discuss compromises for what's not doable. But I do think we could 
>>> do a whole lot better than the current 'needauth'.
>
> +1

You start driving me to the wall.
Where you have been when there was huge discussion about details of these 
mechanisms with Tommasso?

> Meep! Principle 1: "Things don???t become unsafe all by themselves
> (Explicit authority)". This means in particular: the secure settings are
> the default.

Yep. Isn't it the case with needauth right now?

Pavel

Re: Living with shell-escape: Using two LyX instances - critique invited

[Guenter Milde]

On 2017-07-18, Guillaume MM wrote:
> Le 18/07/2017 à 21:29, Christian Ridderström a écrit :

>> If I had to use a converter that requires e.g. shell-escape perhaps the 
>> approach below would be useful. [...]

...

> I find that it would be more cumbersome and error-prone than a good
> needauth implementation. If I understand, what you want is visibility
> and revokability, which people already seem to agree are desirable
> improvements to make to needauth (a red status bar thingy).

Agreed. I am glad to see this consensus emerging.

If I got Christian right, the suggestion was intended as stop-gap measure
for power-users of LyX <= 2.2.x (as is my alternative proposal).

Günter

Re: Going into dangerous mode (Was: Can shell-escape take advantage of needauth framework?)

[Pavel Sanda]

Christian Ridderström wrote:
> - Users uncheck settings all the time, it doesn't seem very "scary"
> 
> Why does disabling something like needauth have to be done from within LyX?

... as I read through the list I see we come to similar conclusions ...
I don't have strong opinion about these.

Pavel

Re: Can shell-escape take advantage of needauth framework?

[Pavel Sanda]

Christian Ridderström wrote:
> I just did a test with gnuplot. In the LyX settings I had unchecked 'Forbid
> of use of needauth converters' and unchecked 'Use needauth option'. Then I
> opened a LyX doc with a gnuplot script. Result: LyX tried to run the script
> due to the preview, without asking or alerting me.
>
> In my opinion this demonstrates a case where the security is _not_ good
> enough, as I don't think it'd very difficult to trick someone into
> unchecking these boxes.

I think at the end it boils down to the question whether we rather want
LyX for unaware users who can't handle any responsibility or we want
to allow advanced features for more hackish crowd of people.

I obviously stay in the hackish campground ground but understand your
fear for the poor. 

I would offer two quick options here:
1. Rename 'Forbid of use of needauth converters' to something scary
   so users have red flag.
2. Let the machinery alive, but move the flags from UI to RC files,
   and forcing people to edit them, so they have time to think
   what they are doing instead of randomly clicking.

I disagree though that we should ban needauth mechanism right now and
if the argument really is that I can trick someone into unchecking
whatever I want, then I can directly trick him into writing rm -rf /
on the commandline.

Pavel

Re: Options for resolving the minted + shell-escape issue

[Jürgen Spitzmüller]

>
> But I'd make one more suggestion: Every time a user opens a
> document for which this
> sort of  thing will be enabled, we pop a warning before we do anything.
> I.e., we do NOT just run
> gnuplot in the background, but we say something like what Jürgen had
> above, with buttons offering
> either to proceed or not. Doing this once per document per session does
> not seem too much to ask.
> (It would streamline things a bit, too, if we could 'inherit' this
> setting for child documents. So you
> would not have to keep clicking through if there were a lot of children.)
>

I agree this is not too much of a burden UI-wise and enhances awareness.

Jürgen



>
> Richard
>
>

Any descriptions of the security aspects (related to needauth and shell-escape)?

[Christian Ridderström]

Hi,

When having tried to contribute to the discussion on needauth and
shell-escape I've felt that it's quite difficult to get a good picture of
things like:
- Goals of design, what are we trying to achieve
- Principle of design and system
- Assumed threat models, and perhaps list threat scenarios we _don't_ try
to protect against

The e-mail threads are ... long, sometimes confusing and I suspect contains
at least a few misunderstandings.  So I would like to ask (not being
optimistic), if there's some design description anywhere?

I wonder because IMHO security requires a system wide approach and that
it's very easy to screw up if only looking at isolated pieces. Further, it
requires continuity so you know what you initially intended to achieve and
what you consider good enough. Otherwise you might later introduce a new
feature that inadvertently opens up a security whole. Without a system
design, it's also easy to get caught in discussions trying to bandaid a
small hole while missing entire walls missing.

I think this kind of information would be good to gather and store in some
kind of design document, which could just be a text file in the repo.  Then
we could add knowledge to this document, and let if include the rationale
behind our choices, as well as letting developers review the system design.

Of course, using a design document isn't something this project is used to
so of course there's a risk it'll e.g. forgotten.  But I feel I should at
least suggest this.

Regards,
Christian