[LyX/2.4.x] Convert $${python} placeholder for graphic conversions

2024-04-20 Thread José Matos
commit cc75da1c3e109e4fd6b1305a707c70929000ddd6
Author: José Matos 
Date:   Sun Apr 14 21:30:02 2024 +0100

Convert $${python} placeholder for graphic conversions

(cherry picked from commit cdcaf0e7b6cc45bc74175667c390c3148c4730e9)
---
 src/graphics/GraphicsConverter.cpp | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/graphics/GraphicsConverter.cpp 
b/src/graphics/GraphicsConverter.cpp
index 669f08d2ff..34d37a4066 100644
--- a/src/graphics/GraphicsConverter.cpp
+++ b/src/graphics/GraphicsConverter.cpp
@@ -362,6 +362,7 @@ static void build_script(string const & doc_fname,
string const token_base  = "$$b";
string const token_to= "$$o";
string const token_todir = "$$d";
+   string const token_python = "$${python}";
 
EdgePath::const_iterator it  = edgepath.begin();
EdgePath::const_iterator end = edgepath.end();
@@ -405,6 +406,7 @@ static void build_script(string const & doc_fname,
command = subst(command, token_base,  "' + '\"' + infile_base + 
'\"' + '");
command = subst(command, token_to,"' + '\"' + outfile + 
'\"' + '");
command = subst(command, token_todir, "' + '\"' + outdir + '\"' 
+ '");
+   command = subst(command, token_python, os::python());
 
build_conversion_command(command, script);
}
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Replace $${python} directly in the converter constructor

2022-01-27 Thread José Matos
commit 79a2ac575cd40291e8cc7e8c4a3f4072670367b7
Author: José Matos 
Date:   Thu Jan 27 18:32:57 2022 +

Replace $${python} directly in the converter constructor
---
 src/Converter.cpp |6 ++
 src/Converter.h   |4 ++--
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/src/Converter.cpp b/src/Converter.cpp
index c00941d..7bb998d 100644
--- a/src/Converter.cpp
+++ b/src/Converter.cpp
@@ -148,6 +148,12 @@ void Converter::readFlags()
 }
 
 
+void Converter::setCommand(std::string const & command)
+{
+   command_ = subst(command, token_python, os::python());
+}
+
+
 Converter const * Converters::getConverter(string const & from,
string const & to) const
 {
diff --git a/src/Converter.h b/src/Converter.h
index 5197e34..091dbcd 100644
--- a/src/Converter.h
+++ b/src/Converter.h
@@ -58,7 +58,7 @@ public:
///
std::string const command() const { return command_; }
///
-   void setCommand(std::string const & command) { command_ = command; }
+   void setCommand(std::string const & command);
///
std::string const flags() const { return flags_; }
///
@@ -144,7 +144,7 @@ public:
FAILURE = 1,
KILLED  = 1000
};
-   
+
///
Converter const & get(int i) const { return converterlist_[i]; }
///
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Fix compilation with gcc-12

2022-01-27 Thread José Matos
commit b73ab0256d113571174fed4b2a3405fe8c44d297
Author: José Matos 
Date:   Thu Jan 27 15:37:45 2022 +

Fix compilation with gcc-12
---
 src/insets/InsetListings.cpp |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/src/insets/InsetListings.cpp b/src/insets/InsetListings.cpp
index 57df06e..67a0462 100644
--- a/src/insets/InsetListings.cpp
+++ b/src/insets/InsetListings.cpp
@@ -41,6 +41,7 @@
 #include "frontends/alert.h"
 #include "frontends/Application.h"
 
+#include 
 #include 
 #include 
 
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Add new placeholder $${python} to configure

2022-01-03 Thread José Matos
commit 109ea2be4a21ca93d22ab25703b3352a50fbbe3b
Author: José Matos 
Date:   Tue Jan 4 00:21:34 2022 +

Add new placeholder $${python} to configure

This ensures that we use a consistent Python interpreter in LyX.

$${python} is replaced by the Python version found.

Users can apply this in preferences and use the same version defined by
LyX.
---
 lib/configure.py   |   96 
 lib/scripts/include_bib.py |2 +-
 src/Converter.cpp  |   13 +++---
 src/Mover.cpp  |1 +
 src/graphics/PreviewLoader.cpp |5 +-
 src/support/ForkedCalls.cpp|   11 +---
 src/support/Systemcall.cpp |2 +-
 src/support/filetools.cpp  |2 +-
 8 files changed, 65 insertions(+), 67 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index 7d9b61c..17ef38f 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -653,7 +653,7 @@ def checkLatex(dtl_tools):
 if dtl_tools:
 # Windows only: DraftDVI
 addToRC(r'''\converter latex  dvi2   "%s"  
"latex,hyperref-driver=dvips"
-\converter dvi2   dvi"python -tt $$s/scripts/clean_dvi.py $$i $$o" 
""''' % LATEX)
+\converter dvi2   dvi"$${python} $$s/scripts/clean_dvi.py $$i $$o" 
""''' % LATEX)
 else:
 addToRC(r'\converter latex  dvi"%s"
"latex,hyperref-driver=dvips"' % LATEX)
 # no latex
@@ -943,7 +943,7 @@ def checkConverterEntries():
 checkProg('an HTML -> LaTeX converter', ['html2latex $$i', 'gnuhtml2latex',
 'htmltolatex -input $$i -output $$o', 'htmltolatex.jar -input $$i 
-output $$o'],
 rc_entry = [ r'\converter html   latex  "%%"   ""',
- r'\converter html   latex  "python -tt 
$$s/scripts/html2latexwrapper.py %% $$i $$o"   ""',
+ r'\converter html   latex  "$${python} 
$$s/scripts/html2latexwrapper.py %% $$i $$o"   ""',
  r'\converter html   latex  "%%"   ""',
  r'\converter html   latex  "%%"   ""', '' ])
 #
@@ -964,8 +964,8 @@ def checkConverterEntries():
 ['elyxer.py --html --nofooter --unicode --directory $$r $$i $$o', 
'elyxer --html --nofooter --unicode --directory $$r $$i $$o'],
 rc_entry = [ r'\converter lyx  word  "%%"  ""' ])
 if elyxer.find('elyxer') >= 0:
-  addToRC(r'''\copierhtml   "python -tt $$s/scripts/ext_copy.py -e 
html,png,jpg,jpeg,css $$i $$o"''')
-  addToRC(r'''\copierwordhtml   "python -tt 
$$s/scripts/ext_copy.py -e html,png,jpg,jpeg,css $$i $$o"''')
+  addToRC(r'''\copierhtml   "$${python} $$s/scripts/ext_copy.py -e 
html,png,jpg,jpeg,css $$i $$o"''')
+  addToRC(r'''\copierwordhtml   "$${python} 
$$s/scripts/ext_copy.py -e html,png,jpg,jpeg,css $$i $$o"''')
 else:
   # search for HTML converters other than eLyXer
   # On SuSE the scripts have a .sh suffix, and on debian they are in 
/usr/share/tex4ht/
@@ -974,17 +974,17 @@ def checkConverterEntries():
   'latex2html -no_subdir -split 0 -show_section_numbers $$i', 'hevea 
-s $$i'],
   rc_entry = [ r'\converter latex  html   "%%" "needaux"' ])
   if htmlconv.find('htlatex') >= 0 or htmlconv == 'latex2html':
-addToRC(r'''\copierhtml   "python -tt $$s/scripts/ext_copy.py 
-e html,png,css $$i $$o"''')
+addToRC(r'''\copierhtml   "$${python} $$s/scripts/ext_copy.py 
-e html,png,css $$i $$o"''')
   else:
-addToRC(r'''\copierhtml   "python -tt $$s/scripts/ext_copy.py 
$$i $$o"''')
+addToRC(r'''\copierhtml   "$${python} $$s/scripts/ext_copy.py 
$$i $$o"''')
   path, htmlconv = checkProg('a LaTeX -> HTML (MS Word) converter', 
["htlatex $$i 'html,word' 'symbol/!' '-cvalidate'",
   "htlatex.sh $$i 'html,word' 'symbol/!' '-cvalidate'",
   "/usr/share/tex4ht/htlatex $$i 'html,word' 'symbol/!' '-cvalidate'"],
   rc_entry = [ r'\converter latex  wordhtml   "%%" "needaux"' ])
   if htmlconv.find('htlatex') >= 0:
-addToRC(r'''\copierwordhtml   "python -tt 
$$s/scripts/ext_copy.py -e html,png,css $$i $$o"''')
+addToRC(r'''\copierwordhtml   "$${python} 
$$s/scripts/ext_copy.py -e html,png,css $$i $$o"''')
   else:
-addToRC(r'''\copierwordhtml   "python -tt 
$$s/scripts/ext_copy.py $$i $$o"''')
+addToRC(r'''\copierwordhtml   "$${python} 
$$s/scripts/ext_copy.py $$

[LyX/master] Make layout2layout compatible with Python 2 and 3

2022-01-03 Thread José Matos
commit 940d3ceeb9e6d8ce216afedf18c898ec075cc27d
Author: José Matos 
Date:   Mon Jan 3 19:59:42 2022 +

Make layout2layout compatible with Python 2 and 3
---
 lib/scripts/layout2layout.py |   14 +++---
 1 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/lib/scripts/layout2layout.py b/lib/scripts/layout2layout.py
index 7e40bfb..88ada12 100644
--- a/lib/scripts/layout2layout.py
+++ b/lib/scripts/layout2layout.py
@@ -256,7 +256,7 @@ currentFormat = 95
 # New textclass tag BibInToc
 
 # Incremented to format 77, 6 August 2019 by spitz
-# New textclass tag PageSize (= default page size) 
+# New textclass tag PageSize (= default page size)
 # and textclass option PageSize (= list of available page sizes)
 
 # Incremented to format 78, 6 August 2019 by spitz
@@ -354,7 +354,7 @@ def error(message):
 
 def trim_bom(line):
 " Remove byte order mark."
-if line[0:3] == "\357\273\277":
+if line[0:3] == b"\357\273\277":
 return line[3:]
 else:
 return line
@@ -444,8 +444,8 @@ def convert(lines, end_format):
 # for categories
 re_Declaration = re.compile(b'^#\\s*\\Declare\\w+Class.*$')
 re_ExtractCategory = 
re.compile(b'^(#\\s*\\Declare\\w+Class(?:\\[[^]]*?\\])?){([^(]+?)\\s+\\(([^)]+?)\\)\\s*}\\s*$')
-ConvDict = {"article": "Articles", "book" : "Books", "letter" : "Letters", 
"report": "Reports",
-"presentation" : "Presentations", "curriculum vitae" : 
"Curricula Vitae", "handout" : "Handouts"}
+ConvDict = {b"article": b"Articles", b"book": b"Books", b"letter": 
b"Letters", b"report": b"Reports",
+b"presentation": b"Presentations", b"curriculum vitae": 
b"Curricula Vitae", b"handout": b"Handouts"}
 # Arguments
 re_OptArgs = re.compile(b'^(\\s*)OptionalArgs(\\s+)(\\d+)\\D*$', 
re.IGNORECASE)
 re_ReqArgs = re.compile(b'^(\\s*)RequiredArgs(\\s+)(\\d+)\\D*$', 
re.IGNORECASE)
@@ -615,7 +615,7 @@ def convert(lines, end_format):
 continue
 col  = match.group(2)
 if col == "collapsable":
-lines[i] = match.group(1) + "collapsible"
+lines[i] = match.group(1) + b"collapsible"
 i += 1
 continue
 
@@ -833,7 +833,7 @@ def convert(lines, end_format):
 # Insert the required number of arguments at the end of the style 
definition
 match = re_End.match(lines[i])
 if match:
-newarg = ['']
+newarg = [b'']
 # First the optionals (this is the required order pre 2.1)
 if opts > 0:
 if opts == 1:
@@ -1283,7 +1283,7 @@ def convert(lines, end_format):
 if latextype == b"item_environment" and label.lower() == 
b"counter_enumi":
 lines[labeltype_line] = 
re_LabelType.sub(b'\\1\\2\\3Enumerate', lines[labeltype_line])
 # Don't add the LabelCounter line later
-counter = ""
+counter = b""
 
 # Replace
 #
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Make all exectuable python scripts use python3

2020-10-30 Thread José Matos
commit c539b57a0e6d21a5b2757d4c8ad522948eb52366
Author: José Matos 
Date:   Fri Oct 30 18:46:13 2020 +

Make all exectuable python scripts use python3

This is only relevant on linux/unix if running the scripts from a shell.
These two were the last where the call still used an unversioned python.

This has no reflex on the way that lyx calls the scripts or the python
version used since the #! "shebang line" is ignored.
---
 lib/configure.py   |2 +-
 lib/scripts/gnuplot2pdf.py |2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index f7941b5..ca62e65 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -1,4 +1,4 @@
-#! /usr/bin/python
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 #
 # file configure.py
diff --git a/lib/scripts/gnuplot2pdf.py b/lib/scripts/gnuplot2pdf.py
index 33fe767..3ada4f5 100755
--- a/lib/scripts/gnuplot2pdf.py
+++ b/lib/scripts/gnuplot2pdf.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python3
 
 from subprocess import Popen, PIPE
 from sys import argv, stderr, exit
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Make script compliant with python 3

2020-09-28 Thread José Matos
commit 0bda5e5b8dcae121893f50af8759ee501224b510
Author: José Matos 
Date:   Mon Sep 28 11:13:47 2020 +0100

Make script compliant with python 3
---
 lib/scripts/listerrors |   18 +-
 1 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/lib/scripts/listerrors b/lib/scripts/listerrors
index 7d6995c..92ffa2c 100755
--- a/lib/scripts/listerrors
+++ b/lib/scripts/listerrors
@@ -12,6 +12,7 @@
 
 Expects to read from stdin and output to stdout.
 """
+from __future__ import print_function
 
 __author__ = "Kayvan A. Sylvan "
 __date__ = "$Date: 2003/10/13 09:50:10 $"
@@ -22,7 +23,6 @@ Bernard Michael Hurley 
 modifications to original listerrors."""
 __copyright__ = "Copyright 2002 - Kayvan A. Sylvan."
 
-from __future__ import print_function
 import sys, string
 
 def write_error(msg, tool = "noweb", line_number = 1):
@@ -75,8 +75,8 @@ def noweb_try(line):
 
   Returns 1 on success, 0 otherwise. Outputs on stdout."""
   retval = 0
-  if string.find(line, ": unescaped << in documentation chunk") != -1:
-line_parts = string.split(line, ':')
+  if line.find(": unescaped << in documentation chunk") != -1:
+line_parts = line.split(':')
 num_str = line_parts[1]
 num_len = len(num_str)
 i = 0
@@ -85,9 +85,9 @@ def noweb_try(line):
   write_error(":" + line_parts[2], "noweb", int(num_str))
   retval = 1
   if (not retval):
-left = string.find(line, "<<")
+left = line.find("<<")
 if (left != -1) and ((left + 2) < len(line)) and \
-   (string.find(line[left+2:], ">>") != -1):
+   (line[left+2:].find(">>") != -1):
   write_error(line, "noweb");
   retval = 1;
   if (not retval):
@@ -104,7 +104,7 @@ def noweb_try(line):
   "This can't happen:",
   "non-numeric line number in")
 for msg in msgs_to_try:
-  if string.find(line, msg) != -1:
+  if line.find(msg) != -1:
 write_error(line, "noweb")
 retval = 1
 break
@@ -115,7 +115,7 @@ def gcc_try(line):
 
   Returns 1 on success, 0 otherwise. Outputs on stdout."""
   retval = 0
-  first_space = string.find(line, ' ')
+  first_space = line.find(' ')
   if first_space > 1: # The smallest would be "X: "
 if line[first_space - 1] == ':':
   header_to_see = line[:first_space - 1]
@@ -147,8 +147,8 @@ def xlc_try(line):
   Returns 1 on success, 0 otherwise. Outputs on stdout."""
   retval = 0
   if line[0] == '"': # This is the first character of all xlc errors
-next_quote = string.find(line, '"', 1)
-first_space = string.find(line, ' ')
+next_quote = line.find('"', 1)
+first_space = line.find(' ')
 if (next_quote != -1) and (first_space > next_quote): # no space inisde 
quotes
   if line[first_space - 1:first_space + 6] == ", line ":
 num_start = num_end = first_space + 6
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Simplify reversion code (keeping the same output)

2020-09-21 Thread José Matos
commit 4419f6e0591bdaed0f6cebfd6b5d108363628f88
Author: José Matos 
Date:   Mon Sep 21 14:36:52 2020 +0100

Simplify reversion code (keeping the same output)
---
 lib/lyx2lyx/lyx_2_4.py |7 ++-
 1 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/lib/lyx2lyx/lyx_2_4.py b/lib/lyx2lyx/lyx_2_4.py
index 9c72406..74f17cf 100644
--- a/lib/lyx2lyx/lyx_2_4.py
+++ b/lib/lyx2lyx/lyx_2_4.py
@@ -3506,16 +3506,13 @@ def revert_totalheight(document):
 special = m.group(1)
 mspecial = special.split(',')
 for spc in mspecial:
-if spc[:7] == "height=":
+if spc.startswith("height="):
 oldheight = spc.split('=')[1]
 ms = rxx.search(oldheight)
 if ms:
-oldval = ms.group(1)
 oldunit = ms.group(2)
-if len(oldval) > 1 and oldval[1] == ".":
-oldval = "0" + oldval
 if oldunit in list(relative_heights.keys()):
-oldval = str(float(oldval) * 100)
+oldval = str(float(ms.group(1)) * 100)
 oldunit = relative_heights[oldunit]
 oldheight = oldval + oldunit
 mspecial.remove(spc)
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Avoid crash when string only has one char

2020-09-17 Thread José Matos
commit 5154400b641a46b27a35e90d97aca725e541a14a
Author: José Matos 
Date:   Fri Sep 18 01:09:36 2020 +0100

Avoid crash when string only has one char
---
 lib/lyx2lyx/lyx_2_4.py |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/lyx2lyx/lyx_2_4.py b/lib/lyx2lyx/lyx_2_4.py
index 3fd876c..9c72406 100644
--- a/lib/lyx2lyx/lyx_2_4.py
+++ b/lib/lyx2lyx/lyx_2_4.py
@@ -3512,7 +3512,7 @@ def revert_totalheight(document):
 if ms:
 oldval = ms.group(1)
 oldunit = ms.group(2)
-if oldval[1] == ".":
+if len(oldval) > 1 and oldval[1] == ".":
 oldval = "0" + oldval
 if oldunit in list(relative_heights.keys()):
 oldval = str(float(oldval) * 100)
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Whitespace cleanup

2020-09-17 Thread José Matos
commit d0a27e3afc3444d10694df10dc4cb8d604287765
Author: José Matos 
Date:   Fri Sep 18 01:07:39 2020 +0100

Whitespace cleanup
---
 lib/lyx2lyx/lyx_2_4.py |   26 +-
 1 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/lib/lyx2lyx/lyx_2_4.py b/lib/lyx2lyx/lyx_2_4.py
index e1a8e6a..3fd876c 100644
--- a/lib/lyx2lyx/lyx_2_4.py
+++ b/lib/lyx2lyx/lyx_2_4.py
@@ -3777,7 +3777,7 @@ def revert_counter_inset(document):
 ert = put_cmd_in_ert("\\setcounter{%s}{\\value{%s}}" % (cnt, 
savecnt))
 else:
 document.warning("Unknown counter command `%s' in inset at line 
%d!" % (cnt, i))
-
+
 if ert:
 document.body[i : j + 1] = ert
 i += 1
@@ -3804,7 +3804,7 @@ def revert_ams_spaces(document):
 subst = put_cmd_in_ert(inset)
 document.body[i : end + 1] = subst
 Found = True
-  
+
 if Found == True:
 # load amsmath in the preamble if not already loaded
 i = find_token(document.header, "\\use_package amsmath 2", 0)
@@ -3815,18 +3815,18 @@ def revert_ams_spaces(document):
 
 def convert_parskip(document):
 " Move old parskip settings to preamble "
-
+
 i = find_token(document.header, "\\paragraph_separation skip", 0)
 if i == -1:
 return
-
+
 j = find_token(document.header, "\\defskip", 0)
 if j == -1:
 document.warning("Malformed LyX document! Missing \\defskip.")
 return
-
+
 val = get_value(document.header, "\\defskip", j)
-
+
 skipval = "\\medskipamount"
 if val == "smallskip" or val == "medskip" or val == "bigskip":
 skipval = "\\" + val + "amount"
@@ -3834,25 +3834,25 @@ def convert_parskip(document):
 skipval = val
 
 add_to_preamble(document, ["\\setlength{\\parskip}{" + skipval + "}", 
"\\setlength{\\parindent}{0pt}"])
-
+
 document.header[i] = "\\paragraph_separation indent"
 document.header[j] = "\\paragraph_indentation default"
 
 
 def revert_parskip(document):
 " Revert new parskip settings to preamble "
-
+
 i = find_token(document.header, "\\paragraph_separation skip", 0)
 if i == -1:
 return
-
+
 j = find_token(document.header, "\\defskip", 0)
 if j == -1:
 document.warning("Malformed LyX document! Missing \\defskip.")
 return
-
+
 val = get_value(document.header, "\\defskip", j)
-
+
 skipval = ""
 if val == "smallskip" or val == "medskip" or val == "bigskip":
 skipval = "[skip=\\" + val + "amount]"
@@ -3860,9 +3860,9 @@ def revert_parskip(document):
 skipval = "[skip=\\baselineskip]"
 elif val != "halfline":
 skipval = "[skip={" + val + "}]"
-
+
 add_to_preamble(document, ["\\usepackage" + skipval + "{parskip}"])
-
+
 document.header[i] = "\\paragraph_separation indent"
 document.header[j] = "\\paragraph_indentation default"
 
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/2.3.x] Consider file encoding for modules in reconfigure

2020-08-15 Thread José Matos
commit f5489036305eae3e63e721e2714746bd48bac1dc
Author: José Matos 
Date:   Sat Aug 15 21:02:38 2020 +0100

Consider file encoding for modules in reconfigure

If the modules are not in utf8 then we warn and skip that file
like it happens for layout files.

This a port of commit a8094051c1ae9c546c76bb0d3300d83e6cdbadef to stable.
---
 lib/configure.py |   18 +-
 status.23x   |6 +-
 2 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index a5a49b7..7fee68d 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -228,7 +228,7 @@ def checkTeXPaths():
 if sys.version_info[0] < 3:
 inpname = shortPath(unicode(tmpfname, encoding)).replace('\\', 
'/')
 else:
-inpname = shortPath(tmpfname).replace('\\', '/') 
+inpname = shortPath(tmpfname).replace('\\', '/')
 else:
 inpname = cmdOutput('cygpath -m ' + tmpfname)
 logname = os.path.basename(re.sub("(?i).ltx", ".log", inpname))
@@ -1174,7 +1174,7 @@ def checkConverterEntries():
 \converter svgz   png"%%"""'''],
 path = ['', inkscape_path])
 #
-checkProg('Gnuplot', ['gnuplot'], 
+checkProg('Gnuplot', ['gnuplot'],
 rc_entry = [ r'''\Format gnuplot "gp, gnuplot""Gnuplot" "" 
"" ""  "vector" "text/plain"
 \converter gnuplot  pdf6  "python -tt $$s/scripts/gnuplot2pdf.py $$i 
$$o""needauth"''' ])
 #
@@ -1584,9 +1584,17 @@ def checkModulesConfig():
   continue
 
   seen.append(filename)
-  retval = processModuleFile(file, filename, bool_docbook)
-  if retval:
-  tx.write(retval)
+  try:
+  retval = processModuleFile(file, filename, bool_docbook)
+  if retval:
+  tx.write(retval)
+  except UnicodeDecodeError:
+  logger.warning("**\n"
+ "Module file '%s'\n"
+ "cannot be decoded in utf-8.\n"
+ "Please check if the file has the correct encoding.\n"
+ "Skipping this file!\n"
+ "**" 
% filename)
   tx.close()
   logger.info('\tdone')
 
diff --git a/status.23x b/status.23x
index f8420a1..0a75945 100644
--- a/status.23x
+++ b/status.23x
@@ -88,7 +88,7 @@ What's new
 
 - Do not allow editing of math package list in Document > Settings > Math 
Options,
   thus preventing a crash (bug 11931).
-  
+
 - Maintain standard layout for separating paragraphs when switching layouts 
(bug 11936).
 
 
@@ -114,3 +114,7 @@ What's new
 
 - Wininstaller, fix bug where previous installations, with emergency version
   greater than 1, could not be seen in registry on overinstall
+
+- Do not break configuration if a module file is not decodable (file
+  not encoded in utf-8). Rather than that, issue a warning and skip
+  that file (a variant of bug 11736).
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Consider file encoding for modules in reconfigure

2020-08-15 Thread José Matos
commit a8094051c1ae9c546c76bb0d3300d83e6cdbadef
Author: José Matos 
Date:   Sat Aug 15 17:49:23 2020 +0100

Consider file encoding for modules in reconfigure

If the modules are not in utf8 then we warn and skip that file
like it happens for layout files.

It would be nice in both cases to have a warn in the gui and not only in 
the config.log
---
 lib/configure.py |   14 +++---
 1 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index b638f93..2b4d61f 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -1562,9 +1562,17 @@ def checkModulesConfig():
   continue
 
   seen.append(filename)
-  retval = processModuleFile(file, filename)
-  if retval:
-  tx.write(retval)
+  try:
+  retval = processModuleFile(file, filename)
+  if retval:
+  tx.write(retval)
+  except UnicodeDecodeError:
+  logger.warning("**\n"
+ "Module file '%s'\n"
+ "cannot be decoded in utf-8.\n"
+ "Please check if the file has the correct encoding.\n"
+ "Skipping this file!\n"
+ "**" 
% filename)
   tx.close()
   logger.info('\tdone')
 
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Trim endline whitespace

2020-08-15 Thread José Matos
commit 5a212823d7bf103b59be0c79f72448a718186494
Author: José Matos 
Date:   Sat Aug 15 16:36:34 2020 +0100

Trim endline whitespace
---
 lib/configure.py |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index cea9bdc..b638f93 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -217,7 +217,7 @@ def checkTeXPaths():
 if sys.version_info[0] < 3:
 inpname = shortPath(unicode(tmpfname, encoding)).replace('\\', 
'/')
 else:
-inpname = shortPath(tmpfname).replace('\\', '/') 
+inpname = shortPath(tmpfname).replace('\\', '/')
 else:
 inpname = cmdOutput('cygpath -m ' + tmpfname)
 logname = os.path.basename(re.sub("(?i).ltx", ".log", inpname))
@@ -1157,7 +1157,7 @@ def checkConverterEntries():
 \converter svgz   png"%%"""'''],
 path = ['', inkscape_path])
 #
-checkProg('Gnuplot', ['gnuplot'], 
+checkProg('Gnuplot', ['gnuplot'],
 rc_entry = [ r'''\Format gnuplot "gp, gnuplot""Gnuplot" "" 
"" ""  "vector" "text/plain"
 \converter gnuplot  pdf6  "python -tt $$s/scripts/gnuplot2pdf.py $$i 
$$o""needauth"''' ])
 #
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/2.3.x] Fix typo in lyx2lyx conversion to 2.1 (taken from master)

2019-12-28 Thread José Matos
commit 78db6194c6f717cf4f72072d232f8aef5df13528
Author: José Matos 
Date:   Sat Dec 28 14:39:49 2019 +

Fix typo in lyx2lyx conversion to 2.1 (taken from master)
---
 lib/lyx2lyx/lyx_2_1.py |2 +-
 status.23x |3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/lyx2lyx/lyx_2_1.py b/lib/lyx2lyx/lyx_2_1.py
index b507151..e831a7e 100644
--- a/lib/lyx2lyx/lyx_2_1.py
+++ b/lib/lyx2lyx/lyx_2_1.py
@@ -180,7 +180,7 @@ def convert_TeX_brace_to_Argument(document, line, n, nmax, 
inset, environment, o
 else:
   beginBrace = find_token(document.body, "{", endBrace, end_layout)
 # assure that the ERTs are consecutive (11 or 12 depending if 
there is a space between the ERTs or not)
-if beginBrance != -1 and (beginBrace == endBrace + 11 or 
beginBrace == endBrace + 12):
+if beginBrace != -1 and (beginBrace == endBrace + 11 or beginBrace 
== endBrace + 12):
   end = find_token(document.body, "\\end_inset", beginBrace)
   document.body[lineERT : end + 1] = ["\\end_layout", "", 
"\\end_inset"]
   if loop == 1:
diff --git a/status.23x b/status.23x
index a2d4b00..b08b265 100644
--- a/status.23x
+++ b/status.23x
@@ -182,7 +182,8 @@ What's new
 
 * LYX2LYX
 
-
+- Fix typo in the code that converts from file formats older than lyx 2.1
+  (thanks to Salvatore Falco).
 
 
 * TEX2LYX
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Fix typo (thanks to Salvatore Falco for the fix).

2019-11-04 Thread José Matos
commit 668aa17ad7567b5969796b9a7a8f57f57a646e92
Author: José Matos 
Date:   Mon Nov 4 12:12:23 2019 +

Fix typo (thanks to Salvatore Falco for the fix).
---
 lib/lyx2lyx/lyx_2_1.py |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/lyx2lyx/lyx_2_1.py b/lib/lyx2lyx/lyx_2_1.py
index 8838a64..133d500 100644
--- a/lib/lyx2lyx/lyx_2_1.py
+++ b/lib/lyx2lyx/lyx_2_1.py
@@ -181,7 +181,7 @@ def convert_TeX_brace_to_Argument(document, line, n, nmax, 
inset, environment, o
 else:
   beginBrace = find_token(document.body, "{", endBrace, end_layout)
 # assure that the ERTs are consecutive (11 or 12 depending if 
there is a space between the ERTs or not)
-if beginBrance != -1 and (beginBrace == endBrace + 11 or 
beginBrace == endBrace + 12):
+if beginBrace != -1 and (beginBrace == endBrace + 11 or beginBrace 
== endBrace + 12):
   end = find_token(document.body, "\\end_inset", beginBrace)
   document.body[lineERT : end + 1] = ["\\end_layout", "", 
"\\end_inset"]
   if loop == 1:
-- 
lyx-cvs mailing list
lyx-cvs@lists.lyx.org
http://lists.lyx.org/mailman/listinfo/lyx-cvs


[LyX/master] Divide the python detection in three functions, with a clear delegation of responsibilities

2019-06-08 Thread José Matos
commit ad96fd835b9719a5af8f742a17156c718320e852
Author: José Matos 
Date:   Sat Jun 8 12:49:30 2019 +0100

Divide the python detection in three functions, with a clear delegation of 
responsibilities

python23_call: determines if the binary given is appropriate and adds the 
necessary calling options

find_python_binary: get a list of candidates and choose the right one using 
python23_call

python: returns the name of the python interpreter that can be found on 
PATH, using find_python_binary
---
 src/support/os.cpp |  103 +++-
 1 files changed, 54 insertions(+), 49 deletions(-)

diff --git a/src/support/os.cpp b/src/support/os.cpp
index 58d9d43..9840ad3 100644
--- a/src/support/os.cpp
+++ b/src/support/os.cpp
@@ -46,13 +46,16 @@ int timeout_min()
 }
 
 
-static string const python23(string const & binary, bool verbose = false)
+static string const python23_call(string const & binary, bool verbose = false)
 {
const string version_info = " -c 'from __future__ import 
print_function;import sys; print(sys.version_info[:2], end=\"\")'";
static regex const python_reg("\\((\\d*), (\\d*)\\)");
+   // Default to "python" if no binary is given.
+   if (binary.empty())
+   return "python -tt";
+
if (verbose)
lyxerr << "Examining " << binary << "\n";
-
// Check whether this is a python 2 or 3 binary.
cmd_ret const out = runCommand(binary + version_info);
 
@@ -67,11 +70,13 @@ static string const python23(string const & binary, bool 
verbose = false)
 
if (verbose)
lyxerr << "Found Python " << out.second << "\n";
-   return binary;
+   // Add the -tt switch so that mixed tab/whitespace
+   // indentation is an error
+   return binary + " -tt";
 }
 
 
-string const python(bool reset)
+static string const find_python_binary()
 {
// This function takes inspiration from PEP 394 and PEP 397
// PEP 394 -- The "python" Command on Unix-Like Systems
@@ -80,67 +85,67 @@ string const python(bool reset)
// https://www.python.org/dev/peps/pep-0397/
 
// Check whether python3 in PATH is the right one.
-   static string command = python23("python3");
-   // FIXME THREAD
-   if (reset) {
-   command = python23("python3");
-   }
+   string command = python23_call("python3");
+   if (!command.empty())
+   return command;
 
// python3 does not exists, let us try to find python3.x in PATH
// the search is probably broader than required
// but we are trying hard to find a valid python binary
-   if (command.empty()) {
-   vector const path = getEnvPath("PATH");
-   lyxerr << "Looking for python 3.x ...\n";
-   for (auto bin: path) {
-   QString const dir = toqstr(bin);
-   string const localdir = dir.toLocal8Bit().constData();
-   QDir qdir(dir);
-   qdir.setFilter(QDir::Files | QDir::Executable);
-   QStringList list = 
qdir.entryList(QStringList("python3*"));
-   for (int i = 0; i < list.size() && command.empty(); 
++i) {
-   string const binary = addName(localdir,
-   list.at(i).toLocal8Bit().constData());
-   command = python23(binary, true);
-   }
+   vector const path = getEnvPath("PATH");
+   lyxerr << "Looking for python 3.x ...\n";
+   for (auto bin: path) {
+   QString const dir = toqstr(bin);
+   string const localdir = dir.toLocal8Bit().constData();
+   QDir qdir(dir);
+   qdir.setFilter(QDir::Files | QDir::Executable);
+   QStringList list = qdir.entryList(QStringList("python3*"));
+   for (auto bin: list) {
+   string const binary = addName(localdir,
+   bin.toLocal8Bit().constData());
+   command = python23_call(binary, true);
+   if (!command.empty())
+   return command;
}
-
}
+
// python 3 was not found let us look for python 2
-   if (command.empty())
-   command = python23("python2");
+   command = python23_call("python2");
+   if (!command.empty())
+   return command;
 
// python2 does not exists, let us try to find python2.x in PATH
// the search is probably broader than required
// 

[LyX/master] Move python related functions together to make it easy to read the code.

2019-06-08 Thread José Matos
commit 02ee471d9d686499ba9d5d24609394388d0754bf
Author: José Matos 
Date:   Sat Jun 8 12:22:26 2019 +0100

Move python related functions together to make it easy to read the code.
---
 src/support/os.cpp |   12 ++--
 1 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/src/support/os.cpp b/src/support/os.cpp
index d435575..58d9d43 100644
--- a/src/support/os.cpp
+++ b/src/support/os.cpp
@@ -40,6 +40,12 @@ namespace lyx {
 namespace support {
 namespace os {
 
+int timeout_min()
+{
+   return 3;
+}
+
+
 static string const python23(string const & binary, bool verbose = false)
 {
const string version_info = " -c 'from __future__ import 
print_function;import sys; print(sys.version_info[:2], end=\"\")'";
@@ -65,12 +71,6 @@ static string const python23(string const & binary, bool 
verbose = false)
 }
 
 
-int timeout_min()
-{
-   return 3;
-}
-
-
 string const python(bool reset)
 {
// This function takes inspiration from PEP 394 and PEP 397


[LyX/master] Remove alternative operator representations (and, or) and improve comments for python binary detection.

2019-06-08 Thread José Matos
commit 35c299f86d35385b5bf3729682eeb8d2eb1c78a4
Author: José Matos 
Date:   Sat Jun 8 09:15:03 2019 +0100

Remove alternative operator representations (and,or) and improve comments 
for python binary detection.
---
 src/support/os.cpp |   26 --
 1 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/src/support/os.cpp b/src/support/os.cpp
index 936e759..d435575 100644
--- a/src/support/os.cpp
+++ b/src/support/os.cpp
@@ -56,7 +56,7 @@ static string const python23(string const & binary, bool 
verbose = false)
 
int major = convert(sm.str(1));
int minor = convert(sm.str(2));
-   if((major == 2 and minor < 7) or (major == 3 and minor < 4))
+   if((major == 2 && minor < 7) || (major == 3 && minor < 4))
return string();
 
if (verbose)
@@ -73,6 +73,12 @@ int timeout_min()
 
 string const python(bool reset)
 {
+   // This function takes inspiration from PEP 394 and PEP 397
+   // PEP 394 -- The "python" Command on Unix-Like Systems
+   // https://www.python.org/dev/peps/pep-0394/
+   // PEP 397 -- Python launcher for Windows
+   // https://www.python.org/dev/peps/pep-0397/
+
// Check whether python3 in PATH is the right one.
static string command = python23("python3");
// FIXME THREAD
@@ -80,10 +86,10 @@ string const python(bool reset)
command = python23("python3");
}
 
-   // python3 does not exists, let us try python3.x
+   // python3 does not exists, let us try to find python3.x in PATH
+   // the search is probably broader than required
+   // but we are trying hard to find a valid python binary
if (command.empty()) {
-   // It was not, so check whether we can find it elsewhere in
-   // PATH, maybe with some suffix appended.
vector const path = getEnvPath("PATH");
lyxerr << "Looking for python 3.x ...\n";
for (auto bin: path) {
@@ -100,16 +106,16 @@ string const python(bool reset)
}
 
}
-   // python 3 not found let us look for python 2
+   // python 3 was not found let us look for python 2
if (command.empty())
command = python23("python2");
 
-   // python2 does not exists, let us try python2.x
+   // python2 does not exists, let us try to find python2.x in PATH
+   // the search is probably broader than required
+   // but we are trying hard to find a valid python binary
if (command.empty()) {
-   // It was not, so check whether we can find it elsewhere in
-   // PATH, maybe with some suffix appended.
vector const path = getEnvPath("PATH");
-   lyxerr << "Looking for python 3.x ...\n";
+   lyxerr << "Looking for python 2.x ...\n";
for (auto bin: path) {
QString const dir = toqstr(bin);
string const localdir = dir.toLocal8Bit().constData();
@@ -126,7 +132,7 @@ string const python(bool reset)
}
 
// Default to "python" if no usable binary was found.
-   // If this happens all hope is lost that there is a sane system
+   // If this happens all hope is lost that this is a sane system
if (command.empty()) {
lyxerr << "Warning: No python v2.x or 3.x binary found.\n";
command = "python";


[LyX/master] Make the lyx2lyx code compatible between python2 and python3.

2019-06-05 Thread José Matos
commit 58734edff688ae81461fffbc1f3de146d4883833
Author: José Matos 
Date:   Wed Jun 5 14:06:09 2019 +0100

Make the lyx2lyx code compatible between python2 and python3.

FWIW this code is important for very old versions of lyx, older than 1.1.5 
(released 19 years ago - 2000/06/06).

Funny fact of the day, byte strings do not behave as regular strings in 
python3 when taking and index.
To get a sub-string we need to pass a range, a integer index will not work 
as it happens in a regular string:

$ ipython3
...
In [30]: line
Out[30]: b'#This file was created by  Tue Jan 25 10:36:51 2000'

In [31]: line[0]
Out[31]: 35

In [32]: line[0:1]
Out[32]: b'#'

The range notations works for both byte and regular strings in python 3, 
and it also works in python 2.
Thus the change is simple and effective. In any case I should confess that 
I was quite surprised by this. :-)
---
 lib/lyx2lyx/LyX.py |   14 +++---
 1 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/lib/lyx2lyx/LyX.py b/lib/lyx2lyx/LyX.py
index 245cc6e..ffaf4f0 100644
--- a/lib/lyx2lyx/LyX.py
+++ b/lib/lyx2lyx/LyX.py
@@ -71,8 +71,8 @@ def minor_versions(major, last_minor_version):
 # Regular expressions used
 format_re = re.compile(r"(\d)[\.,]?(\d\d)")
 fileformat = re.compile(r"\\lyxformat\s*(\S*)")
-original_version = re.compile(r".*?LyX ([\d.]*)")
-original_tex2lyx_version = re.compile(r".*?tex2lyx ([\d.]*)")
+original_version = re.compile(b".*?LyX ([\\d.]*)")
+original_tex2lyx_version = re.compile(b".*?tex2lyx ([\\d.]*)")
 
 ##
 # file format information:
@@ -519,10 +519,10 @@ class LyX_base:
 file, returns the most likely value, or None otherwise."""
 
 for line in self.header:
-if line[0] != "#":
+if line[0:1] != b"#":
 return None
 
-line = line.replace("fix",".")
+line = line.replace(b"fix",b".")
 # need to test original_tex2lyx_version first because tex2lyx
 # writes "#LyX file created by tex2lyx 2.2"
 result = original_tex2lyx_version.match(line)
@@ -530,14 +530,14 @@ class LyX_base:
 result = original_version.match(line)
 if result:
 # Special know cases: reLyX and KLyX
-if line.find("reLyX") != -1 or line.find("KLyX") != -1:
+if line.find(b"reLyX") != -1 or line.find(b"KLyX") != -1:
 return "0.12"
 if result:
 res = result.group(1)
 if not res:
 self.warning(line)
 #self.warning("Version %s" % result.group(1))
-return res
+return res.decode('ascii') if not PY2 else res
 self.warning(str(self.header[:2]))
 return None
 
@@ -642,7 +642,7 @@ class LyX_base:
   self.warning("Malformed LyX document: No \\textclass!!")
   return
 i = j = tclass + 1
-  else: 
+  else:
 j = find_token(self.header, "\\end_modules", i)
 if j == -1:
 self.warning("(set_module_list) Malformed LyX document: No 
\\end_modules.")


[LyX/master] Remove code that is redudant.

2019-06-04 Thread José Matos
commit 4268a9e8121b08f06706d733e8488e163286eab8
Author: José Matos 
Date:   Tue Jun 4 18:26:13 2019 +0100

Remove code that is redudant.

Both for python2 and python3 output is always a string.
---
 lib/scripts/convertDefault.py |7 ---
 1 files changed, 0 insertions(+), 7 deletions(-)

diff --git a/lib/scripts/convertDefault.py b/lib/scripts/convertDefault.py
index cd69709..c927713 100644
--- a/lib/scripts/convertDefault.py
+++ b/lib/scripts/convertDefault.py
@@ -19,8 +19,6 @@
 from __future__ import print_function
 import os, re, sys
 
-PY2 = sys.version_info[0] == 2
-
 # We may need some extra options only supported by recent convert versions
 re_version = re.compile(r'^Version:.*ImageMagick\s*(\d*)\.(\d*)\.(\d*).*$')
 # imagemagick 7
@@ -34,11 +32,6 @@ if fout.close() != None:
 fout = os.popen('convert -version 2>&1')
 output = fout.readline()
 fout.close()
-if not PY2:
-# Ensure we have a (unicode) string object in Python3
-# (not required for version >= 3.5).
-# FIXME: Check whether this is required with any supported 3.x version!
-output = str(output)
 
 version = re_version.match(output)
 


[LyX/master] Fix the remaing issues with comparisons with objects of different types.

2019-06-03 Thread José Matos
commit 639b5da1afe87ffe2fa4df3f738b838d6637f11a
Author: José Matos 
Date:   Mon Jun 3 19:07:20 2019 +0100

Fix the remaing issues with comparisons with objects of different types.

In python it is possible to compare tuples with a lexicographic order.

Take advantage of that since there is no need to resort to the C-trick of 
converting a version in hex format.

We need to set a dummy version in case we are using ImageMagick to ensure 
that version is always an integer 3-tuple.
---
 lib/scripts/convertDefault.py |8 +---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/lib/scripts/convertDefault.py b/lib/scripts/convertDefault.py
index eee533d..cd69709 100644
--- a/lib/scripts/convertDefault.py
+++ b/lib/scripts/convertDefault.py
@@ -50,7 +50,7 @@ if version != None:
 major = int(version.group(1))
 minor = int(version.group(2))
 patch = int(version.group(3))
-version = hex(major * 65536 + minor * 256 + patch)
+version = (major, minor, patch)
 im = True
 else:
 # Try GraphicsMagick
@@ -58,6 +58,8 @@ else:
 version = re_version.match(output)
 if version != None:
 gm = True
+# we need version to be a valid integer 3-tuple
+version = (1,0,0)
 
 # IM >= 5.5.8 separates options for source and target files
 # See http://www.imagemagick.org/Usage/basics/#why
@@ -68,11 +70,11 @@ elif sys.platform == 'darwin':
 command = 'lyxconvert'
 
 # If supported, add the -define option for pdf source formats
-if sys.argv[1] == 'pdf' and (version >= 0x060206 or gm):
+if sys.argv[1] == 'pdf' and (version >= (6,2,6) or gm):
 sopts = '-define pdf:use-cropbox=true ' + sopts
 
 # If supported, add the -flatten option for ppm target formats (see bug 4749)
-if sys.argv[3] == 'ppm' and (im and version >= "0x060305" or gm):
+if sys.argv[3] == 'ppm' and (im and version >= (6,3,5) or gm):
 topts = '-flatten'
 
 # print (command, sys.argv[2], sys.argv[4], file= sys.stdout)


[LyX/master] Fix bug in python comparison.

2019-06-03 Thread José Matos
commit a8937b53ecb054e140036b742e0c529bf8c9937a
Author: José Matos 
Date:   Mon Jun 3 17:15:09 2019 +0100

Fix bug in python comparison.

It worked in python2 but not the way the authors imagined. Because hex 
always returns a string.

From python2:
>>> 1 > "2"
False
>>> "2" > 1
True
>>> "1" > 2
True

The rational is that an integer is always smaller than a string.

In python 3 this because it does not make sense to compare objects of 
different types.
---
 lib/scripts/convertDefault.py |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/scripts/convertDefault.py b/lib/scripts/convertDefault.py
index 418b553..eee533d 100644
--- a/lib/scripts/convertDefault.py
+++ b/lib/scripts/convertDefault.py
@@ -72,7 +72,7 @@ if sys.argv[1] == 'pdf' and (version >= 0x060206 or gm):
 sopts = '-define pdf:use-cropbox=true ' + sopts
 
 # If supported, add the -flatten option for ppm target formats (see bug 4749)
-if sys.argv[3] == 'ppm' and (im and version >= 0x060305 or gm):
+if sys.argv[3] == 'ppm' and (im and version >= "0x060305" or gm):
 topts = '-flatten'
 
 # print (command, sys.argv[2], sys.argv[4], file= sys.stdout)


[LyX/master] Make verbose switch consistent.

2019-06-03 Thread José Matos
commit f9bf53f35abf88aa6dd25c7413e73db105464e17
Author: José Matos 
Date:   Mon Jun 3 07:31:05 2019 +0100

Make verbose switch consistent.

Someday we should probably unify these two switches. Because the debug 
switch is verbose
and the verbose switch is mostly used for debuging.
---
 lib/scripts/lyxpreview2bitmap.py |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index 236010f..eee000d 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -378,8 +378,8 @@ def main(argv):
 if len(dir) != 0:
 os.chdir(dir)
 
-if lyxpreview_tools.debug:
-f_out = open('debug.txt', 'a')
+if lyxpreview_tools.verbose:
+f_out = open('verbose.txt', 'a')
 sys.stdout = f_out
 sys.stderr = f_out
 


[LyX/master] Import lyxpreview_tools directly and not only inside *if* conditions.

2019-06-02 Thread José Matos
commit a810d779d78f2bc4f952321ca05692f9534e0fba
Author: José Matos 
Date:   Sun Jun 2 17:24:40 2019 +0100

Import lyxpreview_tools directly and not only inside *if* conditions.

This is related to the bug #11457 saga and it was my fault.

The debug files should be written only be on if the argument --debug is 
passed and not --verbose as it was done by mistake.
---
 lib/scripts/lyxpreview2bitmap.py |6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index b02c522..236010f 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -79,6 +79,8 @@ from __future__ import print_function
 
 import getopt, glob, os, re, shutil, sys, tempfile
 
+import lyxpreview_tools
+
 from legacy_lyxpreview2ppm import extract_resolution, legacy_conversion_step1
 
 from lyxpreview_tools import bibtex_commands, check_latex_log, copyfileobj, \
@@ -343,7 +345,6 @@ def main(argv):
 elif opt == "--bg":
 bg_color = val
 elif opt in ("-d", "--debug"):
-import lyxpreview_tools
 lyxpreview_tools.debug = True
 elif opt == "--dpi":
 try:
@@ -361,7 +362,6 @@ def main(argv):
 elif opt in ("--png", "--ppm"):
 output_format = opt[2:]
 elif opt in ("-v", "--verbose"):
-import lyxpreview_tools
 lyxpreview_tools.verbose = True
 
 # Determine input file
@@ -378,7 +378,7 @@ def main(argv):
 if len(dir) != 0:
 os.chdir(dir)
 
-if lyxpreview_tools.verbose:
+if lyxpreview_tools.debug:
 f_out = open('debug.txt', 'a')
 sys.stdout = f_out
 sys.stderr = f_out


[LyX/master] Add further debug information fow windows (#11457)

2019-05-31 Thread José Matos
commit 050f0cbc91863ae608bba00c858b357f20198bcc
Author: José Matos 
Date:   Fri May 31 16:07:13 2019 +0100

Add further debug information fow windows (#11457)
---
 lib/scripts/lyxpreview2bitmap.py |2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index 6a7259a..b02c522 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -386,6 +386,8 @@ def main(argv):
 # Echo the settings
 progress("Running Python %s" % str(sys.version_info[:3]))
 progress("Starting %s..." % script_name)
+if os.name == "nt":
+progress("Use win32_modules: %d" % lyxpreview_tools.use_win32_modules)
 progress("Output format: %s" % output_format)
 progress("Foreground color: %s" % fg_color)
 progress("Background color: %s" % bg_color)


[LyX/master] Temporary hack to try to solve #11457 on windows

2019-05-31 Thread José Matos
commit 28f17333ffe73bd22b83ba2267ff1f3c57248b68
Author: José Matos 
Date:   Fri May 31 10:43:02 2019 +0100

Temporary hack to try to solve #11457 on windows

Redirect the standard output and standard error of the script to a file 
called debug.txt in the temporary directory.
---
 lib/scripts/lyxpreview2bitmap.py |   17 +++--
 1 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index ecefdaf..6a7259a 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -372,6 +372,17 @@ def main(argv):
 input_path = args[0]
 dir, latex_file = os.path.split(input_path)
 
+# Check for the input file
+if not os.path.exists(input_path):
+error('File "%s" not found.' % input_path)
+if len(dir) != 0:
+os.chdir(dir)
+
+if lyxpreview_tools.verbose:
+f_out = open('debug.txt', 'a')
+sys.stdout = f_out
+sys.stderr = f_out
+
 # Echo the settings
 progress("Running Python %s" % str(sys.version_info[:3]))
 progress("Starting %s..." % script_name)
@@ -381,12 +392,6 @@ def main(argv):
 progress("Resolution (dpi): %s" % dpi)
 progress("File to process: %s" % input_path)
 
-# Check for the input file
-if not os.path.exists(input_path):
-error('File "%s" not found.' % input_path)
-if len(dir) != 0:
-os.chdir(dir)
-
 # For python > 2 convert strings to bytes
 if not PY2:
 fg_color = bytes(fg_color, 'ascii')


[LyX/master] Attempt to fix #11457

2019-05-28 Thread José Matos
commit 8dd31803b16d1fb6b30772955188c3595437f64a
Author: José Matos 
Date:   Tue May 28 16:22:32 2019 +0100

Attempt to fix #11457

In python 3 the colors need to be strings and not bytes:

This was the equivalent of

>> print("%s" % b"1")
"b'1'"

since the colors were bytes the call to dvipng was something like

dvipng -Ttight -depth -height -D 115 -fg "b'rgb 0.937255 0.941176 
0.945098'" -bg "b'rgb 0.137255 0.149020 0.160784'"  "lyxpreviewxBJEqm.dvi"

Note the "b'rgb after both -fg and -bg that wrecked havoc and thus dvipng 
failed. That was the difference between python2 and python3 calls.
---
 lib/scripts/lyxpreview2bitmap.py |6 ++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index e4bf131..ecefdaf 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -373,6 +373,7 @@ def main(argv):
 dir, latex_file = os.path.split(input_path)
 
 # Echo the settings
+progress("Running Python %s" % str(sys.version_info[:3]))
 progress("Starting %s..." % script_name)
 progress("Output format: %s" % output_format)
 progress("Foreground color: %s" % fg_color)
@@ -394,6 +395,11 @@ def main(argv):
 fg_color_dvipng = make_texcolor(fg_color, False)
 bg_color_dvipng = make_texcolor(bg_color, False)
 
+# For python > 2 convert bytes to string
+if not PY2:
+fg_color_dvipng = fg_color_dvipng.decode('ascii')
+bg_color_dvipng = bg_color_dvipng.decode('ascii')
+
 # External programs used by the script.
 latex = find_exe_or_terminate(latex or latex_commands)
 bibtex = find_exe(bibtex or bibtex_commands)


[LyX/master] Fix configure.py to work with python3 again (it continues to work with python2(.7))

2019-05-16 Thread José Matos
commit 82ee9794f916d956f29c5efcb3ae4ae99756186b
Author: José Matos 
Date:   Thu May 16 09:27:12 2019 +0100

Fix configure.py to work with python3 again (it continues to work with 
python2(.7))
---
 lib/configure.py |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index 9247957..0b3c2a7 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -1587,9 +1587,9 @@ def processModuleFile(file, filename, bool_docbook):
 cm.write(line + '\n')
 cm.close()
 
-local = "true"
+local = b"true"
 if (file.startswith(srcdir)):
-local = "false"
+local = b"false"
 return (b'"%s" "%s" "%s" "%s" "%s" "%s" "%s" "%s"\n'
 % (modname, filename, desc, pkgs, req, excl, catgy, local))
 


[LyX/master] Merge branch 'master' of git.lyx.org:lyx

2018-12-29 Thread José Matos
commit 8663a371f31dc7bb888071c15a0b2cd4c3669c60
Merge: 3f03f0a 69726b8
Author: José Matos 
Date:   Sat Dec 29 19:24:59 2018 +

Merge branch 'master' of git.lyx.org:lyx

 src/support/filetools.cpp |6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)


[LyX/master] Move all python shebangs from /usr/bin/env to python3.

2018-12-29 Thread José Matos
commit 3f03f0a447e0f672025d78881d8a2790133ff47b
Author: José Matos 
Date:   Sat Dec 29 19:14:41 2018 +

Move all python shebangs from /usr/bin/env to python3.

The change is only relevant to development as all the call to python 
scripts is done
calling C++ os::python that invoques the appropriate python version.

The change is two fold, on one hand remove all the uses of /usr/bin/env for 
python.

On the other hand rename all the calls to python from python to python3 
making it explicit
and being compliant with PEP 394 -- The "python" Command on Unix-Like 
Systems:
https://www.python.org/dev/peps/pep-0394/

Remove the sheebang from src/graphics/GraphicsConverter.cpp because it is 
not necessary.

Some small whitespace changes.
---
 development/cmake/doc/ReplaceValues.py   |2 +-
 development/cmake/po/cat.py  |2 +-
 development/cmake/po/unix2dos.py |6 ++--
 development/cmake/po/update-gmo.py   |2 +-
 development/tools/gen_lfuns.py   |   48 +-
 development/tools/generate_symbols_images.py |2 +-
 development/tools/generate_symbols_list.py   |2 +-
 development/tools/mergepo.py |2 +-
 development/tools/unicodesymbols.py  |2 +-
 development/tools/updatedocs.py  |2 +-
 development/tools/updatelayouts.py   |4 +-
 po/lyx_pot.py|4 +--
 po/postats.py|4 +-
 src/graphics/GraphicsConverter.cpp   |3 +-
 src/tex2lyx/test/runtests.py |3 +-
 15 files changed, 42 insertions(+), 46 deletions(-)

diff --git a/development/cmake/doc/ReplaceValues.py 
b/development/cmake/doc/ReplaceValues.py
index da1d42a..18eaab3 100755
--- a/development/cmake/doc/ReplaceValues.py
+++ b/development/cmake/doc/ReplaceValues.py
@@ -1,4 +1,4 @@
-#! /usr/bin/env python
+#! /usr/bin/python3
 
 from __future__ import print_function
 
diff --git a/development/cmake/po/cat.py b/development/cmake/po/cat.py
index 7a1ee26..0462b29 100644
--- a/development/cmake/po/cat.py
+++ b/development/cmake/po/cat.py
@@ -1,4 +1,4 @@
-#! /usr/bin/env python
+#! /usr/bin/python3
 
 from __future__ import print_function
 
diff --git a/development/cmake/po/unix2dos.py b/development/cmake/po/unix2dos.py
index 85af0ec..2812873 100644
--- a/development/cmake/po/unix2dos.py
+++ b/development/cmake/po/unix2dos.py
@@ -1,4 +1,4 @@
-#! /usr/bin/env python
+#! /usr/bin/python3
 
 ###
 import sys
@@ -11,7 +11,7 @@ for fname in sys.argv[1:]:
 
 if len(outstr) == len(instr):
 continue
-
+
 outfile = open( fname , "wb" )
 outfile.write( outstr )
-outfile.close()
\ No newline at end of file
+outfile.close()
diff --git a/development/cmake/po/update-gmo.py 
b/development/cmake/po/update-gmo.py
index 8e0c108..1758755 100644
--- a/development/cmake/po/update-gmo.py
+++ b/development/cmake/po/update-gmo.py
@@ -1,4 +1,4 @@
-#! /usr/bin/env python
+#! /usr/bin/python3
 
 ###
 import sys, os, shutil
diff --git a/development/tools/gen_lfuns.py b/development/tools/gen_lfuns.py
index 64dc7dc..6162bbb 100755
--- a/development/tools/gen_lfuns.py
+++ b/development/tools/gen_lfuns.py
@@ -1,4 +1,4 @@
-#! /usr/bin/env python
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 
 # file gen_lfuns.py
@@ -39,7 +39,7 @@ LFUN_SAMPLE_ID = u"\\li Sample: "
 LFUN_ORIGIN_ID = u"\\li Origin: "
 LFUN_ENDVAR = u"\\endvar"
 
-ID_DICT = dict(name=LFUN_NAME_ID, action=LFUN_ACTION_ID, 
notion=LFUN_NOTION_ID, 
+ID_DICT = dict(name=LFUN_NAME_ID, action=LFUN_ACTION_ID, notion=LFUN_NOTION_ID,
 syntax=LFUN_SYNTAX_ID, params=LFUN_PARAMS_ID, 
sample=LFUN_SAMPLE_ID, origin=LFUN_ORIGIN_ID)
 
 LFUNS_HEADER = u"""# gen_lfuns.py generated this file. For more info see 
http://www.lyx.org/
@@ -153,7 +153,7 @@ About this manual
 \\end_layout
 
 \\begin_layout Standard
-This manual documents the 
+This manual documents the
 \\begin_inset Quotes eld
 \\end_inset
 
@@ -175,7 +175,7 @@ LFUNs are also used in the files that define keyboard 
shortcuts, menu or
  So if you want to change\\SpecialChar breakableslash
 customize the user interface, you need to deal
  with LFUNs.
- Furthermore, external programs can use LFUNs to communicate with and 
+ Furthermore, external programs can use LFUNs to communicate with and
 \\begin_inset Quotes eld
 \\end_inset
 
@@ -186,7 +186,7 @@ remote-control
  \\SpecialChar LyX
  .
  Finally, you can also issue LFUNs directly via the so called mini-buffer
- which can be opened via 
+ which can be opened via
 \\begin_inset Info
 type  "shortcuts"
 arg   "command-execute"
@@ -208,15 +208,15 @@ LFUNS_FOOTER = u"""\\end_body
 
 def parse_lfun(str):
 """Takes a comment block (str) and parses it for fields d

[LyX/master] Replace /usr/bin/env python -> /usr/bin/python3

2018-08-29 Thread José Matos
commit cac27076ead10684270520670adc6bd004793361
Author: José Matos 
Date:   Wed Aug 29 17:26:21 2018 +0100

Replace /usr/bin/env python -> /usr/bin/python3
---
 lib/configure.py  |2 +-
 lib/generate_contributions.py |2 +-
 lib/lyx2lyx/lyx2lyx   |2 +-
 lib/lyx2lyx/profiling.py  |2 +-
 lib/scripts/listerrors|2 +-
 lib/scripts/svg2pdftex.py |2 +-
 lib/scripts/svg2pstex.py  |2 +-
 7 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index 1a4ef55..4c345ca 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -1,4 +1,4 @@
-#! /usr/bin/env python
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 #
 # file configure.py
diff --git a/lib/generate_contributions.py b/lib/generate_contributions.py
index b796457..2427453 100755
--- a/lib/generate_contributions.py
+++ b/lib/generate_contributions.py
@@ -1,4 +1,4 @@
-#! /usr/bin/env python
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 
 '''
diff --git a/lib/lyx2lyx/lyx2lyx b/lib/lyx2lyx/lyx2lyx
index 9d03830..174bcd7 100755
--- a/lib/lyx2lyx/lyx2lyx
+++ b/lib/lyx2lyx/lyx2lyx
@@ -1,4 +1,4 @@
-#! /usr/bin/env python
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Copyright (C) 2002-2011 The LyX Team
 # Copyright (C) 2002-2007 José Matos 
diff --git a/lib/lyx2lyx/profiling.py b/lib/lyx2lyx/profiling.py
index dd5cbee..bd885bb 100755
--- a/lib/lyx2lyx/profiling.py
+++ b/lib/lyx2lyx/profiling.py
@@ -1,4 +1,4 @@
-#! /usr/bin/env python
+#! /usr/bin/python3
 # -*- coding: utf-8 -*-
 # Copyright (C) 2004 José Matos 
 #
diff --git a/lib/scripts/listerrors b/lib/scripts/listerrors
index 232e542..7d6995c 100755
--- a/lib/scripts/listerrors
+++ b/lib/scripts/listerrors
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/python3
 
 # file listerrors
 # This file is part of LyX, the document processor.
diff --git a/lib/scripts/svg2pdftex.py b/lib/scripts/svg2pdftex.py
index d1e0bf3..86564fa 100644
--- a/lib/scripts/svg2pdftex.py
+++ b/lib/scripts/svg2pdftex.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/python3
 # -*- coding: utf-8 -*-
 
 # file svg2pdftex.py
diff --git a/lib/scripts/svg2pstex.py b/lib/scripts/svg2pstex.py
index 97d6ae1..ef6ff1d 100644
--- a/lib/scripts/svg2pstex.py
+++ b/lib/scripts/svg2pstex.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/python3
 # -*- coding: utf-8 -*-
 
 # file svg2pstex.py


[LyX/master] Simplify the backup names when using an older file format.

2018-05-09 Thread José Matos
commit 37c94806566fdd13bef1fe5bfeda64ceb839f6a3
Author: José Matos <jama...@lyx.org>
Date:   Wed May 9 08:53:28 2018 +0100

Simplify the backup names when using an older file format.

Currently if we have a 2.3 file and we open it with a more recent version
we get:

original file: file.lyx
backup file:   file-lyxformat-544.lyx~

After this commit the backup file becomes

file-lyx23.lyx~

If the file is from a development version then we get the same result as 
before.
---
 src/Buffer.cpp |   27 +--
 1 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/src/Buffer.cpp b/src/Buffer.cpp
index 5a0bd74..83a82f8 100644
--- a/src/Buffer.cpp
+++ b/src/Buffer.cpp
@@ -1329,14 +1329,37 @@ Buffer::ReadStatus Buffer::convertLyXFormat(FileName 
const & fn,
 
 
 FileName Buffer::getBackupName() const {
+   map<int, string> const file_formats = {
+ {544, "23"},
+ {508, "22"},
+ {474, "21"},
+ {413, "20"},
+ {345, "16"},
+ {276, "15"},
+ {245, "14"},
+ {221, "13"},
+ {220, "12"},
+ {218, "1163"},
+ {217, "116"},
+ {216, "115"},
+ {215, "11"},
+ {210, "010"},
+ {200, "006"}
+   };
FileName const & fn = fileName();
string const fname = fn.onlyFileNameWithoutExt();
string const fext  = fn.extension() + "~";
string const fpath = lyxrc.backupdir_path.empty() ?
fn.onlyPath().absFileName() :
lyxrc.backupdir_path;
-   string const fform = convert(d->file_format);
-   string const backname = fname + "-lyxformat-" + fform;
+   string backup_suffix;
+   // If file format is from a stable series use version instead of file 
format
+   auto const it = file_formats.find(d->file_format);
+   if (it != file_formats.end())
+   backup_suffix = "-lyx" + it->second;
+   else
+   backup_suffix = "-lyxformat-" + convert(d->file_format);
+   string const backname = fname + backup_suffix;
FileName backup(addName(fpath, addExtension(backname, fext)));
 
// limit recursion, just in case


[LyX/master] Small improvement to pre-historic file format conversion.

2018-04-28 Thread José Matos
commit 91ded82b52723e6f3967cc57a155bd9c04c8d728
Author: José Matos <jama...@lyx.org>
Date:   Sat Apr 28 14:57:57 2018 +0100

Small improvement to pre-historic file format conversion.
---
 lib/lyx2lyx/lyx_0_10.py |   54 --
 1 files changed, 28 insertions(+), 26 deletions(-)

diff --git a/lib/lyx2lyx/lyx_0_10.py b/lib/lyx2lyx/lyx_0_10.py
index dc2d09e..13602ac 100644
--- a/lib/lyx2lyx/lyx_0_10.py
+++ b/lib/lyx2lyx/lyx_0_10.py
@@ -32,18 +32,17 @@ def regularise_header(document):
 
 def find_next_space(line, j):
 """ Return position of next space or backslash, which one comes
-first, starting from position k, if not existing return last
-position in line."""
-l = line.find(' ', j)
-if l == -1:
-l = len(line)
-k = line.find('\\', j)
-if k == -1:
-k = len(line)
+first, starting from position j, if none exists returns last
+position in line (+1)."""
+space_pos = line.find(' ', j)
+if space_pos == -1:
+space_pos = len(line)
 
-if k < l:
-return k
-return l
+bksl_pos = line.find('\\', j)
+if bksl_pos == -1:
+bksl_pos = len(line)
+
+return min(space_pos, bksl_pos)
 
 
 def regularise_body(document):
@@ -65,36 +64,38 @@ def regularise_body(document):
 while i < len(document.body):
 line = document.body[i]
 j = 0
-tmp = []
+new_block = []
 while j < len(line):
 k = line.find('\\', j)
 
 if k == -1:
-tmp += [line[j:]]
+new_block += [line[j:]]
 break
 
 if k != j:
-tmp += [line[j: k]]
+#document.warning("j=%d\tk=%d\t#%s#%s#" % (j,k,line,line[j: 
k]))
+new_block += [line[j: k]]
 j = k
 
 k = find_next_space(line, j+1)
 
-# These tokens take the rest of the line
 token = line[j+1:k]
+# These tokens take the rest of the line
 if token in getline_tokens:
-tmp += [line[j:]]
+#document.warning("getline_token:%s\tj=%d\t\t#%s#%s#" % 
(token,j,line,line[j:]))
+new_block += [line[j:]]
 break
 
 # These tokens take no arguments
 if token in noargs_tokens:
-tmp += [line[j:k]]
+new_block += [line[j:k]]
 j = k
 continue
 
 # These tokens take one argument
 if token in onearg_tokens:
 k = find_next_space(line, k + 1)
-tmp += [line[j:k]]
+new_block += [line[j:k]]
 j = k
 continue
 
@@ -104,29 +105,30 @@ def regularise_body(document):
 inset = line[k+1: l]
 
 if inset == "Latex":
-tmp += [line[j:l]]
+new_block += [line[j:l]]
 j = l
 continue
 
-if inset in ["LatexCommand", "LatexDel"]:
-tmp += [line[j:]]
+if inset in ["LatexCommand", "LatexDel", "Label", "Figure",
+ "Formula"]:
+new_block += [line[j:]]
 break
 
 if inset == "Quotes":
 l = find_next_space(line, l + 1)
-tmp += [line[j:l]]
+new_block += [line[j:l]]
 j = l
 continue
 
-document.warning("unkown inset %s" % line)
+document.warning("unkown inset %s" % inset)
 assert(False)
 
 # We are inside a latex inset, pass the text verbatim
-tmp += [line[j:]]
+new_block += [line[j:]]
 break
 
-document.body[i: i+1] = tmp
-i += len(tmp)
+document.body[i: i+1] = new_block
+i += len(new_block)
 
 
 supported_versions = ["0.10.%d" % i for i in range(8)] + ["0.10"]


[LyX/master] Add new semantic functions to add an remove document options.

2018-04-28 Thread José Matos
commit ea8b0df0e4d1d6f8d08be2ac2a87f1d43959a53d
Author: José Matos <jama...@lyx.org>
Date:   Sat Apr 28 15:10:09 2018 +0100

Add new semantic functions to add an remove document options.

The objective is to identify common operations and place them
in functions in order to improve the readability and correctness of the 
code.

is_document_option(document, option):
  Find if _option_ is a document option (\\options in the header).

insert_document_option(document, option):
  Insert _option_ as a document option.

remove_document_option(document, option):
  Remove _option_ as a document option.
---
 lib/lyx2lyx/lyx2lyx_tools.py |   89 +
 lib/lyx2lyx/lyx_1_5.py   |9 +---
 2 files changed, 82 insertions(+), 16 deletions(-)

diff --git a/lib/lyx2lyx/lyx2lyx_tools.py b/lib/lyx2lyx/lyx2lyx_tools.py
index 7aac890..cb1996e 100644
--- a/lib/lyx2lyx/lyx2lyx_tools.py
+++ b/lib/lyx2lyx/lyx2lyx_tools.py
@@ -65,19 +65,27 @@ lyx2verbatim(document, lines):
   can and return a string containing the translated material.
 
 latex_length(slen):
-Convert lengths (in LyX form) to their LaTeX representation. Returns
-(bool, length), where the bool tells us if it was a percentage, and
-the length is the LaTeX representation.
+  Convert lengths (in LyX form) to their LaTeX representation. Returns
+  (bool, length), where the bool tells us if it was a percentage, and
+  the length is the LaTeX representation.
 
 convert_info_insets(document, type, func):
-Applies func to the argument of all info insets matching certain types
-type : the type to match. This can be a regular expression.
-func : function from string to string to apply to the "arg" field of
-   the info insets.
+  Applies func to the argument of all info insets matching certain types
+  type : the type to match. This can be a regular expression.
+  func : function from string to string to apply to the "arg" field of
+ the info insets.
+
+is_document_option(document, option):
+  Find if _option_ is a document option (\\options in the header).
+
+insert_document_option(document, option):
+  Insert _option_ as a document option.
+
+remove_document_option(document, option):
+  Remove _option_ as a document option.
 '''
 
 import re
-import string
 from parser_tools import find_token, find_end_of_inset
 from unicode_symbols import unicode_reps
 
@@ -318,7 +326,7 @@ def latex_length(slen):
 # Convert relative lengths to LaTeX units
 units = {"col%": "\\columnwidth",
  "text%": "\\textwidth",
- "page%": "\\paperwidth", 
+ "page%": "\\paperwidth",
  "line%": "\\linewidth",
  "theight%": "\\textheight",
  "pheight%": "\\paperheight",
@@ -533,3 +541,66 @@ def convert_info_insets(document, type, func):
 new_arg = func(arg.group(1))
 document.body[i + 2] = 'arg   "%s"' % new_arg
 i += 3
+
+
+def insert_document_option(document, option):
+"Insert _option_ as a document option."
+
+# Find \options in the header
+options_line = find_token(document.header, "\\options", 0)
+
+# if the options does not exists add it after the textclass
+if options_line == -1:
+textclass_line = find_token(document.header, "\\textclass", 0)
+document.header.insert(textclass_line +1,
+   r"\options %s" % option)
+return
+
+# add it to the end of the options
+document.header[options_line] += " ,%s" % option
+
+
+def remove_document_option(document, option):
+""" Remove _option_ as a document option.
+
+It is assumed that option belongs to the \options.
+That can be done running is_document_option(document, option)."""
+
+options_line = find_token(document.header, "\\options", 0)
+option_pos = document.header[options_line].find(option)
+
+# Remove option from \options
+comma_before_pos = document.header[options_line].rfind(',', 0, option_pos)
+comma_after_pos  = document.header[options_line].find(',', option_pos)
+
+# if there are no commas then it is the single option
+# and the options line should be removed since it will be empty
+if comma_before_pos == comma_after_pos == -1:
+del document.header[options_line]
+return
+
+# last option
+options = document.header[options_line]
+if comma_after_pos == -1:
+document.header[options_line] = options[:comma_before_pos].rsplit()
+return
+
+document.header[options_line] = options[comma_before_pos: comma_after_pos]
+
+
+def is_document_option(document, option):
+"F

[LyX/2.3.x] Merge branch '2.3.x' of git.lyx.org:lyx into 2.3.x

2018-04-28 Thread José Matos
commit 0a43aac52474e5b316df9b3679b771fa4c98839c
Merge: f7b9d57 1084e30
Author: José Matos <jama...@lyx.org>
Date:   Sat Apr 28 11:26:16 2018 +0100

Merge branch '2.3.x' of git.lyx.org:lyx into 2.3.x



[LyX/2.3.x] Update scripts to support simultaneously python 2 and 3

2018-04-28 Thread José Matos
commit f7b9d57a454fb3f6275ccfecbe842edd41e9ed49
Author: José Matos <jama...@lyx.org>
Date:   Sat Apr 28 10:51:35 2018 +0100

Update scripts to support simultaneously python 2 and 3

The fixes are simple and on line with the changes made during
the 2.3 development. It was an oversight to leave them out.

With this commit all the python scripts should be supported by
python 2 and 3.

(cherry picked from 5b160e82be3797ae57632174f2510a132a7558a1)
---
 lib/scripts/convertDefault.py |   16 +++-
 lib/scripts/fen2ascii.py  |9 +
 lib/scripts/fig2pdftex.py |   14 +++---
 lib/scripts/fig2pstex.py  |3 ++-
 lib/scripts/fig_copy.py   |   19 ++-
 lib/scripts/include_bib.py|   27 ++-
 lib/scripts/listerrors|   14 --
 lib/scripts/tex_copy.py   |4 
 status.23x|3 +++
 9 files changed, 64 insertions(+), 45 deletions(-)

diff --git a/lib/scripts/convertDefault.py b/lib/scripts/convertDefault.py
index e54b066..9a460b7 100644
--- a/lib/scripts/convertDefault.py
+++ b/lib/scripts/convertDefault.py
@@ -16,8 +16,11 @@
 # replacement in ~/.lyx/scripts
 
 # converts an image $2 (format $1) to $4 (format $3)
+from __future__ import print_function
 import os, re, sys
 
+PY2 = sys.version_info[0] == 2
+
 # We may need some extra options only supported by recent convert versions
 re_version = re.compile(r'^Version:.*ImageMagick\s*(\d*)\.(\d*)\.(\d*).*$')
 # imagemagick 7
@@ -31,6 +34,9 @@ if fout.close() != None:
 fout = os.popen('convert -version 2>&1')
 output = fout.readline()
 fout.close()
+if not PY2:
+output = output.decode()
+
 version = re_version.match(output)
 
 # Imagemagick by default
@@ -63,12 +69,12 @@ if sys.argv[1] == 'pdf' and (version >= 0x060206 or gm):
 if sys.argv[3] == 'ppm' and (im and version >= 0x060305 or gm):
 opts = opts + ' -flatten'
 
-# print >> sys.stdout, command, sys.argv[2], sys.argv[4]
+# print (command, sys.argv[2], sys.argv[4], file= sys.stdout)
 if (im or gm) and os.system(r'%s %s "%s" "%s"' % (command, opts, sys.argv[2], 
sys.argv[3] + ':' + sys.argv[4])) != 0:
-print >> sys.stderr, sys.argv[0], 'ERROR'
-print >> sys.stderr, ('Execution of "%s" failed.' % command)
+print (sys.argv[0], 'ERROR', file= sys.stderr)
+print ('Execution of "%s" failed.' % command, file= sys.stderr)
 sys.exit(1)
 elif not im and not gm and sys.platform == 'darwin' and os.system(r'%s "%s" 
"%s"' % (command, sys.argv[2], sys.argv[4])) != 0:
-print >> sys.stderr, sys.argv[0], 'ERROR'
-print >> sys.stderr, ('Execution of "%s" failed.' % command)
+print (sys.argv[0], 'ERROR', file= sys.stderr)
+print ('Execution of "%s" failed.' % command, file= sys.stderr)
 sys.exit(1)
diff --git a/lib/scripts/fen2ascii.py b/lib/scripts/fen2ascii.py
index d7f0fb3..7408744 100644
--- a/lib/scripts/fen2ascii.py
+++ b/lib/scripts/fen2ascii.py
@@ -9,6 +9,7 @@
 # This script will convert a chess position in the FEN
 # format to an ascii representation of the position.
 
+from __future__ import print_function
 import sys,string,os
 
 os.close(0)
@@ -26,7 +27,7 @@ comp=string.split(line,'/')
 cont=1
 margin= " "*6
 
-print margin+'   +'+"-"*15+'+'
+print (margin+'   +'+"-"*15+'+')
 for i in range(8):
 cont = cont + 1
 tmp=""
@@ -42,7 +43,7 @@ for i in range(8):
 cont = cont + 1
 
 row = 8 - i
-print margin, row, tmp+"|"
+print (margin, row, tmp+"|")
 
-print margin+'   +'+"-"*15+'+'
-print margin+'a b c d e f g h '
+print (margin+'   +'+"-"*15+'+')
+print (margin+'a b c d e f g h ')
diff --git a/lib/scripts/fig2pdftex.py b/lib/scripts/fig2pdftex.py
index 603fd31..b458ccd 100644
--- a/lib/scripts/fig2pdftex.py
+++ b/lib/scripts/fig2pdftex.py
@@ -26,7 +26,7 @@
 #   the real pdf file will be overwritten by a tex file named file.pdf.
 #
 
-
+from __future__ import print_function
 import os, sys, re
 
 
@@ -35,7 +35,7 @@ def runCommand(cmd):
 run a command, quit if fails
 '''
 if os.system(cmd) != 0:
-print "Command '%s' fails." % cmd
+print("Command '%s' fails." % cmd)
 sys.exit(1)
 
 
@@ -78,15 +78,15 @@ else:
 # with tetex.
 epsfile = outbase + '.pstex'
 tmp = mkstemp()
-boundingboxline = 
re.compile('%%BoundingBox:\s+(\d*)\s+(\d*)\s+(\d*)\s+(\d*)')
-for line in open(epsfile).xreadlines():
-if line[:13] == '%%BoundingBox':
-(llx, lly, urx, ury) = map(int, 
boundingboxline.search(line).groups())
+boundingboxline = 
re.compile(b'%%BoundingBox:\s+(\d*)\s+(\d*)\s+(\d*)\s+(\d*)')
+for line in open(epsfile, 'rb'):
+if line[:13] == b'%%BoundingBox':
+

[LyX/2.3.x] Update scripts to support simultaneously python 2 and 3

2018-04-28 Thread José Matos
commit 1084e30f10fa4b4c78ef10a04adb5497a3341376
Author: José Matos <jama...@lyx.org>
Date:   Sat Apr 28 10:51:35 2018 +0100

Update scripts to support simultaneously python 2 and 3

The fixes are simple and on line with the changes made during
the 2.3 development. It was an oversight to leave them out.

With this commit all the python scripts should be supported by
python 2 and 3.
---
 lib/scripts/convertDefault.py |   16 +++-
 lib/scripts/fen2ascii.py  |9 +
 lib/scripts/fig2pdftex.py |   14 +++---
 lib/scripts/fig2pstex.py  |3 ++-
 lib/scripts/fig_copy.py   |   19 ++-
 lib/scripts/include_bib.py|   27 ++-
 lib/scripts/listerrors|   14 --
 lib/scripts/tex_copy.py   |4 
 8 files changed, 61 insertions(+), 45 deletions(-)

diff --git a/lib/scripts/convertDefault.py b/lib/scripts/convertDefault.py
index e54b066..9a460b7 100644
--- a/lib/scripts/convertDefault.py
+++ b/lib/scripts/convertDefault.py
@@ -16,8 +16,11 @@
 # replacement in ~/.lyx/scripts
 
 # converts an image $2 (format $1) to $4 (format $3)
+from __future__ import print_function
 import os, re, sys
 
+PY2 = sys.version_info[0] == 2
+
 # We may need some extra options only supported by recent convert versions
 re_version = re.compile(r'^Version:.*ImageMagick\s*(\d*)\.(\d*)\.(\d*).*$')
 # imagemagick 7
@@ -31,6 +34,9 @@ if fout.close() != None:
 fout = os.popen('convert -version 2>&1')
 output = fout.readline()
 fout.close()
+if not PY2:
+output = output.decode()
+
 version = re_version.match(output)
 
 # Imagemagick by default
@@ -63,12 +69,12 @@ if sys.argv[1] == 'pdf' and (version >= 0x060206 or gm):
 if sys.argv[3] == 'ppm' and (im and version >= 0x060305 or gm):
 opts = opts + ' -flatten'
 
-# print >> sys.stdout, command, sys.argv[2], sys.argv[4]
+# print (command, sys.argv[2], sys.argv[4], file= sys.stdout)
 if (im or gm) and os.system(r'%s %s "%s" "%s"' % (command, opts, sys.argv[2], 
sys.argv[3] + ':' + sys.argv[4])) != 0:
-print >> sys.stderr, sys.argv[0], 'ERROR'
-print >> sys.stderr, ('Execution of "%s" failed.' % command)
+print (sys.argv[0], 'ERROR', file= sys.stderr)
+print ('Execution of "%s" failed.' % command, file= sys.stderr)
 sys.exit(1)
 elif not im and not gm and sys.platform == 'darwin' and os.system(r'%s "%s" 
"%s"' % (command, sys.argv[2], sys.argv[4])) != 0:
-print >> sys.stderr, sys.argv[0], 'ERROR'
-print >> sys.stderr, ('Execution of "%s" failed.' % command)
+print (sys.argv[0], 'ERROR', file= sys.stderr)
+print ('Execution of "%s" failed.' % command, file= sys.stderr)
 sys.exit(1)
diff --git a/lib/scripts/fen2ascii.py b/lib/scripts/fen2ascii.py
index d7f0fb3..7408744 100644
--- a/lib/scripts/fen2ascii.py
+++ b/lib/scripts/fen2ascii.py
@@ -9,6 +9,7 @@
 # This script will convert a chess position in the FEN
 # format to an ascii representation of the position.
 
+from __future__ import print_function
 import sys,string,os
 
 os.close(0)
@@ -26,7 +27,7 @@ comp=string.split(line,'/')
 cont=1
 margin= " "*6
 
-print margin+'   +'+"-"*15+'+'
+print (margin+'   +'+"-"*15+'+')
 for i in range(8):
 cont = cont + 1
 tmp=""
@@ -42,7 +43,7 @@ for i in range(8):
 cont = cont + 1
 
 row = 8 - i
-print margin, row, tmp+"|"
+print (margin, row, tmp+"|")
 
-print margin+'   +'+"-"*15+'+'
-print margin+'a b c d e f g h '
+print (margin+'   +'+"-"*15+'+')
+print (margin+'a b c d e f g h ')
diff --git a/lib/scripts/fig2pdftex.py b/lib/scripts/fig2pdftex.py
index 603fd31..b458ccd 100644
--- a/lib/scripts/fig2pdftex.py
+++ b/lib/scripts/fig2pdftex.py
@@ -26,7 +26,7 @@
 #   the real pdf file will be overwritten by a tex file named file.pdf.
 #
 
-
+from __future__ import print_function
 import os, sys, re
 
 
@@ -35,7 +35,7 @@ def runCommand(cmd):
 run a command, quit if fails
 '''
 if os.system(cmd) != 0:
-print "Command '%s' fails." % cmd
+print("Command '%s' fails." % cmd)
 sys.exit(1)
 
 
@@ -78,15 +78,15 @@ else:
 # with tetex.
 epsfile = outbase + '.pstex'
 tmp = mkstemp()
-boundingboxline = 
re.compile('%%BoundingBox:\s+(\d*)\s+(\d*)\s+(\d*)\s+(\d*)')
-for line in open(epsfile).xreadlines():
-if line[:13] == '%%BoundingBox':
-(llx, lly, urx, ury) = map(int, 
boundingboxline.search(line).groups())
+boundingboxline = 
re.compile(b'%%BoundingBox:\s+(\d*)\s+(\d*)\s+(\d*)\s+(\d*)')
+for line in open(epsfile, 'rb'):
+if line[:13] == b'%%BoundingBox':
+(llx, lly, urx, ury) = list(map(int, 
boundingboxline.search(line).groups()))
 width =

[LyX/master] Update scripts to support simultaneously python 2 and 3

2018-04-28 Thread José Matos
commit 5b160e82be3797ae57632174f2510a132a7558a1
Author: José Matos <jama...@lyx.org>
Date:   Sat Apr 28 10:51:35 2018 +0100

Update scripts to support simultaneously python 2 and 3

The fixes are simple and on line with the changes made during
the 2.3 development. It was an oversight to leave them out.

With this commit all the python scripts should be supported by
python 2 and 3.
---
 lib/scripts/convertDefault.py |   16 +++-
 lib/scripts/fen2ascii.py  |9 +
 lib/scripts/fig2pdftex.py |   14 +++---
 lib/scripts/fig2pstex.py  |3 ++-
 lib/scripts/fig_copy.py   |   19 ++-
 lib/scripts/include_bib.py|   27 ++-
 lib/scripts/listerrors|   14 --
 lib/scripts/tex_copy.py   |4 
 8 files changed, 61 insertions(+), 45 deletions(-)

diff --git a/lib/scripts/convertDefault.py b/lib/scripts/convertDefault.py
index e54b066..9a460b7 100644
--- a/lib/scripts/convertDefault.py
+++ b/lib/scripts/convertDefault.py
@@ -16,8 +16,11 @@
 # replacement in ~/.lyx/scripts
 
 # converts an image $2 (format $1) to $4 (format $3)
+from __future__ import print_function
 import os, re, sys
 
+PY2 = sys.version_info[0] == 2
+
 # We may need some extra options only supported by recent convert versions
 re_version = re.compile(r'^Version:.*ImageMagick\s*(\d*)\.(\d*)\.(\d*).*$')
 # imagemagick 7
@@ -31,6 +34,9 @@ if fout.close() != None:
 fout = os.popen('convert -version 2>&1')
 output = fout.readline()
 fout.close()
+if not PY2:
+output = output.decode()
+
 version = re_version.match(output)
 
 # Imagemagick by default
@@ -63,12 +69,12 @@ if sys.argv[1] == 'pdf' and (version >= 0x060206 or gm):
 if sys.argv[3] == 'ppm' and (im and version >= 0x060305 or gm):
 opts = opts + ' -flatten'
 
-# print >> sys.stdout, command, sys.argv[2], sys.argv[4]
+# print (command, sys.argv[2], sys.argv[4], file= sys.stdout)
 if (im or gm) and os.system(r'%s %s "%s" "%s"' % (command, opts, sys.argv[2], 
sys.argv[3] + ':' + sys.argv[4])) != 0:
-print >> sys.stderr, sys.argv[0], 'ERROR'
-print >> sys.stderr, ('Execution of "%s" failed.' % command)
+print (sys.argv[0], 'ERROR', file= sys.stderr)
+print ('Execution of "%s" failed.' % command, file= sys.stderr)
 sys.exit(1)
 elif not im and not gm and sys.platform == 'darwin' and os.system(r'%s "%s" 
"%s"' % (command, sys.argv[2], sys.argv[4])) != 0:
-print >> sys.stderr, sys.argv[0], 'ERROR'
-print >> sys.stderr, ('Execution of "%s" failed.' % command)
+print (sys.argv[0], 'ERROR', file= sys.stderr)
+print ('Execution of "%s" failed.' % command, file= sys.stderr)
 sys.exit(1)
diff --git a/lib/scripts/fen2ascii.py b/lib/scripts/fen2ascii.py
index d7f0fb3..7408744 100644
--- a/lib/scripts/fen2ascii.py
+++ b/lib/scripts/fen2ascii.py
@@ -9,6 +9,7 @@
 # This script will convert a chess position in the FEN
 # format to an ascii representation of the position.
 
+from __future__ import print_function
 import sys,string,os
 
 os.close(0)
@@ -26,7 +27,7 @@ comp=string.split(line,'/')
 cont=1
 margin= " "*6
 
-print margin+'   +'+"-"*15+'+'
+print (margin+'   +'+"-"*15+'+')
 for i in range(8):
 cont = cont + 1
 tmp=""
@@ -42,7 +43,7 @@ for i in range(8):
 cont = cont + 1
 
 row = 8 - i
-print margin, row, tmp+"|"
+print (margin, row, tmp+"|")
 
-print margin+'   +'+"-"*15+'+'
-print margin+'a b c d e f g h '
+print (margin+'   +'+"-"*15+'+')
+print (margin+'a b c d e f g h ')
diff --git a/lib/scripts/fig2pdftex.py b/lib/scripts/fig2pdftex.py
index 603fd31..b458ccd 100644
--- a/lib/scripts/fig2pdftex.py
+++ b/lib/scripts/fig2pdftex.py
@@ -26,7 +26,7 @@
 #   the real pdf file will be overwritten by a tex file named file.pdf.
 #
 
-
+from __future__ import print_function
 import os, sys, re
 
 
@@ -35,7 +35,7 @@ def runCommand(cmd):
 run a command, quit if fails
 '''
 if os.system(cmd) != 0:
-print "Command '%s' fails." % cmd
+print("Command '%s' fails." % cmd)
 sys.exit(1)
 
 
@@ -78,15 +78,15 @@ else:
 # with tetex.
 epsfile = outbase + '.pstex'
 tmp = mkstemp()
-boundingboxline = 
re.compile('%%BoundingBox:\s+(\d*)\s+(\d*)\s+(\d*)\s+(\d*)')
-for line in open(epsfile).xreadlines():
-if line[:13] == '%%BoundingBox':
-(llx, lly, urx, ury) = map(int, 
boundingboxline.search(line).groups())
+boundingboxline = 
re.compile(b'%%BoundingBox:\s+(\d*)\s+(\d*)\s+(\d*)\s+(\d*)')
+for line in open(epsfile, 'rb'):
+if line[:13] == b'%%BoundingBox':
+(llx, lly, urx, ury) = list(map(int, 
boundingboxline.search(line).groups()))
 width =

[LyX/master] scripts/ext_copy.py: add option to copy directly to the document directory

2017-08-08 Thread José Matos
commit 6b092c8400b21ed3a38d04428c1755b4ecea2ce0
Author: José Matos <jama...@lyx.org>
Date:   Tue Aug 8 15:23:03 2017 +0100

scripts/ext_copy.py: add option to copy directly to the document directory

When this option is not given, as it happened before, the copier will 
create/use
a subdirectory in the target final directory
---
 lib/scripts/ext_copy.py |   35 +++
 1 files changed, 23 insertions(+), 12 deletions(-)

diff --git a/lib/scripts/ext_copy.py b/lib/scripts/ext_copy.py
index fb70d7b..3022d30 100644
--- a/lib/scripts/ext_copy.py
+++ b/lib/scripts/ext_copy.py
@@ -9,36 +9,44 @@
 # Full author contact details are available in file CREDITS
 
 # Usage:
-# ext_copy.py [-e ext1,ext2,...]  
+# ext_copy.py [-d] [-e ext1,ext2,...] [-t target_ext] from_file to_file
 
 # This script is to be used as a "copier" script in the sense needed by
-# the converter definitions. Given a  and , it will copy
-# all files in the directory in which from_file is found that have the 
-# extensions given in the -e argument, or all files in that directory if no 
+# the converter definitions. Given a from_file and to_file, it will copy
+# all files in the directory in which from_file is found that have the
+# extensions given in the -e argument, or all files in that directory if no
 # such argument is given. So, for example, we can do:
 #   python ext_copy.py -e png,html,css /path/from/file.html /path/to/file.html
-# and all html, png, and css files in /path/from/ will be copied to the 
+# and all html, png, and css files in /path/from/ will be copied to the
 # (possibly new) directory /path/to/file.html.LyXconv/.
 # The -t argument determines the extension added, the default being "LyXconv".
 # If just . is given, no extension is added.
+# If the -d option is given then the document directory /path/to/ will be used
+# to copy the files no subdirectory will be created/used.
 
 import getopt, os, shutil, sys
 from lyxpreview_tools import error
 
+
 def usage(prog_name):
-return "Usage: %s [-e extensions] [-t target extension]  " % prog_name
+msg = "Usage: %s [-d] [-e extensions] [-t target extension] from_file 
to_file"
+return  msg % prog_name
+
 
 def main(argv):
 progname = argv[0]
 
 exts = [] #list of extensions for which we're checking
+use_doc_dir = False
 targext = "LyXconv" #extension for target directory
-opts, args = getopt.getopt(sys.argv[1:], "e:t:")
+opts, args = getopt.getopt(sys.argv[1:], "de:t:")
 for o, v in opts:
   if o == "-e":
 exts = v.split(',')
   if o == "-t":
 targext = v
+  if o == "-d":
+use_doc_dir = True
 
 # input directory
 if len(args) != 2:
@@ -49,11 +57,14 @@ def main(argv):
 from_dir = os.path.dirname(abs_from_file)
 
 # output directory
-to_dir = args[1]
-if targext != '.':
-  to_dir += "." + targext
-if not os.path.isabs(to_dir):
-  error("%s is not an absolute file name.\n%s" % to_dir, usage(progname))
+if use_doc_dir:
+  to_dir = os.path.dirname(args[1])
+else:
+  to_dir = args[1]
+  if targext != '.':
+to_dir += "." + targext
+  if not os.path.isabs(to_dir):
+error("%s is not an absolute file name.\n%s" % to_dir, usage(progname))
 
 if not copy_all(from_dir, to_dir, exts):
   # some kind of failure


[LyX/master] Fix call of layout2layout as a module.

2017-05-13 Thread José Matos
commit cc7ca138dabec6090edc9b4e8d18d0239f04aa99
Author: José Matos <jama...@lyx.org>
Date:   Sat May 13 15:25:44 2017 +0100

Fix call of layout2layout as a module.

If we call parser.parse_args(), thus with no arguments, the parser uses
sys.argv (because that is the default). We should pass argv since that was
the purpose of handling argv in the main function.

We pass argv[1:] since when parsing the arguments we always ignore the name
of the program.

Use the full power of argparse to declare the default value of the 
end_format.
---
 lib/scripts/layout2layout.py |   17 ++---
 1 files changed, 6 insertions(+), 11 deletions(-)

diff --git a/lib/scripts/layout2layout.py b/lib/scripts/layout2layout.py
index 0633c81..56b4339 100644
--- a/lib/scripts/layout2layout.py
+++ b/lib/scripts/layout2layout.py
@@ -1172,14 +1172,14 @@ def main(argv):
 
 parser = argparse.ArgumentParser(**args)
 
-parser.add_argument("-t", "--to", type=int, dest="format",
+parser.add_argument("-t", "--to", type=int, dest="format", default= 
currentFormat,
 help=("destination layout format, default %i 
(latest)") % currentFormat)
 parser.add_argument("input_file", nargs='?', type=cmd_arg, default=None,
 help="input file (default stdin)")
 parser.add_argument("output_file", nargs='?', type=cmd_arg, default=None,
 help="output file (default stdout)")
 
-options = parser.parse_args(argv)
+options = parser.parse_args(argv[1:])
 
 # Open files
 if options.input_file:
@@ -1192,19 +1192,14 @@ def main(argv):
 else:
 output = sys.stdout
 
-if options.format:
-end_format = options.format
-else:
-end_format = currentFormat
-
-if end_format > currentFormat:
-error("Format %i does not exist" % end_format);
+if options.format > currentFormat:
+error("Format %i does not exist" % options.format);
 
 # Do the real work
 lines = read(source)
 format = 1
-while (format < end_format):
-format = convert(lines, end_format)
+while (format < options.format):
+format = convert(lines, options.format)
 write(output, lines)
 
 # Close files


[LyX/master] Fix call os lyx2lyx as a module.

2017-05-13 Thread José Matos
commit 99cf71f8ebdaaaf1001963b8f52df91fa6733199
Author: José Matos <jama...@lyx.org>
Date:   Sat May 13 13:39:02 2017 +0100

Fix call os lyx2lyx as a module.

If we call parser.parse_args(), thus with no arguments, the parser uses
sys.argv (because that is the default). We should pass argv since that was
the purpose of handling argv in the main function.
---
 lib/scripts/layout2layout.py |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/scripts/layout2layout.py b/lib/scripts/layout2layout.py
index f16aba0..0633c81 100644
--- a/lib/scripts/layout2layout.py
+++ b/lib/scripts/layout2layout.py
@@ -1179,7 +1179,7 @@ def main(argv):
 parser.add_argument("output_file", nargs='?', type=cmd_arg, default=None,
 help="output file (default stdout)")
 
-options = parser.parse_args()
+options = parser.parse_args(argv)
 
 # Open files
 if options.input_file:


[LyX/master] cleanup: remove unneeded whitespaces

2017-05-13 Thread José Matos
commit 6138880c0845a9eb6e5b9bfee107e4ee4210cd95
Author: José Matos <jama...@lyx.org>
Date:   Sat May 13 13:38:01 2017 +0100

cleanup: remove unneeded whitespaces
---
 lib/scripts/layout2layout.py |   18 +-
 1 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/lib/scripts/layout2layout.py b/lib/scripts/layout2layout.py
index 79c7541..f16aba0 100644
--- a/lib/scripts/layout2layout.py
+++ b/lib/scripts/layout2layout.py
@@ -49,7 +49,7 @@ currentFormat = 63
 # Rename I18NPreamble to BabelPreamble and add LangPreamble
 
 # Incremented to format 15, 28 May 2009 by lasgouttes
-# Add new tag OutputFormat; modules can be conditioned on feature 
+# Add new tag OutputFormat; modules can be conditioned on feature
 # "from->to".
 
 # Incremented to format 16, 5 June 2009 by rgh
@@ -338,7 +338,7 @@ def convert(lines, end_format):
 # Arguments
 re_OptArgs = re.compile(b'^(\\s*)OptionalArgs(\\s+)(\\d+)\\D*$', 
re.IGNORECASE)
 re_ReqArgs = re.compile(b'^(\\s*)RequiredArgs(\\s+)(\\d+)\\D*$', 
re.IGNORECASE)
-
+
 # various changes associated with changing how chapters are handled
 re_LabelTypeIsCounter = re.compile(b'^(\\s*)LabelType(\\s*)Counter\\s*$', 
re.IGNORECASE)
 re_TopEnvironment = 
re.compile(b'^(\\s*)LabelType(\\s+)Top_Environment\\s*$', re.IGNORECASE)
@@ -416,7 +416,7 @@ def convert(lines, end_format):
   lcat = ConvDict[lcat]
   lines[i] = lpre + b"{" + lnam + b"}"
   lines.insert(i+1, b"#  \\DeclareCategory{" + lcat + b"}")
-  i += 1 
+  i += 1
   i += 1
   continue
 
@@ -625,7 +625,7 @@ def convert(lines, end_format):
 lines[i] = b"InsetLayout Caption:Standard"
   i += 1
   continue
-
+
 if format == 41:
 # nothing to do.
 i += 1
@@ -694,7 +694,7 @@ def convert(lines, end_format):
 reqs = 0
 i += 1
 continue
-
+
 if format == 39:
 # There is a conversion with format 40, but it is done within the
 # initial comment block and so is above.
@@ -726,7 +726,7 @@ def convert(lines, end_format):
   i += 1
   continue
 style = match.group(2)
-
+
 if flexstyles.count(style):
   lines[i] = match.group(1) + b"\"Flex:" + style + b"\""
 i += 1
@@ -797,7 +797,7 @@ def convert(lines, end_format):
   lines[i] = b"\tCopyStyle \"Flex:" + match.group(1) + b"\""
   i += 1
   continue
-
+
 # Only new features
 if format >= 24 and format <= 27:
   i += 1
@@ -842,8 +842,8 @@ def convert(lines, end_format):
 cmd = b"listoffigures"
   # else unknown, which is why we're doing this
 i += 1
-  continue  
-  
+  continue
+
 # This just involved new features, not any changes to old ones
 if format >= 14 and format <= 22:
   i += 1


[LyX/master] prefs2prefs.py: whitespace changes

2017-05-12 Thread José Matos
commit 430869a1bdf66392ac79740ab84822534430de6e
Author: José Matos <jama...@lyx.org>
Date:   Fri May 12 15:29:29 2017 +0100

prefs2prefs.py: whitespace changes

Bring this file in line with the other python scripts:
  * use four spaces for indentation instead of a tab

Remove empty whitespace at the end of lines.
---
 lib/scripts/prefs2prefs.py |  264 ++--
 1 files changed, 132 insertions(+), 132 deletions(-)

diff --git a/lib/scripts/prefs2prefs.py b/lib/scripts/prefs2prefs.py
index 064de9c..493ddf5 100644
--- a/lib/scripts/prefs2prefs.py
+++ b/lib/scripts/prefs2prefs.py
@@ -10,26 +10,26 @@
 
 # This is the main file for the user preferences conversion system.
 # There are two subsidiary files:
-#  prefs2prefs_lfuns.py
-#  prefs2prefs_prefs.py
+# prefs2prefs_lfuns.py
+# prefs2prefs_prefs.py
 # The former is used to convert bind and ui files; the latter, to convert
 # the preferences file.
 #
-# I've organized it this way because, in many ways, converting bind and ui 
-# files  lfuns) and converting the preferences file are the same task. It's 
-# very line-by-line, unlike lyx2lyx and layout2layout, where changes can be 
-# more "global". So we read the file, line by line, and give a bunch of 
+# I've organized it this way because, in many ways, converting bind and ui
+# files  lfuns) and converting the preferences file are the same task. It's
+# very line-by-line, unlike lyx2lyx and layout2layout, where changes can be
+# more "global". So we read the file, line by line, and give a bunch of
 # converter functions a chance to see if they want to modify that line.
 
-# The converter functions are all in the subsidiary files. They take a line 
-# as  argument and return a list: (Bool, NewLine), where the Bool says if 
+# The converter functions are all in the subsidiary files. They take a line
+# as  argument and return a list: (Bool, NewLine), where the Bool says if
 # we've modified anything and the NewLine is the new line, if so, which will
 # be used to replace the old line.
 
-# The format of the existing files is format 0, as of 2.0.alpha6. We'll 
+# The format of the existing files is format 0, as of 2.0.alpha6. We'll
 # introduce new format numbers as we proceed, just as with layout2layout.
 # These will be different for the bind and ui files and for the preferences
-# file. 
+# file.
 
 from __future__ import print_function
 import os, re, string, sys
@@ -40,23 +40,23 @@ import io
 # Utility functions, borrowed from layout2layout.py
 
 def trim_bom(line):
-   " Remove byte order mark."
-   if line[0:3] == u"\357\273\277":
-   return line[3:]
-   else:
-   return  line
+" Remove byte order mark."
+if line[0:3] == u"\357\273\277":
+return line[3:]
+else:
+return line
 
 
 def read(source):
-   " Read input file and strip lineendings."
-   lines = source.read().splitlines() or ['']
-   lines[0] = trim_bom(lines[0])
-   return lines
+" Read input file and strip lineendings."
+lines = source.read().splitlines() or ['']
+lines[0] = trim_bom(lines[0])
+return lines
 
 
 def write(output, lines):
-   " Write output file with native lineendings."
-   output.write(os.linesep.join(lines) + os.linesep)
+" Write output file with native lineendings."
+output.write(os.linesep.join(lines) + os.linesep)
 
 
 # for use by find_format_lines
@@ -64,138 +64,138 @@ re_comment = re.compile(r'^#')
 re_empty   = re.compile(r'^\s*$')
 
 def find_format_line(lines):
-   '''
-   Returns (bool, int), where int is number of the line the `Format' 
-   specification is on, or else the number of the first non-blank, 
-   non-comment line. The bool tells whether we found a format line.
-   '''
-   for i in range(len(lines)):
-   l = lines[i]
-   if re_comment.search(l) or re_empty.search(l):
-   continue
-   m = re_format.search(l)
-   if m:
-   return (True, i)
-   # we're done when we have hit a non-comment, non-empty line
-   break
-   return (False, i)
+'''
+Returns (bool, int), where int is number of the line the `Format'
+specification is on, or else the number of the first non-blank,
+non-comment line. The bool tells whether we found a format line.
+'''
+for i in range(len(lines)):
+l = lines[i]
+if re_comment.search(l) or re_empty.search(l):
+continue
+m = re_format.search(l)
+if m:
+return (True, i)
+# we're done when we have hit a non-comment, non-empty line
+break
+return (False, i)
 
 
 # for use by get_format
 re_format  = re.compile(r'^Format\s+(\d+)\s*$')
 
 def get_format(l

[LyX/master] python3: fix the preview framework to work with both python 2 and 3 (part 4)

2017-05-12 Thread José Matos
commit 376cb6763fe1ea047fc3cfaad80969c3a5ebdd7e
Author: José Matos <jama...@lyx.org>
Date:   Fri May 12 10:01:48 2017 +0100

python3: fix the preview framework to work with both python 2 and 3 (part 4)

Remove support for python 1.x (really)

This code has not been used for a long time, probably never, since some code
above requires at least python 2.4 to work.

I got to this code by running futurize from python-future. There are no
significant warnings, mostly are related with the division but since
we are dividing floats there is no change between python 2 and 3.
---
 lib/scripts/legacy_lyxpreview2ppm.py |8 ++--
 lib/scripts/lyxpreview2bitmap.py |6 ++--
 lib/scripts/lyxpreview_tools.py  |   57 +
 3 files changed, 9 insertions(+), 62 deletions(-)

diff --git a/lib/scripts/legacy_lyxpreview2ppm.py 
b/lib/scripts/legacy_lyxpreview2ppm.py
index 36d57ca..99f4e56 100644
--- a/lib/scripts/legacy_lyxpreview2ppm.py
+++ b/lib/scripts/legacy_lyxpreview2ppm.py
@@ -79,11 +79,11 @@
 # If possible, the script will use pdftocairo instead of gs,
 # as it's much faster and gives better results.
 
-import glob, os, pipes, re, sys
+import glob, os, pipes, re, sys, tempfile
 
 from lyxpreview_tools import check_latex_log, copyfileobj, error, 
filter_pages,\
  find_exe, find_exe_or_terminate, join_metrics_and_rename, latex_commands, 
\
- latex_file_re, make_texcolor, mkstemp, pdflatex_commands, progress, \
+ latex_file_re, make_texcolor, pdflatex_commands, progress, \
  run_command, run_latex, warning, write_metrics_info
 
 
@@ -205,7 +205,7 @@ def legacy_latex_file(latex_file, fg_color, bg_color):
 fg_color_gr = make_texcolor(fg_color, True)
 bg_color_gr = make_texcolor(bg_color, True)
 
-tmp = mkstemp()
+tmp = tempfile.TemporaryFile()
 
 success = 0
 try:
@@ -255,7 +255,7 @@ def crop_files(pnmcrop, basename):
 t.append('%s -right' % pnmcrop, '--')
 
 for file in glob.glob("%s*.ppm" % basename):
-tmp = mkstemp()
+tmp = tempfile.TemporaryFile()
 new = t.open(file, "r")
 copyfileobj(new, tmp)
 if not new.close():
diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index 6a49f16..e4bf131 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -77,14 +77,14 @@
 
 from __future__ import print_function
 
-import getopt, glob, os, re, shutil, sys
+import getopt, glob, os, re, shutil, sys, tempfile
 
 from legacy_lyxpreview2ppm import extract_resolution, legacy_conversion_step1
 
 from lyxpreview_tools import bibtex_commands, check_latex_log, copyfileobj, \
  error, filter_pages, find_exe, find_exe_or_terminate, \
  join_metrics_and_rename, latex_commands, latex_file_re, make_texcolor, \
- mkstemp, pdflatex_commands, progress, run_command, run_latex, run_tex, \
+ pdflatex_commands, progress, run_command, run_latex, run_tex, \
  warning, write_metrics_info
 
 PY2 = sys.version_info[0] == 2
@@ -167,7 +167,7 @@ def fix_latex_file(latex_file, pdf_output):
 def_re = re.compile(b"(newcommandx|globallongdef)"
 b"([a-zA-Z]+)")
 
-tmp = mkstemp()
+tmp = tempfile.TemporaryFile()
 
 changed = False
 macros = []
diff --git a/lib/scripts/lyxpreview_tools.py b/lib/scripts/lyxpreview_tools.py
index 8e6dee4..91cc4d6 100644
--- a/lib/scripts/lyxpreview_tools.py
+++ b/lib/scripts/lyxpreview_tools.py
@@ -10,7 +10,7 @@
 #   Paul A. Rubin, ru...@msu.edu.
 
 # A repository of the following functions, used by the lyxpreview2xyz scripts.
-# copyfileobj, error, find_exe, find_exe_or_terminate, make_texcolor, mkstemp,
+# copyfileobj, error, find_exe, find_exe_or_terminate, make_texcolor,
 # progress, run_command, run_latex, warning
 
 # Requires python 2.4 or later (subprocess module).
@@ -196,16 +196,6 @@ def run_command(cmd, stderr2stdout = True):
 return run_command_popen(cmd, stderr2stdout)
 
 
-def get_version_info():
-version_re = re.compile("([0-9])\.([0-9])")
-
-match = version_re.match(sys.version)
-if match == None:
-error("Unable to extract version info from 'sys.version'")
-
-return int(match.group(1)), int(match.group(2))
-
-
 def copyfileobj(fsrc, fdst, rewind=0, length=16*1024):
 """copy data from file-like object fsrc to file-like object fdst"""
 if rewind:
@@ -219,56 +209,13 @@ def copyfileobj(fsrc, fdst, rewind=0, length=16*1024):
 fdst.write(buf)
 
 
-class TempFile:
-"""clone of tempfile.TemporaryFile to use with python < 2.0."""
-# Cache the unlinker so we don't get spurious errors at shutdown
-# when the module-level "os" is None'd out.  Note that this must
-# be referenced as self.unlink, because the name 

[LyX/master] python3: fix the preview framework to work with both python 2 and 3 (part 3)

2017-05-11 Thread José Matos
commit 1f5241f7474b99699a8d1d6a9d60bcdc599790d7
Author: José Matos <jama...@lyx.org>
Date:   Thu May 11 20:56:56 2017 +0100

python3: fix the preview framework to work with both python 2 and 3 (part 3)

trivial fix: xrange -> range
---
 lib/scripts/lyxpreview2bitmap.py |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index 11f2ba9..6a49f16 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -283,7 +283,7 @@ def find_ps_pages(dvi_file):
 
 # Use page ranges, as a list of pages could exceed command line
 # maximum length (especially under Win32)
-for index in xrange(1, page_index + 1):
+for index in range(1, page_index + 1):
 if (not index in ps_or_pdf_pages) and skip:
 # We were skipping pages but current page shouldn't be skipped.
 # Add this page to -pp, it could stay alone or become the


[LyX/master] python3: fix the preview framework to work with both python 2 and 3 (part 2)

2017-05-11 Thread José Matos
commit c546977c6eb7d780a2995b57c468f1e935fc9fe3
Author: José Matos <jama...@lyx.org>
Date:   Thu May 11 15:50:08 2017 +0100

python3: fix the preview framework to work with both python 2 and 3 (part 2)
---
 lib/scripts/legacy_lyxpreview2ppm.py |   32 
 lib/scripts/lyxpreview2bitmap.py |6 ++
 lib/scripts/lyxpreview_tools.py  |6 +++---
 3 files changed, 25 insertions(+), 19 deletions(-)

diff --git a/lib/scripts/legacy_lyxpreview2ppm.py 
b/lib/scripts/legacy_lyxpreview2ppm.py
index 6a91402..36d57ca 100644
--- a/lib/scripts/legacy_lyxpreview2ppm.py
+++ b/lib/scripts/legacy_lyxpreview2ppm.py
@@ -201,7 +201,7 @@ def extract_resolution(log_file, dpi):
 
 
 def legacy_latex_file(latex_file, fg_color, bg_color):
-use_preview_re = re.compile(r"\s*\\usepackage\[([^]]+)\]{preview}")
+use_preview_re = re.compile(b"\\s*usepackage\\[([^]]+)\\]{preview}")
 fg_color_gr = make_texcolor(fg_color, True)
 bg_color_gr = make_texcolor(bg_color, True)
 
@@ -209,7 +209,7 @@ def legacy_latex_file(latex_file, fg_color, bg_color):
 
 success = 0
 try:
-f = open(latex_file, 'r')
+f = open(latex_file, 'rb')
 except:
 # Unable to open the file, but do nothing here because
 # the calling function will act on the value of 'success'.
@@ -227,20 +227,20 @@ def legacy_latex_file(latex_file, fg_color, bg_color):
 success = 1
 # Package order: color should be loaded before preview
 # Preview options: add the options lyx and tightpage
-tmp.write(r"""
-\usepackage{color}
-\definecolor{fg}{rgb}{%s}
-\definecolor{bg}{rgb}{%s}
-\pagecolor{bg}
-\usepackage[%s,tightpage]{preview}
-\makeatletter
-\def\t@a{cmr}
-\if\f@family\t@a
-\IfFileExists{lmodern.sty}{\usepackage{lmodern}}{\usepackage{ae,aecompl}}
-\fi
-\g@addto@macro\preview{\begingroup\color{bg}\special{ps::clippath 
fill}\color{fg}}
-\g@addto@macro\endpreview{\endgroup}
-\makeatother
+tmp.write(b"""
+\\usepackage{color}
+\\definecolor{fg}{rgb}{%s}
+\\definecolor{bg}{rgb}{%s}
+\\pagecolor{bg}
+\\usepackage[%s,tightpage]{preview}
+\\makeatletter
+\\def\\t@a{cmr}
+\\if\\f@family\\t@a
+\\IfFileExists{lmodern.sty}{\\usepackage{lmodern}}{\\usepackage{ae,aecompl}}
+\\fi
+\\g@addto@macro\\preview{\\begingroup\\color{bg}\\special{ps::clippath 
fill}\\color{fg}}
+\\g@addto@macro\\endpreview{\\endgroup}
+\\makeatother
 """ % (fg_color_gr, bg_color_gr, match.group(1)))
 
 if success:
diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index 1848366..11f2ba9 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -87,6 +87,7 @@ from lyxpreview_tools import bibtex_commands, 
check_latex_log, copyfileobj, \
  mkstemp, pdflatex_commands, progress, run_command, run_latex, run_tex, \
  warning, write_metrics_info
 
+PY2 = sys.version_info[0] == 2
 
 def usage(prog_name):
 msg = """
@@ -385,6 +386,11 @@ def main(argv):
 if len(dir) != 0:
 os.chdir(dir)
 
+# For python > 2 convert strings to bytes
+if not PY2:
+fg_color = bytes(fg_color, 'ascii')
+bg_color = bytes(bg_color, 'ascii')
+
 fg_color_dvipng = make_texcolor(fg_color, False)
 bg_color_dvipng = make_texcolor(bg_color, False)
 
diff --git a/lib/scripts/lyxpreview_tools.py b/lib/scripts/lyxpreview_tools.py
index b7a89dc..8e6dee4 100644
--- a/lib/scripts/lyxpreview_tools.py
+++ b/lib/scripts/lyxpreview_tools.py
@@ -72,7 +72,7 @@ def error(message):
 
 def make_texcolor(hexcolor, graphics):
 # Test that the input string contains 6 hexadecimal chars.
-hexcolor_re = re.compile("^[0-9a-fA-F]{6}$")
+hexcolor_re = re.compile(b"^[0-9a-fA-F]{6}$")
 if not hexcolor_re.match(hexcolor):
 error("Cannot convert color '%s'" % hexcolor)
 
@@ -81,9 +81,9 @@ def make_texcolor(hexcolor, graphics):
 blue  = float(int(hexcolor[4:6], 16)) / 255.0
 
 if graphics:
-return "%f,%f,%f" % (red, green, blue)
+return b"%f,%f,%f" % (red, green, blue)
 else:
-return "rgb %f %f %f" % (red, green, blue)
+return b"rgb %f %f %f" % (red, green, blue)
 
 
 def find_exe(candidates):


[LyX/master] make python string compliant with python 2 and 3

2017-05-09 Thread José Matos
commit 6495cd135f15c3775613c79b008ad62f6726573e
Author: José Matos <jama...@lyx.org>
Date:   Tue May 9 16:53:32 2017 +0100

make python string compliant with python 2 and 3

python 2 does not allow to declare a string as raw byte so we double
the backslashes and remove the r preffix

python 3 accepts rb"..." meaning a byte string that is raw. In this context
raw means that the backslash does not has any special meaning and thus it
is not escaped. This is usefull together with regular expressions where the
backslashes are special.

In the worst possible case, like this one, we must use 4 backslashes to 
represent
one in the regular expression...
---
 lib/scripts/lyxpreview2bitmap.py |4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index 68d0a4d..094e8f9 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -161,7 +161,9 @@ def extract_metrics_info(dvipng_stdout):
 
 
 def fix_latex_file(latex_file, pdf_output):
-def_re = re.compile(rb"(\\newcommandx|\\global\\long\\def)(\\[a-zA-Z]+)")
+# python 2 does not allow to declare a string as raw byte so we double
+# the backslashes and remove the r preffix
+def_re = 
re.compile(rb"(newcommandx|globallongdef)([a-zA-Z]+)")
 
 tmp = mkstemp()
 


[LyX/master] python3: fix the preview framework to work with both python 2 and 3

2017-05-09 Thread José Matos
commit 11f2a59ce90b406537ff3379276d5e0bb1bf6c37
Author: José Matos <jama...@lyx.org>
Date:   Tue May 9 11:20:34 2017 +0100

python3: fix the preview framework to work with both python 2 and 3
---
 lib/scripts/legacy_lyxpreview2ppm.py |   16 
 lib/scripts/lyxpreview2bitmap.py |   28 +++-
 lib/scripts/lyxpreview_tools.py  |   12 ++--
 3 files changed, 29 insertions(+), 27 deletions(-)

diff --git a/lib/scripts/legacy_lyxpreview2ppm.py 
b/lib/scripts/legacy_lyxpreview2ppm.py
index 3049b44..6a91402 100644
--- a/lib/scripts/legacy_lyxpreview2ppm.py
+++ b/lib/scripts/legacy_lyxpreview2ppm.py
@@ -79,7 +79,7 @@
 # If possible, the script will use pdftocairo instead of gs,
 # as it's much faster and gives better results.
 
-import glob, os, pipes, re, string, sys
+import glob, os, pipes, re, sys
 
 from lyxpreview_tools import check_latex_log, copyfileobj, error, 
filter_pages,\
  find_exe, find_exe_or_terminate, join_metrics_and_rename, latex_commands, 
\
@@ -118,8 +118,8 @@ def legacy_extract_metrics_info(log_file):
 error("Unexpected data in %s\n%s" % (log_file, line))
 
 if snippet:
-ascent  = string.atof(match.group(2))
-descent = string.atof(match.group(3))
+ascent  = float(match.group(2))
+descent = float(match.group(3))
 
 frac = 0.5
 if ascent == 0 and descent == 0:
@@ -139,8 +139,8 @@ def legacy_extract_metrics_info(log_file):
 results.append((int(match.group(1)), frac))
 
 else:
-tp_descent = string.atof(match.group(2))
-tp_ascent  = string.atof(match.group(4))
+tp_descent = float(match.group(2))
+tp_ascent  = float(match.group(4))
 
 except:
 # Unable to open the file, but do nothing here because
@@ -177,7 +177,7 @@ def extract_resolution(log_file, dpi):
 match = extract_decimal_re.search(line)
 if match == None:
 error("Unable to parse: %s" % line)
-fontsize = string.atof(match.group(1))
+fontsize = float(match.group(1))
 found_fontsize = 1
 continue
 
@@ -187,7 +187,7 @@ def extract_resolution(log_file, dpi):
 match = extract_integer_re.search(line)
 if match == None:
 error("Unable to parse: %s" % line)
-magnification = string.atof(match.group(1))
+magnification = float(match.group(1))
 found_magnification = 1
 continue
 
@@ -275,7 +275,7 @@ def legacy_conversion(argv, skipMetrics = False):
 if len(dir) != 0:
 os.chdir(dir)
 
-dpi = string.atoi(argv[2])
+dpi = int(argv[2])
 
 output_format = argv[3]
 
diff --git a/lib/scripts/lyxpreview2bitmap.py b/lib/scripts/lyxpreview2bitmap.py
index 95bb895..68d0a4d 100755
--- a/lib/scripts/lyxpreview2bitmap.py
+++ b/lib/scripts/lyxpreview2bitmap.py
@@ -75,7 +75,9 @@
 # Moreover dvipng can't work with PDF files, so, if the CONVERTER
 # paramter is pdflatex we have to fallback to legacy route (step 2).
 
-import getopt, glob, os, re, shutil, string, sys
+from __future__ import print_function
+
+import getopt, glob, os, re, shutil, sys
 
 from legacy_lyxpreview2ppm import extract_resolution, legacy_conversion_step1
 
@@ -134,8 +136,8 @@ def extract_metrics_info(dvipng_stdout):
 success = 1
 
 # Calculate the 'ascent fraction'.
-descent = string.atof(match.group(1))
-ascent  = string.atof(match.group(2))
+descent = float(match.group(1))
+ascent  = float(match.group(2))
 
 frac = 0.5
 if ascent < 0:
@@ -159,25 +161,25 @@ def extract_metrics_info(dvipng_stdout):
 
 
 def fix_latex_file(latex_file, pdf_output):
-def_re = re.compile(r"(\\newcommandx|\\global\\long\\def)(\\[a-zA-Z]+)")
+def_re = re.compile(rb"(\\newcommandx|\\global\\long\\def)(\\[a-zA-Z]+)")
 
 tmp = mkstemp()
 
 changed = False
 macros = []
-for line in open(latex_file, 'r').readlines():
-if not pdf_output and line.startswith("\\documentclass"):
+for line in open(latex_file, 'rb').readlines():
+if not pdf_output and line.startswith(b"\\documentclass"):
 changed = True
-line += "\\PassOptionsToPackage{draft}{microtype}\n"
+line += b"\\PassOptionsToPackage{draft}{microtype}\n"
 else:
 match = def_re.match(line)
 if match != None:
 macroname = match.group(2)
 if macroname in macros:
 definecmd = match.group(1)
-if definecmd == "\\newcommandx":
+  

[LyX/master] Get the file system encoding from sys instead of the locale

2017-03-29 Thread José Matos
commit dc7ac99d18c59234ab5e9b12d290e3959e761e11
Author: José Matos <jama...@lyx.org>
Date:   Wed Mar 29 14:24:34 2017 +0100

Get the file system encoding from sys instead of the locale
---
 lib/configure.py |5 +
 1 files changed, 1 insertions(+), 4 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index 7f46ec6..aae6a85 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -196,10 +196,7 @@ def checkTeXPaths():
 from tempfile import mkstemp
 fd, tmpfname = mkstemp(suffix='.ltx')
 if os.name == 'nt':
-from locale import getdefaultlocale
-language, encoding = getdefaultlocale()
-if encoding == None:
-encoding = 'latin1'
+encoding = sys.getfilesystemencoding()
 if sys.version_info[0] < 3:
 inpname = shortPath(unicode(tmpfname, encoding)).replace('\\', 
'/')
 else:


[LyX/master] lyx2lyx: convert iteritems -> items to be compatible with python 2 and 3

2017-03-29 Thread José Matos
commit 67bf174b1016ee46fc2651d1a43ca2f20982fccf
Author: José Matos <jama...@lyx.org>
Date:   Wed Mar 29 12:34:53 2017 +0100

lyx2lyx: convert iteritems -> items to be compatible with python 2 and 3
---
 lib/lyx2lyx/lyx_2_3.py |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/lyx2lyx/lyx_2_3.py b/lib/lyx2lyx/lyx_2_3.py
index 5b770ed..ddf7822 100644
--- a/lib/lyx2lyx/lyx_2_3.py
+++ b/lib/lyx2lyx/lyx_2_3.py
@@ -166,7 +166,7 @@ def revert_ibranches(document):
 i += 1
 
 # now we need to add the new branches to the header
-for old, new in ibranches.iteritems():
+for old, new in ibranches.items():
 i = find_token(document.header, "\\branch " + old, 0)
 if i == -1:
 document.warning("Can't find branch %s even though we found it 
before!" % (old))


[LyX/master] lyx2lyx: fix #9006 (python3 with non-utf8 encoding as the system default)

2017-03-27 Thread José Matos
commit bcf715f398ead2e2c43af65bedcfecad27e48880
Author: José Matos <jama...@lyx.org>
Date:   Mon Mar 27 11:35:29 2017 +0100

lyx2lyx: fix #9006 (python3 with non-utf8 encoding as the system default)

This patch fixes lyx2lyx running on python 3 for those systems
where the default encoding is not UTF-8, since open by default
uses the default system encoding.
---
 lib/lyx2lyx/lyx_1_6.py |   20 
 lib/lyx2lyx/unicode_symbols.py |7 ++-
 2 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/lib/lyx2lyx/lyx_1_6.py b/lib/lyx2lyx/lyx_1_6.py
index b5650e2..636d4b2 100644
--- a/lib/lyx2lyx/lyx_1_6.py
+++ b/lib/lyx2lyx/lyx_1_6.py
@@ -23,12 +23,7 @@ import unicodedata
 import sys, os
 
 from parser_tools import find_token, find_end_of, find_tokens, get_value
-
-# Provide support for both python 2 and 3
-PY2 = sys.version_info[0] == 2
-if not PY2:
-unichr = chr
-# End of code to support for both python 2 and 3
+from unicode_symbols import read_unicodesymbols
 
 
 # Private helper functions
@@ -151,9 +146,18 @@ def set_option(document, m, option, value):
 return l
 
 
-# FIXME: Use the version in unicode_symbols.py which has some bug fixes
-def read_unicodesymbols():
+# FIXME: Remove this function if the version imported from unicode_symbols 
works.
+# This function was the predecessor from that function, that in the meanwhile 
got
+# new fixes.
+def read_unicodesymbols2():
 " Read the unicodesymbols list of unicode characters and corresponding 
commands."
+
+# Provide support for both python 2 and 3
+PY2 = sys.version_info[0] == 2
+if not PY2:
+unichr = chr
+# End of code to support for both python 2 and 3
+
 pathname = os.path.abspath(os.path.dirname(sys.argv[0]))
 fp = open(os.path.join(pathname.strip('lyx2lyx'), 'unicodesymbols'))
 spec_chars = []
diff --git a/lib/lyx2lyx/unicode_symbols.py b/lib/lyx2lyx/unicode_symbols.py
index f24f3a9..d9eeff9 100644
--- a/lib/lyx2lyx/unicode_symbols.py
+++ b/lib/lyx2lyx/unicode_symbols.py
@@ -29,7 +29,12 @@ if not PY2:
 def read_unicodesymbols():
 " Read the unicodesymbols list of unicode characters and corresponding 
commands."
 pathname = os.path.abspath(os.path.dirname(sys.argv[0]))
-fp = open(os.path.join(pathname.strip('lyx2lyx'), 'unicodesymbols'))
+filename = os.path.join(pathname.strip('lyx2lyx'), 'unicodesymbols')
+
+# For python 3+ we have to specify the encoding for those systems
+# where the default is not UTF-8
+fp = open(filename, encoding="utf8") if (not PY2) else open(filename)
+
 spec_chars = []
 # A backslash, followed by some non-word character, and then a character
 # in brackets. The idea is to check for constructs like: \"{u}, which is 
how


[LyX/master] lyx2lyx: remove trailing whitespaces from lyx_1_6.py

2017-03-27 Thread José Matos
commit 1269860b45646dac3a3425307dbdd56c99005732
Author: José Matos <jama...@lyx.org>
Date:   Mon Mar 27 11:26:58 2017 +0100

lyx2lyx: remove trailing whitespaces from lyx_1_6.py
---
 lib/lyx2lyx/lyx_1_6.py |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/lib/lyx2lyx/lyx_1_6.py b/lib/lyx2lyx/lyx_1_6.py
index f8409a0..b5650e2 100644
--- a/lib/lyx2lyx/lyx_1_6.py
+++ b/lib/lyx2lyx/lyx_1_6.py
@@ -430,7 +430,7 @@ def lyx2latex(document, lines):
 continue
 inert = ert_end >= curline
 content += lyxline2latex(document, lines[curline], inert)
-  
+
 return content
 
 


[LyX/master] lyx2lyx: more descriptive member name (document.start -> document.initial_format)

2017-03-27 Thread José Matos
commit 463e12827ce4466bd679d3f8c3fe3c6f5aff39e2
Author: José Matos <jama...@lyx.org>
Date:   Mon Mar 27 10:36:30 2017 +0100

lyx2lyx: more descriptive member name (document.start -> 
document.initial_format)
---
 lib/lyx2lyx/LyX.py |4 ++--
 lib/lyx2lyx/lyx_2_0.py |2 +-
 lib/lyx2lyx/lyx_2_3.py |2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/lyx2lyx/LyX.py b/lib/lyx2lyx/LyX.py
index b924690..c76fc58 100644
--- a/lib/lyx2lyx/LyX.py
+++ b/lib/lyx2lyx/LyX.py
@@ -393,6 +393,7 @@ class LyX_base:
 self.inputencoding = get_value(self.header, b"\\inputencoding", 0,
default = b"auto").decode('ascii')
 self.format = self.read_format()
+self.initial_format = self.format
 self.encoding = get_encoding(self.language,
  self.inputencoding, self.format,
  self.cjk_encoding)
@@ -698,7 +699,6 @@ class LyX_base:
 conversion are taken.  It returns a list of modules needed to
 convert the LyX file from self.format to self.end_format"""
 
-self.start =  self.format
 format = self.format
 correct_version = 0
 
@@ -732,7 +732,7 @@ class LyX_base:
 
 # Convertion mode, back or forth
 steps = []
-if (initial_step, self.start) < (final_step, self.end_format):
+if (initial_step, self.initial_format) < (final_step, self.end_format):
 mode = "convert"
 full_steps = []
 for step in format_relation:
diff --git a/lib/lyx2lyx/lyx_2_0.py b/lib/lyx2lyx/lyx_2_0.py
index 6f897d5..ad56364 100644
--- a/lib/lyx2lyx/lyx_2_0.py
+++ b/lib/lyx2lyx/lyx_2_0.py
@@ -820,7 +820,7 @@ def revert_suppress_date(document):
 
 def convert_mhchem(document):
 "Set mhchem to off for versions older than 1.6.x"
-if document.start < 277:
+if document.initial_format < 277:
 # LyX 1.5.x and older did never load mhchem.
 # Therefore we must switch it off: Documents that use mhchem have
 # a manual \usepackage anyway, and documents not using mhchem but
diff --git a/lib/lyx2lyx/lyx_2_3.py b/lib/lyx2lyx/lyx_2_3.py
index 1da16f2..5b770ed 100644
--- a/lib/lyx2lyx/lyx_2_3.py
+++ b/lib/lyx2lyx/lyx_2_3.py
@@ -1846,7 +1846,7 @@ def convert_dashligatures(document):
 
 i = find_token(document.header, "\\use_microtype", 0)
 if i != -1:
-if document.start > 474 and document.start < 509:
+if document.initial_format > 474 and document.initial_format < 509:
 # This was created by LyX 2.2
 document.header[i+1:i+1] = ["\\use_dash_ligatures false"]
 else:


[LyX/master] lyx2lyx: Cosmetic clean up of the lyx_2_?.py files.

2017-03-27 Thread José Matos
commit 37901e2e25da922fe7eab8736a3824188f2ec06c
Author: José Matos <jama...@lyx.org>
Date:   Mon Mar 27 10:16:31 2017 +0100

lyx2lyx: Cosmetic clean up of the lyx_2_?.py files.

Remove the duplicated \# -*- coding: utf-8 -*-
lines that show in the third line.
As far as I know they are not used, the first line though is used
by python and text editors (emacs, vi & co) to get the file encoding.
That will also work in the second lines (because of the shebang convention)
but not in other lines.

Remove trailing whitespaces.
---
 lib/lyx2lyx/lyx_2_0.py |  155 +--
 lib/lyx2lyx/lyx_2_1.py |  279 
 lib/lyx2lyx/lyx_2_2.py |   25 ++---
 lib/lyx2lyx/lyx_2_3.py |   15 ++--
 4 files changed, 235 insertions(+), 239 deletions(-)

diff --git a/lib/lyx2lyx/lyx_2_0.py b/lib/lyx2lyx/lyx_2_0.py
index 53ad3ea..6f897d5 100644
--- a/lib/lyx2lyx/lyx_2_0.py
+++ b/lib/lyx2lyx/lyx_2_0.py
@@ -1,6 +1,5 @@
 # -*- coding: utf-8 -*-
 # This file is part of lyx2lyx
-# -*- coding: utf-8 -*-
 # Copyright (C) 2011 The LyX team
 #
 # This program is free software; you can redistribute it and/or
@@ -27,7 +26,7 @@ from parser_tools import find_token, find_end_of, 
find_tokens, \
   find_token_exact, find_end_of_inset, find_end_of_layout, \
   find_token_backwards, is_in_inset, get_value, get_quoted_value, \
   del_token, check_token, get_option_value
-  
+
 from lyx2lyx_tools import add_to_preamble, insert_to_preamble, \
   put_cmd_in_ert, lyx2latex, latex_length, revert_flex_inset, \
   revert_font_attrs, hex2ratio, str2bool
@@ -156,7 +155,7 @@ def revert_phantom_types(document, ptype, cmd):
   i = end
   continue
   substi = ["\\begin_inset ERT", "status collapsed", "",
-"\\begin_layout Plain Layout", "", "", "\\backslash", 
+"\\begin_layout Plain Layout", "", "", "\\backslash",
 cmd + "{", "\\end_layout", "", "\\end_inset"]
   substj = ["\\size default", "", "\\begin_inset ERT", "status collapsed", 
"",
 "\\begin_layout Plain Layout", "", "}", "\\end_layout", "", 
"\\end_inset"]
@@ -168,7 +167,7 @@ def revert_phantom_types(document, ptype, cmd):
 
 def revert_phantom(document):
 revert_phantom_types(document, "Phantom", "phantom")
-
+
 def revert_hphantom(document):
 revert_phantom_types(document, "HPhantom", "hphantom")
 
@@ -200,7 +199,7 @@ def revert_xetex(document):
 roman = sans = typew = "default"
 osf = False
 sf_scale = tt_scale = 100.0
-
+
 i = find_token(document.header, "\\font_roman", 0)
 if i == -1:
 document.warning("Malformed LyX document: Missing \\font_roman.")
@@ -214,7 +213,7 @@ def revert_xetex(document):
 else:
 sans = get_value(document.header, "\\font_sans", i)
 document.header[i] = "\\font_sans default"
-
+
 i = find_token(document.header, "\\font_typewriter", 0)
 if i == -1:
 document.warning("Malformed LyX document: Missing \\font_typewriter.")
@@ -235,7 +234,7 @@ def revert_xetex(document):
 else:
 # we do not need this value.
 document.header[i] = "\\font_sc false"
-
+
 i = find_token(document.header, "\\font_sf_scale", 0)
 if i == -1:
 document.warning("Malformed LyX document: Missing \\font_sf_scale.")
@@ -286,7 +285,7 @@ def revert_xetex(document):
 
 def revert_outputformat(document):
 " Remove default output format param "
-
+
 if not del_token(document.header, '\\default_output_format', 0):
 document.warning("Malformed LyX document: Missing 
\\default_output_format.")
 
@@ -344,7 +343,7 @@ def revert_splitindex(document):
 preamble = []
 if useindices:
  preamble.append("\\usepackage{splitidx})")
-
+
 # deal with index declarations in the preamble
 i = 0
 while True:
@@ -355,7 +354,7 @@ def revert_splitindex(document):
 if k == -1:
 document.warning("Malformed LyX document: Missing \\end_index.")
 return
-if useindices:
+if useindices:
   line = document.header[i]
   l = re.compile(r'\\index (.*)$')
   m = l.match(line)
@@ -366,7 +365,7 @@ def revert_splitindex(document):
 del document.header[i:k + 1]
 if preamble:
 insert_to_preamble(document, preamble)
-
+
 # deal with index insets
 # these need to have the argument removed
 i = 0
@@ -392,7 +391,7 @@ def revert_

[LyX/master] lyx2lyx: Remove UTF-8 BOM if present in the lyx file

2017-03-15 Thread José Matos
commit ef6ff7fb53b0888f4f33f75814a0ba331fffcac1
Author: José Matos <jama...@lyx.org>
Date:   Wed Mar 15 10:06:30 2017 +

lyx2lyx: Remove UTF-8 BOM if present in the lyx file
---
 lib/lyx2lyx/LyX.py |9 +
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/lib/lyx2lyx/LyX.py b/lib/lyx2lyx/LyX.py
index 77ccdd0..b924690 100644
--- a/lib/lyx2lyx/LyX.py
+++ b/lib/lyx2lyx/LyX.py
@@ -29,6 +29,7 @@ import sys
 import re
 import time
 import io
+import codecs
 
 try:
 import lyx2lyx_version
@@ -304,12 +305,20 @@ class LyX_base:
 # use latin1. This works since a) the parts we are interested in are
 # pure ASCII (subset of latin1) and b) in contrast to pure ascii or
 # utf8, one can decode any 8byte string using latin1.
+first_line = True
 while True:
 line = self.input.readline()
 if not line:
 # eof found before end of header
 self.error("Invalid LyX file: Missing body.")
 
+if first_line:
+# Remove UTF8 BOM marker if present
+if line.startswith(codecs.BOM_UTF8):
+line = line[len(codecs.BOM_UTF8):]
+
+first_line = False
+
 if PY2:
 line = trim_eol(line)
 decoded = line


[LyX/2.2.x] White space change only (remove spaces at the end of lines)

2016-08-01 Thread José Matos
commit 41666164f7b9d6036660e8a1c3ada269e3d138fd
Author: José Matos <jama...@lyx.org>
Date:   Mon Aug 1 10:53:10 2016 +0100

White space change only (remove spaces at the end of lines)
---
 lib/configure.py |   28 ++--
 1 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index 5303ebf..93ee409 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -106,7 +106,7 @@ def setEnviron():
 
 def copy_tree(src, dst, preserve_symlinks=False, level=0):
 ''' Copy an entire directory tree 'src' to a new location 'dst'.
- 
+
 Code inspired from distutils.copy_tree.
 Copying ignores non-regular files and the cache directory.
 Pipes may be present as leftovers from LyX for lyx-server.
@@ -115,7 +115,7 @@ def copy_tree(src, dst, preserve_symlinks=False, level=0):
 copied as symlinks (on platforms that support them!); otherwise
 (the default), the destination of the symlink will be copied.
 '''
- 
+
 if not os.path.isdir(src):
 raise FileError, \
   "cannot copy tree '%s': not a directory" % src
@@ -124,12 +124,12 @@ def copy_tree(src, dst, preserve_symlinks=False, level=0):
 except os.error, (errno, errstr):
 raise FileError, \
   "error listing files in '%s': %s" % (src, errstr)
- 
+
 if not os.path.isdir(dst):
 os.makedirs(dst)
- 
+
 outputs = []
- 
+
 for name in names:
 src_name = os.path.join(src, name)
 dst_name = os.path.join(dst, name)
@@ -147,7 +147,7 @@ def copy_tree(src, dst, preserve_symlinks=False, level=0):
 outputs.append(dst_name)
 else:
 logger.info("Ignore non-regular file %s", src_name)
- 
+
 return outputs
 
 
@@ -766,7 +766,7 @@ def checkConverterEntries():
 \converter knitr   xetex  "%%" ""
 \converter knitr   luatex "%%" ""'''])
 #
-checkProg('a Sweave -> R/S code converter', ['Rscript --verbose --no-save 
--no-restore $$s/scripts/lyxstangle.R $$i $$e $$r'], 
+checkProg('a Sweave -> R/S code converter', ['Rscript --verbose --no-save 
--no-restore $$s/scripts/lyxstangle.R $$i $$e $$r'],
 rc_entry = [ r'\converter sweave  r  "%%"""' ])
 #
 checkProg('a knitr -> R/S code converter', ['Rscript --verbose --no-save 
--no-restore $$s/scripts/lyxknitr.R $$p$$i $$p$$o $$e $$r tangle'],
@@ -863,7 +863,7 @@ def checkConverterEntries():
 #
 checkProg('a RTF -> HTML converter', ['unrtf --html  $$i > $$o'],
 rc_entry = [ r'\converter rtf  html"%%"""' ])
-# Do not define a converter to pdf6, ps is a pure export format 
+# Do not define a converter to pdf6, ps is a pure export format
 checkProg('a PS to PDF converter', ['ps2pdf $$i $$o'],
 rc_entry = [ r'\converter ps pdf"%%"   ""' ])
 #
@@ -1162,19 +1162,19 @@ def processLayoutFile(file, bool_docbook):
 Declare lines look like this:
 
 \DeclareLaTeXClass[]{}
-
+
 Optionally, a \DeclareCategory line follows:
-
+
 \DeclareCategory{}
-
+
 So for example (article.layout, scrbook.layout, svjog.layout)
-
+
 \DeclareLaTeXClass{article}
 \DeclareCategory{Articles}
-
+
 \DeclareLaTeXClass[scrbook]{book (koma-script)}
 \DeclareCategory{Books}
-
+
 \DeclareLaTeXClass[svjour,svjog.clo]{article (Springer - svjour/jog)}
 
 we'd expect this output:


[LyX/2.2.x] Fix bug #10273

2016-08-01 Thread José Matos
commit 61a9f04754921abb28d9189c4449ae170d0b994f
Author: José Matos <jama...@lyx.org>
Date:   Mon Aug 1 11:04:39 2016 +0100

Fix bug #10273

The export of compressed files to previous lyx versions failed on windows.

The change in this case is to explicitly pass the output file since we know 
it.

At the same time take care of the fact that lyx2lyx knows what file format
versions corresponds to each lyx version.
---
 lib/configure.py |   24 
 status.22x   |5 -
 2 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index 93ee409..d604fad 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -1089,18 +1089,18 @@ def checkConverterEntries():
 \converter date   dateout"python -tt $$s/scripts/date.py %d-%m-%Y > 
$$o"   ""
 \converter docbookdocbook-xml "cp $$i $$o" "xml"
 \converter fenasciichess "python -tt $$s/scripts/fen2ascii.py $$i $$o" 
""
-\converter lyxlyx13x "python -tt $$s/lyx2lyx/lyx2lyx -t 221 $$i > 
$$o" ""
-\converter lyxlyx14x "python -tt $$s/lyx2lyx/lyx2lyx -t 245 $$i > 
$$o" ""
-\converter lyxlyx15x "python -tt $$s/lyx2lyx/lyx2lyx -t 276 $$i > 
$$o" ""
-\converter lyxlyx16x "python -tt $$s/lyx2lyx/lyx2lyx -t 345 $$i > 
$$o" ""
-\converter lyxlyx20x "python -tt $$s/lyx2lyx/lyx2lyx -t 413 $$i > 
$$o" ""
-\converter lyxlyx21x "python -tt $$s/lyx2lyx/lyx2lyx -t 474 $$i > 
$$o" ""
-\converter lyxclyx   "python -tt $$s/lyx2lyx/lyx2lyx -c big5 -t 
245 $$i > $$o" ""
-\converter lyxjlyx   "python -tt $$s/lyx2lyx/lyx2lyx -c euc_jp -t 
245 $$i > $$o"   ""
-\converter lyxklyx   "python -tt $$s/lyx2lyx/lyx2lyx -c euc_kr -t 
245 $$i > $$o"   ""
-\converter clyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c big5 $$i > 
$$o"""
-\converter jlyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c euc_jp $$i 
> $$o"  ""
-\converter klyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c euc_kr $$i 
> $$o"  ""
+\converter lyxlyx13x "python -tt $$s/lyx2lyx/lyx2lyx -V 1.3 -o $$o 
$$i"""
+\converter lyxlyx14x "python -tt $$s/lyx2lyx/lyx2lyx -V 1.4 -o $$o 
$$i"""
+\converter lyxlyx15x "python -tt $$s/lyx2lyx/lyx2lyx -V 1.5 -o $$o 
$$i"""
+\converter lyxlyx16x "python -tt $$s/lyx2lyx/lyx2lyx -V 1.6 -o $$o 
$$i"""
+\converter lyxlyx20x "python -tt $$s/lyx2lyx/lyx2lyx -V 2.0 -o $$o 
$$i"""
+\converter lyxlyx21x "python -tt $$s/lyx2lyx/lyx2lyx -V 2.1 -o $$o 
$$i"""
+\converter lyxclyx   "python -tt $$s/lyx2lyx/lyx2lyx -V 1.4 -o $$o 
-c big5   $$i"  ""
+\converter lyxjlyx   "python -tt $$s/lyx2lyx/lyx2lyx -V 1.4 -o $$o 
-c euc_jp $$i"  ""
+\converter lyxklyx   "python -tt $$s/lyx2lyx/lyx2lyx -V 1.4 -o $$o 
-c euc_kr $$i"  ""
+\converter clyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c big5   -o 
$$o $$i" ""
+\converter jlyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c euc_jp -o 
$$o $$i" ""
+\converter klyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c euc_kr -o 
$$o $$i" ""
 \converter lyxpreview png"python -tt $$s/scripts/lyxpreview2bitmap.py 
--png"   ""
 \converter lyxpreview ppm"python -tt $$s/scripts/lyxpreview2bitmap.py 
--ppm"   ""
 ''')
diff --git a/status.22x b/status.22x
index 1b4b576..3afded9 100644
--- a/status.22x
+++ b/status.22x
@@ -24,7 +24,7 @@ What's new
 * When changing zoom level, show current value in status bar (bug
   10212).
 
-  
+
 * DOCUMENTATION AND LOCALIZATION
 
 
@@ -52,6 +52,9 @@ What's new
 
 - Fix hang with some IEEEtran documents (bug 10307).
 
+- Fix export of compressed files to previous versions that were not
+  recognised on windows (bug 10273).
+
 
 * USER INTERFACE
 


[LyX/master] Improve the file format upgrade and downgrade for microtype

2016-07-13 Thread José Matos
commit cca93dacae12aeb50ce5ea63bd4ee8b7c3e01db0
Author: José Matos <jama...@lyx.org>
Date:   Wed Jul 13 15:01:17 2016 +0100

Improve the file format upgrade and downgrade for microtype

Add get_bool_value to parser_tools to be used in other places.
Make the upgrade convertion seamless from lyx writing the file (regarding
microtypes, there are still other dragons to chase. :-)
---
 lib/lyx2lyx/lyx_2_3.py  |   16 +---
 lib/lyx2lyx/parser_tools.py |   19 +++
 2 files changed, 28 insertions(+), 7 deletions(-)

diff --git a/lib/lyx2lyx/lyx_2_3.py b/lib/lyx2lyx/lyx_2_3.py
index fca8508..37386a3 100644
--- a/lib/lyx2lyx/lyx_2_3.py
+++ b/lib/lyx2lyx/lyx_2_3.py
@@ -28,9 +28,10 @@ import sys, os
 #from parser_tools import find_token, find_end_of, find_tokens, \
 #  find_token_exact, find_end_of_inset, find_end_of_layout, \
 #  find_token_backwards, is_in_inset, get_value, get_quoted_value, \
-#  del_token, check_token, get_option_value
+#  del_token, check_token, get_option_value, get_bool_value
 
-from parser_tools import find_token, find_end_of_inset, get_value
+from parser_tools import find_token, find_end_of_inset, get_value, \
+ get_bool_value
 
 #from lyx2lyx_tools import add_to_preamble, put_cmd_in_ert, get_ert, 
lyx2latex, \
 #  lyx2verbatim, length_in_bp, convert_info_insets
@@ -55,12 +56,13 @@ def convert_microtype(document):
 i = find_token(document.header, "\\font_tt_scale" , 0)
 if i == -1:
 document.warning("Malformed LyX document: Can't find \\font_tt_scale.")
-return;
+i = len(document.header) - 1
+
 j = find_token(document.preamble, "\\usepackage{microtype}", 0)
 if j == -1:
-document.header.insert(i + 1, "\\use_microtype 0")
+document.header.insert(i + 1, "\\use_microtype false")
 else:
-document.header.insert(i + 1, "\\use_microtype 1")
+document.header.insert(i + 1, "\\use_microtype true")
 del document.preamble[j]
 
 
@@ -69,9 +71,9 @@ def revert_microtype(document):
 i = find_token(document.header, "\\use_microtype", 0)
 if i == -1:
 return
-value = get_value(document.header, "\\use_microtype" , i).split()[0]
+use_microtype = get_bool_value(document.header, "\\use_microtype" , i)
 del document.header[i]
-if value == "1":
+if use_microtype:
 add_to_preamble(document, ["\\usepackage{microtype}"])
 
 
diff --git a/lib/lyx2lyx/parser_tools.py b/lib/lyx2lyx/parser_tools.py
index 85e2b6a..50ad3a5 100644
--- a/lib/lyx2lyx/parser_tools.py
+++ b/lib/lyx2lyx/parser_tools.py
@@ -315,6 +315,25 @@ def get_quoted_value(lines, token, start, end = 0, default 
= ""):
 return val.strip('"')
 
 
+def get_bool_value(lines, token, start, end = 0, default = None):
+""" get_value(lines, token, start[[, end], default]) -> string
+
+Find the next line that looks like:
+  token bool_value
+
+Returns True if bool_value is 1 or true and
+False if bool_value is 0 or false
+"""
+
+val = get_value(lines, token, start, end, "")
+
+if val == "1" or val == "true":
+return True
+if val == "0" or val == "false":
+return False
+return default
+
+
 def get_option_value(line, option):
 rx = option + '\s*=\s*"([^"]+)"'
 rx = re.compile(rx)


[LyX/master] Fix typo

2016-07-13 Thread José Matos
commit b41300d81984a6b7f4e4784c6dbc763bac49d5ee
Author: José Matos <jama...@lyx.org>
Date:   Wed Jul 13 16:46:52 2016 +0100

Fix typo
---
 src/BufferParams.h |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/src/BufferParams.h b/src/BufferParams.h
index 7314b42..aecf76b 100644
--- a/src/BufferParams.h
+++ b/src/BufferParams.h
@@ -382,7 +382,7 @@ public:
bool use_bibtopic;
/// Split the index?
bool use_indices;
-   /// Save trensient properties?
+   /// Save transient properties?
bool save_transient_properties;
/// revision tracking for this buffer ? (this is a transient property)
bool track_changes;


[LyX/master] Fix bug #10273

2016-07-08 Thread José Matos
commit 7848bdd4fe17a0e0ec33d39d77ceee26a09dea4c
Author: José Matos <jama...@lyx.org>
Date:   Fri Jul 8 16:06:03 2016 +0100

Fix bug #10273

Although this a problem that only manifests itself on windows the change is 
general
and it works anywhere.

The major change is to change the file redirection > to -o the specifies 
the output file.

At the same time it makes the call to lyx2lyx less cryptic, e.g. to revert 
to the 1.3
format we have:

\converter lyx lyx13x "python -tt $$s/lyx2lyx/lyx2lyx -t 221 $$i > $$o" 
""

now instead of
python -tt $$s/lyx2lyx/lyx2lyx -t 221 $$i > $$o

we use a call where the version to revert is explicit
python -tt $$s/lyx2lyx/lyx2lyx -V 1.3 -o $$o $$i

or we could write a longer, but more understandable form:
python -tt $$s/lyx2lyx/lyx2lyx --final_version 1.3 --output $$o $$i

FWIW I shuffled the order of the arguments just for the sake of readability,
to let $$i be the last argument.
---
 lib/configure.py |   26 +-
 1 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/lib/configure.py b/lib/configure.py
index ccf7226..6f13356 100644
--- a/lib/configure.py
+++ b/lib/configure.py
@@ -1086,19 +1086,19 @@ def checkConverterEntries():
 \converter csvlyx"python -tt $$s/scripts/csv2lyx.py $$i $$o"   
""
 \converter docbookdocbook-xml "cp $$i $$o" "xml"
 \converter fenasciichess "python -tt $$s/scripts/fen2ascii.py $$i $$o" 
""
-\converter lyxlyx13x "python -tt $$s/lyx2lyx/lyx2lyx -t 221 $$i > 
$$o" ""
-\converter lyxlyx14x "python -tt $$s/lyx2lyx/lyx2lyx -t 245 $$i > 
$$o" ""
-\converter lyxlyx15x "python -tt $$s/lyx2lyx/lyx2lyx -t 276 $$i > 
$$o" ""
-\converter lyxlyx16x "python -tt $$s/lyx2lyx/lyx2lyx -t 345 $$i > 
$$o" ""
-\converter lyxlyx20x "python -tt $$s/lyx2lyx/lyx2lyx -t 413 $$i > 
$$o" ""
-\converter lyxlyx21x "python -tt $$s/lyx2lyx/lyx2lyx -t 474 $$i > 
$$o" ""
-\converter lyxlyx22x "python -tt $$s/lyx2lyx/lyx2lyx -t 508 $$i > 
$$o" ""
-\converter lyxclyx   "python -tt $$s/lyx2lyx/lyx2lyx -c big5 -t 
245 $$i > $$o" ""
-\converter lyxjlyx   "python -tt $$s/lyx2lyx/lyx2lyx -c euc_jp -t 
245 $$i > $$o"   ""
-\converter lyxklyx   "python -tt $$s/lyx2lyx/lyx2lyx -c euc_kr -t 
245 $$i > $$o"   ""
-\converter clyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c big5 $$i > 
$$o"""
-\converter jlyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c euc_jp $$i 
> $$o"  ""
-\converter klyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c euc_kr $$i 
> $$o"  ""
+\converter lyxlyx13x "python -tt $$s/lyx2lyx/lyx2lyx -V 1.3 -o $$o 
$$i"""
+\converter lyxlyx14x "python -tt $$s/lyx2lyx/lyx2lyx -V 1.4 -o $$o 
$$i"""
+\converter lyxlyx15x "python -tt $$s/lyx2lyx/lyx2lyx -V 1.5 -o $$o 
$$i"""
+\converter lyxlyx16x "python -tt $$s/lyx2lyx/lyx2lyx -V 1.6 -o $$o 
$$i"""
+\converter lyxlyx20x "python -tt $$s/lyx2lyx/lyx2lyx -V 2.0 -o $$o 
$$i"""
+\converter lyxlyx21x "python -tt $$s/lyx2lyx/lyx2lyx -V 2.1 -o $$o 
$$i"""
+\converter lyxlyx22x "python -tt $$s/lyx2lyx/lyx2lyx -V 2.2 -o $$o 
$$i"""
+\converter lyxclyx   "python -tt $$s/lyx2lyx/lyx2lyx -V 1.4 -o $$o 
-c big5   $$i"  ""
+\converter lyxjlyx   "python -tt $$s/lyx2lyx/lyx2lyx -V 1.4 -o $$o 
-c euc_jp $$i"  ""
+\converter lyxklyx   "python -tt $$s/lyx2lyx/lyx2lyx -V 1.4 -o $$o 
-c euc_kr $$i"  ""
+\converter clyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c big5   -o 
$$o $$i" ""
+\converter jlyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c euc_jp -o 
$$o $$i" ""
+\converter klyx   lyx"python -tt $$s/lyx2lyx/lyx2lyx -c euc_kr -o 
$$o $$i" ""
 \converter lyxpreview png"python -tt $$s/scripts/lyxpreview2bitmap.py 
--png"   ""
 \converter lyxpreview ppm"python -tt $$s/scripts/lyxpreview2bitmap.py 
--ppm"   ""
 ''')


[LyX/master] Fix bug #9279

2015-03-31 Thread José Matos
commit 908e93df5060b29678cc51ad9a83c82fba1997cc
Author: José Matos jama...@lyx.org
Date:   Tue Mar 31 15:03:48 2015 +0100

Fix bug #9279

Make lyx2lyx parser a bit more robust.

Ignore if the first line of a lyx file does not start with a #
Add such a line if it is missing..

diff --git a/lib/lyx2lyx/LyX.py b/lib/lyx2lyx/LyX.py
index d96b7b4..9b8dd55 100644
--- a/lib/lyx2lyx/LyX.py
+++ b/lib/lyx2lyx/LyX.py
@@ -438,8 +438,22 @@ class LyX_base:
 
 def set_version(self):
  Set the header with the version used.
-self.header[0] =  .join([#LyX %s created this file. % version__,
-  For more info see http://www.lyx.org/;])
+
+initial_comment =  .join([#LyX %s created this file. % version__,
+For more info see http://www.lyx.org/;])
+
+# Simple heuristic to determine the comment that always starts
+# a lyx file
+if self.header[0].startswith(#):
+self.header[0] = initial_comment
+else:
+self.header.insert(0, initial_comment)
+
+# Old lyx files had a two lines comment header:
+# 1) the first line had the user who had created it
+# 2) the second line had the lyx version used
+# later we decided that 1) was a privacy risk for no gain
+# here we remove the second line effectively erasing 1)
 if self.header[1][0] == '#':
 del self.header[1]
 


[LyX/2.1.x] Make lyx2lyx parser a bit more robust.

2015-03-31 Thread José Matos
commit 5ed74b4446636e785aec380651f3c901c5fb53d2
Author: José Matos jama...@lyx.org
Date:   Tue Mar 31 15:29:41 2015 +0100

Make lyx2lyx parser a bit more robust.

Ignore if the first line of a lyx file does not start with a #
Add such a line if it is missing..

Fixes bug #9279

diff --git a/lib/lyx2lyx/LyX.py b/lib/lyx2lyx/LyX.py
index e3a57b7..7e52238 100644
--- a/lib/lyx2lyx/LyX.py
+++ b/lib/lyx2lyx/LyX.py
@@ -432,8 +432,22 @@ class LyX_base:
 
 def set_version(self):
  Set the header with the version used.
-self.header[0] =  .join([#LyX %s created this file. % version__,
-  For more info see http://www.lyx.org/;])
+
+initial_comment =  .join([#LyX %s created this file. % version__,
+For more info see http://www.lyx.org/;])
+
+# Simple heuristic to determine the comment that always starts
+# a lyx file
+if self.header[0].startswith(#):
+self.header[0] = initial_comment
+else:
+self.header.insert(0, initial_comment)
+
+# Old lyx files had a two lines comment header:
+# 1) the first line had the user who had created it
+# 2) the second line had the lyx version used
+# later we decided that 1) was a privacy risk for no gain
+# here we remove the second line effectively erasing 1)
 if self.header[1][0] == '#':
 del self.header[1]
 
diff --git a/status.21x b/status.21x
index 488f6a1..040616d 100644
--- a/status.21x
+++ b/status.21x
@@ -110,6 +110,8 @@ What's new
 
 - Fix handling of the TEXINPUTS environment variable on Windows (bug 9453).
 
+- Make lyx2lyx more robust, add the initial comment line that says what is the
+  lyx version that created the document if none is present (bug 9279).
 
 
 * USER INTERFACE


[LyX/master] Fix bug #9279

2015-03-31 Thread José Matos
commit 908e93df5060b29678cc51ad9a83c82fba1997cc
Author: José Matos <jama...@lyx.org>
Date:   Tue Mar 31 15:03:48 2015 +0100

Fix bug #9279

Make lyx2lyx parser a bit more robust.

Ignore if the first line of a lyx file does not start with a #
Add such a line if it is missing..

diff --git a/lib/lyx2lyx/LyX.py b/lib/lyx2lyx/LyX.py
index d96b7b4..9b8dd55 100644
--- a/lib/lyx2lyx/LyX.py
+++ b/lib/lyx2lyx/LyX.py
@@ -438,8 +438,22 @@ class LyX_base:
 
 def set_version(self):
 " Set the header with the version used."
-self.header[0] = " ".join(["#LyX %s created this file." % version__,
-  "For more info see http://www.lyx.org/;])
+
+initial_comment = " ".join(["#LyX %s created this file." % version__,
+"For more info see http://www.lyx.org/;])
+
+# Simple heuristic to determine the comment that always starts
+# a lyx file
+if self.header[0].startswith("#"):
+self.header[0] = initial_comment
+else:
+self.header.insert(0, initial_comment)
+
+# Old lyx files had a two lines comment header:
+# 1) the first line had the user who had created it
+# 2) the second line had the lyx version used
+# later we decided that 1) was a privacy risk for no gain
+# here we remove the second line effectively erasing 1)
 if self.header[1][0] == '#':
 del self.header[1]
 


[LyX/2.1.x] Make lyx2lyx parser a bit more robust.

2015-03-31 Thread José Matos
commit 5ed74b4446636e785aec380651f3c901c5fb53d2
Author: José Matos <jama...@lyx.org>
Date:   Tue Mar 31 15:29:41 2015 +0100

Make lyx2lyx parser a bit more robust.

Ignore if the first line of a lyx file does not start with a #
Add such a line if it is missing..

Fixes bug #9279

diff --git a/lib/lyx2lyx/LyX.py b/lib/lyx2lyx/LyX.py
index e3a57b7..7e52238 100644
--- a/lib/lyx2lyx/LyX.py
+++ b/lib/lyx2lyx/LyX.py
@@ -432,8 +432,22 @@ class LyX_base:
 
 def set_version(self):
 " Set the header with the version used."
-self.header[0] = " ".join(["#LyX %s created this file." % version__,
-  "For more info see http://www.lyx.org/;])
+
+initial_comment = " ".join(["#LyX %s created this file." % version__,
+"For more info see http://www.lyx.org/;])
+
+# Simple heuristic to determine the comment that always starts
+# a lyx file
+if self.header[0].startswith("#"):
+self.header[0] = initial_comment
+else:
+self.header.insert(0, initial_comment)
+
+# Old lyx files had a two lines comment header:
+# 1) the first line had the user who had created it
+# 2) the second line had the lyx version used
+# later we decided that 1) was a privacy risk for no gain
+# here we remove the second line effectively erasing 1)
 if self.header[1][0] == '#':
 del self.header[1]
 
diff --git a/status.21x b/status.21x
index 488f6a1..040616d 100644
--- a/status.21x
+++ b/status.21x
@@ -110,6 +110,8 @@ What's new
 
 - Fix handling of the TEXINPUTS environment variable on Windows (bug 9453).
 
+- Make lyx2lyx more robust, add the initial comment line that says what is the
+  lyx version that created the document if none is present (bug 9279).
 
 
 * USER INTERFACE


[LyX/master] First step to make lyx2lyx code support both python 2 and python 3 (3.3+)

2015-03-11 Thread José Matos
commit 9a0d70a45fde2d2a5d7af84f39f91e3c2ea91dff
Author: José Matos jama...@lyx.org
Date:   Wed Mar 11 12:04:46 2015 +

First step to make lyx2lyx code support both python 2 and python 3 (3.3+)

Most of the changes are related with dictionaries returning views instead of
lists.

xrange - range (since xrange is gone in python 3)

The code that is special to support both python 2 and 3 is enclosed in a 
comment
 \# Provide support for both python 2 and 3
and
 \# End of code to support for both python 2 and 3

And so later it can be removed safely when python 2 is no longer supported.

diff --git a/lib/lyx2lyx/LyX.py b/lib/lyx2lyx/LyX.py
index fdfa34d..89b08c8 100644
--- a/lib/lyx2lyx/LyX.py
+++ b/lib/lyx2lyx/LyX.py
@@ -80,12 +80,12 @@ format_relation = [(0_06,[200], minor_versions(0.6 
, 4)),
(1_1_6_3, [218], [1.1, 1.1.6.3,1.1.6.4]),
(1_2, [220], minor_versions(1.2 , 4)),
(1_3, [221], minor_versions(1.3 , 7)),
-   (1_4, range(222,246), minor_versions(1.4 , 5)),
-   (1_5, range(246,277), minor_versions(1.5 , 7)),
-   (1_6, range(277,346), minor_versions(1.6 , 10)),
-   (2_0, range(346,414), minor_versions(2.0, 8)),
-   (2_1, range(414,475), minor_versions(2.1, 0)),
-   (2_2, range(475,483), minor_versions(2.2, 0))
+   (1_4, list(range(222,246)), minor_versions(1.4 , 5)),
+   (1_5, list(range(246,277)), minor_versions(1.5 , 7)),
+   (1_6, list(range(277,346)), minor_versions(1.6 , 10)),
+   (2_0, list(range(346,414)), minor_versions(2.0, 8)),
+   (2_1, list(range(414,475)), minor_versions(2.1, 0)),
+   (2_2, list(range(475,483)), minor_versions(2.2, 0))
   ]
 
 
diff --git a/lib/lyx2lyx/generate_encoding_info.py 
b/lib/lyx2lyx/generate_encoding_info.py
index 6766ef0..50725b7 100644
--- a/lib/lyx2lyx/generate_encoding_info.py
+++ b/lib/lyx2lyx/generate_encoding_info.py
@@ -19,6 +19,7 @@
  This module parses lib/languages and prints it as a python
 dictionary, ready to use by other python modules
 
+from __future__ import print_function
 import pprint
 
 def parse_line(line):
@@ -55,8 +56,8 @@ if __name__ == '__main__':
 lang[tmp[0]] = tmp[1:]
 
 
-print # This file is generated by generate_incoding_info.py from 
lib/languages file.
-print # Do not change this file directly.
-print
-print lang = ,
+print (# This file is generated by generate_incoding_info.py from 
lib/languages file.)
+print (# Do not change this file directly.)
+print ()
+print (lang = , end =  )
 pprint.pprint(lang)
diff --git a/lib/lyx2lyx/lyx2lyx_tools.py b/lib/lyx2lyx/lyx2lyx_tools.py
index fefc53f..9c69439 100644
--- a/lib/lyx2lyx/lyx2lyx_tools.py
+++ b/lib/lyx2lyx/lyx2lyx_tools.py
@@ -283,7 +283,7 @@ def latex_length(slen):
 units = {text%:\\textwidth, col%:\\columnwidth,
  page%:\\paperwidth, line%:\\linewidth,
  theight%:\\textheight, pheight%:\\paperheight}
-for unit in units.keys():
+for unit in list(units.keys()):
 i = slen.find(unit)
 if i == -1:
 continue
diff --git a/lib/lyx2lyx/lyx_1_2.py b/lib/lyx2lyx/lyx_1_2.py
index a47fcaf..19b6647 100644
--- a/lib/lyx2lyx/lyx_1_2.py
+++ b/lib/lyx2lyx/lyx_1_2.py
@@ -154,7 +154,7 @@ def remove_oldfloat(document):
 j = find_token(lines, \\end_float, i+1)
 
 floattype = lines[i].split()[1]
-if not floats.has_key(floattype):
+if floattype not in floats:
 document.warning(Error! Unknown float type  + floattype)
 floattype = fig
 
@@ -284,7 +284,7 @@ def remove_pextra(document):
 
 def is_empty(lines):
  Are all the lines empty?
-return filter(is_nonempty_line, lines) == []
+return list(filter(is_nonempty_line, lines)) == []
 
 
 move_rexp =  
re.compile(r\\(family|series|shape|size|emph|numeric|bar|noun|end_deeper))
@@ -358,7 +358,7 @@ def remove_oldert(document):
 tmp.append(line)
 
 if is_empty(tmp):
-if filter(lambda x:x != , tmp) != []:
+if [x for x in tmp if x != ] != []:
 if new == []:
 # This is not necessary, but we want the output to be
 # as similar as posible to the lyx format
diff --git a/lib/lyx2lyx/lyx_1_4.py b/lib/lyx2lyx/lyx_1_4.py
index 9fb583f..c1a4591 100644
--- a/lib/lyx2lyx/lyx_1_4.py
+++ b/lib/lyx2lyx/lyx_1_4.py
@@ -1495,7 +1495,7 @@ def convert_len(len, special):
 len = '%f\\' % len2value(len) + special
 
 # Convert LyX units to LaTeX units
-for unit in units.keys():
+for unit in list(units.keys()):
 if len.find(unit) != -1:
 len

[LyX/master] First step to make po python files to support both python 2 and python 3 (3.3+)

2015-03-11 Thread José Matos
commit 0a035283765d3d5d476430cb635694336c442c87
Author: José Matos jama...@lyx.org
Date:   Wed Mar 11 13:41:10 2015 +

First step to make po python files to support both python 2 and python 3 
(3.3+)

Most of the changes are related with dictionaries returning iterators 
instead of
lists. (Iterators and not views as I wrote in the previous commit 
message)

xrange - range (since xrange is gone in python 3)

diff --git a/po/lyx_pot.py b/po/lyx_pot.py
index a0f98b9..c92939c 100755
--- a/po/lyx_pot.py
+++ b/po/lyx_pot.py
@@ -16,9 +16,9 @@
 # This script will extract translatable strings from input files and write
 # to output in gettext .pot format.
 #
+from __future__ import print_function
+
 import sys, os, re, getopt
-if sys.version_info  (2, 4, 0):
-from sets import Set as set
 
 def relativePath(path, base):
 '''return relative path from top source dir'''
@@ -26,7 +26,7 @@ def relativePath(path, base):
 path1 = os.path.normpath(os.path.realpath(path)).split(os.sep)
 path2 = os.path.normpath(os.path.realpath(base)).split(os.sep)
 if path1[:len(path2)] != path2:
-print Path %s is not under top source directory % path
+print(Path %s is not under top source directory % path)
 path3 = os.path.join(*path1[len(path2):]);
 # replace all \ by / such that we get the same comments on Windows and *nix
 path3 = path3.replace('\\', '/')
@@ -37,8 +37,8 @@ def writeString(outfile, infile, basefile, lineno, string):
 string = string.replace('\\', '').replace('', '')
 if string == :
 return
-print  outfile, '#: %s:%d\nmsgid %s\nmsgstr \n' % \
-(relativePath(infile, basefile), lineno, string)
+print('#: %s:%d\nmsgid %s\nmsgstr \n' % \
+(relativePath(infile, basefile), lineno, string), file=outfile)
 
 
 def ui_l10n(input_files, output, base):
@@ -70,8 +70,8 @@ def ui_l10n(input_files, output, base):
 continue
 string = string.replace('', '')
 if string != :
-print  output, '#: %s:%d\nmsgid %s\nmsgstr \n' % \
-(relativePath(src, base), lineno+1, string)
+print('#: %s:%d\nmsgid %s\nmsgstr \n' % \
+(relativePath(src, base), lineno+1, string), file=output)
 input.close()
 output.close()
 
@@ -153,11 +153,11 @@ def layouts_l10n(input_files, output, base, 
layouttranslations):
 oldtrans[lang][key] = val
 keyset.add(key)
 continue
-print Error: Unable to handle line:
-print line
+print(Error: Unable to handle line:)
+print(line)
 except IOError:
-print Warning: Unable to open %s for reading. % output
-print  Old translations will be lost.
+print(Warning: Unable to open %s for reading. % output)
+print( Old translations will be lost.)
 
 # walon is not a known document language
 # FIXME: Do not hardcode, read from lib/languages!
@@ -374,16 +374,16 @@ def layouts_l10n(input_files, output, base, 
layouttranslations):
 
 ContextRe = re.compile(r'(.*)(\[\[.*\]\])')
 
-print  out, '''# This file has been automatically generated by 
po/lyx_pot.py.
+print('''# This file has been automatically generated by po/lyx_pot.py.
 # PLEASE MODIFY ONLY THE LAGUAGES HAVING NO .po FILE! If you want to regenerate
 # this file from the translations, run `make ../lib/layouttranslations' in po.
 # Python polib library is needed for building the output file.
 #
 # This file should remain fixed during minor LyX releases.
-# For more comments see README.localization file.'''
+# For more comments see README.localization file.''', file=out)
 for lang in languages:
-print  out, '\nTranslation %s' % lang
-if lang in oldtrans.keys():
+print('\nTranslation %s' % lang, file=out)
+if lang in list(oldtrans.keys()):
 trans = oldtrans[lang]
 else:
 trans = dict()
@@ -402,14 +402,14 @@ def layouts_l10n(input_files, output, base, 
layouttranslations):
 if val != key:
 trans[key] = val
 for key in keys:
-if key in trans.keys():
+if key in list(trans.keys()):
 val = trans[key].replace('\\', '').replace('', '\\')
 res = ContextRe.search(val)
 if res != None:
 val = res.group(1)
 key = key.replace('\\', '').replace('', '\\')
-print  out, '\t%s %s' % \
- (key.encode('utf-8'), val.encode('utf-8'))
+print('\t%s %s' % \
+ (key.encode('utf-8'), val.encode('utf-8')), 
file=out)
 # also

[LyX/master] First step to make lyx2lyx code support both python 2 and python 3 (3.3+)

2015-03-11 Thread José Matos
commit 9a0d70a45fde2d2a5d7af84f39f91e3c2ea91dff
Author: José Matos <jama...@lyx.org>
Date:   Wed Mar 11 12:04:46 2015 +

First step to make lyx2lyx code support both python 2 and python 3 (3.3+)

Most of the changes are related with dictionaries returning views instead of
lists.

xrange -> range (since xrange is gone in python 3)

The code that is special to support both python 2 and 3 is enclosed in a 
comment
 \# Provide support for both python 2 and 3
and
 \# End of code to support for both python 2 and 3

And so later it can be removed safely when python 2 is no longer supported.

diff --git a/lib/lyx2lyx/LyX.py b/lib/lyx2lyx/LyX.py
index fdfa34d..89b08c8 100644
--- a/lib/lyx2lyx/LyX.py
+++ b/lib/lyx2lyx/LyX.py
@@ -80,12 +80,12 @@ format_relation = [("0_06",[200], minor_versions("0.6" 
, 4)),
("1_1_6_3", [218], ["1.1", "1.1.6.3","1.1.6.4"]),
("1_2", [220], minor_versions("1.2" , 4)),
("1_3", [221], minor_versions("1.3" , 7)),
-   ("1_4", range(222,246), minor_versions("1.4" , 5)),
-   ("1_5", range(246,277), minor_versions("1.5" , 7)),
-   ("1_6", range(277,346), minor_versions("1.6" , 10)),
-   ("2_0", range(346,414), minor_versions("2.0", 8)),
-   ("2_1", range(414,475), minor_versions("2.1", 0)),
-   ("2_2", range(475,483), minor_versions("2.2", 0))
+   ("1_4", list(range(222,246)), minor_versions("1.4" , 5)),
+   ("1_5", list(range(246,277)), minor_versions("1.5" , 7)),
+   ("1_6", list(range(277,346)), minor_versions("1.6" , 10)),
+   ("2_0", list(range(346,414)), minor_versions("2.0", 8)),
+   ("2_1", list(range(414,475)), minor_versions("2.1", 0)),
+   ("2_2", list(range(475,483)), minor_versions("2.2", 0))
   ]
 
 
diff --git a/lib/lyx2lyx/generate_encoding_info.py 
b/lib/lyx2lyx/generate_encoding_info.py
index 6766ef0..50725b7 100644
--- a/lib/lyx2lyx/generate_encoding_info.py
+++ b/lib/lyx2lyx/generate_encoding_info.py
@@ -19,6 +19,7 @@
 """ This module parses lib/languages and prints it as a python
 dictionary, ready to use by other python modules"""
 
+from __future__ import print_function
 import pprint
 
 def parse_line(line):
@@ -55,8 +56,8 @@ if __name__ == '__main__':
 lang[tmp[0]] = tmp[1:]
 
 
-print "# This file is generated by generate_incoding_info.py from 
lib/languages file."
-print "# Do not change this file directly."
-print
-print "lang = ",
+print ("# This file is generated by generate_incoding_info.py from 
lib/languages file.")
+print ("# Do not change this file directly.")
+print ()
+print ("lang = ", end = " ")
 pprint.pprint(lang)
diff --git a/lib/lyx2lyx/lyx2lyx_tools.py b/lib/lyx2lyx/lyx2lyx_tools.py
index fefc53f..9c69439 100644
--- a/lib/lyx2lyx/lyx2lyx_tools.py
+++ b/lib/lyx2lyx/lyx2lyx_tools.py
@@ -283,7 +283,7 @@ def latex_length(slen):
 units = {"text%":"\\textwidth", "col%":"\\columnwidth",
  "page%":"\\paperwidth", "line%":"\\linewidth",
  "theight%":"\\textheight", "pheight%":"\\paperheight"}
-for unit in units.keys():
+for unit in list(units.keys()):
 i = slen.find(unit)
 if i == -1:
 continue
diff --git a/lib/lyx2lyx/lyx_1_2.py b/lib/lyx2lyx/lyx_1_2.py
index a47fcaf..19b6647 100644
--- a/lib/lyx2lyx/lyx_1_2.py
+++ b/lib/lyx2lyx/lyx_1_2.py
@@ -154,7 +154,7 @@ def remove_oldfloat(document):
 j = find_token(lines, "\\end_float", i+1)
 
 floattype = lines[i].split()[1]
-if not floats.has_key(floattype):
+if floattype not in floats:
 document.warning("Error! Unknown float type " + floattype)
 floattype = "fig"
 
@@ -284,7 +284,7 @@ def remove_pextra(document):
 
 def is_empty(lines):
 " Are all the lines empty?"
-return filter(is_nonempty_line, lines) == []
+return list(filter(is_nonempty_line, lines)) == []
 
 
 move_rexp =  
re.compile(r"\\(family|series|shape|size|emph|numeric|bar|noun|end_deeper)")
@@ -358,7 +358,7 @@ def remove_oldert(document):
 tmp.append

[LyX/master] First step to make po python files to support both python 2 and python 3 (3.3+)

2015-03-11 Thread José Matos
commit 0a035283765d3d5d476430cb635694336c442c87
Author: José Matos <jama...@lyx.org>
Date:   Wed Mar 11 13:41:10 2015 +

First step to make po python files to support both python 2 and python 3 
(3.3+)

Most of the changes are related with dictionaries returning iterators 
instead of
lists. (Iterators and not views as I wrote in the previous commit 
message)

xrange -> range (since xrange is gone in python 3)

diff --git a/po/lyx_pot.py b/po/lyx_pot.py
index a0f98b9..c92939c 100755
--- a/po/lyx_pot.py
+++ b/po/lyx_pot.py
@@ -16,9 +16,9 @@
 # This script will extract translatable strings from input files and write
 # to output in gettext .pot format.
 #
+from __future__ import print_function
+
 import sys, os, re, getopt
-if sys.version_info < (2, 4, 0):
-from sets import Set as set
 
 def relativePath(path, base):
 '''return relative path from top source dir'''
@@ -26,7 +26,7 @@ def relativePath(path, base):
 path1 = os.path.normpath(os.path.realpath(path)).split(os.sep)
 path2 = os.path.normpath(os.path.realpath(base)).split(os.sep)
 if path1[:len(path2)] != path2:
-print "Path %s is not under top source directory" % path
+print("Path %s is not under top source directory" % path)
 path3 = os.path.join(*path1[len(path2):]);
 # replace all \ by / such that we get the same comments on Windows and *nix
 path3 = path3.replace('\\', '/')
@@ -37,8 +37,8 @@ def writeString(outfile, infile, basefile, lineno, string):
 string = string.replace('\\', '').replace('"', '')
 if string == "":
 return
-print >> outfile, '#: %s:%d\nmsgid "%s"\nmsgstr ""\n' % \
-(relativePath(infile, basefile), lineno, string)
+print('#: %s:%d\nmsgid "%s"\nmsgstr ""\n' % \
+(relativePath(infile, basefile), lineno, string), file=outfile)
 
 
 def ui_l10n(input_files, output, base):
@@ -70,8 +70,8 @@ def ui_l10n(input_files, output, base):
 continue
 string = string.replace('"', '')
 if string != "":
-print >> output, '#: %s:%d\nmsgid "%s"\nmsgstr ""\n' % \
-(relativePath(src, base), lineno+1, string)
+print('#: %s:%d\nmsgid "%s"\nmsgstr ""\n' % \
+(relativePath(src, base), lineno+1, string), file=output)
 input.close()
 output.close()
 
@@ -153,11 +153,11 @@ def layouts_l10n(input_files, output, base, 
layouttranslations):
 oldtrans[lang][key] = val
 keyset.add(key)
 continue
-print "Error: Unable to handle line:"
-print line
+print("Error: Unable to handle line:")
+print(line)
 except IOError:
-print "Warning: Unable to open %s for reading." % output
-print " Old translations will be lost."
+print("Warning: Unable to open %s for reading." % output)
+print(" Old translations will be lost.")
 
 # walon is not a known document language
 # FIXME: Do not hardcode, read from lib/languages!
@@ -374,16 +374,16 @@ def layouts_l10n(input_files, output, base, 
layouttranslations):
 
 ContextRe = re.compile(r'(.*)(\[\[.*\]\])')
 
-print >> out, '''# This file has been automatically generated by 
po/lyx_pot.py.
+print('''# This file has been automatically generated by po/lyx_pot.py.
 # PLEASE MODIFY ONLY THE LAGUAGES HAVING NO .po FILE! If you want to regenerate
 # this file from the translations, run `make ../lib/layouttranslations' in po.
 # Python polib library is needed for building the output file.
 #
 # This file should remain fixed during minor LyX releases.
-# For more comments see README.localization file.'''
+# For more comments see README.localization file.''', file=out)
 for lang in languages:
-print >> out, '\nTranslation %s' % lang
-if lang in oldtrans.keys():
+print('\nTranslation %s' % lang, file=out)
+if lang in list(oldtrans.keys()):
 trans = oldtrans[lang]
 else:
 trans = dict()
@@ -402,14 +402,14 @@ def layouts_l10n(input_files, output, base, 
layouttranslations):
 if val != key:
 trans[key] = val
 for key in keys:
-if key in trans.keys():
+if key in list(trans.keys()):
 val = trans[key].replace('\\', '').replace('"', '\\"')
 res = ContextRe.search(val)
 if res != None:
 val = res.group(1)
 key = key.replace('\\', '').r

Re: [LyX/master] Simplify a bunch of lyx2lyx code.

2013-07-24 Thread José Matos
On Wednesday 24 July 2013 15:56:21 Richard Heck wrote:
  def convert_IEEEtran(document):
 @@ -1656,13 +1654,11 @@ def revert_AASTeX(document):
if document.textclass == aastex:
  i = 0
  while True:
 -  if i != -1:
 -i = find_token(document.body, \\begin_layout Altaffilation, i)
 -  if i != -1:
 -revert_Argument_to_TeX_brace(document, i, 0, 1, 1, False, False)
 -i = i + 1
 +  i = find_token(document.body, \\begin_layout Altaffilation, i)
if i == -1:
  return
 +  revert_Argument_to_TeX_brace(document, i, 0, 1, 1, False, False)
 +  i += i + 1

Hi Richard,
the last line is clearly a mistake, according to the previous code, and 
in line with your changes, it should be

i += 1

Regards, 
-- 
José Abílio


Re: [LyX/master] Simplify a bunch of lyx2lyx code.

2013-07-24 Thread José Matos
On Wednesday 24 July 2013 15:56:21 Richard Heck wrote:
>  def convert_IEEEtran(document):
> @@ -1656,13 +1654,11 @@ def revert_AASTeX(document):
>if document.textclass == "aastex":
>  i = 0
>  while True:
> -  if i != -1:
> -i = find_token(document.body, "\\begin_layout Altaffilation", i)
> -  if i != -1:
> -revert_Argument_to_TeX_brace(document, i, 0, 1, 1, False, False)
> -i = i + 1
> +  i = find_token(document.body, "\\begin_layout Altaffilation", i)
>if i == -1:
>  return
> +  revert_Argument_to_TeX_brace(document, i, 0, 1, 1, False, False)
> +  i += i + 1

Hi Richard,
the last line is clearly a mistake, according to the previous code, and 
in line with your changes, it should be

i += 1

Regards, 
-- 
José Abílio


Re: r40668 - lyx-devel/trunk/lib/lyx2lyx

2012-01-24 Thread José Matos
On 01/23/2012 09:22 PM, uwesto...@lyx.org wrote:
 Author: uwestoehr
 Date: Mon Jan 23 22:22:34 2012
 New Revision: 40668
 URL: http://www.lyx.org/trac/changeset/40668

 Log:
 lyx_2_1.py: there is no need for a comma here, it is already the last entry 
 of the list

 Modified:
lyx-devel/trunk/lib/lyx2lyx/lyx_2_1.py

 Modified: lyx-devel/trunk/lib/lyx2lyx/lyx_2_1.py
 ==
 --- lyx-devel/trunk/lib/lyx2lyx/lyx_2_1.pyMon Jan 23 18:20:07 2012
 (r40667)
 +++ lyx-devel/trunk/lib/lyx2lyx/lyx_2_1.pyMon Jan 23 22:22:34 2012
 (r40668)
 @@ -488,7 +488,7 @@
 [422, [convert_use_packages]],
 [423, [convert_use_mathtools]],
 [424, [convert_cite_engine_type]],
 -   [425, []],
 +   [425, []]
]
  
  revert =  [


The comma in the last line is not necessary but it helps the difference
between different versions because then it only changes one line instead
of two. Sure it is a style issue but then it allows for nicer diffs.

-- 
José Matos



Re: r40668 - lyx-devel/trunk/lib/lyx2lyx

2012-01-24 Thread José Matos
On 01/23/2012 09:22 PM, uwesto...@lyx.org wrote:
> Author: uwestoehr
> Date: Mon Jan 23 22:22:34 2012
> New Revision: 40668
> URL: http://www.lyx.org/trac/changeset/40668
>
> Log:
> lyx_2_1.py: there is no need for a comma here, it is already the last entry 
> of the list
>
> Modified:
>lyx-devel/trunk/lib/lyx2lyx/lyx_2_1.py
>
> Modified: lyx-devel/trunk/lib/lyx2lyx/lyx_2_1.py
> ==
> --- lyx-devel/trunk/lib/lyx2lyx/lyx_2_1.pyMon Jan 23 18:20:07 2012
> (r40667)
> +++ lyx-devel/trunk/lib/lyx2lyx/lyx_2_1.pyMon Jan 23 22:22:34 2012
> (r40668)
> @@ -488,7 +488,7 @@
> [422, [convert_use_packages]],
> [423, [convert_use_mathtools]],
> [424, [convert_cite_engine_type]],
> -   [425, []],
> +   [425, []]
>]
>  
>  revert =  [


The comma in the last line is not necessary but it helps the difference
between different versions because then it only changes one line instead
of two. Sure it is a style issue but then it allows for nicer diffs.

-- 
José Matos



Re: r38842 - lyx-devel/trunk/src

2011-05-26 Thread José Matos
On Thursday 26 May 2011 15:16:53 rgh...@lyx.org wrote:
 Fix bug reported on user list. Children included more than once in
 the same document could not be rendered.
 
 Problem was that we were using children_positions to deal with the
 included documents. That structure only includes each child once.
 We need to use position_to_children, instead.

Although this is your call I would say that this is OK to go in branch... me 
ducks and cover. ;-)

-- 
José Abílio


Re: r38842 - lyx-devel/trunk/src

2011-05-26 Thread José Matos
On Thursday 26 May 2011 15:16:53 rgh...@lyx.org wrote:
> Fix bug reported on user list. Children included more than once in
> the same document could not be rendered.
> 
> Problem was that we were using children_positions to deal with the
> included documents. That structure only includes each child once.
> We need to use position_to_children, instead.

Although this is your call I would say that this is OK to go in branch... me 
ducks and cover. ;-)

-- 
José Abílio