Re: Support for python 2 and 3 in lyx2lyx for 2.3

[José Abílio Matos]

On Wednesday, 29 March 2017 15.25.12 WEST Kornel Benko wrote:
> > > There is one task remaining. Creation of lyxrc.defaults uses string
> > > 'python' as executable even if lyx is configured for use with python3.
> 
> > I agree that this could change.

Actually here I meant to say not just that it could but the it *should* 
change.

> > > I had to edit this file to get the desired python3 to run the converts.
> > > (Example: \converter figpstex  "python -tt
> > > $$s/scripts/fig2pstex.py $$i $$o""")
> 
> > Honestly here I suggest us to change this automatically for users.
> > If the first command is python we should change it to the python version
> > configured. If the user wants to use a different python version the user
> > is
> > free to use say "python2 -tt ..."
> > 
> > For example in my system where python is python2 I have:
> > $ ll /usr/bin/python
> > lrwxrwxrwx. 1 root root 7 Jan 12 18:00 /usr/bin/python -> python2
> > $ ll /usr/bin/python2
> > lrwxrwxrwx. 1 root root 9 Jan 12 18:00 /usr/bin/python2 -> python2.7
> > 
> > And that has been like this for ages. 
> 
> yes, it is the same here. But I don't want to change the symbolic link
> because it may influence some other programs.
> Instead I'd prefer to have an option to tell lyx to use python3 for handling
> of lyx-local python scripts.

In case it was not clear I mean that we should do it inside lyx because the 
python2 and python3 names are *always* available.

So in every place where we have the python name as the first command we should 
change it to python2 or python3, like if in a sense python meant $$python that 
we replace by the appropriate python name.

This way it will work for everyone and someone that needs can pin the python 
version by directly placing the major version in front of the python name.

I hope that this is clear now. :-)

Regards,
-- 
José Abílio

Re: LyX master still not ready for Python 3

[José Abílio Matos]

On Wednesday, 29 March 2017 01.12.06 WEST Uwe Stöhr wrote:
> As requested by José, I installed Python 3.6.1 and tested today's master
> with it. As result LyX cannot be started because the reconfiguration fails:
> 
> 02:05:53.187: Checking whether TeX allows spaces in file names... yes
> 02:05:53.188: Traceback (most recent call last):
> 02:05:53.189:   File "D:/LyXGit/Master/lib/configure.py", line 1814, in
> 
> 02:05:53.192: windows_style_tex_paths = checkTeXPaths()
> 02:05:53.193:   File "D:/LyXGit/Master/lib/configure.py", line 206, in
> checkTeXPaths
> 02:05:53.194: inpname = shortPath(str(tmpfname,
> encoding)).replace('\\', '/')
> 02:05:53.196: TypeError: decoding str is not supported
> 02:05:53.198: configuration will be reloaded...
> D:\LyXGit\Master\src\support\Systemcall.cpp (295): Systemcall: 'python
> -tt "D:/LyXGit/Master/lib/configure.py"
> --binary-dir="D:/LyXGit/Master/compile-2015/bin/Release/"' finished with
> exit code 1
> 
> regards Uwe

OK, I committed a change that should fix this problem.

Looking into that function I find it strange that we have to change the path 
separators if it is windows... I would expect this to be unnecessary...

I hope that this works.

Regards,
-- 
José Abílio

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[José Abílio Matos]

On Wednesday, 29 March 2017 13.19.23 WEST Kornel Benko wrote:
> So, should we make python3 be the default for lyx2.3?

We want it to work as well as it works python 2. If it is the default or not I 
have no strong opinion.
 
> There is one task remaining. Creation of lyxrc.defaults uses string 'python'
> as executable even if lyx is configured for use with python3.

I agree that this could change.

> I had to edit this file to get the desired python3 to run the converts.
> (Example: \converter figpstex  "python -tt
> $$s/scripts/fig2pstex.py $$i $$o""")

Honestly here I suggest us to change this automatically for users.
If the first command is python we should change it to the python version 
configured. If the user wants to use a different python version the user is 
free to use say "python2 -tt ..."

For example in my system where python is python2 I have:
$ ll /usr/bin/python
lrwxrwxrwx. 1 root root 7 Jan 12 18:00 /usr/bin/python -> python2
$ ll /usr/bin/python2
lrwxrwxrwx. 1 root root 9 Jan 12 18:00 /usr/bin/python2 -> python2.7

And that has been like this for ages. :-)

> Kornel

-- 
José Abílio

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[Kornel Benko]

Am Mittwoch, 29. März 2017 um 14:50:46, schrieb José Abílio Matos 

> On Wednesday, 29 March 2017 13.19.23 WEST Kornel Benko wrote:
> > So, should we make python3 be the default for lyx2.3?
> 
> We want it to work as well as it works python 2. If it is the default or not 
> I 
> have no strong opinion.
>  
> > There is one task remaining. Creation of lyxrc.defaults uses string 'python'
> > as executable even if lyx is configured for use with python3.
> 
> I agree that this could change.
> 
> > I had to edit this file to get the desired python3 to run the converts.
> > (Example: \converter figpstex  "python -tt
> > $$s/scripts/fig2pstex.py $$i $$o""")
> 
> Honestly here I suggest us to change this automatically for users.
> If the first command is python we should change it to the python version 
> configured. If the user wants to use a different python version the user is 
> free to use say "python2 -tt ..."
> 
> For example in my system where python is python2 I have:
> $ ll /usr/bin/python
> lrwxrwxrwx. 1 root root 7 Jan 12 18:00 /usr/bin/python -> python2
> $ ll /usr/bin/python2
> lrwxrwxrwx. 1 root root 9 Jan 12 18:00 /usr/bin/python2 -> python2.7
> 
> And that has been like this for ages. :-)
> 
> > Kornel

yes, it is the same here. But I don't want to change the symbolic link because
it may influence some other programs.
Instead I'd prefer to have an option to tell lyx to use python3 for handling of 
lyx-local python scripts.

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[Kornel Benko]

Am Mittwoch, 29. März 2017 um 15:52:22, schrieb José Abílio Matos 

> On Wednesday, 29 March 2017 15.25.12 WEST Kornel Benko wrote:
> > > > There is one task remaining. Creation of lyxrc.defaults uses string
> > > > 'python' as executable even if lyx is configured for use with python3.
> > 
> > > I agree that this could change.
> 
> Actually here I meant to say not just that it could but the it *should* 
> change.

Cmake part is ready.

> > > > I had to edit this file to get the desired python3 to run the converts.
> > > > (Example: \converter figpstex  "python -tt
> > > > $$s/scripts/fig2pstex.py $$i $$o""")
> > 
> > > Honestly here I suggest us to change this automatically for users.
> > > If the first command is python we should change it to the python version
> > > configured. If the user wants to use a different python version the user
> > > is
> > > free to use say "python2 -tt ..."
> > > 
> > > For example in my system where python is python2 I have:
> > > $ ll /usr/bin/python
> > > lrwxrwxrwx. 1 root root 7 Jan 12 18:00 /usr/bin/python -> python2
> > > $ ll /usr/bin/python2
> > > lrwxrwxrwx. 1 root root 9 Jan 12 18:00 /usr/bin/python2 -> python2.7
> > > 
> > > And that has been like this for ages. 
> > 
> > yes, it is the same here. But I don't want to change the symbolic link
> > because it may influence some other programs.
> > Instead I'd prefer to have an option to tell lyx to use python3 for handling
> > of lyx-local python scripts.
> 
> In case it was not clear I mean that we should do it inside lyx because the 
> python2 and python3 names are *always* available.
> 
> So in every place where we have the python name as the first command we 
> should 
> change it to python2 or python3, like if in a sense python meant $$python 
> that 
> we replace by the appropriate python name.
> 
> This way it will work for everyone and someone that needs can pin the python 
> version by directly placing the major version in front of the python name.
> 
> I hope that this is clear now. :-)
> 
> Regards,

Thanks for clarification. Yes, better as an option since it does not need user 
intervention.

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: LyX master still not ready for Python 3

[Enrico Forestieri]

On Wed, Mar 29, 2017 at 02:41:23PM +0100, José Abílio Matos wrote:
> 
> OK, I committed a change that should fix this problem.

I hope so ;)

C:\python>python
Python 2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getdefaultlocale()
('it_IT', 'cp1252')
>>> import sys
>>> sys.getfilesystemencoding()
'mbcs'

Hm?

> Looking into that function I find it strange that we have to change the path 
> separators if it is windows... I would expect this to be unnecessary...

No, it is necessary because the path is going to be used in a latex file.

> I hope that this works.

Let's see...

-- 
Enrico

Re: LyX master still not ready for Python 3

[Uwe Stöhr]

Am 29.03.2017 um 23:05 schrieb Uwe Stöhr:


('de_DE', 'cp1252')


OK, using

inpname = shortPath(str(tmpfname).encode(encoding)).replace('\\', '/')
or
inpname = shortPath(str(tmpfname).encode('cp1252')).replace('\\', '/')

gives this error:

TypeError: a bytes-like object is required, not 'str'

I googled this:
"
You are using Python 2 methodology instead of Python 3.
Change:
outfile=open('./immates.csv','wb')
To:
outfile=open('./immates.csv','w')
.
In Python 3 csv takes the input in text mode, whereas in Python 2 it 
took it in binary mode.

"

And indeed by using i n line 208

os.write(fd, b'\relax')
instead of
os.write(fd, r'\relax')

I get with
inpname = shortPath(str(tmpfname)).replace('\\', '/')
no error. But then it will of course fail with Python 2.

regards Uwe

Re: Formal tables

[Richard Heck]

On 03/29/2017 04:22 AM, Andrew Parsloe wrote:
>
>
> On 29/03/2017 1:04 p.m., Joel Kulesza wrote:
>>
>> On Tue, Mar 28, 2017 at 3:28 PM, Andrew Parsloe > > wrote:
>>
>> The booktabs package produces much cleaner tables than "native"
>> LaTeX ones, but I wonder why LyX only "half converts" a LaTeX table
>> to formal style? In the attachment LyX produces the middle table
>> from the left-hand one, but the right-hand one is what should be
>> produced according to the booktabs documentation:
>>
>> 1. Never, ever use vertical rules.
>> 2. Never use double rules.
>> ...
>> In the simplest of cases a table begins with a \toprule, has a
>> single row of column headings, then a dividing rule called here a
>> \midrule; after the columns of data we finish off with a
>> \bottomrule.
>>
>> A user might wonder what the benefits of the Formal Style are when
>> faced with the middle table. It takes significantly more space and
>> is still fussy with horizontal rules.
>>
>> Why not convert according to the scheme:
>>
>> 1. the initial \hline -> \toprule
>> 2. the final \hline -> \bottomrule
>> 3. \hline \hline -> \midrule
>> 4. any other \hline -> (nothing)
>>
>> Andrew
>>
>>
>> Andrew:
>>
>> See also tickets for varying degrees of discussion/disposition:
>>
>> https://www.lyx.org/trac/ticket/8360
>> https://www.lyx.org/trac/ticket/9901
>> https://www.lyx.org/trac/ticket/10105
>> https://www.lyx.org/trac/ticket/10106
>>
>> - Joel
>>
> Thank you Joel. I should have searched on trac before spouting. I
> fully support the sentiments in 9901, 10105, 10106.

It might be worth a meta-bug that draws these together.

rh

Re: LyX master still not ready for Python 3

[Uwe Stöhr]

Am 29.03.2017 um 15:41 schrieb José Abílio Matos:


OK, I committed a change that should fix this problem.


Thanks José,

this does not fix the bug. I get still the same error message:

22:59:50.996:   File "D:/LyXGit/Master/lib/configure.py", line 1811, in 


22:59:50.999: windows_style_tex_paths = checkTeXPaths()
22:59:51.000:   File "D:/LyXGit/Master/lib/configure.py", line 203, in 
checkTeXPaths
22:59:51.003: inpname = shortPath(str(tmpfname, 
encoding)).replace('\\', '/')

22:59:51.004: TypeError: decoding str is not supported

> Could you, please, open a terminal/power shell to call directly
> python and then run:
> import locale
> locale.getdefaultlocale()
> I am interested in the output of the second line.

I get only one line of output. It is

('de_DE', 'cp1252')

Here is the complete content of the Python console:

D:\LyXGit\Master\compile-2015\bin\Release>python
Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 17:54:52) [MSC v.1900 32 bit 
(Intel)] on win32

Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getdefaultlocale()
('de_DE', 'cp1252')
>>>

regards Uwe

Re: LyX master still not ready for Python 3

[José Abílio Matos]

On Wednesday, 29 March 2017 18.24.26 WEST Enrico Forestieri wrote:
> On Wed, Mar 29, 2017 at 02:41:23PM +0100, José Abílio Matos wrote:
> > OK, I committed a change that should fix this problem.
> 
> I hope so ;)

When dealing with windows I always hope, after all hope is the last to die. 
:-)
 
> C:\python>python
> Python 2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit (Intel)]
> on win32 Type "help", "copyright", "credits" or "license" for more
> information.
> >>> import locale
> >>> locale.getdefaultlocale()
> 
> ('it_IT', 'cp1252')
> 
> >>> import sys
> >>> sys.getfilesystemencoding()
> 
> 'mbcs'
> 
> Hm?

multi-byte character set, I suspect that it means UTF-16 and not UCS2. :-)

> > Looking into that function I find it strange that we have to change the
> > path separators if it is windows... I would expect this to be
> > unnecessary...
> No, it is necessary because the path is going to be used in a latex file.

OK, that makes sense.
 
> > I hope that this works.
> 
> Let's see...

I am a man of faith.

-- 
José Abílio

Re: Formal tables

[Andrew Parsloe]

On 30/03/2017 11:11 a.m., Richard Heck wrote:

On 03/29/2017 04:22 AM, Andrew Parsloe wrote:



On 29/03/2017 1:04 p.m., Joel Kulesza wrote:


On Tue, Mar 28, 2017 at 3:28 PM, Andrew Parsloe > wrote:

 The booktabs package produces much cleaner tables than "native"
 LaTeX ones, but I wonder why LyX only "half converts" a LaTeX table
 to formal style? In the attachment LyX produces the middle table
 from the left-hand one, but the right-hand one is what should be
 produced according to the booktabs documentation:

 1. Never, ever use vertical rules.
 2. Never use double rules.
 ...
 In the simplest of cases a table begins with a \toprule, has a
 single row of column headings, then a dividing rule called here a
 \midrule; after the columns of data we finish off with a
\bottomrule.

 A user might wonder what the benefits of the Formal Style are when
 faced with the middle table. It takes significantly more space and
 is still fussy with horizontal rules.

 Why not convert according to the scheme:

 1. the initial \hline -> \toprule
 2. the final \hline -> \bottomrule
 3. \hline \hline -> \midrule
 4. any other \hline -> (nothing)

 Andrew


Andrew:

See also tickets for varying degrees of discussion/disposition:

https://www.lyx.org/trac/ticket/8360
https://www.lyx.org/trac/ticket/9901
https://www.lyx.org/trac/ticket/10105
https://www.lyx.org/trac/ticket/10106

- Joel


Thank you Joel. I should have searched on trac before spouting. I
fully support the sentiments in 9901, 10105, 10106.


It might be worth a meta-bug that draws these together.

rh



There is already a tabular meta-bug, #6011, although none of the bugs 
collected there are about formal tables. It is not clear to me how to 
add the four that Joel mentions to the Description in the ticket. What 
is the magic button to click? Is it the Reply button to the right of the 
Description field? The Preview suggests not.


Andrew



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Re: Formal tables

[Andrew Parsloe]

On 29/03/2017 1:04 p.m., Joel Kulesza wrote:


On Tue, Mar 28, 2017 at 3:28 PM, Andrew Parsloe > wrote:

The booktabs package produces much cleaner tables than "native"
LaTeX ones, but I wonder why LyX only "half converts" a LaTeX table
to formal style? In the attachment LyX produces the middle table
from the left-hand one, but the right-hand one is what should be
produced according to the booktabs documentation:

1. Never, ever use vertical rules.
2. Never use double rules.
...
In the simplest of cases a table begins with a \toprule, has a
single row of column headings, then a dividing rule called here a
\midrule; after the columns of data we finish off with a \bottomrule.

A user might wonder what the benefits of the Formal Style are when
faced with the middle table. It takes significantly more space and
is still fussy with horizontal rules.

Why not convert according to the scheme:

1. the initial \hline -> \toprule
2. the final \hline -> \bottomrule
3. \hline \hline -> \midrule
4. any other \hline -> (nothing)

Andrew


Andrew:

See also tickets for varying degrees of discussion/disposition:

https://www.lyx.org/trac/ticket/8360
https://www.lyx.org/trac/ticket/9901
https://www.lyx.org/trac/ticket/10105
https://www.lyx.org/trac/ticket/10106

- Joel

Thank you Joel. I should have searched on trac before spouting. I fully 
support the sentiments in 9901, 10105, 10106.


Andrew

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Re: LyX 2.2.3, Finally?

[Guenter Milde]

On 2017-03-28, Richard Heck wrote:
> On 03/28/2017 04:55 PM, Guenter Milde wrote:
>> On 2017-03-28, Richard Heck wrote:

>>> It looks as if we're now ready for 2.2.3, after some of the recent work.
>>> Any reason not to freeze strings now?

>> Yes, ...

> Sorry, think there is some confusion. I was talking about 2.2.3, NOT
> 2.3.0. Correct me if I'm wrong, but those all look like issues for 2.3.0.

You are right, I mixed up 2.3. and 2.2.3.



For 2.2.3, I propose a patch to work around the overfull lines with em-dashes:

As lyx2lyx converts --- and -- to the intermediate representation
\threehyphens and \twohyphens, we can add a "line break opportunity" (ZWSP)
after the literal dashes in the second conversion step:

--- a/src/Text.cpp
+++ b/src/Text.cpp
@@ -506,9 +506,11 @@ void Text::readParToken(Paragraph & par, Lexer & lex,
par.insert(par.size(), from_ascii("---"), font, 
change);
} else {
if (token == "\\twohyphens")
-   par.insertChar(par.size(), 0x2013, font, 
change);
-   else
-   par.insertChar(par.size(), 0x2014, font, 
change);
+   par.insertChar(par.size(), 0x2013, font, 
change); // EN DASH
+   else {
+   par.insertChar(par.size(), 0x2014, font, 
change); // EM DASH
+   par.insertChar(par.size(), 0x200b, font, 
change); // ZWSP
+   }
}
} else if (token == "\\backslash") {
par.appendChar('\\', font, change);


Of course, this cannot restore lost information if converted documents
are already modified and saved with 2.2.

Günter

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[Kornel Benko]

Am Mittwoch, 29. März 2017 um 12:41:37, schrieb José Abílio Matos 

> On Wednesday, 29 March 2017 12.14.29 WEST Kornel Benko wrote:
> > Export tests to previous lyx-versions using python3 are failing.
> 
> It should be working now.


Yes, all 990 tests succeeded. (e.g. 'ctest -R '_lyx(16|21|22)$')

> Using iteritems() returns an iterator while items() returns a list in python 
> 2.
> 
> For python 3 items() returns and iterator and iteritems() does not exists.
> 
> Since the list of branches should be small (where I mean < 1000) the 
> performance penalty will be negligible and it will work for both versions.
> 
> Regards,


So, should we make python3 be the default for lyx2.3?

There is one task remaining. Creation of lyxrc.defaults uses string 'python'
as executable even if lyx is configured for use with python3.
I had to edit this file to get the desired python3 to run the converts.
(Example: \converter figpstex  "python -tt $$s/scripts/fig2pstex.py 
$$i $$o""")

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[Kornel Benko]

Am Mittwoch, 29. März 2017 um 01:32:05, schrieb Scott Kostyshak 

> On Tue, Mar 28, 2017 at 11:43:10AM +0100, José Abílio Matos wrote:
> 
> > At the same time and do not expect pressure from the users who want to 
> > convert 
> > lyx-0.10 documents so I do no expect that Scott makes this a release 
> > blocker 
> > for 2.3. :-D
> 
> Thanks for the investigation on this issue, José.
> 
> Scott

Export tests to previous lyx-versions using python3 are failing.
Example:
Warning: An error ocurred in 510, 
Traceback (most recent call last):
  File "/usr2/src/lyx/lyx-git/lib/lyx2lyx/lyx2lyx", line 96, in 
main()
  File "/usr2/src/lyx/lyx-git/lib/lyx2lyx/lyx2lyx", line 90, in main
doc.convert()
  File "/usr2/src/lyx/lyx-git/lib/lyx2lyx/LyX.py", line 679, in convert
conv(self)
  File "/usr2/src/lyx/lyx-git/lib/lyx2lyx/lyx_2_3.py", line 169, in 
revert_ibranches
for old, new in ibranches.iteritems():
AttributeError: 'dict' object has no attribute 'iteritems'
support/Systemcall.cpp (295): Systemcall: 'python3 -tt 
"/usr2/src/lyx/lyx-git/lib/lyx2lyx/lyx2lyx" -V 1.6 -o "splash.16.lyx" 
"splash.lyx"' finished with exit code 1

Kornel


signature.asc
Description: This is a digitally signed message part.

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[José Abílio Matos]

On Wednesday, 29 March 2017 12.14.29 WEST Kornel Benko wrote:
> Export tests to previous lyx-versions using python3 are failing.

It should be working now.

Using iteritems() returns an iterator while items() returns a list in python 
2.

For python 3 items() returns and iterator and iteritems() does not exists.

Since the list of branches should be small (where I mean < 1000) the 
performance penalty will be negligible and it will work for both versions.

Regards,
-- 
José Abílio

Re: LyX master still not ready for Python 3

[José Abílio Matos]

On Wednesday, 29 March 2017 01.12.06 WEST Uwe Stöhr wrote:
> As requested by José, I installed Python 3.6.1 and tested today's master
> with it. As result LyX cannot be started because the reconfiguration fails:
> 
> 02:05:53.187: Checking whether TeX allows spaces in file names... yes
> 02:05:53.188: Traceback (most recent call last):
> 02:05:53.189:   File "D:/LyXGit/Master/lib/configure.py", line 1814, in
> 
> 02:05:53.192: windows_style_tex_paths = checkTeXPaths()
> 02:05:53.193:   File "D:/LyXGit/Master/lib/configure.py", line 206, in
> checkTeXPaths
> 02:05:53.194: inpname = shortPath(str(tmpfname,
> encoding)).replace('\\', '/')
> 02:05:53.196: TypeError: decoding str is not supported
> 02:05:53.198: configuration will be reloaded...
> D:\LyXGit\Master\src\support\Systemcall.cpp (295): Systemcall: 'python
> -tt "D:/LyXGit/Master/lib/configure.py"
> --binary-dir="D:/LyXGit/Master/compile-2015/bin/Release/"' finished with
> exit code 1
> 
> regards Uwe

Could you, please, open a terminal/power shell to call directly python and 
then run:

import locale
locale.getdefaultlocale()

I am interested in the output of the second line.

It should be something like, I am guessing here,
('de_DE', 'cp1252') 

but in this case since there is an error I suspect that the outcome is

(None, None)

FWIW in Fedora 25 I get

(None, None) for python 2

and

('en_US', 'UTF-8') for python 3 # the correct answer is this case. :-)

Regards,
-- 
José Abílio