Re: Support for python 2 and 3 in lyx2lyx for 2.3

[Kornel Benko]

Am Mittwoch, 29. März 2017 um 15:52:22, schrieb José Abílio Matos 

> On Wednesday, 29 March 2017 15.25.12 WEST Kornel Benko wrote:
> > > > There is one task remaining. Creation of lyxrc.defaults uses string
> > > > 'python' as executable even if lyx is configured for use with python3.
> > 
> > > I agree that this could change.
> 
> Actually here I meant to say not just that it could but the it *should* 
> change.

Cmake part is ready.

> > > > I had to edit this file to get the desired python3 to run the converts.
> > > > (Example: \converter figpstex  "python -tt
> > > > $$s/scripts/fig2pstex.py $$i $$o""")
> > 
> > > Honestly here I suggest us to change this automatically for users.
> > > If the first command is python we should change it to the python version
> > > configured. If the user wants to use a different python version the user
> > > is
> > > free to use say "python2 -tt ..."
> > > 
> > > For example in my system where python is python2 I have:
> > > $ ll /usr/bin/python
> > > lrwxrwxrwx. 1 root root 7 Jan 12 18:00 /usr/bin/python -> python2
> > > $ ll /usr/bin/python2
> > > lrwxrwxrwx. 1 root root 9 Jan 12 18:00 /usr/bin/python2 -> python2.7
> > > 
> > > And that has been like this for ages. 
> > 
> > yes, it is the same here. But I don't want to change the symbolic link
> > because it may influence some other programs.
> > Instead I'd prefer to have an option to tell lyx to use python3 for handling
> > of lyx-local python scripts.
> 
> In case it was not clear I mean that we should do it inside lyx because the 
> python2 and python3 names are *always* available.
> 
> So in every place where we have the python name as the first command we 
> should 
> change it to python2 or python3, like if in a sense python meant $$python 
> that 
> we replace by the appropriate python name.
> 
> This way it will work for everyone and someone that needs can pin the python 
> version by directly placing the major version in front of the python name.
> 
> I hope that this is clear now. :-)
> 
> Regards,

Thanks for clarification. Yes, better as an option since it does not need user 
intervention.

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[José Abílio Matos]

On Wednesday, 29 March 2017 15.25.12 WEST Kornel Benko wrote:
> > > There is one task remaining. Creation of lyxrc.defaults uses string
> > > 'python' as executable even if lyx is configured for use with python3.
> 
> > I agree that this could change.

Actually here I meant to say not just that it could but the it *should* 
change.

> > > I had to edit this file to get the desired python3 to run the converts.
> > > (Example: \converter figpstex  "python -tt
> > > $$s/scripts/fig2pstex.py $$i $$o""")
> 
> > Honestly here I suggest us to change this automatically for users.
> > If the first command is python we should change it to the python version
> > configured. If the user wants to use a different python version the user
> > is
> > free to use say "python2 -tt ..."
> > 
> > For example in my system where python is python2 I have:
> > $ ll /usr/bin/python
> > lrwxrwxrwx. 1 root root 7 Jan 12 18:00 /usr/bin/python -> python2
> > $ ll /usr/bin/python2
> > lrwxrwxrwx. 1 root root 9 Jan 12 18:00 /usr/bin/python2 -> python2.7
> > 
> > And that has been like this for ages. 
> 
> yes, it is the same here. But I don't want to change the symbolic link
> because it may influence some other programs.
> Instead I'd prefer to have an option to tell lyx to use python3 for handling
> of lyx-local python scripts.

In case it was not clear I mean that we should do it inside lyx because the 
python2 and python3 names are *always* available.

So in every place where we have the python name as the first command we should 
change it to python2 or python3, like if in a sense python meant $$python that 
we replace by the appropriate python name.

This way it will work for everyone and someone that needs can pin the python 
version by directly placing the major version in front of the python name.

I hope that this is clear now. :-)

Regards,
-- 
José Abílio

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[Kornel Benko]

Am Mittwoch, 29. März 2017 um 14:50:46, schrieb José Abílio Matos 

> On Wednesday, 29 March 2017 13.19.23 WEST Kornel Benko wrote:
> > So, should we make python3 be the default for lyx2.3?
> 
> We want it to work as well as it works python 2. If it is the default or not 
> I 
> have no strong opinion.
>  
> > There is one task remaining. Creation of lyxrc.defaults uses string 'python'
> > as executable even if lyx is configured for use with python3.
> 
> I agree that this could change.
> 
> > I had to edit this file to get the desired python3 to run the converts.
> > (Example: \converter figpstex  "python -tt
> > $$s/scripts/fig2pstex.py $$i $$o""")
> 
> Honestly here I suggest us to change this automatically for users.
> If the first command is python we should change it to the python version 
> configured. If the user wants to use a different python version the user is 
> free to use say "python2 -tt ..."
> 
> For example in my system where python is python2 I have:
> $ ll /usr/bin/python
> lrwxrwxrwx. 1 root root 7 Jan 12 18:00 /usr/bin/python -> python2
> $ ll /usr/bin/python2
> lrwxrwxrwx. 1 root root 9 Jan 12 18:00 /usr/bin/python2 -> python2.7
> 
> And that has been like this for ages. :-)
> 
> > Kornel

yes, it is the same here. But I don't want to change the symbolic link because
it may influence some other programs.
Instead I'd prefer to have an option to tell lyx to use python3 for handling of 
lyx-local python scripts.

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[José Abílio Matos]

On Wednesday, 29 March 2017 13.19.23 WEST Kornel Benko wrote:
> So, should we make python3 be the default for lyx2.3?

We want it to work as well as it works python 2. If it is the default or not I 
have no strong opinion.
 
> There is one task remaining. Creation of lyxrc.defaults uses string 'python'
> as executable even if lyx is configured for use with python3.

I agree that this could change.

> I had to edit this file to get the desired python3 to run the converts.
> (Example: \converter figpstex  "python -tt
> $$s/scripts/fig2pstex.py $$i $$o""")

Honestly here I suggest us to change this automatically for users.
If the first command is python we should change it to the python version 
configured. If the user wants to use a different python version the user is 
free to use say "python2 -tt ..."

For example in my system where python is python2 I have:
$ ll /usr/bin/python
lrwxrwxrwx. 1 root root 7 Jan 12 18:00 /usr/bin/python -> python2
$ ll /usr/bin/python2
lrwxrwxrwx. 1 root root 9 Jan 12 18:00 /usr/bin/python2 -> python2.7

And that has been like this for ages. :-)

> Kornel

-- 
José Abílio

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[Kornel Benko]

Am Mittwoch, 29. März 2017 um 12:41:37, schrieb José Abílio Matos 

> On Wednesday, 29 March 2017 12.14.29 WEST Kornel Benko wrote:
> > Export tests to previous lyx-versions using python3 are failing.
> 
> It should be working now.


Yes, all 990 tests succeeded. (e.g. 'ctest -R '_lyx(16|21|22)$')

> Using iteritems() returns an iterator while items() returns a list in python 
> 2.
> 
> For python 3 items() returns and iterator and iteritems() does not exists.
> 
> Since the list of branches should be small (where I mean < 1000) the 
> performance penalty will be negligible and it will work for both versions.
> 
> Regards,


So, should we make python3 be the default for lyx2.3?

There is one task remaining. Creation of lyxrc.defaults uses string 'python'
as executable even if lyx is configured for use with python3.
I had to edit this file to get the desired python3 to run the converts.
(Example: \converter figpstex  "python -tt $$s/scripts/fig2pstex.py 
$$i $$o""")

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[José Abílio Matos]

On Wednesday, 29 March 2017 12.14.29 WEST Kornel Benko wrote:
> Export tests to previous lyx-versions using python3 are failing.

It should be working now.

Using iteritems() returns an iterator while items() returns a list in python 
2.

For python 3 items() returns and iterator and iteritems() does not exists.

Since the list of branches should be small (where I mean < 1000) the 
performance penalty will be negligible and it will work for both versions.

Regards,
-- 
José Abílio

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[Kornel Benko]

Am Mittwoch, 29. März 2017 um 01:32:05, schrieb Scott Kostyshak 

> On Tue, Mar 28, 2017 at 11:43:10AM +0100, José Abílio Matos wrote:
> 
> > At the same time and do not expect pressure from the users who want to 
> > convert 
> > lyx-0.10 documents so I do no expect that Scott makes this a release 
> > blocker 
> > for 2.3. :-D
> 
> Thanks for the investigation on this issue, José.
> 
> Scott

Export tests to previous lyx-versions using python3 are failing.
Example:
Warning: An error ocurred in 510, 
Traceback (most recent call last):
  File "/usr2/src/lyx/lyx-git/lib/lyx2lyx/lyx2lyx", line 96, in 
main()
  File "/usr2/src/lyx/lyx-git/lib/lyx2lyx/lyx2lyx", line 90, in main
doc.convert()
  File "/usr2/src/lyx/lyx-git/lib/lyx2lyx/LyX.py", line 679, in convert
conv(self)
  File "/usr2/src/lyx/lyx-git/lib/lyx2lyx/lyx_2_3.py", line 169, in 
revert_ibranches
for old, new in ibranches.iteritems():
AttributeError: 'dict' object has no attribute 'iteritems'
support/Systemcall.cpp (295): Systemcall: 'python3 -tt 
"/usr2/src/lyx/lyx-git/lib/lyx2lyx/lyx2lyx" -V 1.6 -o "splash.16.lyx" 
"splash.lyx"' finished with exit code 1

Kornel


signature.asc
Description: This is a digitally signed message part.

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[Scott Kostyshak]

On Tue, Mar 28, 2017 at 11:43:10AM +0100, José Abílio Matos wrote:

> At the same time and do not expect pressure from the users who want to 
> convert 
> lyx-0.10 documents so I do no expect that Scott makes this a release blocker 
> for 2.3. :-D

Thanks for the investigation on this issue, José.

Scott


signature.asc
Description: PGP signature

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[José Abílio Matos]

On Tuesday, 28 March 2017 23.57.39 WEST José Abílio Matos wrote:
> FWIW the culprit seems to be normalize_font_whitespace in lyx_1_5.py, that
> is used for formats 259 and 274.

I found the reason.

In that function the is a variable called changes that is a dictionary. We 
access to the list of keys of that dictionary with list(changes.keys()) but 
this function does not guarantee the same order return between two different 
python versions. BTW this also applies to python 2 versions.

After reading the code I see that both versions are functionally equivalent. 
:-D

OK this means that we do not have a bug between the two versions, but 
something that we can improve by making the return order stable.

One possible option would be to use

from collections import OrderedDict

and then

changes = OrderedDict()

this guarantees a stable return order...

-- 
José Abílio

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[José Abílio Matos]

On Tuesday, 28 March 2017 15.07.17 WEST Jean-Marc Lasgouttes wrote:
> It looks like something that can be handled by a bisect, either a real
> one or a hand-made one where a dichotomy is done on the intermediate
> commits.

Nah... I prefer to follow Ken Thompson's advice:
"When in doubt, use brute force."

Just in case I remember this quote from Brian W. Kernighan and P. J. Plauger 
in "The Elements of Programming Style".

:-)

FWIW the culprit seems to be normalize_font_whitespace in lyx_1_5.py, that is 
used for formats 259 and 274.

> Looks good!

I agree. :-)

I have noticed that python 3 fails to correctly detect the version used to 
produce the original document. In any case this information is redundant since 
it correctly gets the file format and so the conversion works as expected.

My purpose is to get all the different behaviours between the two versions to 
get any possible error. Just like your work with coverity reports. :-)

> JMarc

Regards,
-- 
José Abílio

Re: Support for python 2 and 3 in lyx2lyx for 2.3

[Jean-Marc Lasgouttes]

Le 28/03/2017 à 12:43, José Abílio Matos a écrit :

Using the 0.12.0 User Guide as the starting point the only differences between
the two conversions is that python 2 way gets at some points:

\bar default
\family default

while the python 3 gets

\family default
\bar default

This happens in 12 places.

This is good because it means that the difference comes likely from a single
place and with so many conversions being done this is better than OK.


It looks like something that can be handled by a bisect, either a real 
one or a hand-made one where a dichotomy is done on the intermediate 
commits.


Looks good!

JMarc