Re: Migration of preferences

[Jean-Marc Lasgouttes]

As someone who actually met José, I can testify that he actually _is_ a smiley.

JMarc

Le 13 mai 2017 12:32:33 GMT+02:00, "José Abílio Matos"  a 
écrit :
>On Saturday, 13 May 2017 10.50.10 WEST Stephan Witt wrote:
>> Yes, I put in in. (And you luckily hit a Friday to place your smiley 
>)
>
>That has never stopped me before. :-D
>
>My daughter and son follow the same footsteps, some of the other
>parents said 
>that never saw them not smiling in the morning. That is obviously an 
>exaggeration but the point remains. :-)

Re: Migration of preferences

[Stephan Witt]

Am 13.05.2017 um 12:32 schrieb José Abílio Matos :
> 
> On Saturday, 13 May 2017 10.50.10 WEST Stephan Witt wrote:
>> Yes, I put in in. (And you luckily hit a Friday to place your smiley  )
> 
> That has never stopped me before. :-D
> 
> My daughter and son follow the same footsteps, some of the other parents said 
> that never saw them not smiling in the morning. That is obviously an 
> exaggeration but the point remains. :-)
> 
>> I’m not so easy with your proposal to remove the error handler.
>> I think we have a real problem here: the encoding of the path names
>> in preferences is undefined if I’m not mistaken.
> 
> I understand and share your concern. But we were placing those safeguards at 
> the wrong place.
> 
> The encoding of the path names is related with the name of the file and not 
> with its content. The error handler was placed when reading the file and thus 
> is related with its content.

I meant the name of a directory as part of a preference entry.
Isn’t this read and converted by prefs2prefs too?

> 
>> Of course we have to fix it in some way - but I don’t know enough
>> of python (obviously) to make the right proposal.
> 
> FWIW note the we could restore the previous behaviour by doing:
> 
> if PY2:
>source = io.open(args[0], 'r', encoding='utf_8')
> else:
>source = io.open(args[0], 'r', encoding='utf_8', errors='surrogateescape')
> 
> with PY2 being defined as usual in the other python files.

Ok, I can take a look - if you want I should do it. (As an exercise?)

Stephan

Re: Migration of preferences

[José Abílio Matos]

On Saturday, 13 May 2017 10.50.10 WEST Stephan Witt wrote:
> Yes, I put in in. (And you luckily hit a Friday to place your smiley  )

That has never stopped me before. :-D

My daughter and son follow the same footsteps, some of the other parents said 
that never saw them not smiling in the morning. That is obviously an 
exaggeration but the point remains. :-)
 
> I’m not so easy with your proposal to remove the error handler.
> I think we have a real problem here: the encoding of the path names
> in preferences is undefined if I’m not mistaken.

I understand and share your concern. But we were placing those safeguards at 
the wrong place.

The encoding of the path names is related with the name of the file and not 
with its content. The error handler was placed when reading the file and thus 
is related with its content.

> Of course we have to fix it in some way - but I don’t know enough
> of python (obviously) to make the right proposal.

FWIW note the we could restore the previous behaviour by doing:

if PY2:
source = io.open(args[0], 'r', encoding='utf_8')
else:
source = io.open(args[0], 'r', encoding='utf_8', errors='surrogateescape')

with PY2 being defined as usual in the other python files.

> Stephan

-- 
José Abílio

Re: Migration of preferences

[Stephan Witt]

Am 12.05.2017 um 11:32 schrieb José Abílio Matos :
> 
> On Thursday, 11 May 2017 14.26.37 WEST Jürgen Spitzmüller wrote:
>> 2017-05-11 12:51 GMT+02:00 José Abílio Matos :
>>> I will look into this problem if we have a bug report for it and a simple
>>> example file.
>> 
>> http://www.lyx.org/trac/ticket/10660
>> 
>> Jürgen
>> 
>>> --
>>> José Abílio
> 
> Thank you.
> 
> First I will blame Stephan (at least according to git). ;-)

Yes, I put in in. (And you luckily hit a Friday to place your smiley :-) )

I’m not so easy with your proposal to remove the error handler.
I think we have a real problem here: the encoding of the path names
in preferences is undefined if I’m not mistaken.

Of course we have to fix it in some way - but I don’t know enough
of python (obviously) to make the right proposal.

Stephan

> 
> The problem is:
> 
> LookupError: unknown error handler name 'surrogateescape'
> 
> according to PEP383:
> https://www.python.org/dev/peps/pep-0383/
> 
> this error handler was only introduced in python 3 and so it makes sense the 
> error seen by Andrew.
> 
> 
> OTHO both Stephan and I (and probably others) do not see this problem with 
> python 2, so probably the error handler was backported to python 2.
> 
> 
> So I suggest to remove for the moment the error argument from the io.open 
> call.
> 
> Also according to Nick Coghlan (python core member):
> http://python-notes.curiousefficiency.org/en/latest/python3/
> text_file_processing.html
> 
> "surrogate escape ... is the error handler that Python uses for most OS 
> facing 
> APIs to gracefully cope with encoding problems in the data supplied by the 
> OS."
> 
> so I think that we should be safe here.
> 
> -- 
> José Abílio

Re: Migration of preferences

[Jean-Marc Lasgouttes]

Le 10/05/2017 à 07:11, Stephan Witt a écrit :

On Mac the preferences are in a separate directory for every major release too. 
2014 I added the feature to create a copy of previous major releases 
automatically if they’re present as first step of the configure process. Not 
all developers were friends of this change - so I made it limited to platform 
darwin (Mac OS X). If you’re interested in doing likewise on Windows too it 
shouldn’t be impossible. That way the old major release and the new one exists 
in parallel - exactly the same as without the additional copy step.


It would not be too difficult when the is no user directory to make a 
list of the existing ones and show them to the user so that s/he can 
choose whether to import settings. When exporting settings, one should 
also run pref2pref and layout2layout in order to speed up LyX startup.


JMarc

Re: Migration of preferences

[José Abílio Matos]

On Thursday, 11 May 2017 14.26.37 WEST Jürgen Spitzmüller wrote:
> 2017-05-11 12:51 GMT+02:00 José Abílio Matos :
> > I will look into this problem if we have a bug report for it and a simple
> > example file.
> 
> http://www.lyx.org/trac/ticket/10660
> 
> Jürgen
> 
> > --
> > José Abílio

Thank you.

First I will blame Stephan (at least according to git). ;-)

The problem is:

 LookupError: unknown error handler name 'surrogateescape'

according to PEP383:
https://www.python.org/dev/peps/pep-0383/

this error handler was only introduced in python 3 and so it makes sense the 
error seen by Andrew.


OTHO both Stephan and I (and probably others) do not see this problem with 
python 2, so probably the error handler was backported to python 2.


So I suggest to remove for the moment the error argument from the io.open 
call.

Also according to Nick Coghlan (python core member):
http://python-notes.curiousefficiency.org/en/latest/python3/
text_file_processing.html

"surrogate escape ... is the error handler that Python uses for most OS facing 
APIs to gracefully cope with encoding problems in the data supplied by the 
OS."

so I think that we should be safe here.

-- 
José Abílio

Re: Migration of preferences

[Jürgen Spitzmüller]

2017-05-11 12:51 GMT+02:00 José Abílio Matos :

> I will look into this problem if we have a bug report for it and a simple
> example file.
>

http://www.lyx.org/trac/ticket/10660

Jürgen


> --
> José Abílio
>

Re: Migration of preferences

[José Abílio Matos]

On Tuesday, 9 May 2017 12.08.31 WEST Jürgen Spitzmüller wrote:
> We (José and Enrico) fixed a similar problem with encodings in layouts in
> lyx2lyx, so I suppose prefs2prefs can be fixed as well.
> 
> Jürgen

I agree with your point of view. :-)

OTOH the difference between lyx2lyx and prefs2prefs is that at some point we 
take care of the file encoding while I am not sure that we do it in 
prefs2prefs.

I will look into this problem if we have a bug report for it and a simple 
example file.
-- 
José Abílio

Re: Migration of preferences

[Stephan Witt]

Am 10.05.2017 um 02:27 schrieb Uwe Stöhr :
> 
> El 08.05.2017 a las 16:42, Scott Kostyshak escribió:
> 
>> On Mon, May 08, 2017 at 06:39:38PM +1200, Andrew Parsloe wrote:
> 
>>> alpha1-1 installed on windows 7. I can't remember if preferences are
>>> supposed to migrate on a major release.
> 
> They won't be migrated. The idea is to be able to run major versions side by 
> side. This way you can check our LyX 2.3.x as long as you want and can go 
> back to LyX 2.2.x if you encounter problems.

On Mac the preferences are in a separate directory for every major release too. 
2014 I added the feature to create a copy of previous major releases 
automatically if they’re present as first step of the configure process. Not 
all developers were friends of this change - so I made it limited to platform 
darwin (Mac OS X). If you’re interested in doing likewise on Windows too it 
shouldn’t be impossible. That way the old major release and the new one exists 
in parallel - exactly the same as without the additional copy step.

Stephan

Re: Migration of preferences

[Uwe Stöhr]

El 08.05.2017 a las 16:42, Scott Kostyshak escribió:


On Mon, May 08, 2017 at 06:39:38PM +1200, Andrew Parsloe wrote:



alpha1-1 installed on windows 7. I can't remember if preferences are
supposed to migrate on a major release.


They won't be migrated. The idea is to be able to run major versions 
side by side. This way you can check our LyX 2.3.x as long as you want 
and can go back to LyX 2.2.x if you encounter problems.


regards Uwe

Re: Migration of preferences

[Jürgen Spitzmüller]

2017-05-09 13:03 GMT+02:00 Scott Kostyshak:

> Nice work! I have no idea how to improve our read of different
> encodings. I'm guessing this issue is not so rare for users who manually
> edit the preferences files with various text editors that for some
> reason do not preserve encodings. If you happen to have your old
> preferences file, or can recreate the situation, you might want to post
> a bug report with a small preferences file.
>

We (José and Enrico) fixed a similar problem with encodings in layouts in
lyx2lyx, so I suppose prefs2prefs can be fixed as well.

Jürgen


>
> Scott
>

Re: Migration of preferences

[Scott Kostyshak]

On Tue, May 09, 2017 at 08:22:08PM +1200, Andrew Parsloe wrote:

> The problem was with a customized copy of cua.bind in my LyX2.3/bind folder.
> prefs2prefs.py was stumbling on it. After much blundering about I found the
> problem was an encoding one. The customized cua.bind dates from many years
> ago and (according to my text editor) the encoding is ANSI rather utf8.
> prefs2prefs.py choked at
> 
> line 52 in read: lines = source.read().splitlines() or ['']
> 
> Python27/lib/codecs.py, line 314, in decode
>   (result, consumed) = self._buffer_decode(data, self.errors, final)
> LookupError: unknown error handler name 'surrogateescape'
> 
> I have updated the customized cua.bind, now encoded in utf8, and the problem
> has vanished.

Nice work! I have no idea how to improve our read of different
encodings. I'm guessing this issue is not so rare for users who manually
edit the preferences files with various text editors that for some
reason do not preserve encodings. If you happen to have your old
preferences file, or can recreate the situation, you might want to post
a bug report with a small preferences file.

Scott


signature.asc
Description: PGP signature

Re: Migration of preferences

[Andrew Parsloe]

On 9/05/2017 2:42 a.m., Scott Kostyshak wrote:

On Mon, May 08, 2017 at 06:39:38PM +1200, Andrew Parsloe wrote:

alpha1-1 installed on windows 7. I can't remember if preferences are
supposed to migrate on a major release. They didn't. I renamed my LyX2.2
folder LyX2.3 and then had to manually increment the format numbers of the
files in bind and ui.


Thanks for testing, Andrew. I forget whether we migrate preferences on
Windows.

For any Windows-related emails, you might want to CC Uwe since he
doesn't have time to read every message on the mailing list but he's
interested in the Windows-related ones. (I add him to CC this time).

If you do not manually increment the format numbers of the bind and ui
files, what error do you get?

Scott



The problem was with a customized copy of cua.bind in my LyX2.3/bind 
folder. prefs2prefs.py was stumbling on it. After much blundering about 
I found the problem was an encoding one. The customized cua.bind dates 
from many years ago and (according to my text editor) the encoding is 
ANSI rather utf8. prefs2prefs.py choked at


line 52 in read: lines = source.read().splitlines() or ['']

Python27/lib/codecs.py, line 314, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
LookupError: unknown error handler name 'surrogateescape'

I have updated the customized cua.bind, now encoded in utf8, and the 
problem has vanished.


Andrew

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Re: Migration of preferences

[Scott Kostyshak]

On Mon, May 08, 2017 at 06:39:38PM +1200, Andrew Parsloe wrote:
> alpha1-1 installed on windows 7. I can't remember if preferences are
> supposed to migrate on a major release. They didn't. I renamed my LyX2.2
> folder LyX2.3 and then had to manually increment the format numbers of the
> files in bind and ui.

Thanks for testing, Andrew. I forget whether we migrate preferences on
Windows.

For any Windows-related emails, you might want to CC Uwe since he
doesn't have time to read every message on the mailing list but he's
interested in the Windows-related ones. (I add him to CC this time).

If you do not manually increment the format numbers of the bind and ui
files, what error do you get?

Scott


signature.asc
Description: PGP signature