On 13/10/13 22:15, Andrew Gregory wrote: > On 10/13/13 at 05:11pm, Allan McRae wrote: >> On 13/10/13 13:14, Martin Panter wrote: >>> On 13 October 2013 13:14, Jeremy Heiner <[email protected]> wrote: >>>> On Sat, Oct 12, 2013 at 9:16 PM, Allan McRae <[email protected]> wrote: >>>>> On 13/10/13 02:44, Jeremy Heiner wrote: >>>>>> There was one hiccup in encoding in test sync600: the unicode strings >>>>>> which work in other tests for some reason cause problems here. This >>>>>> hiccup is only in 2.7, so a try block is used to fall back to use the >>>>>> unencoded string on that runtime only. >>>>> >>>>> Using "for some reason" in the description is a clear flag for me not to >>>>> apply the patch. This tends to result in an underlying issue being >>>>> missed due to lack of understanding. >>>> >>>> Hi, Allan! >>>> Yeah, I was not happy at all about having to put that exception >>>> handler in. I tried a crazy number of other workarounds, and in doing >>>> so I got quite a good understanding. But describing it adequately will >>>> take a lot of typing. Like several pages. It is absolutely right of >>>> you to ask, and I'm happy and able to provide that, but I doubt I can >>>> get to it before tomorrow night. The short answer is that Unicode >>>> support in Python 2 is fundamentally broken. Fixing that is why Python >>>> 3 got a major version bump and they had to give up on backwards >>>> compatibility. Sorry I don't have more time right now to go into all >>>> the gory details... and I hope the other readers won't be too bored by >>>> my post when I do. >>> >>> Looking at the “sync600” test you mentioned, it has a bunch of file >>> names with non-ASCII characters. Perhaps you might have better luck >>> putting a “u” prefix in front of the strings, or using “from >>> __future__ import unicode_literals”. Be aware that the u"" syntax was >>> only added back in Python 3.3. >>> >> >> I get the feeling that this should be dealt with in sync600 rather than >> what this patch does. >> >> Comments? >> >> Allan > > This is the reason I never submitted my own patchset. I don't think > there is a good way to do this. There doesn't appear to be a way to > use either a unicode or bytes literal in the test that is compatible > with both python2 and python3 unless we also drop support for versions > of python3 < 3.3, which are still supported. The only solution > I found was to test for the python version in the test suite. But, > given that our build process is tied to python2 anyway until asciidoc > supports python3 it didn't seem worth it to make the transition until > we could drop python2 entirely. >
Just a note. Our build process for git is tied to asciidoc, but not for release tarballs. I am going to merge all these patches apart from this one and the final patch. If a consensus can be found on how to deal with this issue, I will pull it in - I am not familiar enough with python issues to make the decision myself. Allan
