Hi Vaclav, I think that it would make much more sense to have the GRASS python libraries using unicode, and to add an interface managing the translation to/from bytes when dealing with C code. Python programmers using the GRASS libraries will expect unicode strings.
Laurent 2017-11-26 21:21 GMT-06:00 Vaclav Petras <[email protected]>: > Dear all, > > after looking at different Python 2 to 3 porting issues, doing r71849, and > reading #3392, I understand the following: > > * Several solutions for poring exist. Most recent one is python-future > project, but only from __future__ import ... is part of the library and thus > guaranteed with recent Python 2.7. (We can discuss concrete steps > separately.) > > * However, the most challenging part of the porting will be the unicode. > > * There is no way around the unicode when using Python 3. Unicode is > inherent part of the language even things such as os.environ or > sys.stdout.write() work only with unicode. I'm not sure what exactly the > rule is here, but it seems to be everywhere. > > * I haven't seen any simple fix which would limit the changes in the code in > a way, e.g., in which print statement can be fixed. > > * GUI will always use unicode because that's how the libraries and > interfaces as set. > > * In relation to the previous point, one of the reasons why unicode is used > that thinks like text[:10] actually return 10 characters to display. > > * C library will not use unicode for now. > > * Users of the Python API who are using Python 3 will expect unicode strings > to work, i.e. expect run_command('g.region', flags='p') to work (not just > run_command(b'g.region', flags=b'p')). > > * If Python libraries are unicode, there will need to be an interface to > work with ctypes which would add to existing code for transferring from C > world to Python and back. > > * If Python libraries are bytes, there will need to be an interface to work > with GUI in unicode as well as with users of the API who will expect unicode > to work. In other words, internally it would use bytes, but interface must > be both bytes (for modules and internal use) and unicode (for GUI and > users). > > * Having unicode-based library means encoding and decoding on any "external" > interface such as file reading or ctypes. > > * Having bytes-based library means encoding and decoding on any interface > such as Python 3 interface such as os.environ and additionally rewriting all > string literals ("abc") to bytes (b"abc"). > > * It seems hard to predict when we will know the right encoding of the text. > It seems that we will need it with any solution since garbage-in-garbage > stops when you need to use some system interface function in Python 3 which > requires unicode. Although e.g. sys.stdout.write() has a (less generic) > sys.stdout.buffer.write() alternative, os.environb does not work on MS > Windows. > > An example fix in r71849 is done using a (custom) decode function which > creates unicode (standard string in Python3) when file content is read. > Alternative to this change would be changing all the strings in the file to > bytes (b'abc' as opposed to 'abc'). > > Please comment or link other related discussions. > > Thanks, > Vaclav > > > python3 -c "import os; os.environ[b'abc'] = b'def'" > python3 -c "import os; os.environb[b'abc'] = b'def'" > python3 -c "import sys; sys.stdout.write(b'abc\n')" > python3 -c "import sys; sys.stdout.buffer.write(b'abc\n')" > python3 -c "import os; print(type(os.name))" > https://trac.osgeo.org/grass/changeset/71849 > https://trac.osgeo.org/grass/ticket/2708 > https://trac.osgeo.org/grass/ticket/3392 > https://trac.osgeo.org/grass/query?status=!closed&keywords=~python3 > https://trac.osgeo.org/grass/query?status=!closed&keywords=~encoding > https://trac.osgeo.org/grass/query?status=!closed&keywords=~unicode > > _______________________________________________ > grass-dev mailing list > [email protected] > https://lists.osgeo.org/mailman/listinfo/grass-dev _______________________________________________ grass-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/grass-dev
