[issue19459] Python does not support the GEORGIAN-PS charset

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19459] Python does not support the GEORGIAN-PS charset

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

On 19.10.2021 10:44, Serhiy Storchaka wrote:
> 
> Possible solutions (they can be combined):
> 
> 1. Add support for the GEORGIAN-PS charset and all other encodings used in 
> libc (issue22679). The problem is that it is difficult to get the official 
> information about these encodings.

As with all encodings we add: there has to be a real need to support
them natively in Python (as opposed to installing codecs via PyPI)
and we need a definite source for the encoding, e.g. a standards
document from an official body.

IMO, we should not really add more encodings to the stdlib, but instead
point people to e.g. the iconv package:

https://pypi.org/project/python-iconv/

Perhaps we ought to make it easier for such packages to provide
additional codecs even during the startup phase, e.g. via a special
env var which points Python to a list of codec packages to load
prior to initializing the I/O encoding... not sure whether this is
possible, though.

> 2. Falls back to utf-8 or ascii+surrogateescape in case of unsupported locale 
> encoding. But typos can slip unnoticed.

I think this would be a more general solution to such cases, provided
the startup logic issues a visible warning about the fallback.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19459] Python does not support the GEORGIAN-PS charset

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Possible solutions (they can be combined):

1. Add support for the GEORGIAN-PS charset and all other encodings used in libc 
(issue22679). The problem is that it is difficult to get the official 
information about these encodings.

2. Falls back to utf-8 or ascii+surrogateescape in case of unsupported locale 
encoding. But typos can slip unnoticed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19459] Python does not support the GEORGIAN-PS charset

[STINNER Victor]

STINNER Victor  added the comment:

Python uses UTF-8 if the locale is not supported:

$ LANG=xxx python3.9 -c "import sys; print(sys.flags.utf8_mode)"
1

On Fedora 34, the locale is still supported, and Python 3.11 still fails:

vstinner@apu$ LANG=ka_GE.georgianps locale
LANG=ka_GE.georgianps
LC_CTYPE="ka_GE.georgianps"
LC_NUMERIC="ka_GE.georgianps"
LC_TIME="ka_GE.georgianps"
LC_COLLATE="ka_GE.georgianps"
LC_MONETARY="ka_GE.georgianps"
LC_MESSAGES="ka_GE.georgianps"
LC_PAPER="ka_GE.georgianps"
LC_NAME="ka_GE.georgianps"
LC_ADDRESS="ka_GE.georgianps"
LC_TELEPHONE="ka_GE.georgianps"
LC_MEASUREMENT="ka_GE.georgianps"
LC_IDENTIFICATION="ka_GE.georgianps"
LC_ALL=

vstinner@apu$ LANG=ka_GE.georgianps python3.11 -c "import sys; 
print(sys.flags.utf8_mode)"
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = './python'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  stdlib dir = '/home/vstinner/python/main/Lib'
  sys._base_executable = '/home/vstinner/python/main/python'
  sys.base_prefix = '/usr/local'
  sys.base_exec_prefix = '/usr/local'
  sys.platlibdir = 'lib'
  sys.executable = '/home/vstinner/python/main/python'
  sys.prefix = '/usr/local'
  sys.exec_prefix = '/usr/local'
  sys.path = [
'/usr/local/lib/python311.zip',
'/home/vstinner/python/main/Lib',
'/home/vstinner/python/main/build/lib.linux-x86_64-3.11-pydebug',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the 
filesystem encoding
Python runtime state: core initialized
LookupError: unknown encoding: GEORGIAN-PS

Current thread 0x7ff89b81d2c0 (most recent call first):
  

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19459] Python does not support the GEORGIAN-PS charset

[Tal Einat]

Tal Einat  added the comment:

With recent versions of Python (e.g. 3.9) this no longer causes a crash. Python 
apparently falls back to UTF-8, at least on my system:

$ LANG=ka_GE.georgianps python3.9
Python 3.9.7 (default, Sep  9 2021, 23:20:13) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale; print(locale.getpreferredencoding())
UTF-8

I'm marking this as fixed. If someone still has issues with this encoding, 
please open a new issue with up-to-date information.

--
nosy: +taleinat
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19459] Python does not support the GEORGIAN-PS charset

[Jakub Wilk]

Changes by Jakub Wilk jw...@jwilk.net:


--
nosy: +jwilk

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19459
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19459] Python does not support the GEORGIAN-PS charset

[STINNER Victor]

Changes by STINNER Victor victor.stin...@gmail.com:


--
title: Fatal Python error: Py_Initialize: Unable to get the locale encoding: 
GEORGIAN-PS - Python does not support the GEORGIAN-PS charset
versions: +Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19459
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19459] Python does not support the GEORGIAN-PS charset

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
nosy: +lemburg, loewis, serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19459
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com