[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Inada Naoki]

Inada Naoki  added the comment:


New changeset efefe25443c56988841ab96cdac01352123ba268 by Inada Naoki (wim 
glenn) in branch 'master':
bpo-27413: json.tool: Add --no-ensure-ascii option. (GH-17472)
https://github.com/python/cpython/commit/efefe25443c56988841ab96cdac01352123ba268


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[wim glenn]

Change by wim glenn :


--
pull_requests: +16953
pull_request: https://github.com/python/cpython/pull/17472

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[R. David Murray]

Changes by R. David Murray :


--
versions: +Python 3.7 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[R. David Murray]

Changes by R. David Murray :


--
nosy: +qingyunha

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Daniel Himmelstein]

Changes by Daniel Himmelstein :


--
pull_requests: +2779

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Daniel Himmelstein]

Changes by Daniel Himmelstein :


--
pull_requests: +232

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The last change just sweeps a problem under a rug.

For now json.tool never fails with valid data. But with the --no-ensure-ascii 
option it can fail when output a string not encodable with the locale encoding. 
All can work with common cases on common UTF-8 environment, but unexpectedly 
fail on nonstandard environment. It would be better to output encodable 
characters as is and represent unencodable characters with \u encoding.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Berker Peksag]

Berker Peksag added the comment:

> If I'm not misreading your comment, this will change the original behavior, 
> right?

Assuming you also change

   ensure_ascii = not options.no_ensure_ascii

to

   ensure_ascii = options.no_ensure_ascii

no, it won't change the original behavior. That way you won't need to invert 
the value of ``options.no_ensure_ascii`` in line 37.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
Removed message: https://bugs.python.org/msg273813

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
assignee: rhettinger -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Wei-Cheng Pan]

Wei-Cheng Pan added the comment:

1. Replaced non-ASCII literals to \u
2. Removed failed assertion

Test passed with LC_ALL=en_US and LC_ALL=en_US.UTF-8 .

I've tried to use locale.getdefaultlocale(), but seems the output string will 
vary in different locales.

--
Added file: 
https://bugs.python.org/file44246/json-add-an-option-to-bypass-non-ascii-characters-v4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Wei-Cheng Pan]

Changes by Wei-Cheng Pan :


Removed file: 
https://bugs.python.org/file44243/json-add-an-option-to-bypass-non-ascii-characters-v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Test fails on non-utf8 locale.

$ LC_ALL=en_US ./python -m test.regrtest -v -m test_no_ensure_ascii_flag 
test_json
...
==
FAIL: test_no_ensure_ascii_flag (test.test_json.test_tool.TestTool)
--
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/test_json/test_tool.py", line 115, in 
test_no_ensure_ascii_flag
self.assertEqual(out.splitlines(), b'"\\u6e2c\\u8a66"\n'.splitlines())
AssertionError: Lists differ: [b'"\\u00e6\\u00b8\\u00ac\\u00e8\\u00a9\\u00a6"'] 
!= [b'"\\u6e2c\\u8a66"']

First differing element 0:
b'"\\u00e6\\u00b8\\u00ac\\u00e8\\u00a9\\u00a6"'
b'"\\u6e2c\\u8a66"'

- [b'"\\u00e6\\u00b8\\u00ac\\u00e8\\u00a9\\u00a6"']
+ [b'"\\u6e2c\\u8a66"']

--

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Raymond Hettinger]

Raymond Hettinger added the comment:

The patch looks good.  If no one else objects soon, I will apply it.

--
assignee:  -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Wei-Cheng Pan]

Changes by Wei-Cheng Pan :


Removed file: 
https://bugs.python.org/file43596/json-add-an-option-to-bypass-non-ascii-characters.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Wei-Cheng Pan]

Wei-Cheng Pan added the comment:

Added doc and test.

--
Added file: 
https://bugs.python.org/file44243/json-add-an-option-to-bypass-non-ascii-characters-v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Wei-Cheng Pan]

Wei-Cheng Pan added the comment:

> The patch needs tests and documentation.

Ok, I'll update it later.

>> +parser.add_argument('--no-ensure-ascii', action='store_true', 
>> default=False,
>I'd go with ``action='store_false', default=True``.

If I'm not misreading your comment, this will change the original behavior, 
right? (because options.no_ensure_ascii will be set to True by default)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Berker Peksag]

Berker Peksag added the comment:

The patch needs tests and documentation.

> +parser.add_argument('--no-ensure-ascii', action='store_true', 
> default=False,

I'd go with ``action='store_false', default=True``.

--
nosy: +berker.peksag
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Wei-Cheng Pan]

Wei-Cheng Pan added the comment:

Use "--no-ensure-ascii" instead.

--
Added file: 
http://bugs.python.org/file43596/json-add-an-option-to-bypass-non-ascii-characters.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Wei-Cheng Pan]

Changes by Wei-Cheng Pan :


Removed file: 
http://bugs.python.org/file43578/json-add-an-option-to-bypass-non-ascii-characters.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[R. David Murray]

R. David Murray added the comment:

Sorry, yes, that's what I meant.  I think it will make it easier to understand 
and remember if the option uses the same terminology as the function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Wei-Cheng Pan]

Wei-Cheng Pan added the comment:

If the arguments should be aligned with those in dump/load, then maybe 
"--no-ensure-ascii" is an option?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[R. David Murray]

R. David Murray added the comment:

Maybe name it --no-insure-ascii?  Or --insure-ascii=no/yes. (or true/false).

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27413] Add an option to json.tool to bypass non-ASCII characters.

[Wei-Cheng Pan]

New submission from Wei-Cheng Pan:

This patch adds a command line option "--no-escape" that allows json.tool to 
display non-ASCII characters.

e.g.:

$ echo '"測試"' | python -m json.tool
"\u6e2c\u8a66"

$ echo '"測試"' | python -m json.tool --no-escape
"測試"

--
components: Library (Lib)
files: json-add-an-option-to-bypass-non-ascii-characters.patch
keywords: patch
messages: 269479
nosy: Wei-Cheng.Pan
priority: normal
severity: normal
status: open
title: Add an option to json.tool to bypass non-ASCII characters.
type: enhancement
versions: Python 3.6
Added file: 
http://bugs.python.org/file43578/json-add-an-option-to-bypass-non-ascii-characters.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com