[issue39758] StreamWriter.wait_closed() can hang indefinately.

2020-02-26 Thread Tom Christie


New submission from Tom Christie :

Raising an issue that's impacting us on `httpx`.

It appears that in some cases SSL unwrapping can cause `.wait_closed()` to hang 
indefinately.

Trio are particularly careful to work around this case, and have an extensive 
comment on it: 
https://github.com/python-trio/trio/blob/31e2ae866ad549f1927d45ce073d4f0ea9f12419/trio/_ssl.py#L779-L829
 

Originally raised via https://github.com/encode/httpx/issues/634

Tested on:

* Python 3.7.6
* Python 3.8.1

```
import asyncio
import ssl
import certifi

hostname = 'login.microsoftonline.com'
context = ssl.create_default_context()
context.load_verify_locations(cafile=certifi.where())

async def main():
reader, writer = await asyncio.open_connection(hostname, 443, ssl=context)
print('opened')
writer.close()
print('close started')
await writer.wait_closed()
print('close completed')

asyncio.run(main())
```

--
components: asyncio
messages: 362688
nosy: asvetlov, tomchristie, yselivanov
priority: normal
severity: normal
status: open
title: StreamWriter.wait_closed() can hang indefinately.
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue39758>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

2019-05-30 Thread Tom Christie


Tom Christie  added the comment:

Right, and `requests` *does* provide both those styles.

The point more being that *not* having closed the transport at the point of 
exit shouldn't end up raising a hard error. It doesn't raise errors in 
sync-land, and it shouldn't do so in async-land.

Similarly, we wouldn't expect an open file resource to cause errors to be 
raised at the point of exit.

--

___
Python tracker 
<https://bugs.python.org/issue36709>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

2019-05-28 Thread Tom Christie


Tom Christie  added the comment:

> From my understanding, the correct code should close all transports and wait 
> for their connection_lost() callbacks before closing the loop.

Ideally, yes, although we should be able to expect that an SSL connection that 
hasn't been gracefully closed wouldn't loudly error on teardown like that.

In standard sync code, the equivelent would running something like this...

```python
session = requests.Session()
session.get('https://example.com/')
```

We wouldn't expect a traceback to be raised on exiting. (Even though the user 
*hasn't* explicitly closed the session, and even though a keep alive SSL 
connection will be open at the point of exit.)

--

___
Python tracker 
<https://bugs.python.org/issue36709>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

2019-04-24 Thread Tom Christie


Tom Christie  added the comment:

This appears somewhat related: https://bugs.python.org/issue34506

As it *also* logs exceptions occuring during `_fatal_error` and `_force_close`.

--

___
Python tracker 
<https://bugs.python.org/issue36709>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

2019-04-24 Thread Tom Christie


New submission from Tom Christie :

If an asyncio SSL connection is left open (eg. any kind of keep-alive 
connection) then after closing the event loop, an exception will be raised...

Python:

```
import asyncio
import ssl
import certifi


async def f():
ssl_context = ssl.create_default_context()
ssl_context.load_verify_locations(cafile=certifi.where())
await asyncio.open_connection('example.org', 443, ssl=ssl_context)


loop = asyncio.get_event_loop()
loop.run_until_complete(f())
loop.close()
```

Traceback:

```
$ python example.py 
Fatal write error on socket transport
protocol: 
transport: <_SelectorSocketTransport fd=8>
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py",
 line 868, in write
n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor
Fatal error on SSL transport
protocol: 
transport: <_SelectorSocketTransport closing fd=8>
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py",
 line 868, in write
n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py",
 line 676, in _process_write_backlog
self._transport.write(chunk)
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py",
 line 872, in write
self._fatal_error(exc, 'Fatal write error on socket transport')
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py",
 line 681, in _fatal_error
self._force_close(exc)
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py",
 line 693, in _force_close
self._loop.call_soon(self._call_connection_lost, exc)
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py",
 line 677, in call_soon
self._check_closed()
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py",
 line 469, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
```

It looks to me like the original "OSError: [Errno 9] Bad file descriptor" 
probably shouldn't be raised in any case - if when attempting to tear down the 
SSL connection, then we should probably pass silently in the case that the 
socket has already been closed uncleanly.

Bought to my attention via: https://github.com/encode/httpcore/issues/16

--
assignee: christian.heimes
components: SSL, asyncio
messages: 340764
nosy: asvetlov, christian.heimes, tomchristie, yselivanov
priority: normal
severity: normal
status: open
title: Asyncio SSL keep-alive connections raise errors after loop close.
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue36709>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33366] `contextvars` documentation incorrectly refers to "non-local state".

2018-04-27 Thread Tom Christie

Tom Christie <t...@tomchristie.com> added the comment:

Refs: https://github.com/python/cpython/pull/6617

--

___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue33366>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33366] `contextvars` documentation incorrectly refers to "non-local state".

2018-04-26 Thread Tom Christie

New submission from Tom Christie <t...@tomchristie.com>:

The `contextvars` documentation, at 
https://docs.python.org/3.7/library/contextvars.html starts with the following:

"This module provides APIs to manage, store, and access non-local state."

I assume that must be a documentation bug, right. The module isn't for managing 
non-local state, it's for managing state that *is* local.

I'd assume it ought to read...

"This module provides APIs to manage, store, and access context-local state."

(ie. for managing state that is transparently either thread-local or task-local 
depending on the current execution context.)

--
assignee: docs@python
components: Documentation, asyncio
messages: 315792
nosy: asvetlov, docs@python, tomchristie, yselivanov
priority: normal
severity: normal
status: open
title: `contextvars` documentation incorrectly refers to "non-local state".
type: enhancement
versions: Python 3.7, Python 3.8

___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue33366>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1043134] Add preferred extensions for MIME types

2016-09-20 Thread Tom Christie

Tom Christie added the comment:

Confirming that I've also bumped into this for Python 3.5.

A docs update would seem to be the lowest-cost option to start with.

Right now `mimetypes.guess_extension()` isn't terribly useful, and it'd be 
better to at least know that upfront.

--
nosy: +Tom.Christie

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue1043134>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24033] Update _test_multiprocessing.py to use script helpers

2015-06-28 Thread Christie

Christie added the comment:

If #18576 gets merged before this does, I have some updated docs for the new 
helper functions.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24033
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-06-28 Thread Christie

Christie added the comment:

Hello all!

So the following are waiting for review:
* #24033
* #23981
* #24279

I'm going to hold off on continuing to refactor any other modules for the 
moment, would be great to get the above wrapped up and eventually merged when 
possible.

Thanks!
- Christie

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18576] Document test.support.script_helper

2015-06-28 Thread Christie

Christie added the comment:

Hey @vadmium!

I've finally responded to your review feedback. I've attached a new patch, 
hopefully this time actually review-able. I've removed the docs for methods I 
added in @24033 so that this could be merged without it.

Let me know if you have any more feedback, or if this could be merged.

Thanks!

--
Added file: http://bugs.python.org/file39827/issue18576.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18576
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23981] Update test_unicodedata.py to use script_helpers

2015-06-14 Thread Christie

Christie added the comment:

Uploading a new patch, diffed against a public revision so it's reviewable.

--
Added file: http://bugs.python.org/file39708/issue23981_test_unicode.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23981
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24033] Update _test_multiprocessing.py to use script helpers

2015-06-14 Thread Christie

Christie added the comment:

Now using existing function spawn_python instead of creating a new function.

--
Added file: http://bugs.python.org/file39710/issue24033_multi.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24033
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24279] Update test_base64 to use test.support.script_helper

2015-06-14 Thread Christie

Christie added the comment:

Uploading a new patch, diffed against a public revision so it's reviewable.

--
Added file: http://bugs.python.org/file39709/issue24279_base64.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24279
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18576] Document test.support.script_helper

2015-06-07 Thread Christie

Christie added the comment:

 If you diffed your patch from a public revision in the main repository, there 
 should be a nice “Review” link.

Ah, thanks @vadmium!!! I've been confused about why some patches get review 
links and others do not .

I'll have a new patch up shortly which is diffed against a public revision and 
has your suggested changes.

 For autodoc in particular, we prefer to avoid it for the main docs because 
 good docstring and good prose documentation don't necessarily look alike. 

@ncoghlan for some reason I thought that the main docs were also auto generated 
from docstrings in the actual modules, but I can see now that that isn't the 
case.

I'll reach out to d...@python.org to see how they feel about autogenerating the 
test.support docs.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18576
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24398] Update test_capi to use test.support.script_helper

2015-06-06 Thread Christie

Christie added the comment:

I can't seem to get test_no_FatalError_infinite_loop to fail - I tried 
reverting the fix that this was verifying, but the test still passed and no 
infinite loop to be seen. I'm wondering if this is because I can't reproduce it 
on a mac.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24398
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24033] Update _test_multiprocessing.py to use script helpers

2015-06-06 Thread Christie

Christie added the comment:

run_python and spawn_python are basically the same, I need to reconcile those.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24033
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18576] Document test.support.script_helper

2015-06-06 Thread Christie

Christie added the comment:

@ncoghlan I've taken the test.rst changes from the previous diff and reviewed 
them against the latest script_helper (including updates from issue24033). 
Nothing had fallen behind, but I've added what was missing.

* I've realized that spawn_python and the helper run_python I added in 
issue24033 are basically the same so I will reconcile those
* Is there some way to load the docstrings from script_helper.py in test.rst 
instead of writing explanations for each method twice?

--
Added file: http://bugs.python.org/file39645/issue18576.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18576
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-06-06 Thread Christie

Christie added the comment:

Created issue24398 for updating test_capi.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24398] Update test_capi to use test.support.script_helper

2015-06-06 Thread Christie

New submission from Christie:

As described in Issue9517, many test modules do not make use of the helpers in 
script_helpers.py to invoke the python interpreter in a subprocess. Issue9517 
will be broken down into several smaller issues so we can address smaller 
change sets.

This issue is to update test_capi.py to use script_helpers.py.

--
messages: 244923
nosy: bobcatfish
priority: normal
severity: normal
status: open
title: Update test_capi to use test.support.script_helper
versions: Python 3.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24398
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24279] Update test_base64 to use test.support.script_helper

2015-05-24 Thread Christie

New submission from Christie:

As described in Issue9517, many test modules do not make use of the helpers in 
script_helpers.py to invoke the python interpreter in a subprocess. Issue9517 
will be broken down into several smaller issues so we can address smaller 
change sets.

This issue is to update test_base64.py to use script_helpers.py.

--
messages: 243999
nosy: bobcatfish
priority: normal
severity: normal
status: open
title: Update test_base64 to use test.support.script_helper
versions: Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24279
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-05-24 Thread Christie

Christie added the comment:

Created Issue24279 for updating test_base64.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24279] Update test_base64 to use test.support.script_helper

2015-05-24 Thread Christie

Christie added the comment:

Updated TestMain in test_base64 to use test.support.script_helpers. Split 
`test_encode_file` into two tests.

--
keywords: +patch
Added file: http://bugs.python.org/file39485/issue24279_base64.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24279
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18576] Document test.support.script_helper

2015-05-18 Thread Christie

Christie added the comment:

I'm going to take a look at bringing these docs up to date and including the 
changes from issue24033.

--
nosy: +bobcatfish

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18576
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15376] Refactor the test_runpy walk_package support code into a common location

2015-05-18 Thread Christie

Christie added the comment:

@BreamoreBoy, it looks like the next step would be to revisit issue15403 and go 
through the steps outlined by @ncoghlan.

--
nosy: +bobcatfish

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15376
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-05-12 Thread Christie

Christie added the comment:

Cool, thanks @ncoghlan! Would you like someone to take on the work of updating 
the latest patch on issue 18576?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-05-11 Thread Christie

Christie added the comment:

Hey @berker.peksag, @r.david.murray, @serhiy.storchaka,

If you get a chance I've got some changes up for review at:
* http://bugs.python.org/issue24033
* http://bugs.python.org/issue23981

Thanks!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24033] Update _test_multiprocessing.py to use script helpers

2015-05-11 Thread Christie

Christie added the comment:

Updated both tests in _test_mulitprocessing.py which were using sys.exectuable 
and Popen to use script_helpers instead, refactored script_helpers a bit to 
make this work.

--
keywords: +patch
Added file: 
http://bugs.python.org/file39347/issue24033_script_helper_multi_both.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24033
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-05-06 Thread Christie

Christie added the comment:

Thanks very much @berker.peksag!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-26 Thread Christie

Christie added the comment:

@serhiy.storchaka - just double checking, do you guys need me to make any more 
changes to the patch?

And is there any more review needed, or is it possible for this to be merged?

Thanks very much!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-23 Thread Christie

Christie added the comment:

Hey @berker.peksag, I've added a new patch in response to your review!

 You should create the patch with hg diff --git to preserve moving. And then 
 apply it with hg import, not the patch command. Unfortunately Rietveld 
 don't like patches in git format.

Ick, I'm guessing it's okay to just use plain old hg diff then, if Rietveld 
doesn't like the patches in git format.

--
Added file: 
http://bugs.python.org/file39189/iss9517_move_script_helpers_review1.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-22 Thread Christie

Christie added the comment:

Hey @berker.peksag, @r.david.murray!

Here's another change set where script_helpers.py is moved with an hg mv (still 
looks in the patch like the file was deleted and re-added, not sure if that's 
expected).

I've ALSO removed some unused imports from script_helpers.py - temp_dir was 
being imported from script_helpers in a few places tho, so I had to fix that. 
If you guys would rather not include that change here, please let me know!

--
Added file: http://bugs.python.org/file39176/iss9517_move_script_helpers.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24033] Update _test_multiprocessing.py to use script helpers

2015-04-22 Thread Christie

New submission from Christie:

As described in Issue9517, many test modules do not make use of the helpers in 
script_helpers.py to invoke the python interpreter in a subprocess. Issue9517 
will be broken down into several smaller issues so we can address smaller 
change sets.

This issue is to update _test_multiprocessing.py to use script_helpers.py.

--
components: Tests
messages: 241837
nosy: bobcatfish
priority: normal
severity: normal
status: open
title: Update _test_multiprocessing.py to use script helpers
versions: Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24033
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24033] Update _test_multiprocessing.py to use script helpers

2015-04-22 Thread Christie

Christie added the comment:

I am working on this!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24033
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-22 Thread Christie

Christie added the comment:

Created issue24033. If it and issue23981 could be added as dependencies of this 
issue (or I could get permission to add them?) that would be swell!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-16 Thread Christie

Christie added the comment:

So @flipmcf we're sticking with the test.support.script_helper namespace?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23981] Update test_unicodedata.py to use script_helpers

2015-04-16 Thread Christie

New submission from Christie:

As described in Issue9517, many test modules do not make use of the helpers in 
script_helpers.py to invoke the python interpreter in a subprocess. Issue9517 
will be broken down into several smaller issues so we can address smaller 
change sets.

This issue is to update test_unicodedata.py to use script_helpers.py.

--
components: Tests
messages: 241293
nosy: bobcatfish
priority: normal
severity: normal
status: open
title: Update test_unicodedata.py to use script_helpers
type: enhancement
versions: Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23981
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23981] Update test_unicodedata.py to use script_helpers

2015-04-16 Thread Christie

Christie added the comment:

I am working on this now.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23981
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-16 Thread Christie

Christie added the comment:

@r.david.murray I've created issue23981, I don't think I have permissions to 
add it as a dependency to this issue.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23981] Update test_unicodedata.py to use script_helpers

2015-04-16 Thread Christie

Christie added the comment:

@r.david.murray, just bugging you b/c you were commenting on Issue9517, this is 
ready for a review. It assumes the patch from Issue9517 has been applied.

I'm thinking for the next issue I'll tackle a few test modules at once, since 
this is such a tiny tiny change set (though these tiny change sets would be 
easier to review - @r.david.murray what do you think?).

--
keywords: +patch
Added file: http://bugs.python.org/file39080/issue23981_test_unicode.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23981
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-15 Thread Christie

Christie added the comment:

 I would recommend still moving script_helper into test.support , and adding a 
 list of symbols from script_helper.py into __init__

Sounds like a great plan to me @flipmcf!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-15 Thread Christie

Christie added the comment:

@r.david.murray and @flipmcf:

I've audited all test modules using sys.executable for a starting point, and 
these tests look like they could benefit from helper methods to invoke python 
(probably the ones in script_helper, but we may want to create others if the 
use cases don’t quite fit):
* _test_multiprocessing.py
* test_base64.py
* test_capi.py - some of this looks legitimate, but some of it could maybe 
benefit from the helpers
* test_cmd_line.py
* test_faulthandler.py
* test_file_eintr.py
* test_gc.py
* test_keyword.py
* test_pdb.py
* test_popen.py
* test_quopri.py
* test_site.py
* test_source_encoding.py
* test_sys.py
* test_sysconfig.py
* test_tcl.py - something complex is going on in here, not sure if it’s 
necessary or not
* test_threading.py
* test_pindent.py
* test_traceback.py
* test_unicodedata.py
* test_json/test_tool.py

@r.david.murray I'm going to start with test_unicodedata.py for a nice easy 
starting point so I'll create a separate issue for it.

@flipmcf should I start using your patch set as-is or is there more to come?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-14 Thread Christie

Christie added the comment:

Hey there @flipmcf, is the change which adds `script_helpers` to test.support 
is missing from your patch?

 import test.support.script_helper
Traceback (most recent call last):
  File stdin, line 1, in module
ImportError: No module named 'test.support.script_helper'

I'm thinking also, maybe instead of putting script_helper into the test.support 
namespace, we could move the individual methods and helpers from script_helper 
into test.support, e.g.:

test.support.run_python_until_end
vs.
test.support.script_helper.run_python_until_end

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-13 Thread Christie

Christie added the comment:

Okee dokee, I'll tackle that first!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-13 Thread Christie

Christie added the comment:

Okay thanks @r.david.murray, I'll take a look at doing this. I'll open an issue 
when I figure out which module to start on.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue9517] Make test.script_helper more comprehensive, and use it in the test suite

2015-04-11 Thread Christie

Christie added the comment:

Hello @r.david.murray!

 We probably don't want a multi-module changeset, though, so this could become 
 a meta issue for new issues for converting particular test files to use 
 script_helpers.

Does this mean you'd like to leave this issue open until all of the test files 
are updated in separate issues?

Or is it that you'd like separate changesets for each module, but all 
associated with this issue?

Thanks!

--
nosy: +bobcatfish

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9517
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18290] json encoder does not support JSONP/JavaScript safe escaping

2014-12-02 Thread Tom Christie

Tom Christie added the comment:

I believe the status of this should be reassessed and that python should 
default to escaping '\u2028' and '\u2029'. *Strictly* speaking this isn't a bug 
and is per the JSON spec.

*However* this *is* a bug in the JSON spec - which *should* be a strict subset 
of JSON. Given that both escaped and unescaped are valid, ensuring that those 
two characters *are* always escaped would clearly be more user-friendly 
behavior on our part, and *would* lead to less bugs in, say web frameworks that 
use the JSON module and then pass the output to template (eg populating a 
javscript variable with some JSON output).

--
nosy: +tomchristie

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18290
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18290] json encoder does not support JSONP/JavaScript safe escaping

2014-12-02 Thread Tom Christie

Tom Christie added the comment:

 There is explicit note in the documentation about incompatibility with 
 JavaScript.

That may be, but we're still unnecessarily making for a poorer user experience. 
There's no good reason why we shouldn't just treat \u2028 and \u2029 as control 
characters - it's only going to making things better for developers using the 
json module. It is an unnecessary usability bug as it stands.

Just because JSON has a bug in its spec wrt those two characters, doesn't mean 
we can't help our users avoid ever having to know about that or work around it 
in user code.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18290
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22767] `separators` argument to json.dumps() behaves unexpectedly across 2.x vs 3.x

2014-10-30 Thread Tom Christie

New submission from Tom Christie:

This is one of those behavioural issues that is a borderline bug.

The seperators argument to `json.dumps()` behaves differently across python 2 
and 3.

* In python 2 it should be provided as a bytestring, and can cause a 
UnicodeDecodeError otherwise.
* In python 3 it should be provided as unicode,and can cause a TypeError 
otherwise.

Examples:

Python 2.7.2
 print json.dumps({'snowman': '☃'}, separators=(':', ','), 
ensure_ascii=False)
{snowman,☃}
 print json.dumps({'snowman': '☃'}, separators=(u':', u','), 
ensure_ascii=False)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: 
ordinal not in range(128)

And:

Python 3.4.0
 print(json.dumps({'snowman': '☃'}, separators=(':', ','), 
ensure_ascii=False))
{snowman,☃}
 print(json.dumps({'snowman': '☃'}, separators=(b':', b','), 
ensure_ascii=False))
...
TypeError: sequence item 2: expected str instance, bytes found

Technically this isn't out of line with the documentation - in both cases it 
uses `separators=(':', ',')` which is indeed the correct type in both v2 and 
v3. However it's unexpected behaviour that it changes types between versions, 
without being called out.

Working on a codebase with `from __future__ import unicode_literals` this is 
particularly unexpected because we get a `UnicodeDecodeError` when running code 
that otherwise looks correct.

It's also slightly awkward to fix because it's a bit of a weird branch 
condition.

The fix would probably be to forcibly coerce it to the correct type regardless 
of if it is supplied as unicode or a bytestring, or at least to do so for 
python 2.7.

Possibly related to http://bugs.python.org/issue22701 but wasn't able to 
understand if that ticket was in fact a different user error.

--
messages: 230274
nosy: Tom.Christie
priority: normal
severity: normal
status: open
title: `separators` argument to json.dumps() behaves unexpectedly across 2.x vs 
3.x
type: behavior
versions: Python 2.7, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22767
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22767] `separators` argument to json.dumps() behaves unexpectedly across 2.x vs 3.x

2014-10-30 Thread Tom Christie

Tom Christie added the comment:

Not too fussed if this is addressed or not, but I think this is closed a little 
prematurely.

I don't think there's a problem under Python 3, that's entirely reasonable.

However under Python 2, `json.dumps()` will normally handle *either* bytes or 
unicode transparently for you (just altering the return type accordingly).

If you happen to be using unicode separators, then the normally lax behaviour 
of either unicode or bytes that stops being the case.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22767
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22767] `separators` argument to json.dumps() behaves unexpectedly across 2.x vs 3.x

2014-10-30 Thread Tom Christie

Tom Christie added the comment:

 But only if you use non-ascii in the binary input, in which case you get an 
 encoding error, which is a correct error.

Kind of, except that this (python 2.7) works just fine:

 data = {'snowman': '☃'}
 json.dumps(data, ensure_ascii=False)
'{snowman: \xe2\x98\x83}'

Whereas this raises an exception:

 json.dumps(data, separators=(u':', u','), ensure_ascii=False)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: 
ordinal not in range(128)

If it was the same in both cases then I wouldn't consider it a problem.
As it is, introducing the `seperators` parameter changes the behaviour.

Anyways, I'll get off my high horse now. :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22767
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22767] `separators` argument to json.dumps() behaves unexpectedly across 2.x vs 3.x

2014-10-30 Thread Tom Christie

Tom Christie added the comment:

 So, as soon as (but only as soon as) you mix unicode with your non-ascii 
 data, your program blows up.

Indeed. For context tho my example of running into this the unicode literals 
used as seperators weren't even in the same package as the non-ASCII binary 
strings. (JSONRenderer in Django REST framework, being excersized by some third 
party test code. End result non-obvious exception.

Anyways, okay with this resolution, although I am now using a compat branch to 
ensure that we use binary seperators in py2 to continue to get the more lax 
rendering style.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22767
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14042] json.dumps() documentation is slightly incorrect.

2012-02-17 Thread Tom Christie

New submission from Tom Christie t...@tomchristie.com:

json.dumps() documentation is slightly incorrect.

http://docs.python.org/library/json.html#json.dumps

Reads:

  If ensure_ascii is False, then the return value will be a unicode instance.

Should read:

  If ensure_ascii is False, then the return value MAY BE be a unicode 
instance.

(Without the caps of course)

bash: python
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type help, copyright, credits or license for more information.
 import json
 type(json.dumps({'a': 1}, ensure_ascii=False))
type 'str'

Tested against 2.6 and 2.7.

--
components: Library (Lib)
messages: 153558
nosy: tomchristie
priority: normal
severity: normal
status: open
title: json.dumps() documentation is slightly incorrect.
type: behavior
versions: Python 2.6, Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14042
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com