[issue32606] Email Header Injection Protection Bypass

[Dalton Campbell]

Change by Dalton Campbell :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32606] Email Header Injection Protection Bypass

[Dalton Campbell]

New submission from Dalton Campbell :

The protection's implemented in 
https://github.com/python/cpython/blob/master/Lib/email/header.py to prevent 
Email Header injection can be bypassed by specifying an injected additional 
header in the following format:
exam...@python.org\ncc :injec...@python.org

The white space bypasses the current regex protection (_embedded_header = 
re.compile(r'\n[^ \t]+:')) and is still accepted by the smtp server. 

Attached is a proof of concept script

--
components: Library (Lib)
files: PythonEmailHeaderInjectionPOC.py
messages: 310367
nosy: r.david.murray, thedoctorsoup
priority: normal
severity: normal
status: open
title: Email Header Injection Protection Bypass
type: security
Added file: https://bugs.python.org/file47396/PythonEmailHeaderInjectionPOC.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32232] building extensions as builtins is broken in 3.7

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

It seems that Microsoft compiler does not handle well the circular dependencies 
between Include/pystate.h and Include/internal/pystate.h so I have included 
guards for Windows in the new include for Include/pystate.h and deactivated the 
test in that case. Some better ideas on how to handle that case?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32232] building extensions as builtins is broken in 3.7

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I have put together a PR with Matthias' patch and a test.

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32232] building extensions as builtins is broken in 3.7

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +5103
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32603] Deprecation warning on strings used in re module

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +5102
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Can't get python running

[Chris Angelico]

On Sun, Jan 21, 2018 at 12:46 PM, Steven D'Aprano
 wrote:
> On Sun, 21 Jan 2018 12:21:40 +1100, Chris Angelico wrote:
>
>> Jim, let the installer put it where it wants to, and make sure you've
>> added it to PATH. Then you should be able to type "py" to start Python.
>
> Shouldn't the installer ensure that it puts "py" somewhere on the path?
>

Only if you tick the box to do that (or at least, that was the case a
few versions ago). You don't have to do it manually, but you do have
to make sure the box is ticked. (Which might just mean making sure you
don't UNtick it; I don't know what the default is.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is this the right way to write a codec error handler?

[Steven D'Aprano]

On Sat, 20 Jan 2018 12:57:45 +0200, Serhiy Storchaka wrote:

> Just `end` instead of `end+1`.

Oops!

> And it is safer to use `bytes.decode(obj[start:end], 'latin1')` or
> `str(obj[start:end], 'latin1')` instead of
> `obj[start:end].decode('latin1')`. Just for the case if obj has
> overridden decode() method.
> 
> Otherwise LGTM.

Thanks.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can't get python running

[Chris Angelico]

On Sun, Jan 21, 2018 at 12:40 PM, bartc  wrote:
> On 21/01/2018 01:21, Chris Angelico wrote:
>>
>> On Sun, Jan 21, 2018 at 12:08 PM, bartc  wrote:
>>>
>>> On 20/01/2018 17:16, Jim Sadler wrote:


 I downloaded python 3.6.4 and although everything about the installation
 seems correct (path, file size, checking on cmd to see if file installed
 correctly-it is)
>>>
>>>
>>>
>>> What do you mean by 'checking on cmd'?
>>>
>>> I would install it somewhere like c:\python364, then it is easier to find
>>> than having it buried in its default installation path (I assume this is
>>> Windows).
>>
>>
>> python36 is a better choice of name
>
>
> That's what I use; I assumed everyone else used the 3-digit version in the
> path.

Terrible assumption, given that it's not the default NOR a good idea :)

> , as it doesn't cause problems when
>>
>> you upgrade to a new bugfix release. But there are permissions issues
>> with dropping stuff straight into the root directory, which is why the
>> default installers now put Python into Program Files.
>
>
> It's not in the root, the executable will be:
>
>   c:\python36\python.exe
>
> Not c:\python.exe. Yes, Windows allows you to have your own directories
> within /. It's not Unix.

I meant putting the pythonXY directory straight into the root. Yes,
Windows allows it... but only if you are administrator. I think.
Depends on the Windows version. And just FYI, Unix allows you to have
your own directories within / too, so I don't know what your point is.
Both OSes - recent versions, at least - restrict the creation of
directories and files straight in the root.

>> Jim, let the installer put it where it wants to, and make sure you've
>> added it to PATH. Then you should be able to type "py" to start
>> Python.
>
>
> If I try to install 3.7 (as I already have 3.6) it suggests putting it in:
>
>  c:\Users\users\AppData\Local\Programs\Python\Python37-32
>
> Not actually a snappy path if you need to get there in a hurry to sort out
> problems. That is, in a location 7 levels deep.
>
> Get it working anywhere first to find out what the problem is.
>

Get it working in the default location before you change things. The
%PATH% environment variable exists to save you from typing seven
levels of directory names.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can't get python running

[Steven D'Aprano]

On Sun, 21 Jan 2018 12:21:40 +1100, Chris Angelico wrote:

> Jim, let the installer put it where it wants to, and make sure you've
> added it to PATH. Then you should be able to type "py" to start Python.

Shouldn't the installer ensure that it puts "py" somewhere on the path?



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can't get python running

[bartc]

On 21/01/2018 01:21, Chris Angelico wrote:

On Sun, Jan 21, 2018 at 12:08 PM, bartc  wrote:

On 20/01/2018 17:16, Jim Sadler wrote:


I downloaded python 3.6.4 and although everything about the installation
seems correct (path, file size, checking on cmd to see if file installed
correctly-it is)



What do you mean by 'checking on cmd'?

I would install it somewhere like c:\python364, then it is easier to find
than having it buried in its default installation path (I assume this is
Windows).


python36 is a better choice of name


That's what I use; I assumed everyone else used the 3-digit version in 
the path.


, as it doesn't cause problems when

you upgrade to a new bugfix release. But there are permissions issues
with dropping stuff straight into the root directory, which is why the
default installers now put Python into Program Files.


It's not in the root, the executable will be:

  c:\python36\python.exe

Not c:\python.exe. Yes, Windows allows you to have your own directories 
within /. It's not Unix.




Jim, let the installer put it where it wants to, and make sure you've
added it to PATH. Then you should be able to type "py" to start
Python.


If I try to install 3.7 (as I already have 3.6) it suggests putting it in:

 c:\Users\users\AppData\Local\Programs\Python\Python37-32

Not actually a snappy path if you need to get there in a hurry to sort 
out problems. That is, in a location 7 levels deep.


Get it working anywhere first to find out what the problem is.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

[issue32574] asyncio.Queue, put() leaks memory if the queue is full

[José Melero Fernández]

Change by José Melero Fernández :


--
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Can't get python running

[Chris Angelico]

On Sun, Jan 21, 2018 at 12:08 PM, bartc  wrote:
> On 20/01/2018 17:16, Jim Sadler wrote:
>>
>> I downloaded python 3.6.4 and although everything about the installation
>> seems correct (path, file size, checking on cmd to see if file installed
>> correctly-it is)
>
>
> What do you mean by 'checking on cmd'?
>
> I would install it somewhere like c:\python364, then it is easier to find
> than having it buried in its default installation path (I assume this is
> Windows).

python36 is a better choice of name, as it doesn't cause problems when
you upgrade to a new bugfix release. But there are permissions issues
with dropping stuff straight into the root directory, which is why the
default installers now put Python into Program Files.

Jim, let the installer put it where it wants to, and make sure you've
added it to PATH. Then you should be able to type "py" to start
Python.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can't get python running

[bartc]

On 20/01/2018 17:16, Jim Sadler wrote:

I downloaded python 3.6.4 and although everything about the installation
seems correct (path, file size, checking on cmd to see if file installed
correctly-it is)


What do you mean by 'checking on cmd'?

I would install it somewhere like c:\python364, then it is easier to 
find than having it buried in its default installation path (I assume 
this is Windows).


If it's installed properly, then python.exe should be present in 
c:\python364, and you can start it with:


  c:\python364\python

or:

  c:
  cd \python364
  python

I assume you can open a command prompt. If not in python364, then 
substitute the actual installation path.


Or click Start, and type python.exe into the search box. If it finds it, 
click it.



--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to use asyncore with SSL?

[Grant Edwards]

On 2018-01-20, Marko Rauhamaa  wrote:
> Grant Edwards :
[...]
>> I won't argue with that.  I think that non-blocking ssl-wrapped
>> sockets _should_ have the same select/poll/send/recv API/semantics
>> that normal sockets do.  I thought about writing my own
>> wrapped-ssl-socket class that does that, but using stunnel was just so
>> much easier.  If you _did_ want to wrap sockets like that, I think
>> you'd need to actually run a thread to deal with the SSL socket and
>> provide a "proxy" socket or pipe for use with select/poll.
>>
>> Basically you'd be doing what stunnel does only doing it in-process.
>
> Stunnel is fine for many applications but not for our needs. Also, a
> subsidiary thread is not necessary. Everything can be done within an
> async framework (in C anyway).

Of course it can.  But (ISTM) either the async framework has to be
SSL-aware, or the sockets have to be wrapped and "proxied" using
another socket or pipe -- I don't see how else you can provide
something that has the same select/poll and send/recv semantacs that
an unwrapped, non-blocking socket provides.

-- 
Grant


-- 
https://mail.python.org/mailman/listinfo/python-list

[issue32591] Deprecate sys.set_coroutine_wrapper and replace it with more focused API(s)

[Guido van Rossum]

Guido van Rossum  added the comment:

Yes, I'm OK with this plan. Note that I've only read Nathaniel's original
message and Yury's last message, and I've not reviewed the PR, but getting
rid of that eyesore sounds good, and it seems you two have reached
agreement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32502] uuid1() fails if only 64-bit interface addresses are available

[bbayles]

Change by bbayles :


--
keywords: +patch
pull_requests: +5101
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32378] test_npn_protocols broken with LibreSSL 2.6.1+

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +5100

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27682] wsgiref: Windows Error 10053, ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine

[William Hingston]

William Hingston  added the comment:

I'm still seeing this with

Python 3.6.3
Django 2.0.1
Windows 10 Pro Version 1709

Was this determined to be a Django bug?

--
nosy: +William Hingston

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32424] Synchronize copy methods between Python and C implementations of xml.etree.ElementTree.Element

[Gordon P. Hemsley]

Gordon P. Hemsley  added the comment:

>> As discussed above, starting with msg309074, __deepcopy__() is being added 
>> to the Python implementation because it already exists in the C 
>> implementation.
> 
> Python implementation shouldn't copy all implementation details of the C 
> implementation. This is cumbersome and often impossible. Both implementation 
> should have the same behavior, and they do have.

As it stands, calling Element.__deepcopy__() will succeed with the C
implementation and fail with the Python implementation. That seems
problematic to me, and making that *not* be the case is trivial, so I
don't understand the generalization to "all implementation details".

Additionally, if copy.deepcopy() already works well with the Python
implementation, why would it not work on the C implementation? What is
different about the C implementation that requires the definition of
__deepcopy__() in the first place?

>> And additional tests have in fact uncovered further discrepancies between 
>> the Python and C implementations with regard to copying behavior.
> 
> What are these discrepancies?

As I mentioned in the PR, the C implementation does not make use of
Element.__init__() when it creates new instances of the class, opting
instead to use create_new_element() (which is not used in the C
implementation of Element.__init__()). However, create_new_element()
does not make a copy of the attrib dict that is passed in, meaning that
the internal attrib dict of an Element instance is mutable by changing
the dict that was passed in.

I have modified the C implementation to fix this problem by adding a
copy line in create_new_element(). Without this change, some of the new
tests in the PR will fail in the C implementation.

The Python implementation does not have this problem because it
inherently always goes through Element.__init__().

> In any case it is better to extend existing tests by adding missed checks 
> than duplicate tests.

As I mentioned in the PR:

I have added the unit tests in the way that I have because the existing
checks are generally integration tests, and most of the test coverage
for this module comes as a side effect of pickle tests. It is my goal to
expand the unit tests later to test each method individually, so
starting that work here makes it easier.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Can't get python running

[Igor Korot]

Hi, Jim,

On Sat, Jan 20, 2018 at 11:16 AM, Jim Sadler  wrote:
> I downloaded python 3.6.4 and although everything about the installation
> seems correct (path, file size, checking on cmd to see if file installed
> correctly-it is) the only window that will pop up when I attempt to run it
> is the set-up window offering to modify, restore, or uninstall.  I have
> uninstalled and re-installed the program several times, run the repair
> function many times and nothing changes.  Would someone please help me or
> direct me to someone who can?

How exactly are attempting to run python?

Thank you.

>  Thank you.
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Can't get python running

[Terry Reedy]

On 1/20/2018 12:16 PM, Jim Sadler wrote:

I downloaded python 3.6.4 and although everything about the installation
seems correct (path, file size, checking on cmd to see if file installed
correctly-it is) the only window that will pop up when I attempt to run it
is the set-up window offering to modify, restore, or uninstall.


The only time that should pop up is when you rerun the installer instead 
of the installed binary.  (Or when you invoke it from the installed 
programs control panel.)  *Exactly* how are you trying to run the 
installed Python.  What version of Windows?


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Can't get python running

[Jim Sadler]

I downloaded python 3.6.4 and although everything about the installation
seems correct (path, file size, checking on cmd to see if file installed
correctly-it is) the only window that will pop up when I attempt to run it
is the set-up window offering to modify, restore, or uninstall.  I have
uninstalled and re-installed the program several times, run the repair
function many times and nothing changes.  Would someone please help me or
direct me to someone who can?  Thank you.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue32603] Deprecation warning on strings used in re module

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

+1 to this. I have seen a lot of regular expressions not using raw strings that 
makes use of "\d" and friends. I will work on a patch to the docs if that's ok.

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31687] test_semaphore_tracker() of test_multiprocessing_spawn fails randomly (race condition?)

[STINNER Victor]

STINNER Victor  added the comment:

To reproduce the issue, someone should increase the system load to make the 
system slower and make timeouts more likely, and change the order in which 
processes are run. I'm using the "stress" utility on Linux for that. But 
sometimes, the best stress tool... is the Python test suite, since running the 
full test suite in multiple processes in parallel (./python -m test -j0 -r) 
runs various workloads, and so sometimes trigger bugs which would be very hard 
to trigger othewise.

Terminal 1: run "./python -m test -j0 -r", stress or anything else to make the 
system slower

Terminal 2: run "./python -m test test_multiprocessing_fork -m 
test_semaphore_tracker -F -v" which runs the test in a loop until it fails.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How to use asyncore with SSL?

[Marko Rauhamaa]

Grant Edwards :

> On 2018-01-20, Marko Rauhamaa  wrote:
>> OpenSSL isn't the easiest beast to deal with, but I have been able to
>> abstract it (in C) so it behaves very close to TCP. The one blemish
>> is in the fact that the TLS protocol does not support a half-duplex
>> connection. Shame.
>>
>> The WANT_READ/WANT_WRITE silliness should be abstracted out of the
>> non-blocking TLS library so the application doesn't need to know
>> anything about it.
>
> I won't argue with that.  I think that non-blocking ssl-wrapped
> sockets _should_ have the same select/poll/send/recv API/semantics
> that normal sockets do.  I thought about writing my own
> wrapped-ssl-socket class that does that, but using stunnel was just so
> much easier.  If you _did_ want to wrap sockets like that, I think
> you'd need to actually run a thread to deal with the SSL socket and
> provide a "proxy" socket or pipe for use with select/poll.
>
> Basically you'd be doing what stunnel does only doing it in-process.

Stunnel is fine for many applications but not for our needs. Also, a
subsidiary thread is not necessary. Everything can be done within an
async framework (in C anyway).


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue32583] Crash during decoding using UTF-16/32 and custom error handler

[Xiang Zhang]

Xiang Zhang  added the comment:

Another way to crash:

>>> import codecs
>>> def replace_with_longer(exc):
... exc.object = b'\xa0\x00' * 100
... return ('\ufffd', exc.end)
...
>>> codecs.register
codecs.register(   codecs.register_error(
>>> codecs.register_error('replace_with_longer', rep
replace_with_longer( repr(
>>> codecs.register_error('replace_with_longer', replace_with_longer)
>>> b'\xd8\xd8'.decode('utf-16-le', 'replace_with_longer')
Debug memory block at address p=0x10b3b8c40: API 'o'
92 bytes originally requested
The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
The 8 pad bytes at tail=0x10b3b8c9c are not all FORBIDDENBYTE (0xfb):
at tail+0: 0xa0 *** OUCH
at tail+1: 0x00 *** OUCH
at tail+2: 0xa0 *** OUCH
at tail+3: 0x00 *** OUCH
at tail+4: 0xa0 *** OUCH
at tail+5: 0x00 *** OUCH
at tail+6: 0xa0 *** OUCH
at tail+7: 0x00 *** OUCH
The block was made by call #11529390970613309440 to debug malloc/realloc.
Data at p: 00 00 00 00 00 00 00 00 ... 00 00 00 00 fd ff a0 00

Fatal Python error: bad trailing pad byte

Current thread 0x7fffab9b4340 (most recent call first):
  File "/Users/angwer/Repositories/cpython/Lib/encodings/utf_16_le.py", line 16 
in decode
  File "", line 1 in 
[1]64081 abort  ~/Repositories/cpython/python.exe

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32597] Bad detection of clang

[Stefan Krah]

Stefan Krah  added the comment:

> https://bugs.python.org/issue20767 was showing the issue and imho badly fixed.

You do realize that the fix was proposed by FreeBSD people and that the 
upstream Clang issue is still open?

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32583] Crash during decoding using UTF-16/32 and custom error handler

[Xiang Zhang]

Change by Xiang Zhang :


--
stage: patch review -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32583] Crash during decoding using UTF-16/32 and custom error handler

[Xiang Zhang]

Xiang Zhang  added the comment:

The problem is utf16 decoder almost always assumes that two bytes decodes to 
one unicode character, so when allocating memory, it assumes (bytes_number+1)/2 
unicode slots is enough, there is even a comment in the code. And in 
unicode_decode_call_errorhandler_writer, it only allocates more memory when the 
error handler returns a unicode longer than 1, but doesn't take care pace by 
one, in which case one byte to one unicode character. So it's possible for the 
decoder to write out of bound.

This example could steadily crash on my Mac with debug version, it writes 
across the bound of the internal unicode buffer:

>>> import codecs
>>> def pace_by_one(exc):
... return ('\ufffd', exc.start+1)
...
>>> codecs.register_error('pace_by_one', pace_by_one)
>>> b'\xd8\xd8\xd8\xd8\xd8\xd8\x00\x00\x00'.decode('utf-16-le', 'pace_by_one')
Debug memory block at address p=0x10210c260: API 'o'
100 bytes originally requested
The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
The 8 pad bytes at tail=0x10210c2c4 are not all FORBIDDENBYTE (0xfb):
at tail+0: 0x00 *** OUCH
at tail+1: 0x00 *** OUCH
at tail+2: 0xfb
at tail+3: 0xfb
at tail+4: 0xfb
at tail+5: 0xfb
at tail+6: 0xfb
at tail+7: 0xfb
The block was made by call #30672 to debug malloc/realloc.
Data at p: 00 00 00 00 00 00 00 00 ... fd ff fd ff fd ff d8 00

Fatal Python error: bad trailing pad byte

Current thread 0x7fffab9b4340 (most recent call first):
  File "/Users/angwer/Repositories/cpython/Lib/encodings/utf_16_le.py", line 16 
in decode
  File "", line 1 in 
[1]63997 abort  ~/Repositories/cpython/python.exe

I'll try to make a fix tomorrow.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29833] Avoid raising OverflowError if possible

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
dependencies: +chr raises OverflowError

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24334] SSLSocket extra level of indirection

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +5099
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How to use asyncore with SSL?

[Grant Edwards]

On 2018-01-20, Marko Rauhamaa  wrote:
> Grant Edwards :
>
>> Asyncore seems to be based on fundamental assumptions that aren't true
>> for non-blocking ssl sockets.
>
> Pot calling kettle black.
>
> OpenSSL isn't the easiest beast to deal with, but I have been able to
> abstract it (in C) so it behaves very close to TCP. The one blemish is
> in the fact that the TLS protocol does not support a half-duplex
> connection. Shame.
>
> The WANT_READ/WANT_WRITE silliness should be abstracted out of the
> non-blocking TLS library so the application doesn't need to know
> anything about it.

I won't argue with that.  I think that non-blocking ssl-wrapped
sockets _should_ have the same select/poll/send/recv API/semantics
that normal sockets do.  I thought about writing my own
wrapped-ssl-socket class that does that, but using stunnel was just so
much easier.  If you _did_ want to wrap sockets like that, I think
you'd need to actually run a thread to deal with the SSL socket and
provide a "proxy" socket or pipe for use with select/poll.

Basically you'd be doing what stunnel does only doing it in-process.

--
Grant



-- 
https://mail.python.org/mailman/listinfo/python-list

[issue32472] Mention of __await__ missing in Coroutine Abstract Methods

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution: not a bug -> 
stage: resolved -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32589] Statistics as a result from timeit

[Terry J. Reedy]

Terry J. Reedy  added the comment:

If we agree that the patch is wrong, we should reject it and close this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32476] Add concat functionality to ElementTree xpath find

[John Jolly]

Change by John Jolly :


--
pull_requests: +5098
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: How to use asyncore with SSL?

[Marko Rauhamaa]

Grant Edwards :

> Asyncore seems to be based on fundamental assumptions that aren't true
> for non-blocking ssl sockets.

Pot calling kettle black.

OpenSSL isn't the easiest beast to deal with, but I have been able to
abstract it (in C) so it behaves very close to TCP. The one blemish is
in the fact that the TLS protocol does not support a half-duplex
connection. Shame.

The WANT_READ/WANT_WRITE silliness should be abstracted out of the
non-blocking TLS library so the application doesn't need to know
anything about it.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Speeding up the implementation of Stochastic Gradient Ascent in Python

[Jugurtha Hadjar]

Aside from the obvious imports I've added (import numpy as np, etc), 
there are still undefined objects (`names`, `times_`, `feat`, 
`bind_ind`) and indentation errors. Can you post a *working* code to be 
sped up and a benchmarking test?


As you're using Python 2, you can improve the code with the following:

 - Use `xrange` in place of `range`.
 - Printing in loops dramatically slows things down.
 - Loop directly over an iterable (and maybe use 
`collections.defaultdict`):


    # This:

    def pos_per_user_(self):
    pos_per_user = {}
    for k, v in self.userItems.items():
    for i in range(len(self.userItems[k])):
    item__ = self.userItems[k][i][0]
    if k not in pos_per_user:
    pos_per_user[k] = [item__]
    else:
    pos_per_user[k].append(item__)

    return pos_per_user


    # Becomes like this:

    from collections import defaultdict

    def pos_per_user(self):
    pos_per_user = defaultdict(list)
    for k, v in self.userItems.iteritems():
    for item in items:
    pos_per_user[k].append(item[0])
    return pos_per_user


  - You also have too many list creations inside loops, which is also slow:

    # If you do nothing, use xrange here for it will not construct
    # a list. Avoid materializing a list if all you do is iterate.

    for bin in range(10):
    for i in range(len(self.list_of_items)):
    for k in range(self.K2):
    for f in range(4096):




--
~ Jugurtha Hadjar,

--
https://mail.python.org/mailman/listinfo/python-list

[issue32591] Deprecate sys.set_coroutine_wrapper and replace it with more focused API(s)

[Yury Selivanov]

Yury Selivanov  added the comment:

Guido,

I'd like to go forward with this and merge Nathaniel's PR.

Quick summary:

* This issue adds new APIs: a sys.set_coroutine_origin_tracking_depth(depth) 
function and a 'cr_origin' property to native coroutine objects.

* By using this APIs, users can opt-in to save traceback (serialized as tuples 
of `(funcname, filename, line)`) of where native coroutines are created.

* This allows us to deprecate the sys.set_coroutine_wrapper function, which I 
never liked.  It's too powerful as it allows anyone to intercept/override 
native coroutine construction logic.

* In turn, we are removing a lot of complexity from asyncio code: starting with 
3.7 we'll use CoroWrapper only for generator-based corotuines; at some point 
(Python 3.9?) when we remove @asyncio.coroutine we'll be able to remove 
CoroWrapper.  This will also make asyncio debug mode quite a bit faster, 
allowing people to use it in production.

* Finally, sys.set_coroutine_wrapper was always documented as a debug-only API 
that might disappear in future CPython releases.  We'll deprecate it in 3.7 and 
remove it in 3.8.

Are you OK with this?

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Why is there no functional xml?

[Rustom Mody]

Looking around for how to create (l)xml one sees typical tutorials like this:

https://www.blog.pythonlibrary.org/2013/04/30/python-101-intro-to-xml-parsing-with-elementtree/



Given the requirement to build up this xml:


1181251680
04008200E000
1181572063


1800
Bring pizza home





the way I would rather do it is thus:

[Note in actual practice the 'contents' such as 1181251680 etc would come
from suitable program variables/function-calls
]


ex = Ea("zAppointments",  {'reminder':'15'},
E("appointment",
  En("begin", 1181251680),
  Et("uid", "04008200E000"),
  En("alarmTime", 1181572063),
  E("state"),
  E("location"),
  En("duration",1800),
  Et("subject", "Bring pizza home")))


with the following obvious definitions:

[The function names are short so that the above becomes correspondingly 
readable]


from lxml.etree import Element

def Ea(tag, attrib=None, *subnodes):
"xml node constructor"
root = Element(tag, attrib)
for n in subnodes:
root.append(n)
return root

def E(tag, *subnodes):
"Like E but without attributes"
root = Element(tag)
for n in subnodes:
root.append(n)
return root

def Et(tag, text):
"A pure text node"
root = E(tag)
root.text = text
return root

def En(tag, text):
"A node containing a integer"
root = E(tag)
root.text = str(text)
return root


This approach seems so obvious that I find it hard to believe its not there 
somewhere…
Am I missing something??
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to use asyncore with SSL?

[Grant Edwards]

On 2018-01-18, Grant Edwards  wrote:

[regarding secure-smtpd -- a module based on smtpd and asyncore]

> That makes the SSL support pretty much useless.
>
> I'm trying to fix that, but I can't find any information or
> documentation about using asyncore with SSL.

Asyncore seems to be based on fundamental assumptions that aren't true
for non-blocking ssl sockets.  After looking into it for a couple
hours, the way you use ssl with asyncore is like this:

pid = subprocess.Pipe(["stunnel","stunnel.conf"]).pid
# insert asyncore app here
os.kill(pid,signal.SIGTERM)

-- 
Grant

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue32550] STORE_ANNOTATION bytecode is unnecessary and can be removed.

[Guido van Rossum]

Guido van Rossum  added the comment:

There is very little code that depends on __annotations__ at all, and none
of it is very subtle. All actual type checking is done by separate tools
(mypy, pytype, PyCharm) that don't import the code and don't care about
__annotations__. The __annotations__ variable exists so that other people
*may* do useful stuff with it, e.g. design run-time type-checking
decorators. So please don't worry about interning.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29302] add contextlib.AsyncExitStack

[Nick Coghlan]

Nick Coghlan  added the comment:

Explicitly noting some API design decisions from the review of 
https://github.com/python/cpython/pull/4790:

1. AsyncExitStack will define __aenter__ and __aexit__ but *not* __enter__ and 
__exit. This makes it unambiguous which form of with statement you need to be 
using for a given stack.
2. AsyncExitStack will use the same methods to add synchronous context managers 
as ExitStack does
3. The only common method with clearly distinct external behaviour will be 
stack.close(), which will be a coroutine in the AsyncExitStack case (the other 
common methods will all still be synchronous in AsyncExitStack)
4. AsyncExitStack will gain 3 new methods for working with async context 
managers:

- enter_async_context(cm): coroutine to enter an async context manager and adds 
its __aexit__ to the context stack if __aenter__ succeeds
- push_async_exit(cm_or_exit): push an exit callback directly onto the stack. 
The argument must either be an object implementing __aexit__, or else a 
callback with the same signature as __aexit__ (use stack.push() to register a 
synchronous CM or exit callback)
- push_async_callback(callback): push the given callback onto the stack. The 
callback should return an awaitable (use stack.callback() to register 
synchronous callbacks)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32594] File object 'name' attribute inconsistent type and not obviously documented

[R. David Murray]

R. David Murray  added the comment:

Ah, yes.  When I said "this has been discussed previously", that's the issue I 
was thinking of but couldn't find.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> State clearly that open() 'file' param is "name" attr of the 
result

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31429] TLS cipher suite compile time option for downstream

[Christian Heimes]

Christian Heimes  added the comment:

My PR is ready for review.

--
nosy: +alex, dstufft, janssen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32503] Avoid creating small frames in pickle protocol 4

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32594] File object 'name' attribute inconsistent type and not obviously documented

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Isn't this a duplicate of issue18534?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32503] Avoid creating small frames in pickle protocol 4

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 1211c9a9897a174b7261ca258cabf289815a40d8 by Serhiy Storchaka in 
branch 'master':
bpo-32503: Avoid creating too small frames in pickles. (#5127)
https://github.com/python/cpython/commit/1211c9a9897a174b7261ca258cabf289815a40d8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32430] Simplify Modules/Setup{,.dist,.local}

[Julien Palard]

Julien Palard  added the comment:

I don't see a use case where one want to edit a file without having his work 
being tracked by git, can someone shed a light to me on this?

I understand that to modify a single configuration point (a path, whatever) a 
configure option is the right way and is not versioned by git, but 
Modules/Setup is a file, not a flag.

But, I assumed the whole time Modules/Setup modifications are not trivial, but 
the result of some work worth versioning, unlike a single configure option, am 
I wrong?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32602] Test ECDSA and dual mode context

[Christian Heimes]

Change by Christian Heimes :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32602] Test ECDSA and dual mode context

[Christian Heimes]

Christian Heimes  added the comment:


New changeset bd5c7d238c01b90fbfae8ea45b47bd601900abaf by Christian Heimes in 
branch 'master':
bpo-32602: Test ECDSA certs (#5247)
https://github.com/python/cpython/commit/bd5c7d238c01b90fbfae8ea45b47bd601900abaf


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32583] Crash during decoding using UTF-16/32 and custom error handler

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
components: +Interpreter Core
nosy: +serhiy.storchaka
stage: test needed -> patch review
versions: +Python 3.6 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32430] Simplify Modules/Setup{,.dist,.local}

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 20/01/2018 à 14:22, STINNER Victor a écrit :
> 
> I don't see how adding an option to make one path configurable would make the 
> build process more complicated.

Do you think implementing and maintaing an option has no cost?

> It seems like Xavier and Thomas do use Modules/Setup for their need, and for 
> me it's strange to have to modify Modules/Setup (or Modules/Setup.dist) if 
> it's tracked by Git.

Distributors routinely modify files that are tracked by git.
Modules/Setup doesn't have to be an exception.

See Anaconda example I posted above.  Also you can look up packaging by
RedHat or any other distro.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32430] Simplify Modules/Setup{,.dist,.local}

[STINNER Victor]

STINNER Victor  added the comment:

> Because we don't want to maintain more config options?  Really, I don't to 
> want to further complicate our build process.

I don't see how adding an option to make one path configurable would make the 
build process more complicated.

It seems like Xavier and Thomas do use Modules/Setup for their need, and for me 
it's strange to have to modify Modules/Setup (or Modules/Setup.dist) if it's 
tracked by Git. If I modify it, IMHO it's a configuration file, and it 
shouldn't be part of Python code base. It's fine to have a default 
configuration, but if you don't want to  use the default, just pass the path to 
your own config. No?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25910] Fixing links in documentation

[R. David Murray]

R. David Murray  added the comment:

@guido: in roundup, by default anyone can change the issue title.  I imagine he 
is a new user who just didn't understand that he was doing so.  (And indeed, 
the account was created on 1/19).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32295] User friendly message when invoking bdist_wheel sans wheel package.

[TestUserEWDurbin]

Change by TestUserEWDurbin :


--
status: pending -> open
title: User friendly message when invoking bdist_wheel sans wheel package -> 
User friendly message when invoking bdist_wheel sans wheel package.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32430] Simplify Modules/Setup{,.dist,.local}

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> Why not adding an option in Makefile or configure to allow to specify a 
> different path to a custom Setup file?

Because we don't want to maintain more config options?  Really, I don't to want 
to further complicate our build process.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30693] tarfile add uses random order

[STINNER Victor]

STINNER Victor  added the comment:

I now agree to leave Python 2.7 and 3.6 unchanged.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32589] Statistics as a result from timeit

[STINNER Victor]

STINNER Victor  added the comment:

> For you particular need it is easy to write an explicit loop and gather the 
> statistics that your need.

I concur with Serhiy:

>>> loops=2**15; values = [value/loops for value in t.repeat(10, loops)]
>>> import statistics
>>> statistics.mean(values)
3.439414190675727e-06
>>> statistics.stdev(values)
1.444941323254124e-07


Matthias Gilch: "I think it's inconvenient to write those stubs every time when 
we have a method for this in the standard library that could be easily used 
with a few changes in it."

Hum, maybe a module on PyPI which uses timeit.Timer internally would be more 
appropriate, than extending the timeit module.

(Or use my existing perf module, it's up to you ;-))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32589] Statistics as a result from timeit

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I concur with Victor. This will defeat the purpose of timeit. And making the 
type of the result depending on arguments usually means a bad design.

For you particular need it is easy to write an explicit loop and gather the 
statistics that your need.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32598] Use autoconf to detect OpenSSL and libssl features

[Christian Heimes]

Christian Heimes  added the comment:

Thanks, Andrew

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32598] Use autoconf to detect OpenSSL and libssl features

[Christian Heimes]

Christian Heimes  added the comment:


New changeset ff5be6e8100276647e0077e80869fc022d1bb53f by Christian Heimes in 
branch 'master':
bpo-32598: Use autoconf to detect usable OpenSSL (#5242)
https://github.com/python/cpython/commit/ff5be6e8100276647e0077e80869fc022d1bb53f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Fourth example from PEP 342

[Léo El Amri]

On 20/01/2018 11:55, Thomas Jollans wrote:
> control is returned to t.resume. nonblocking_accept is supposed to be a
> coroutine
> Ergo, it schedules the nonblocking_accept coroutine (‘value’) to be
> called on the next iteration, and keeps the running listen_on coroutine
> (‘coroutine’) on the stack for safekeeping.

Hello Thomas,

I missed THE point of the examples. nonblocking_accept is a coroutine. I
was stuck in the C way of doing non blocking, and I was wondering what
was the magic behind the examples.

Perfect explanation, now I understand. Still, I also wonder how theses
"nonblocking_" functions are meant to be implemented. But it's another
topic.

Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue30693] tarfile add uses random order

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

If make this change you need to make similar changes in other places that 
recursively add files to archives: shutil, zipapp, distutils, and maybe more.

--
versions:  -Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32226] Implement PEP 560: Core support for typing module and generic types

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:


New changeset d911e40e788fb679723d78b6ea11cabf46caed5a by Ivan Levkivskyi in 
branch 'master':
bpo-32226: PEP 560: improve typing module (#4906)
https://github.com/python/cpython/commit/d911e40e788fb679723d78b6ea11cabf46caed5a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25910] Fixing links in documentation

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
title: How many "critical"-level open issues are there? -> Fixing links in 
documentation

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Fourth example from PEP 342

[Thomas Jollans]

On 19/01/18 22:50, Léo El Amri wrote:
> Hello list,
> 
> I am currently trying to learn co-routine/asynchronous mechanisms in
> Python. I read the PEP 342, but I stumble on the fourth example.
> I don't understand what the lines "data = yield nonblocking_read(sock)"
> in echo_handler() and "connected_socket = yield
> nonblocking_accept(sock)" in listen_on() are trying to do.
> 
> For example, can someone explain me how the returned value in the line
> "connected_socket = yield nonblocking_accept(sock)" can be used on the
> next line ("trampoline.add(handler(connected_socket))") ? To me, it
> looks like the returned value is lost in the Trampoline, when resume()
> gets the returned value of the yield expression.
> 
> Léo
> 

Let's see.

Upon the line

  connected_socket = yield nonblocking_accept(sock)

control is returned to t.resume. nonblocking_accept is supposed to be a
coroutine, so t.resume does this:

  if isinstance(value, types.GeneratorType):
  # Yielded to a specific coroutine, push the
  # current one on the stack, and call the new
  # one with no args
  self.schedule(value, (coroutine,stack))

Ergo, it schedules the nonblocking_accept coroutine (‘value’) to be
called on the next iteration, and keeps the running listen_on coroutine
(‘coroutine’) on the stack for safekeeping.

On a subsequent iteration (jump?) of the trampoline, nonblocking_accept
will yield a socket. Then, resume will

  elif stack:
  # Yielded a result, pop the stack and send the
  # value to the caller
  self.schedule(stack[0], stack[1], value)

plan to return control to listen_on (‘stack[0]’), and (the next time
around) send the value back in.

  if exc:
  value = coroutine.throw(value,*exc)
  else:
  value = coroutine.send(value)

Now, listen on is running again and has the value yielded by
nonblocking_accept.

I wish I knew how exactly nonblocking_accept was supposed to work here,
but that's beside the point as the world of Python coroutines and async
I/O has moved on since 2005.

Hope this helps
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is this the right way to write a codec error handler?

[Serhiy Storchaka]

20.01.18 10:32, Steven D'Aprano пише:

I want an error handler that falls back on Latin-1 for anything which
cannot be decoded. Is this the right way to write it?


def latin1_fallback(exception):
 assert isinstance(exception, UnicodeError)
 start, end = exception.start, exception.end
 obj = exception.object
 if isinstance(exception, UnicodeDecodeError):
 return (obj[start:end].decode('latin1'), end+1)
 elif isinstance(exception, UnicodeEncodeError):
 return (obj[start:end].encode('latin1'), end+1)
 else:
 raise


Just `end` instead of `end+1`.

And it is safer to use `bytes.decode(obj[start:end], 'latin1')` or 
`str(obj[start:end], 'latin1')` instead of 
`obj[start:end].decode('latin1')`. Just for the case if obj has 
overridden decode() method.


Otherwise LGTM.

--
https://mail.python.org/mailman/listinfo/python-list

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

[Nathaniel Smith]

Nathaniel Smith  added the comment:

I have a patch for this, but it will be simplest if we can merge 
https://github.com/python/cpython/pull/5250 (for bpo-32591) first, because they 
touch a lot of overlapping code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32589] Statistics as a result from timeit

[STINNER Victor]

STINNER Victor  added the comment:

"but after
looking more closely at the PR, I see that the statistics gathered
are from the (default) 100 runs of each call to timer.timeit()."

Ah, don't do that. Timeit runs the same code multiple times and computes
the average to compute one value on purpose. Reading a timer takes time,
and measures smaller than 1 ms are unstable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32589] Statistics as a result from timeit

[STINNER Victor]

STINNER Victor  added the comment:

For minimum vs average, see: https://bugs.python.org/issue28240

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25910] How many "critical"-level open issues are there?

[STINNER Victor]

STINNER Victor  added the comment:

"I will issue a separate PR for it, and then we're good to close this
issue."

Ok.

--
title: Fixing links in documentation -> How many "critical"-level open issues 
are there?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32591] Deprecate sys.set_coroutine_wrapper and replace it with more focused API(s)

[Nathaniel Smith]

Change by Nathaniel Smith :


--
keywords: +patch
pull_requests: +5097
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29708] support reproducible Python builds

[Chih-Hsuan Yen]

Change by Chih-Hsuan Yen :


--
nosy:  -yan12125

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Is this the right way to write a codec error handler?

[Steven D'Aprano]

I want an error handler that falls back on Latin-1 for anything which 
cannot be decoded. Is this the right way to write it?


def latin1_fallback(exception):
assert isinstance(exception, UnicodeError)
start, end = exception.start, exception.end
obj = exception.object
if isinstance(exception, UnicodeDecodeError):
return (obj[start:end].decode('latin1'), end+1)
elif isinstance(exception, UnicodeEncodeError):
return (obj[start:end].encode('latin1'), end+1)
else:
raise



Comments, corrections and criticisms welcome.

Thanks.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue32589] Statistics as a result from timeit

[Steven D'Aprano]

Steven D'Aprano  added the comment:

On Fri, Jan 19, 2018 at 11:48:46PM +, STINNER Victor wrote:

> The timeit module of the stdlib computes 5 values by default... I'm 
> not sure that it's revelant to compute the standard deviation only on 
> 5 values.

I made the same mistake as that: I too thought Matthias was talking 
about calculating the statistics based on timer.repeat(), but after 
looking more closely at the PR, I see that the statistics gathered 
are from the (default) 100 runs of each call to timer.timeit().

(Matthias: in future, please don't just dump a PR into the bug tracker 
with barely a word of explanation. We shouldn't have to read the source 
to understand what it does, let alone why it is done. Thank you.)

Given that Mattias is calculating the stats of each call to timeit(), 
I'm not sure whether that makes more or less sense than what I thought.

His patch effectively converts timeit() to calculate the time for one 
run a million times, instead of the total time for one million runs. 
That is one of the traps that Tim Peters warns about in the O'Reilly 
book: timing *one* run may give unrealistic results due to the timer's 
quantization. So I think this will be much less accurate.

(Is this still true now that timeit() uses perf_counter() as the default 
timer, instead of clock() or time()? I don't know.)

> About the PR itself, I dislike providing a fixed list of statistical 
> functions. For example, what if someone needs the geometric mean? What 
> if you want to count outliers? etc.

I'm not sure that the geometric mean would be physically meaningful. But 
your point is well taken. For example, I think the median might be a 
better choice than mean -- but if some user disagrees, that's okay.

> If someone really wants timeit to evolve, I suggest to return all 
> values rather than only the minimum: as repeat() does.

timeit() doesn't return the minimum.

timeit() calculates the total time for `number` (default is 100) 
runs, and returns that total; repeat() calls timeit() `repeat` (default 
is 3, not 5) times and returns them as a list.

> By the way, using the minimum is IMHO a bad idea, but I already 
> proposed to change timeit, and my change was rejected because of the 
> backward compatibility.

I am interested in why you think it is a bad idea to use the minimum as 
the best estimate of the actual execution time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32582] chr raises OverflowError

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Perhaps this issue should be raised on Python-Dev, since it changes the old and 
well-known behavior of buildins. I think that ValueError is more appropriate, 
but in past we decided to keep OverflowError for compatibility.

If change chr(), other code needs to be changed. "%c", PyUnicode_FromOrdinal(), 
PyUnicode_Format(), and possible the same for bytes.

See also issue29833 and issue15988.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com