[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

Oops, I had a local commit for testing purpose. I regenerated the patch: 
tp_fastcall-4.patch.

> Is PR 65 is same to the third patch?

Yes, patches are generated from my Git pull request:
https://github.com/python/cpython/pull/65

The Git history became a little bit more ugly after many changes :-) I should 
try to rework it to have better changes. I will likely do that before pushing 
changes once the change will be accepted.

--
Added file: http://bugs.python.org/file46326/tp_fastcall-4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[INADA Naoki]

INADA Naoki added the comment:

@haypo, Rietveld didn't accept your patch.  (I don't know what format Rietveld 
accepts...)

Is PR 65 is same to the third patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

[INADA Naoki]

INADA Naoki added the comment:

I've attached wrong file.  This patch is first version I wanted to post.
It dedupe all five functions.

--
Added file: http://bugs.python.org/file46325/dict-refactor.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

[INADA Naoki]

Changes by INADA Naoki :


Removed file: http://bugs.python.org/file46324/dictlook-refactoring.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

[INADA Naoki]

INADA Naoki added the comment:

Raymond, I understand your worries.  I won't commit this until I do more 
precise survey.

But why I trying this is not only I find duplicated code.
I think benefit of this technique is reduced by historical reason.


In Python 2.7, there are only two lookup functions: lookdict and 
lookdict_string.
Both functions support dict containing dummy entry.

In the first comparing part in the lookdict_string():

if (ep->me_key == NULL || ep->me_key == key)
return ep;
if (ep->me_key == dummy)
freeslot = ep;
else {
if (ep->me_hash == hash && _PyString_Eq(ep->me_key, key))
return ep;
freeslot = NULL;
}

And similar code in same function but in loop:

if (ep->me_key == NULL)
return freeslot == NULL ? ep : freeslot;
if (ep->me_key == key
|| (ep->me_hash == hash
&& ep->me_key != dummy
&& _PyString_Eq(ep->me_key, key)))
return ep;
if (ep->me_key == dummy && freeslot == NULL)
freeslot = ep;

First part can ignore freeslot variable is NULL or not.  But second lookup 
can't.
Since most lookup is done at first try, this technique may had significant 
effect.


But for now, we're having five lookdict functions (if we include 
lookdict_index).

Three of them (lookdict_unicode_nodummy, lookdict_split, and lookdict_index) 
cares only dict without dummies.
They don't have freeslot variable.  So first part and second part is almost 
same.
Performance benefit from this technique may be negligible.

About remaining two functions (lookdict_unicode and lookdict), this technique 
may have still effect.
But performance of them are not so important now, compared with Python 2.7.
They were used for all name lookup in Python 2.7, but they are fallback of 
lookdict_split and lookdict_unicode_nodummy now.

On the other hand, having 2.5x (2 -> 5) lookdict function means maintenance 
cost of duplicated code is increased.


At first, I'll start three functions which doesn't take care of dummies.
I think there are almost zero performance regression.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28339] "TypeError: Parameterized generics cannot be used with class or instance checks" in test_functools after importing typing module

[Larry Hastings]

Changes by Larry Hastings :


--
nosy:  -larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28339] "TypeError: Parameterized generics cannot be used with class or instance checks" in test_functools after importing typing module

[Guido van Rossum]

Guido van Rossum added the comment:

@levkivskyi Do you want to attempt a better fix in tome for 3.6.1?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29246] typing.Union raises RecursionError when comparing Union to other type

[Guido van Rossum]

Guido van Rossum added the comment:

Fixed, see http://bugs.python.org/issue28556#msg281317

New changeset 75c7bc2c1ad8 by Guido van Rossum in branch '3.5':
Issue #28556: upstream improvements to docstrings and error messages by Ivan 
Levkivskyi (#331)
https://hg.python.org/cpython/rev/75c7bc2c1ad8

New changeset 294525aac5eb by Guido van Rossum in branch '3.6':
Issue #28556: upstream improvements to docstrings and error messages by Ivan 
Levkivskyi (#331) (3.5->3.6)
https://hg.python.org/cpython/rev/294525aac5eb

New changeset 30f154d9abf0 by Guido van Rossum in branch 'default':
Issue #28556: upstream improvements to docstrings and error messages by Ivan 
Levkivskyi (#331) (3.6->3.7)
https://hg.python.org/cpython/rev/30f154d9abf0

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28556] typing.py upgrades

[Roundup Robot]

Roundup Robot added the comment:

New changeset 1a9e12a852b2 by Guido van Rossum in branch '3.5':
Issue #28556: merge 5 more typing changes from upstream (#340, #344, #348, 
#349, #350)
https://hg.python.org/cpython/rev/1a9e12a852b2

New changeset fd889884fe08 by Guido van Rossum in branch '3.6':
Issue #28556: merge 5 more typing changes from upstream (#340, #344, #348, 
#349, #350) (3.5->3.6)
https://hg.python.org/cpython/rev/fd889884fe08

New changeset 794dad4b849f by Guido van Rossum in branch 'default':
Issue #28556: merge 5 more typing changes from upstream (#340, #344, #348, 
#349, #350) (3.6->3.7)
https://hg.python.org/cpython/rev/794dad4b849f

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

[Raymond Hettinger]

Raymond Hettinger added the comment:

I worry you guys are rapidly sloshing around and churning code that was 
developed slowly and methodically over many years.  A lot of people have looked 
at an tweaked the dictionary code, but now it seems like there is a 
rewrite-everything festival.  When looking at potential changes, please 
consider that the people who came before you knew what they were doing.  If 
there is a pre-loop step that could have been folded into a single loop to save 
a little code, it isn't there because the people who wrote it are daft.  That 
expansion was made for a reason.

Even if you get an improved timing here or there, consider that it may vary 
from compiler to compiler, from os to os, and may vary widely across different 
kinds of applications.  Branch prediction and cache effect issues don't readily 
show up in simple timing loops.

Particularly as a new core dev, I recommend resisting the urge to rewrite 
everything you touch.  That will only destabilize Python.  In the past couple 
of years, two or three devs have churned an enormous amount of code.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

R. David Murray added the comment:

I think this constitutes the heuristic I was talking about in that comment, 
that will get it right 99+% of the time.  Strict mode should raise an error, 
but strict is not the default in the email package.

I probably won't have time to do any review for a while yet, I'm afraid.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

[Xiang Zhang]

Xiang Zhang added the comment:

Ahh, I got the same idea before. But then I persuaded myself that the first 
"lookup()" was deliberately extracted from the loop since it's highly possible 
there is no collision and you can hit the result (empty or not) the first time.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup function

[INADA Naoki]

New submission from INADA Naoki:

All lookdict* functions are implemented like pseudo language:

```
lookup()
if not collision:
return result

while True:
perturb_shift()
lookup()
if not collision:
return result
```

This patch changes it as:

```
while True:
lookup()
if not collision:
return result
perturb_shift()
```

It removes 100 lines of code. Good. But how about performance?

When this patch is applied to 4a534c45bbf6:

```
$ ../python.patched -m perf compare_to default.json patched2.json -G 
--min-speed=2
Slower (4):
- xml_etree_generate: 271 ms +- 6 ms -> 283 ms +- 9 ms: 1.04x slower (+4%)
- nqueens: 263 ms +- 4 ms -> 272 ms +- 3 ms: 1.04x slower (+4%)
- scimark_monte_carlo: 272 ms +- 10 ms -> 280 ms +- 14 ms: 1.03x slower (+3%)
- scimark_lu: 435 ms +- 23 ms -> 446 ms +- 32 ms: 1.03x slower (+3%)

Faster (7):
- call_method: 15.2 ms +- 0.2 ms -> 14.7 ms +- 0.4 ms: 1.04x faster (-4%)
- call_simple: 14.4 ms +- 0.2 ms -> 13.9 ms +- 0.3 ms: 1.04x faster (-4%)
- xml_etree_iterparse: 227 ms +- 9 ms -> 219 ms +- 7 ms: 1.04x faster (-3%)
- scimark_sor: 527 ms +- 10 ms -> 510 ms +- 11 ms: 1.03x faster (-3%)
- call_method_slots: 14.7 ms +- 0.5 ms -> 14.3 ms +- 0.2 ms: 1.03x faster (-3%)
- genshi_text: 90.2 ms +- 1.1 ms -> 87.8 ms +- 1.1 ms: 1.03x faster (-3%)
- django_template: 403 ms +- 5 ms -> 394 ms +- 4 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (53): 2to3, ...
```

And when this patch applied to 1a97b10cb420 :

```
$ ../python.patched -m perf compare_to default.json patched.json -G 
--min-speed=2   

Slower (6):
- call_simple: 13.5 ms +- 0.5 ms -> 14.4 ms +- 0.4 ms: 1.07x slower (+7%)
- xml_etree_generate: 270 ms +- 6 ms -> 287 ms +- 5 ms: 1.06x slower (+6%)
- xml_etree_process: 240 ms +- 6 ms -> 247 ms +- 4 ms: 1.03x slower (+3%)
- regex_compile: 429 ms +- 3 ms -> 440 ms +- 5 ms: 1.03x slower (+3%)
- call_method_unknown: 16.1 ms +- 0.2 ms -> 16.5 ms +- 0.3 ms: 1.02x slower 
(+2%)
- logging_simple: 31.2 us +- 0.4 us -> 32.0 us +- 0.3 us: 1.02x slower (+2%)

Faster (8):
- genshi_text: 90.6 ms +- 1.4 ms -> 87.6 ms +- 1.2 ms: 1.03x faster (-3%)
- scimark_sor: 513 ms +- 11 ms -> 497 ms +- 12 ms: 1.03x faster (-3%)
- genshi_xml: 200 ms +- 2 ms -> 194 ms +- 2 ms: 1.03x faster (-3%)
- unpickle_pure_python: 857 us +- 21 us -> 835 us +- 13 us: 1.03x faster (-3%)
- python_startup_no_site: 9.95 ms +- 0.02 ms -> 9.74 ms +- 0.02 ms: 1.02x 
faster (-2%)
- json_dumps: 29.7 ms +- 0.4 ms -> 29.1 ms +- 0.4 ms: 1.02x faster (-2%)
- xml_etree_iterparse: 225 ms +- 9 ms -> 220 ms +- 5 ms: 1.02x faster (-2%)
- chameleon: 31.1 ms +- 0.3 ms -> 30.5 ms +- 0.5 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (50): 2to3, ...
```

I can't see any stable and significant performance regression.
I'll try to create some micro benchmarks.

--
files: dictlook-refactoring.patch
keywords: patch
messages: 285695
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: dict: simplify lookup function
Added file: http://bugs.python.org/file46324/dictlook-refactoring.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi all,

I've received an answer from the IETF Pete Resnick.  The answer does say 
however that there is no guaranteed way of getting the right headers if the 
message doesn't respect the standard; 

"[...] 
In this case, if the trace fields were not present, you would not be able to 
clearly distinguish. Certainly you know which blocks the Resent-Date and 
Resent-From belong to, and in this case you know which blocks the 
Resent-Message-ID belong to (since there are two of them and only two Resent-* 
blocks), and you know which block the Resent-Bcc belongs to (because it comes 
between the first Resent-Date and Resent-Message-ID), but you can't tell which 
block the Resent-To: belongs to. And if the either Resent-Message-ID was 
missing, you would be unable to tell where the Resent-Bcc or Resent-Message-ID 
belongs. This is simply a weakness in the standard.

However, the trace fields should exist, and that should divide the Resent-* 
blocks. As it says in 3.6:

header fields SHOULD NOT be reordered when a message is transported
or transformed. More importantly, the trace header fields and resent
header fields MUST NOT be reordered, and SHOULD be kept in blocks
prepended to the message.

I hope that helps.


pr
-- 
Pete Resnick http://www.qualcomm.com/~presnick/
Qualcomm Technologies, Inc. - +1 (858)651-4478
"
__

In other words, if we were to take an e-mail that would have more than one 
Resent- headers, there should be traces in between the resent block.  If there 
are no traces for a Resent- block, and we can detect there are 2 blocks in what 
is supposed to be a block , we should raise an error.  

However, if we were to do the implementation and make it to detect "blocks" of 
resents (with the position of the resent for example).  We could use the first 
block as the one we should be using and see if there are 2 Resent-Dates or 2 
Resent-From in that block only.

What do you think David?  I would like to know :).

P.s. I can forward the e-mail to does who want to have it :).

Regards,
Eric Lafontaine

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

Ok, I fixed all remaining issues. The new patch version 3 is now waiting for 
your review ;-)


Patch version 3:

* Add Py_TPFLAGS_HAVE_FINALIZE and Py_TPFLAGS_HAVE_FASTCALL to 
Py_TPFLAGS_DEFAULT
* Don't read or write tp_fastcall if the type doesn't have the 
Py_TPFLAGS_HAVE_FASTCALL feature
* fastcall_wrapper() looks into base classes to find a tp_fastcall field: use 
the first base class with the Py_TPFLAGS_HAVE_FASTCALL feature
* Fix how tp_call and tp_fastcall are inherited
* Use designated initializers for  tp_fastcall, as asked by Naoki (and 
discussed on python-dev)
* Rebase the change and revert now uselesss or unwanted changes
* No more FIXME ;-)

About the stable ABI: since the version 3, it should just work thanks to the 
change on Py_TPFLAGS_DEFAULT. Code compiled with Python 3.7 benefit directly of 
tp_fastcall. Code compiled with Python 3.6 or older will get fastcall_wrapper() 
which finds tp_fastcall in base classes.

--
Added file: http://bugs.python.org/file46323/tp_fastcall-3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29296] convert print() to METH_FASTCALL

[STINNER Victor]

STINNER Victor added the comment:

Serhiy Storchaka: "Please wait until the support of var-positional arguments be 
added in Argument Clinic. After that print() could be just converted to 
Argument Clinic."

Why can't we start with METH_FASTCALL and convert to AC later? It seems like 
print-fastcall.patch will be a major performance enhancement (at least for the 
very specific bm_telco benchmark), once tp_fastcall field will be added.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29303] asyncio.Lock.acquire() does not always yield

[Guido van Rossum]

Guido van Rossum added the comment:

No, `yield from` (or, in Python 3.5+, `await`) is not meant to bound back to 
the scheduler. If the target is a coroutine that itself doesn't yield, it is a 
*feature* that the scheduler is bypassed.

If you want to force a trip through the scheduler, use `asyncio.sleep(0)`.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29303] asyncio.Lock.acquire() does not always yield

[anonymous2017]

anonymous2017 added the comment:

I may have mis-characterized the original report. Rephrasing just to make sure 
it is correctly addressed:

First, it is not about fairness. Sorry for the original characterization. I was 
trying to understand what was happening.

Second, the precise issue is whether a `yield from` can be relied upon for 
cooperative multitasking. Should other co-routines should be prevented from 
running or delayed if one routine is running and acquiring and releasing a lock 
using `yield from`.

It seemed at first sight that `yield from ` should give the scheduler 
a chance to consider other routines. But the lock code above and including line 
#171 does not give control to the scheduler in a special case: 
https://github.com/python/cpython/blob/master/Lib/asyncio/locks.py#L171

Code Sample 1 (shows b() being delayed until a() is fully complete despite a() 
yielding many times)
=
import asyncio

lock = asyncio.Lock()

def a ():
 yield from lock.acquire()
 for i in range(10):
  print('b: ' + str(i))
  if i % 2 == 0:
   lock.release()
   yield from lock.acquire()
 lock.release()

async def b ():
 print('hello')

asyncio.get_event_loop().run_until_complete(asyncio.gather(a(), b()))

print('done')

Code Sample 2
=
(shows interleaving if an additional INITIAL yield from asyncio.sleep is 
inserted; removing the initial sleep removes the interleaving)

import asyncio

lock = asyncio.Lock()

def a ():
 yield from lock.acquire()
 yield from asyncio.sleep(0)
 for i in range(10):
  print('a: ' + str(i))
  if i % 2 == 0:
   lock.release()
   yield from lock.acquire()
 lock.release()

def b ():
 yield from lock.acquire()
 yield from asyncio.sleep(0)
 for i in range(10):
  print('b: ' + str(i))
  if i % 2 == 0:
   lock.release()
   yield from lock.acquire()
 lock.release()

asyncio.get_event_loop().run_until_complete(asyncio.gather(a(), b()))

print('done')


Thank you for your kind consideration.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29303] asyncio.Lock.acquire() does not always yield

[Guido van Rossum]

Guido van Rossum added the comment:

Locks are not meant to have predictable behavior like that. They are meant to 
protect against concurrent access. If you want fairness you have to look 
elsewhere.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29303] asyncio.Lock.acquire() does not always yield

[anonymous2017]

New submission from anonymous2017:

Details here: 
http://stackoverflow.com/questions/41708609/unfair-scheduling-bad-lock-optimization-in-asyncio-event-loop

Essentially there should be a `yield` before this line otherwise a coroutine 
that only acquires and releases a lock will never yield to other coroutines.

--
components: asyncio
messages: 285688
nosy: anonymous2017, gvanrossum, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.Lock.acquire() does not always yield
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29302] add contextlib.AsyncExitStack

[Alexander Mohr]

New submission from Alexander Mohr:

ExitStack is a really useful class and would be a great to have an async 
version.  I've gone ahead and created an implementation based on the existing 
Python 3.5.2 implementation.  Let me know what you guys think. I think it would 
be possible to combine most of the two classes together if you guys think it 
would be useful.  Let me know if I can/should create a github PR and where to 
do that.

--
components: Library (Lib)
files: exit_stack.py
messages: 285687
nosy: thehesiod
priority: normal
severity: normal
status: open
title: add contextlib.AsyncExitStack
type: enhancement
versions: Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46322/exit_stack.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi all,

The IETF didn't answer yet :(.  

I'll await your news regarding this patch ("issue_28879_V4.patch").

I would like to have feedback if I need to change something.

Thanks a lot in advance,
Eric Lafontaine

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29282] Fused multiply-add: proposal to add math.fma()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

LGTM except that needed the versionadded directive and What's New entry. And 
maybe few additional tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29282] Fused multiply-add: proposal to add math.fma()

[Mark Dickinson]

Changes by Mark Dickinson :


--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Doug Hellmann]

Doug Hellmann added the comment:

@Antoine - The idea behind introducing some API mechanism is exactly as you 
say, to let the developer say "this use of this algorithm is not related to 
security" to tell FIPS systems to not be pedantic.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Doug Hellmann]

Doug Hellmann added the comment:

@Robert, I thought you were proposing a hashlib.fips module that did not 
include md5() at all. If it does include the function, and the function does 
whatever is needed to disable the "die when using MD5" on a FIPS system, then I 
agree it would work. 

Your point about the FIPS standard changing and needing to include more hash 
types in the future is good.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29282] Fused multiply-add: proposal to add math.fma()

[Mark Dickinson]

Mark Dickinson added the comment:

And here's a patch.

--
keywords: +patch
Added file: http://bugs.python.org/file46321/math_fma.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Yolanda]

Yolanda added the comment:

@rbtcollins, so you mean the apps using it, shall be "fips aware" ? That will 
be the point of your separate module?
So...

if fips_enabled then
use fips.md5
else
use normal.md5

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29301] decimal: Use FASTCALL and/or Argument Clinic

[STINNER Victor]

STINNER Victor added the comment:

Raymond Hettinger: "This should not go in without Stephan Krah's approval.  
This code is finely tuned and carefully arrang"

Sure, that's why I opened this issue.


Serhiy Storchaka: "See msg207652 in issue20177."

Oh, I didn't know that Stefan Krah already gave his opinion.

Since the performance impact is unclear, I prefer to close this issue. I will 
come back if argument parsing is more clearly identified as a performance 
bottleneck.

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Antoine Pitrou]

Antoine Pitrou added the comment:

AFAICT from David's patch, there isn't a new argument in all hashlib functions 
but only in the digest constructors.  Someone might want to correct me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Ian Cordasco]

Ian Cordasco added the comment:

So I see the argument on both sides of this discussion. Having those optional 
arguments for all the functions seems like an obvious blocker. If a submodule 
is a blocker, what if we provide a context-manager to signal this?

--
nosy: +icordasc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Antoine Pitrou]

Antoine Pitrou added the comment:

The separate module idea is an interesting one, though I wonder if it aligns 
with users' goals.  Perhaps some users simply want to set the 
OPENSSL_FORCE_FIPS_MODE environment variable and then run existing Python code 
with it to ensure that code is FIPS-compliant.  A separate module assumes that 
the developer is the one who makes the decision of running in FIPS compliance 
mode or not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Robert Collins]

Robert Collins added the comment:

@doug - I don't see how a separate fips module *wouldn't* solve it:
 - code that uses md5 in security contexts wouldn't be able to call it from the 
fips module, which is the needed outcome
 - code that uses md5 and isn't fips compliant would be importing from the 
non-fips module, and thats as auditable as looking for a 
'usedforsecurity=False' flag
 - auditors can assume that code that doesn't use the fips module


And its way less messy: remember we're going to have this flag passed to every 
hashlib invocation from every project in order to *opt out* of the FIPS 
restrictions. Because, over time, FIPS will change, so noone can assume that 
any given function is and will remain FIPS compatible: and this flag is going 
to percolate up into e.g. the HMAC module.

I think thats pretty ugly: want to calculate the sha of a blob to look it up in 
git? sha1sum(file.read(), usedforsecurity=False)

Separately I wonder about the impact on higher layers - are they ready to be 
parameterised by objects, or do they look things up by name - and thus need to 
start accepting this new parameter and passing it down?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29285] Unicode errors occur inside of multi-line comments

[John Taylor]

John Taylor added the comment:

OP here, thanks for replying to this.  I used Zach's suggestion of placing an 
'r' in front of triple-quotes.  This accomplishes my goal.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[INADA Naoki]

INADA Naoki added the comment:

> Maybe it's the expensive price of creating a temporary dictionary?

Please see the patch on issue29295.
PyDict_GetItemString() is bad part, too.
It's create str object, calc hash, lookup dict, and dealloc the str object.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29301] decimal: Use FASTCALL and/or Argument Clinic

[Raymond Hettinger]

Raymond Hettinger added the comment:

This should not go in without Stephan Krah's approval.  This code is finely 
tuned and carefully arranged.

--
assignee:  -> skrah
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29296] convert print() to METH_FASTCALL

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Please wait until the support of var-positional arguments be added in Argument 
Clinic. After that print() could be just converted to Argument Clinic.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29301] decimal: Use FASTCALL and/or Argument Clinic

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See msg207652 in issue20177.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29301] decimal: Use FASTCALL and/or Argument Clinic

[STINNER Victor]

STINNER Victor added the comment:

Oh wait, I'm not sure that attached patch has a significant impact on 
performances. It seems like the speedup mostly comes from the print patch:
http://bugs.python.org/issue29296#msg285668

But well, it is still interesting to use METH_FASTCALL in decimal ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29300] Modify the _struct module to use FASTCALL and Argument Clinic

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Why not just convert the struct module to Argument Clinic?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29296] convert print() to METH_FASTCALL

[STINNER Victor]

STINNER Victor added the comment:

I reviewed print-fastcall.patch: LGTM, but I proposed a minor change.


Serhiy Storchaka: "The performance of print() is not critical. It usually 
involves slow formatting and IO."

I also had the same understanding of print(), but I just analyzed performances 
of the bm_telco benchmark, and it seems just like handling function parameters 
of print() take 20% of the runtime!?

   http://bugs.python.org/issue29259#msg285667

bm_telco reference (unpatched) => with issue #29259 tp_fastcall-2.patch and 
print-fastcall.patch:

   20.9 ms +- 0.5 ms => 16.7 ms +- 0.2 ms

print-fastcall.patch makes bm_telco 20% faster! Just to make sure, I ran again 
bm_telco only with tp_fastcall-2.patch:

   telco: Median +- std dev: 21.4 ms +- 0.8 ms

Maybe we should optimize _PyStack_AsDict(), but that's a different topic ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

I checked the effect of individual patches:

* tp_fastcall-2
* print
* struct
* decimal

tp_fastcall-2 + print + struct + decimal: 16.3 ms +- 0.6 ms
tp_fastcall-2 + struct + decimal: 21.2 ms +- 0.3 ms
tp_fastcall-2 + print: 16.7 ms +- 0.2 ms

Oh wow, I didn't expect that print would be the bottleneck of this benchmark!? 
There is a single print() in the hotcode of bm_telco:

   print(t, file=outfil)

Maybe it's the expensive price of creating a temporary dictionary?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29259] Add tp_fastcall to PyTypeObject: support FASTCALL calling convention for all callable objects

[STINNER Victor]

STINNER Victor added the comment:

Back on performances: if you combined tp_fastcall-2.patch + 3 minor changes to 
use METH_FASTCALL, the bm_telco benchmark becomes 22% faster, which is 
significant for a benchmark:
http://bugs.python.org/issue29301#msg285665

"20.9 ms +- 0.5 ms => 16.4 ms +- 0.5 ms"

It also means that 22% of the benchmark time is currently spent in handling 
function arguments! I expected that most of the time would be spent on maths 
and I/O!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29301] decimal: Use FASTCALL and/or Argument Clinic

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29301] decimal: Use FASTCALL and/or Argument Clinic

[STINNER Victor]

New submission from STINNER Victor:

I'm currently working on the isuse #29259: "Add tp_fastcall to PyTypeObject: 
support FASTCALL calling convention for all callable objects". I used bm_telco 
of the performance benchmark suite to check which functions still require to 
create a temporary tuple for positional arguments and a temporary dict for 
keyword arguments. I found 3 remaining functions which have an impact on the 
result of the benchmark:

* print(): optimized by the issue #29296
* _struct.unpack(): I just created the issue #29300 "Modify the _struct module 
to use FASTCALL and Argument Clinic"
* _decimal.Decimal.quantize()

I would like to know if Stephan would be ok to modify the _decimal module to 
use FASTCALL. I know that recently he reverted changes to keep the same code 
base on Python 3.5, 3.6 and 3.7.

With 4 changes (tp_fastcall #29259, print #29296, unpack #29300 and this 
issue), bm_telco becomes 22% faster which is not negligible!

   20.9 ms +- 0.5 ms => 16.4 ms +- 0.5 ms

Attached decimal.patch patch is the strict minimum change to optimize bm_telco, 
but I would prefer to change all _decimal functions and methods using 
METH_VARARGS and METH_VARARGS|METH_KEYWORDS to convert them to METH_FASTCALL.

The best would be to use Argument Clinic. AC exists since Python 3.5, so it 
should be possible to keep the same code base on Python 3.5-3.7, only generated 
code would be different.

--
files: decimal.patch
keywords: patch
messages: 285665
nosy: haypo, skrah
priority: normal
severity: normal
status: open
title: decimal: Use FASTCALL and/or Argument Clinic
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46320/decimal.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29292] Missing a parameter in PyEval_EvalCodeEx doc

[Xiang Zhang]

Xiang Zhang added the comment:

> "keywords and defaults," however, the kwdefs argument was never added to the 
> prototype.

I don't think this is about the missing "kwdefs". I think "arrays of arguments, 
keywords and defaults" is a whole part describing "PyObject **args, int 
argcount, PyObject **kws, int kwcount, PyObject **defs, int defcount". "kwdefs" 
is a dict containing the default values of keyword-only parameters in Py3? I am 
not sure.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29300] Modify the _struct module to use FASTCALL and Argument Clinic

[STINNER Victor]

New submission from STINNER Victor:

Attached patch modify the _struct module to use FASTCALL and Argument Clinic.

AC requires a summary line, so I duplicated the first sentence of iter_unpack() 
docstring:

iter_unpack(fmt, buffer, /)
Return an iterator yielding tuples unpacked from the given bytes.

Return an iterator yielding tuples unpacked from the given bytes
source according to the format string, like a repeated invocation of
unpack_from().  Requires that the bytes length be a multiple of the
format struct size.

--
components: Argument Clinic
files: struct_fastcall.patch
keywords: patch
messages: 285663
nosy: haypo, inada.naoki, larry, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Modify the _struct module to use FASTCALL and Argument Clinic
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46319/struct_fastcall.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29292] Missing a parameter in PyEval_EvalCodeEx doc

[Ammar Askar]

Ammar Askar added the comment:

It looks like a basic description of kwdefs was added as part of this commit: 
https://github.com/python/cpython/commit/7811a5d1c93f2aa0b357444eeb3f1ddc242ac57a

"keywords and defaults," however, the kwdefs argument was never added to the 
prototype.

I've attached a patch which adds the argument to the prototype and clarifies 
the language a little bit.

--
keywords: +patch
nosy: +ammar2
Added file: http://bugs.python.org/file46318/kwdefs_docs.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29299] Argument Clinic: Fix signature of optional positional-only arguments

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Please don't change this part of Argument Clinic without Larry. There were 
several attempts to solve this problem, I don't like current status, but this 
is Larry's decision.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20291] Argument Clinic should understand *args and **kwargs parameters

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thank you for your examples Raymond, but they don't directly related to this 
issue, implementing support of var-positional and var-keyword parameters. I 
believe that it is possible to solve it, and the solution is complex, but is 
not extremal hard. I'm working on the first part of this.

Your examples show other, more hard issue. It looks to me that the only 
solution of that issue is to add support of multiple signatures for functions. 
But this can break the API of the inspect module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29299] Argument Clinic: Fix signature of optional positional-only arguments

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The problem is that

  func(mandatory_arg1, mandatory_arg2[, optional_arg3[, optinal_arg4]])

is not compatible with the inspect module.

In many case a meaningful default value was added if this is possible. For 
example the Python default shown in the signature can be set to '', 'utf-8' or 
'strict' while the C default value is NULL for performance. If the parameter is 
upper index in the sequence it can be set to sys.maxsize (Py_SSIZE_T_MAX in C).

This is not always possible. For example there is not default value for 
dict.pop().

--
nosy: +yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29285] Unicode errors occur inside of multi-line comments

[Eryk Sun]

Eryk Sun added the comment:

> they do not appear in the byte code files

It's simple to coonfirm that unassigned string literals get thrown away.:

>>> code = compile('"doc"\n"unused"\n"us"+"ed"', '', 'exec')
>>> code.co_consts
('doc', 'us', 'ed', None, 'used')
>>> dis.dis(code)
  1   0 LOAD_CONST   0 ('doc')
  3 STORE_NAME   0 (__doc__)

  3   6 LOAD_CONST   4 ('used')
  9 POP_TOP
 10 LOAD_CONST   3 (None)
 13 RETURN_VALUE

However, they're retained up to the abstract syntax tree:

>>> print(ast.dump(ast.parse('"doc"\n"unused"\n"us"+"ed"', '', 'exec')))
Module(body=[
Expr(value=Str(s='doc')),
Expr(value=Str(s='unused')),
Expr(value=BinOp(left=Str(s='us'),
 op=Add(),
 right=Str(s='ed')))])

Sans the doc string behavior, this applies to literals in general. Thus even 
though the compiler discards these objects, the literal still has to be parsed 
like any other.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29299] Argument Clinic: Fix signature of optional positional-only arguments

[STINNER Victor]

STINNER Victor added the comment:

This issue is blocking me to convert more functions to Argument Clinic. See for 
example attached getattr_ac.patch which converts getattr() to AC. Without 
ac_optional_positional.patch, AC generates the signature:

   "getattr($module, object, name, default=None, /)\n"

whereas the following signature is expected:

   "getattr($module, object, name[, default])\n"

--
Added file: http://bugs.python.org/file46317/getattr_ac.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29299] Argument Clinic: Fix signature of optional positional-only arguments

[STINNER Victor]

STINNER Victor added the comment:

See also issue #20291: "Argument Clinic should understand *args and **kwargs 
parameters".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29289] Convert OrderedDict methods to Argument Clinic

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I have added other comments on Rietveld (mostly about docstrings).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20291] Argument Clinic should understand *args and **kwargs parameters

[STINNER Victor]

STINNER Victor added the comment:

FYI I started to work on a different Argument Clinic enhancement, issue #29299: 
"Argument Clinic: Fix signature of optional positional-only arguments".

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29297] python3 open() does not check argument type before attempting to read() or write()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Builtin open() in Python 3 (and io.open() in Python 2.7) accept unicode 
strings, byte strings and integers as the first arguments (general path-like 
objects also are supported in Python 3.6, but it doesn't matter). bool is a 
subtype of int, and False is equal to integer 0. 0 is a file descriptor of 
sys.stdin.

>>> import sys
>>> sys.stdin.fileno()
0

Thus open(False) is equivalent to open(sys.stdin.fileno()). It creates a file 
object that wraps the file descriptor 0. This is legitimate operation.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29289] Convert OrderedDict methods to Argument Clinic

[STINNER Victor]

STINNER Victor added the comment:

For OrderedDict.pop(), I created the issue #29299: "Argument Clinic: Fix 
signature of optional positional-only arguments".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29299] Argument Clinic: Fix signature of optional positional-only arguments

[STINNER Victor]

STINNER Victor added the comment:

List of functions modified when Argument Clinic is run to regenerate .c.h files:

builtin_format
builtin_getattr
builtin_input
builtin_sum
cmath_log
_codecs_ascii_decode
_codecs_ascii_encode
_codecs_charmap_decode
_codecs_charmap_encode
_codecs_code_page_encode
_codecs_escape_decode
_codecs_escape_encode
_codecs_latin_1_decode
_codecs_latin_1_encode
_codecs_mbcs_encode
_codecs_oem_encode
_codecs_raw_unicode_escape_decode
_codecs_raw_unicode_escape_encode
_codecs_readbuffer_encode
_codecs_unicode_escape_decode
_codecs_unicode_escape_encode
_codecs_unicode_internal_decode
_codecs_unicode_internal_encode
_codecs_utf_16_be_encode
_codecs_utf_16_le_encode
_codecs_utf_32_be_encode
_codecs_utf_32_le_encode
_codecs_utf_7_encode
_codecs_utf_8_encode
_dbm_dbm_get
_dbm_dbm_setdefault
fcntl_fcntl
_imp_create_dynamic
_io_FileIO_truncate
os_putenv
os_unsetenv
pyexpat_xmlparser_ExternalEntityParserCreate
_sre_SRE_Match_end
_sre_SRE_Match_span
_sre_SRE_Match_start
_tkinter_create
unicodedata_UCD_decimal
unicodedata_UCD_digit
unicodedata_UCD_name
unicodedata_UCD_numeric
unicode_lstrip
unicode_maketrans
unicode_rstrip

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29299] Argument Clinic: Fix signature of optional positional-only arguments

[STINNER Victor]

New submission from STINNER Victor:

When a function has only positional arguments and at least one argument is 
optional, the expected signature is:

  func(mandatory_arg1, mandatory_arg2[, optional_arg3[, optinal_arg4]])

For example, the signature of format() is inconsistent with its documentation.

Signature:
---
$ python3 -c 'help(format)'|cat
Help on built-in function format in module builtins:

format(value, format_spec='', /)
Return value.__format__(format_spec)

format_spec defaults to the empty string
---

Documentation:
---
.. function:: format(value[, format_spec])
---

Attached patch is a first attempt to fix the issue. The problem is that my 
heuristic to check if an argument is "optional" doesn't seem to work as 
expected in all cases. I chose to check if the C default is NULL.

The problem is that some functions defines a C default to NULL whereas the 
Python default is set to a different value and is correct.

Example with _io.FileIO.truncate:

/*[clinic input]
_io.FileIO.truncate
size as posobj: object = NULL
/

whereas the documentation says that the default is None:

   .. method:: truncate(size=None)

It's easy to fix the default, but in this case my change doesn't fix the 
signature anymore since the C default is still NULL:

/*[clinic input]
_io.FileIO.truncate
size as posobj: object(c_default="NULL") = None
/

We need a different heuristic than C default is NULL, or we should fix 
functions where the heuristic fails.

--
components: Argument Clinic
files: ac_optional_positional.patch
keywords: patch
messages: 285650
nosy: haypo, larry, rhettinger, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Argument Clinic: Fix signature of optional positional-only arguments
versions: Python 3.7
Added file: http://bugs.python.org/file46316/ac_optional_positional.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29297] python3 open() does not check argument type before attempting to read() or write()

[Mo Ali]

Mo Ali added the comment:

Serhiy,

I expected a type error or a filenotfound like you received, however mine 
doesn't return the same. It just hangs.  I've attached a picture.  Also, I 
meant this to be for 3.6 not 3.5.

>>> test = False
>>> with open(test) as f:
... fail = f.read()
...

--
versions: +Python 3.6 -Python 3.5
Added file: http://bugs.python.org/file46315/Screen Shot 2017-01-17 at 10.09.35 
AM.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29297] python3 open() does not check argument type before attempting to read() or write()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The first example raises an error:

Traceback (most recent call last):
  File "", line 1, in 
FileNotFoundError: [Errno 2] No such file or directory: 'False'

Unless you have a file named "False" in the current directory of course.

What behavior you expected?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20291] Argument Clinic should understand *args and **kwargs parameters

[Raymond Hettinger]

Raymond Hettinger added the comment:

In case it is helpful, here's my list of examples where the AC and existing 
signature objects are insufficiently expressive:


type(obj)
type(name, bases, mapping)
two different signatures depending on type   

range(stop)
range(start, stop)
range(start, stop, step)

dict.pop(key[, default])
   default of None has different meaning than missing default
   which raises KeyError when the key is missing

itertools.permutations(iterable[, r])
   where the absence of *r* implies r=len(iterable)

bisect.bisect_right(a, x[, lo[, hi]]) -> index
   where the absence of *hi* implies hi=len(a)

min(iterable, *[, default=obj, key=func]) -> value
min(arg1, arg2, *args, *[, key=func]) -> value
   has two signatures depending on the number of
   positional arguments and a keyword argument
   only used in the first signature.  It's implementation
   is also shared with max().


dict() -> new empty dictionary
dict(mapping) -> new dictionary initialized from a mapping object's
   (key, value) pairs
dict(iterable) -> new dictionary initialized as if via:
   d = {}
   for k, v in iterable:
   d[k] = v
dict(**kwargs) -> new dictionary initialized with the name=value pairs
   in the keyword argument list.  For example:  dict(one=1, two=2)

def sumseq(seq, a=0, b=None):
# Pure python code with nullable int
if b is None:
b = len(seq)
return sum(seq[a:b])

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29298] argparse fails with required subparsers, un-named dest, and empty argv

[zachrahan]

New submission from zachrahan:

In python 3.6 (and several versions back), using argparse with required 
subparsers will cause an unhelpful TypeError if the 'dest' parameter is not 
explicitly specified, and no arguments are provided.

Test case:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers()
subparsers.required = True
args = parser.parse_args([])

Observed result:
TypeError: sequence item 0: expected str instance, NoneType found

If the line above is changed to:
subparsers = parser.add_subparsers(dest='function')

Then the following is printed to stderr:
usage: python [-h] {} ...
python: error: the following arguments are required: function

This issue goes back at least several years:
http://stackoverflow.com/questions/23349349/argparse-with-required-subparser/23354355

Though it seems odd to not specify a dest in the add_subparsers line, the 
pattern is not completely useless. The below works fine without setting a 
'dest' in add_subparsers, except when argv is empty:
sub1 = subparsers.add_parser('print')
sub1.set_defaults(function=print)

However, an empty argv produces the unexpected TypeError above. I'm not sure if 
argparse should provide a more useful exception in this case, or if there is a 
clean way to do the right thing without a dest specified.

--
components: Library (Lib)
messages: 285646
nosy: zachrahan
priority: normal
severity: normal
status: open
title: argparse fails with required subparsers, un-named dest, and empty argv
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29285] Unicode errors occur inside of multi-line comments

[R. David Murray]

R. David Murray added the comment:

@zach: actually, triple quoted strings *are* suggested as a way to do multiline 
comments, and are often used for that.  In particular, note that they do not 
appear in the byte code files if they are not assigned to anything or appear in 
the docstring position.  This is something I had to prove to myself at one 
point when I had the same reaction you just did to their use as comments :)  

That said, it is true that they are, first and foremost, strings, so the 
quoting behavior is not going to change.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29297] python3 open() does not check argument type before attempting to read() or write()

[Mo Ali]

New submission from Mo Ali:

Python3 open(), read(), or write()doesn't check argument type before action 
causing a hang.  Would like to catch exceptions but not without an exception to 
return.  See below.

Python3.6:
Python 3.6.0 (default, Dec 24 2016, 08:01:42)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> test = False
>>> with open(str(test)) as f:
... fail = f.read()
...


python2.7:
╰─λ python2 

0 < 09:35:31
Python 2.7.13 (default, Dec 18 2016, 07:03:39)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> test = False
>>> with open(test) as f:
... fail = f.read()
...
Traceback (most recent call last):
  File "", line 1, in 
TypeError: coercing to Unicode: need string or buffer, bool found

--
messages: 285644
nosy: Mo Ali
priority: normal
severity: normal
status: open
title: python3 open() does not check argument type before attempting to read() 
or write()
type: crash
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Yolanda]

Yolanda added the comment:

I agree with Doug. From my understanding, the intention of the patch is to 
allow the usage of md5 for non-security purposes, without being blocked by FIPS.
Right now, md5() calls running on a FIPS enabled kernel, are blocked without 
discrimination of the usage, that shall be ok for hashing purposes, and more 
performant than other methods. This patch provides the ability to continue 
using md5 just flagging it properly.

--
nosy: +yolanda.robla

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20291] Argument Clinic should understand *args and **kwargs parameters

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I'll try to implement the support of var-positional parameters.

--
assignee: larry -> serhiy.storchaka
versions: +Python 3.7 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16623] argparse help formatter does not honor non-breaking space

[Peter Otten]

Changes by Peter Otten <__pete...@web.de>:


--
nosy: +peter.otten

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29290] argparse breaks long lines on NO-BREAK SPACE

[Peter Otten]

Changes by Peter Otten <__pete...@web.de>:


--
nosy: +peter.otten

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29287] IDLE needs syntax highlighting for f-strings

[Peter Otten]

Changes by Peter Otten <__pete...@web.de>:


--
nosy: +peter.otten

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29280] gdbm & ndbm support missing in Windows

[R. David Murray]

R. David Murray added the comment:

This has always been the case (that gdbm and ndbm are not supported on 
windows).  See also issue 3769 for background and dbm and windows, and issue 
3783 for something to work on if you want to contribute to making the dbm story 
on windows better.

Googling found a gdbm port to windows, so if you are trying to use source files 
from linux, that might help you.  We're open to suggestions on improving the 
situation here, but someone has to figure out what would make sense and 
champion it.  Bundling the gdbm windows port is probably not an option, but 
making it possible to install something via PIP that will work is, if there's 
anything in core preventing that currently.

--
nosy: +r.david.murray
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9216] FIPS support for hashlib

[Doug Hellmann]

Doug Hellmann added the comment:

@rbcollins, I don't think providing a hashlib.fips module without md5() solves 
the problem. The idea is to have a way to call md5() in non-secure situations, 
and to signal to the FIPS system that the call is OK. A separate module would 
work if it included an md5() function that always did that signaling. But 
creating a separate module just to wrap one function like that seems like 
overkill, doesn't it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29295] dict: Optimize PyDict_GetItemString()

[STINNER Victor]

STINNER Victor added the comment:

> Close this issue for now, until profiler shows me PyDict_XString.

I like Serhiy's rationale. We should try to avoid PyDict_GetItemString() 
wheneve possible. If PyDict_GetItemString() becomes a clear bottleneck, we can 
discuss again optimizing it. But in the meanwhile, I prefer to use anothe rule: 
an optimization should not modify the behaviour of a function. That's why I 
didn't optimize OrderedDict.pop() yet (it changed the docstring, AC should be 
enhanced for this case).

So yeah, let's close this one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29296] convert print() to METH_FASTCALL

[STINNER Victor]

STINNER Victor added the comment:

Hi,

Serhiy Storchaka: "The performance of print() is not critical. It usually 
involves slow formatting and IO."

Oh, please, I want a print() using METH_FASTCALL! Not for the performance, but 
just to help me in my development!

To identify code not using FASTCALL yet, I'm putting a breakpoint on 
_PyStack_AsTuple() and _PyStack_AsDict(). It's common to stop on print() when 
testing the performance benchmark suite.

FYI currnently, I'm adding the following Python code to python-gdb.py:
---
class MyBreakpoint (gdb.Breakpoint):
def stop(self):
# frame of the caller
frame = gdb.selected_frame()
frame = frame.older()

name = frame.name()
if name == '_PyCFunction_FastCallKeywords':
pass
else:
# callable->ob_type
obj = frame.read_var('callable')
obj = obj.referenced_value()['ob_type']

obj2 = gdb.lookup_global_symbol('PyType_Type').value()

if obj == obj2.address:
return False

print("don't skip", obj, obj2.address)

gdb.execute("up")
return True


MyBreakpoint("_PyStack_AsTuple")
MyBreakpoint("_PyStack_AsDict")
---

These macros make gdb 40x slower :-/

The code is used to skip type_call(): this function is known to not use 
FASTCALL yet. I plan to add tp_fastnew and tp_fastinit to optimize type_call(), 
but it's already hard enough to implement tp_fastcall, so I will do it later.

> _PyArg_ParseStackAndKeywords() is a part of unstable, fast-evolved API. I 
> would beware of using it in common code.

Ah. I have a different opinion on that. Since _PyArg_ParseStackAndKeywords() 
and print() are part of the Python core, we have free to use private 
fast-moving APIs.

By the way, _PyArg_ParseStackAndKeywords() already uses the new efficient 
_PyArg_Parser object. Do you plan other enhancements?


INADA Naoki: "I see.  AC should support this."

Right, but IMHO it can be done later.

I tried to contribute to AC but the code is super complex. I understood that AC 
is not complex, but Python is complex :-D AC is full of corner cases. It's a 
very long list of special cases :-)

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29294] ctypes.windll.LoadLibrary refuses unicode argument

[Eryk Sun]

Changes by Eryk Sun :


--
resolution:  -> duplicate
stage:  -> resolved
superseder:  -> In 2.7.13, _ctypes.LoadLibrary no longer accepts Unicode objects

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20291] Argument Clinic should understand *args and **kwargs parameters

[INADA Naoki]

Changes by INADA Naoki :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29296] convert print() to METH_FASTCALL

[INADA Naoki]

INADA Naoki added the comment:

I see.  AC should support this.

--
dependencies: +Argument Clinic should understand *args and **kwargs parameters
resolution:  -> later
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29295] dict: Optimize PyDict_GetItemString()

[INADA Naoki]

INADA Naoki added the comment:

Close this issue for now, until profiler shows me PyDict_XString.

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29295] dict: Optimize PyDict_GetItemString()

[INADA Naoki]

INADA Naoki added the comment:

This patch checks passed C string is ascii or not.

But I don't want make dict complex too.  telco is more faster with issue29296.
Most common builtin functions are not METH_KEYWORDS when it merged.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29296] convert print() to METH_FASTCALL

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The performance of print() is not critical. It usually involves slow formatting 
and IO.

_PyArg_ParseStackAndKeywords() is a part of unstable, fast-evolved API. I would 
beware of using it in common code.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29295] dict: Optimize PyDict_GetItemString()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

It looks to me that PyDict_GetItemString(), PyObject_GetAttrString(), etc are 
mainly for backward compatibility and for using in performance non-critical 
code. Performance critical code caches string objects.

The only code that heavily used PyDict_GetItemString() was parsing keyword 
arguments in PyArg_ParseTupleAndKeywords(). But this API was replaced with more 
efficient _PyArg_ParseTupleAndKeywordsFast() and _PyArg_ParseStackAndKeywords() 
for internal use. I think something similar will be exposed as public API when 
it become enough mature. Issue29029 made PyArg_ParseTupleAndKeywords() much 
less using PyDict_GetItemString().

PyDict_GetItemString() can be used with non-ASCII C strings. They are decoded 
with UTF-8. The patch works incorrectly in this case.

I afraid that adding more and more specialized code in Objects/dictobject.c can 
slow down other functions in this file. And this makes the maintenance harder.

--
nosy: +haypo, rhettinger, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29296] convert print() to METH_FASTCALL

[INADA Naoki]

New submission from INADA Naoki:

Median +- std dev: [default] 24.2 ms +- 0.4 ms -> [patched] 19.2 ms +- 0.4 ms: 
1.26x faster (-21%)

--
files: print-fastcall.patch
keywords: patch
messages: 285632
nosy: haypo, inada.naoki
priority: normal
severity: normal
status: open
title: convert print() to METH_FASTCALL
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46314/print-fastcall.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29295] dict: Optimize PyDict_GetItemString()

[INADA Naoki]

New submission from INADA Naoki:

PyDict_GetItemString() is heavily used, especially from keyword argument 
parsing.
Current implementation creates temporary string for key object.
This patch avoid the temporary key string when passed C string is ASCII.

This benchmark is based on a8563ef0eb8a, so PyDict_GetItemString() calls for
parsing positional arguments is reduced already.


$ ../python -m perf compare_to -G --min-speed 2 default.json patched.json
Slower (1):
- scimark_lu: 430 ms +- 21 ms -> 446 ms +- 23 ms: 1.04x slower (+4%)

Faster (11):
- telco: 24.2 ms +- 0.4 ms -> 21.8 ms +- 0.7 ms: 1.11x faster (-10%)
- xml_etree_parse: 315 ms +- 17 ms -> 302 ms +- 14 ms: 1.04x faster (-4%)
- logging_simple: 31.6 us +- 0.3 us -> 30.4 us +- 0.3 us: 1.04x faster (-4%)
- mako: 41.6 ms +- 0.7 ms -> 40.3 ms +- 0.4 ms: 1.03x faster (-3%)
- logging_format: 36.5 us +- 0.3 us -> 35.5 us +- 0.4 us: 1.03x faster (-3%)
- float: 297 ms +- 4 ms -> 289 ms +- 4 ms: 1.03x faster (-3%)
- scimark_monte_carlo: 276 ms +- 10 ms -> 269 ms +- 7 ms: 1.02x faster (-2%)
- regex_effbot: 5.31 ms +- 0.37 ms -> 5.19 ms +- 0.06 ms: 1.02x faster (-2%)
- pickle_pure_python: 1.32 ms +- 0.02 ms -> 1.29 ms +- 0.02 ms: 1.02x faster 
(-2%)
- scimark_sor: 525 ms +- 9 ms -> 514 ms +- 8 ms: 1.02x faster (-2%)
- richards: 180 ms +- 3 ms -> 176 ms +- 2 ms: 1.02x faster (-2%)

Benchmark hidden because not significant (52): ...


Performance difference of telco is bit surprising.
Profiler shows the difference is from `print(t, file=outfil)` (here: 
https://github.com/python/performance/blob/master/performance/benchmarks/bm_telco.py#L79
 )

Until most common builtin functions are converted to FASTCALL, this patch has 
significant
performance gain.

--
components: Interpreter Core
files: dict_getitemascii.patch
keywords: patch
messages: 285631
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: dict: Optimize PyDict_GetItemString()
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46313/dict_getitemascii.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29294] ctypes.windll.LoadLibrary refuses unicode argument

[Thomas Heller]

Changes by Thomas Heller :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29294] ctypes.windll.LoadLibrary refuses unicode argument

[Thomas Heller]

Thomas Heller added the comment:

Yes, seems that it is fixed in the repository.  So I have to use 2.7.12
or wait for 2.7.14.  Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29294] ctypes.windll.LoadLibrary refuses unicode argument

[Xiang Zhang]

Xiang Zhang added the comment:

#29082 fixed it?

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29294] ctypes.windll.LoadLibrary refuses unicode argument

[Thomas Heller]

New submission from Thomas Heller:

ctypes.windll.LoadLibrary refuses unicode argument; this is a regression in 
Python 2.7.13:

Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import windll
>>> windll.LoadLibrary(u"kernel32.dll")
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python27\lib\ctypes\__init__.py", line 440, in LoadLibrary
return self._dlltype(name)
  File "C:\Python27\lib\ctypes\__init__.py", line 362, in __init__
self._handle = _dlopen(self._name, mode)
TypeError: LoadLibrary() argument 1 must be string, not unicode
>>>

It did work in Python 2.7.12 and before:

Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import windll
>>> windll.LoadLibrary(u"kernel32")

>>>

--
components: ctypes
messages: 285628
nosy: theller
priority: normal
severity: normal
status: open
title: ctypes.windll.LoadLibrary refuses unicode argument
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29291] Misleading text in the documentation of re library for non-greedy match

[Raymond Hettinger]

Raymond Hettinger added the comment:

I concur with Serhiy and Xiang that the docs are correct as-is.  Also IMO, the 
proposed rewording will cause more confusion than it cures.  Since this issue 
hasn't previously come up in the rather long life span of the re module, it 
suggests that most folks are getting the understanding they need from the 
existing docs.

I think it is the job for regex tutorials (rather than the main docs) to show 
when to use "<.*?>" versus "<[^>]*>".

--
nosy: +rhettinger
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29291] Misleading text in the documentation of re library for non-greedy match

[Xiang Zhang]

Xiang Zhang added the comment:

This doesn't look like a bug in the doc to me.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29293] Missing parameter "n" on multiprocessing.Condition.notify()

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
assignee:  -> davin
nosy: +davin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29293] Missing parameter "n" on multiprocessing.Condition.notify()

[Victor de la Fuente]

Victor de la Fuente added the comment:

Sorry for the typo, I meant:

#Replacing the call with condition.notify(n=2):
#TypeError: notify() got an unexpected keyword argument 'n'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29291] Misleading text in the documentation of re library for non-greedy match

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The documentation doesn't look incorrect to me. The non-greedy match doesn't 
fallback to the greedy match, it always matches as few characters as *possible* 
will be matched. For example a.match(" b  e ") matches 
" b ", not " b  e ".

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29293] Missing parameter "n" on multiprocessing.Condition.notify()

[Victor de la Fuente]

New submission from Victor de la Fuente:

Versions:
Darwin MacBook-Pro-de-Victor.local 16.3.0 Darwin Kernel Version 16.3.0: Thu Nov 
17 20:23:58 PST 2016; root:xnu-3789.31.2~1/RELEASE_X86_64 x86_64
Python 3.6.0 (v3.6.0:41df79263a11, Dec 22 2016, 17:23:13) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

import multiprocessing as mp

condition = mp.Condition()

with condition:
condition.notify(2) #Docs show notify(n=1)
#TypeError: notify() takes 1 positional argument but 2 were given

#Replacing the call with condition.notify(2):
#TypeError: notify() got an unexpected keyword argument 'n'


# Docs show signature: def notify(self, n=1)
# But found this debugging:
 
# multiprocessing/synchronize.py
# line 211: class Condition(object):
# .
# line 271: def notify(self): #<-- ¿There is no n paramater?
# .

--
components: Library (Lib)
messages: 285623
nosy: Victor de la Fuente
priority: normal
severity: normal
status: open
title: Missing parameter "n" on multiprocessing.Condition.notify()
type: compile error
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29292] Missing a parameter in PyEval_EvalCodeEx doc

[Xiang Zhang]

Changes by Xiang Zhang :


--
keywords: +easy
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29029] Faster positional arguments parsing in PyArg_ParseTupleAndKeywords

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thank you for your reviews Inada and Victor. Thank you for rebasing the patch 
Victor.

I were going first to try an alternative patch, maybe less efficient, but 
allowing to share more code. But since the code is changed so fast, it is 
better to commit the ready patch.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29029] Faster positional arguments parsing in PyArg_ParseTupleAndKeywords

[Roundup Robot]

Roundup Robot added the comment:

New changeset a8563ef0eb8a by Serhiy Storchaka in branch 'default':
Issue #29029: Speed up processing positional arguments in
https://hg.python.org/cpython/rev/a8563ef0eb8a

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29292] Missing a parameter in PyEval_EvalCodeEx doc

[Xiang Zhang]

New submission from Xiang Zhang:

The signature of PyEval_EvalCodeEx now gets a "PyObject *kwdefs" parameter but 
the doc doesn't mention it.

--
assignee: docs@python
components: Documentation
messages: 285620
nosy: docs@python, xiang.zhang
priority: normal
severity: normal
status: open
title: Missing a parameter in PyEval_EvalCodeEx doc
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29291] Misleading text in the documentation of re library for non-greedy match

[ipolcak]

New submission from ipolcak:

The text about non-greedy match in the documentation for re library is 
misleading.

The docs for py2.7 (https://docs.python.org/2.7/library/re.html) and 3.6 
(https://docs.python.org/3.6/library/re.html) says: "The '*', '+', and '?' 
qualifiers are all greedy; they match as much text as possible. Sometimes this 
behaviour isn’t desired; if the RE <.*> is matched against  b , it will 
match the entire string, and not just . Adding ? after the qualifier makes 
it perform the match in non-greedy or minimal fashion; as few characters as 
possible will be matched. Using the RE <.*?> will match only ."

The docs for py3.4 (https://docs.python.org/3.4/library/re.html) offers a 
little bit different example:
"The '*', '+', and '?' qualifiers are all greedy; they match as much text as 
possible. Sometimes this behaviour isn’t desired; if the RE <.*> is matched 
against 'title', it will match the entire string, and not just ''. 
Adding '?' after the qualifier makes it perform the match in non-greedy or 
minimal fashion; as few characters as possible will be matched. Using .*? in 
the previous expression will match only ''."

However, in reality if the non-greedy match is not successful, it might 
fallback to the greedy match, see:

>>> import re
>>> a = re.compile(r"<.*?>")
>>> a.match(" b ")
<_sre.SRE_Match object; span=(0, 15), match=' b '>
>>> a.search(" b ")
<_sre.SRE_Match object; span=(0, 15), match=' b '>

So the '<.*?>' part of the regex matches ' b ' in this example. I propose 
to add to the documentation the following text:

"However, note that even the non-greedy version can match additional text, for 
example consider the RE '(<.*>)' to be matched against ' b '. The 
match is successful and the unnamed group contains ' b '."

--
assignee: docs@python
components: Documentation
messages: 285619
nosy: docs@python, ipolcak
priority: normal
severity: normal
status: open
title: Misleading text in the documentation of re library for non-greedy match
type: behavior
versions: Python 2.7, Python 3.4, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28518] execute("begin immediate") throwing OperationalError

[Ma Lin]

Ma Lin added the comment:

some comments on Aviv Palivoda's patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com