date:20200109



Change by Karthikeyan Singaravelan :


--
Removed message: https://bugs.python.org/msg359706

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32545] Unable to install Python 3.7.0a4 on Windows 10 - Error 0x80070643: Failed to install MSI package.

2020-01-09 Thread printerhelp center



printerhelp center  added the comment:

As of late, numerous clients have griped concerning the sibling printer 
disconnected issue as it goes detached naturally or after some idle time which 
ranges from 10 to 30 minutes. This issue makes inconvenience as clients are not 
ready to print pages because the Printer itself will be recorded as 
disconnected. Furthermore, when you coordinated to write at precisely that 
point, the reports will be added to the line. Brother printer offline problem 
was held because of many explanations which are referenced underneath. To 
realize the statement in regards to that, please guarantee you will peruse the 
post totally without getting away from any means.

https://printerhelpcenter.com/how-to-fix-canon-printer-error-b200/
https://printerhelpcenter.com/fix-brother-printer-offline-to-online/

--
nosy: +printerhelpcenter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34311] locale.format() and locale.format_string() cast Decimals to float

2020-01-09 Thread Inada Naoki



Inada Naoki  added the comment:

Does the name "locale.localize" have some origin?

I feel the name doesn't represent well about it is for formatting numeric based 
on LC_NUMERIC.

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39283] Add ability to inherit unittest arguement parser

2020-01-09 Thread Ajay Tripathi



New submission from Ajay Tripathi :

I am currently writing a unittest script that requires argparser but since the 
unittest module already has a ArgumentParser instance, I cannot create and use 
my own ArgumentParser instance.


Possible solution:

The problem would be solved I could inherit the parent ArgumentParser instance 
created here:
https://github.com/python/cpython/blob/master/Lib/unittest/main.py#L162

Please let me know if it's feasible / acceptable to change `parent_parser` as 
`self.parent_parent` for inheritance.

I would love to create a pull request for it.

--
components: Library (Lib)
messages: 359704
nosy: atb00ker
priority: normal
severity: normal
status: open
title: Add ability to inherit unittest arguement parser
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38973] Shared Memory List Returns 0 Length

2020-01-09 Thread Derek Frombach



Derek Frombach  added the comment:

Documentation on Global Descriptor Lock needs re-writing.  Found documentation 
in version 2, much more helpful.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

2020-01-09 Thread Senthil Kumaran



Change by Senthil Kumaran :


--
nosy: +orsenthil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

2020-01-09 Thread Tim Peters



Tim Peters  added the comment:

Let's stir this up a bit ;-)  I recently had occasion to exchange ideas with 
Larry Hastings about topsorts for use in a package manager he's writing.  I 
thought his API for adding edges was ... perfect:

add(node, *dependson)

So, e.g., add(A, B, C) says A depends on B, and on C, but says nothing else 
about B and C.  This is almost always the way topsorts show up in real life:  
you know what a thing depends *on* directly, but have scant idea how things may 
be in the opposite direction.  For example, you know that baking a cake 
requires (among other things) flour, but have no real idea of the universe of 
other things that require flour.  Likewise Larry knows which packages each 
package requires, but not the reverse.  Etc.

Nodes with no edges are trivial to add then:  add(A).

If you're building input to a topsort from a graph, also trivial:

for n, p in node2predecessors.items():
topsort_instance.add(n, *p)

and it doesn't matter whether the predecessors in the original graph were 
stored in a list, set, tuple, or any other iterable container.  Nothing special 
about an empty collection of predecessors either.

The other big thing that came up is that most topsort programs were useless for 
his goal:  downloading and installing packages takes significant wall clock 
time, and there's huge opportunity for exploiting parallelism.  But a flat 
sequence in topsort order gives no clue about what _can_ be done in parallel.  
Instead you really want several methods, like

prepare()

to say that you're done building the graph; and,

get_ready()

to get all nodes ready to go, which haven't already been returned by 
get_ready() calls (initially, this is the collection of nodes with no 
predecessors, which prepare() can know); and,

done(node)

to say that `node` (returned by a previous call to get_ready()) is finished 
now, so that the next call to get_ready() can return all (if any) nodes for 
which `node` was the last non-done predecessor; and,

is_active()

to say whether the topsort can make more progress (is_active() returns True iff 
there are still nodes ready to go that haven't yet been passed out by 
get_ready(), or if the number of nodes marked done() is less than the number 
that have been passed out by get_ready()).

These are all easy to code, and allow the user to extract all the opportunities 
for parallelism that theoretically exist.  There is no static order that can do 
so, since the opportunities that exist at any time depend on the times and 
order in which nodes are marked done() in real life - and that may vary from 
one run to the next.

Of course a deterministic static order can be derived from those, like, e.g.,

def static_order(self):
self.prepare()
while self.is_active():
for node in self.get_ready():
yield node
self.done(node)

For parallel use, e.g.,

self.prepare()
while instance.is_active():
for node in instance.get_ready():
inq.put(node)
node = outq.get()
instance.done(node)

where worker threads or processes take nodes to work on off of queue `inq`, 
then, when the work for a node is done, put the node on queue `outq`.

Am I seriously suggesting this for Python?  Sure.  It's fun to advance the 
practical state of the art :-)

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39282] python-config --embed documentation

2020-01-09 Thread Milo D Kerr



Change by Milo D Kerr :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39282] python-config --embed documentation

2020-01-09 Thread Milo D Kerr



New submission from Milo D Kerr :

Update 3.8 embedding documentation to reflect changes in PR 13500
bpo 36721

--
messages: 359701
nosy: M.Kerr
priority: normal
severity: normal
status: open
title: python-config --embed documentation
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22213] Make pyvenv style virtual environments easier to configure when embedding Python

2020-01-09 Thread Milo D Kerr



Change by Milo D Kerr :


--
nosy: +M.Kerr

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39281] The CO_NESTED flag is associated with a significant performance cost

2020-01-09 Thread Dan Snider



New submission from Dan Snider :

The title was carefully worded as I have no idea how or why what is happening 
is happening, only that it has been like this since a least 3.6.0. That version 
in particular, by the way, is able to execute a call to a python function with 
1 argument 25% faster than 3.8.0 but that may be due at least in part by 
whatever it is that makes it much faster to a call a unary function wrapped by 
functools.partial by utilizing the subcript operator on an instance of a 
partial subtype whose __getitem__ has been set to the data descriptor 
partial.func... Eg:

class Party(partial): __getitem__ = partial.func
fast = Party(hash)
slow = partial(hash)

# the expression `fast[""]` runs approximately 28% faster than
# the expression `slow("")`, and partial.func as __getitem__ is
# confusingly 139% faster than partial.__call__...

That rather large digression aside, here's a demonstration of two functions 
identical in every way except the CO_NESTED bit and perhaps the names:

if 1:
def Slow():
global Slow
class  Slow:
global slow
def slow(self): return self
return Slow
if  Slow():
class Fast:
global fast
def fast(self): return self
import dis
dis.show_code(slow)
print()
dis.show_code(fast)

--
messages: 359700
nosy: bup
priority: normal
severity: normal
status: open
title: The CO_NESTED flag is associated with a significant performance cost
type: performance
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37224] test__xxsubinterpreters fails randomly

2020-01-09 Thread Kyle Stanley



Kyle Stanley  added the comment:

> For a struct-specific getter we usually end the prefix with an
underscore: _PyInterpreter_IsFinalizing.  Otherwise, that's the same
name I would have used. :)

Good to know, thanks!

> No worries (or hurries).  Just request a review from me when you're
ready.  Thanks again for working on this!

No problem, and thanks for your continued patience. I've been a bit 
additionally busy recently with the holidays and preparing for an upcoming 
technical interview w/ Amazon, so I didn't have time to thoroughly test my 
proposed fix. This would potentially be my first full-time job in the Software 
Engineering industry. As a result, I have a lot to prepare for.

When I get the chance to open a PR after ensuring it at least 100% fixes the 
failure I was targeting, I'll be sure to add you as a reviewer. 

If any deadline comes up for this, feel free to move forward with implementing 
any part of my suggestions (or an entirely different fix that resolves the 
other failures at the same time). I should have time to work on this in the 
near future though.

Thanks again for the guidance and patience with this! I've learned a lot about 
the C-API and C in general (which has been a weaker area for me) from my time 
spent working on the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39280] Don't allow datetime parsing to accept non-Ascii digits



Ram Rachum  added the comment:

> To me, this seems like a pretty thin justification for calling this a 
> security vulnerability.

I know that lots of exploits were made because of bugs in Python's URL parsing, 
and security releases of Python were made because of that. It's possible that 
similar exploits could be done for bugs in parsing datetimes. It's different, 
and it's a thin justification, I agree, so if it's treated as a non-security 
bug, I'm okay with that. 


> ISO 8601 specifies dozens of ways to represent the same datetime

Yes, but not within the same format. If someone were to choose the format 
'2014-04-10T24:00:00', they would have a reasonable expectation that there is 
only one unique string that corresponds with that datetime.

> As far as the inconsistency, I think that's an argument for being less 
> strict, not more, and allowing non-ASCII digits in more places not just the 
> first. Why shouldn't (let's say) a Bengali user specify the day of the month 
> using Bengali digits?

That's an interesting direction. It might be a good thing to allow users in 
different locales to specify datetime with their native digits. But, if we were 
to go in that direction, we would have to: 

1. Define that use case more rigorously, consider whether it has any effects we 
need to consider, like clashes with how the local culture expresses dates. 
Maybe we accept the year 2020 in Elbonian digits, but the Elbonian people count 
their years from the birth of Saint Elbon, and for them the year 2020 in 
Elbonian digits means -1553 BC for us?
2. Enable it in other fields except year.
3. Add tests for it. 

Since we're not going to do that ever, I don't think there's a point in leaving 
a 10% implementation of non-Ascii datetime parsing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39280] Don't allow datetime parsing to accept non-Ascii digits

2020-01-09 Thread Steven D'Aprano



Steven D'Aprano  added the comment:

> If user code were to check for uniqueness of a datetime by comparing it as a 
> string, this is where an attacker could fool this logic, by using a non-Ascii 
> digit.

To me, this seems like a pretty thin justification for calling this a security 
vulnerability.

Using the exact same reasoning, one could argue that "If user code were to 
check for uniqueness of a float by comparing it as a string, this is where an 
attacker could fool this logic, by using leading or trailing spaces, extra 
non-significant digits, upper- or lowercase 'E', etc."

py> float(" +00012.14509e00 ") == float("12.145")
True

Referring specifically to strptime(), there are many format codes which break 
uniqueness by allowing optional leading zeroes, and month names are case 
insensitive e.g. %b accepts 'jAn' as well as 'Jan'.

https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior

As far as the inconsistency, I think that's an argument for being less strict, 
not more, and allowing non-ASCII digits in more places not just the first. Why 
shouldn't (let's say) a Bengali user specify the day of the month using Bengali 
digits?

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39280] Don't allow datetime parsing to accept non-Ascii digits

2020-01-09 Thread Paul Ganssle

Paul Ganssle added the comment:

I don't love the inconsistency, but can you elaborate on the actual *danger*
posed by this? What security vulnerabilities involve parsing a datetime using a
non-ascii digit?

The reason that `fromisoformat` doesn't accept non-ASCII digits is actually
because it's the inverse of `datetime.isoformat`, which never *emits* non-ASCII
digits. For `strptime`, we're really going more for a general specifier for
parsing datetime strings in a given format. I'll note that we do accept any
valid unicode character for the date/time separator.

>From my perspective, there are a few goals, some of which may be in conflict
>with the others:

1. Mitigating security vulnerabilities, if they exist.
2. Supporting international locales if possible.
3. Improving consistency in the API.

If no one ever actually specifies datetimes in non-ascii locales (and this
gravestone that includes the date in both Latin and Chinese/Japanese characters
seems to suggest otherwise:
https://jbnewall.com/wp-content/uploads/2017/02/LEE-MONUMENT1-370x270.jpg ),
then I don't see a problem dropping our patchy support, but I think we need to
carefully consider the backwards compatibility implications if we go through
with that.

One bit of evidence in favor of "no one uses this anyway" is that no one has
yet complained that apparently this doesn't work for "%d" even if it works for
"%y", so presumably it's not heavily used. If our support for this sort of
thing is so broken that no one could possibly be using it, I suppose we may as
well break it all the way, but it would be nice to try and identify some
resources that the documentation can point to for how to handle international
date parsing.

> Note the "unique and unambiguous". By accepting non-Ascii digits, we're
> breaking the uniqueness requirement of ISO 8601.

I think in this case "but the standard says X" is probably not a very strong
argument.
Even if we were coding to the ISO 8601 standard (I don't think we claim to be,
we're just using that convention), I don't really know how to interpret the
"unique" portion of that claim, considering that ISO 8601 specifies dozens of
ways to represent the same datetime. Here's an example from [my
`dateutil.parse.isoparse` test
suite](https://github.com/dateutil/dateutil/blob/110a09b4ad46fb87ae858a14bfb5a6b92557b01d/dateutil/test/test_isoparser.py#L150):

```
'2014-04-11T00',
'2014-04-10T24',
'2014-04-11T00:00',
'2014-04-10T24:00',
'2014-04-11T00:00:00',
'2014-04-10T24:00:00',
'2014-04-11T00:00:00.000',
'2014-04-10T24:00:00.000',
'2014-04-11T00:00:00.00',
'2014-04-10T24:00:00.00'
```

All of these represent the exact same moment in time, and this doesn't even get
into using the week-number/day-number configurations or anything with time
zones. They also allow for the use of `,` as the subsecond-component separator
(so add 4 more variants for that) and they allow you to leave out the dashes
between the date components and the colons between time components, so you can
multiply the possible variants by 4.

Just a random aside - I think there may be strong arguments for doing this even
if we don't care about coding to a specific standard.

--
nosy: +p-ganssle

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39280] Don't allow datetime parsing to accept non-Ascii digits



Change by Ram Rachum :


--
keywords: +patch
pull_requests: +17337
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17931

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39279] Don't allow non-Ascii digits in platform.py



Change by Ram Rachum :


--
pull_requests: +17336
pull_request: https://github.com/python/cpython/pull/17931

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39280] Don't allow datetime parsing to accept non-Ascii digits


New submission from Ram Rachum :

I've been doing some research into the use of `\d` in regular expressions in 
CPython, and any security vulnerabilities that might happen as a result of the 
fact that it accepts non-Ascii digits like ٢ and ５.

In most places in the CPython codebase, the `re.ASCII` flag is used for such 
cases, thus ensuring the `re` module prohibits these non-Ascii digits. 
Personally, my preference is to never use `\d` and always use `[0-9]`. I think 
that it's rule that's more easy to enforce and less likely to result in a 
slipup, but that's a matter of personal taste.

I found a few places where we don't use the `re.ASCII` flag and we do accept 
non-Ascii digits.

The first and less interesting place is platform.py, where we define patterns 
used for detecting versions of PyPy and IronPython. I don't know how anyone 
would exploit that, but personally I'd change that to a [0-9] just to be safe. 
I've opened bpo-39279 for that. 

The more sensitive place is the `datetime` module. 

Happily, the `datetime.datetime.fromisoformat` function rejects non-Ascii 
digits. But the `datetime.datetime.strptime` function does not: 

from datetime import datetime

time_format = '%Y-%m-%d'
parse = lambda s: datetime.strptime(s, time_format)
   
x = '٢019-12-22'
y = '2019-12-22'
assert x != y
assert parse(x) == parse(y)
print(parse(x))
# Output: 2019-12-22 00:00:00

If user code were to check for uniqueness of a datetime by comparing it as a 
string, this is where an attacker could fool this logic, by using a non-Ascii 
digit.

Two more interesting points about this: 

1. If you'd try the same trick, but you'd insert ٢ in the day section instead 
of the year section, Python would reject that. So we definitely have 
inconsistent behavior.
2. In the documentation for `strptime`, we're referencing the 1989 C standard. 
Since the first version of Unicode was published in 1991, it's reasonable not 
to expect the standard to support digits that were introduced in Unicode.

If you'd scroll down in that documentation, you'll see that we also implement 
the less-known ISO 8601 standard, where `%G-%V-%u` represents a year, week 
number, and day of week. The `%G` is vulnerable:

from datetime import datetime

time_format = '%G-%V-%u'
parse = lambda s: datetime.strptime(s, time_format)
   
x = '٢019-53-4'
y = '2019-53-4'
assert x != y
assert parse(x) == parse(y)
print(parse(x))
# Output: 2020-01-02 00:00:00

I looked at the ISO 8601:2004 document, and under the "Fundamental principles" 
chapter, it says:

This International Standard gives a set of rules for the representation of
time points
time intervals
recurring time intervals.
Both accurate and approximate representations can be identified by 
means of unique and unambiguous expressions specifying the relevant dates, 
times of day and durations.  

Note the "unique and unambiguous". By accepting non-Ascii digits, we're 
breaking the uniqueness requirement of ISO 8601.

--
components: Library (Lib)
messages: 359695
nosy: cool-RR
priority: normal
severity: normal
status: open
title: Don't allow datetime parsing to accept non-Ascii digits
type: security
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39279] Don't allow non-Ascii digits in platform.py



Change by Ram Rachum :


--
keywords: +patch
pull_requests: +17335
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17928

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39279] Don't allow non-Ascii digits in platform.py


New submission from Ram Rachum :

The platform.py module takes non-Ascii digits in regexes in places it 
shouldn't. e.g. digits like ٢ and ５ and accepted, when only the Ascii digits 
between 0-9 should be accepted.

--
components: Library (Lib)
messages: 359694
nosy: cool-RR
priority: normal
severity: normal
status: open
title: Don't allow non-Ascii digits in platform.py
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39235] Generator expression has wrong line/col info when inside a Call object



miss-islington  added the comment:


New changeset 33e033da3c1472b0aa2ae3cff06649a1ae4aa37f by Miss Islington (bot) 
in branch '3.8':
bpo-39235: Fix end location for genexp in call args (GH-17925)
https://github.com/python/cpython/commit/33e033da3c1472b0aa2ae3cff06649a1ae4aa37f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39235] Generator expression has wrong line/col info when inside a Call object



Change by miss-islington :


--
pull_requests: +17334
pull_request: https://github.com/python/cpython/pull/17927

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39235] Generator expression has wrong line/col info when inside a Call object



miss-islington  added the comment:


New changeset a796d8ef9dd1af65f7e4d7a857b56f35b7cb6e78 by Miss Islington (bot) 
(Guido van Rossum) in branch 'master':
bpo-39235: Fix end location for genexp in call args (GH-17925)
https://github.com/python/cpython/commit/a796d8ef9dd1af65f7e4d7a857b56f35b7cb6e78


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39235] Generator expression has wrong line/col info when inside a Call object

2020-01-09 Thread Serhiy Storchaka



Change by Serhiy Storchaka :


--
pull_requests: +17333
pull_request: https://github.com/python/cpython/pull/17926

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39275] Traceback off by one line when



Karthikeyan Singaravelan  added the comment:

This looks similar to issue35405 . See also issue16482 with a patch.

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26495] super() does not work in nested functions, genexps, listcomps, and gives misleading exceptions

2020-01-09 Thread Josh Lee



Change by Josh Lee :


--
nosy: +jleedev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39235] Generator expression has wrong line/col info when inside a Call object

2020-01-09 Thread Guido van Rossum



Change by Guido van Rossum :


--
keywords: +patch
pull_requests: +17332
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17925

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39235] Generator expression has wrong line/col info when inside a Call object

2020-01-09 Thread Guido van Rossum



Guido van Rossum  added the comment:

OK, so I looked into this (sine we can't wait for Serhiy).

The problem was probably introduced when Ivan added end lineno/column 
attributes to parse tree nodes. The offending copy_location() call copies the 
begin and end locations from the given node (maybegenbeg). But in the 
problematic case, maybegenbeg is just the left paren in the function call, not 
the node representing the surrounding () group.

But there's a separate argument to ast_for_call(), closepar, that should be 
used for the end lineno/offset.

Looking at the various calls to ast_for_call() in Python/ast.c, we find these:

- ast_for_decorator -- passes the open and close parentheses- ast_for_trailer 
-- passes the open and close parentheses
- ast_for_classdef -- passes NULL (because no genexpr allowed here)

So instead of calling copy_location() we should manually set the begin and end 
positions.

An alternative would be to add a second node* argument to copy_location(). That 
would require us to modify all 5 calls to it, but it still feels the better way 
to go. I'll see if I can produce a PR for that.

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39278] add docstrings to functions in pdb module

2020-01-09 Thread Carl Bordum Hansen



Change by Carl Bordum Hansen :


--
keywords: +patch
pull_requests: +17331
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17924

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25172] Unix-only crypt should not be present on Windows.

2020-01-09 Thread Steve Dower



Change by Steve Dower :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39278] add docstrings to functions in pdb module

2020-01-09 Thread Carl Bordum Hansen



New submission from Carl Bordum Hansen :

The functions are documented, but not in doc strings which means you cannot 
call help() on them.

>From this twitter thread: 
>https://twitter.com/raymondh/status/1211414561468952577

--
assignee: docs@python
components: Documentation
messages: 359689
nosy: carlbordum, docs@python
priority: normal
severity: normal
status: open
title: add docstrings to functions in pdb module
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25172] Unix-only crypt should not be present on Windows.



miss-islington  added the comment:


New changeset 8c08518c255747a06d00479f21087f0c934d0ad6 by Miss Islington (bot) 
in branch '3.8':
bpo-25172: Reduce scope of crypt import tests (GH-17881)
https://github.com/python/cpython/commit/8c08518c255747a06d00479f21087f0c934d0ad6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39276] type() cause segmentation fault in callback function called from C extension

2020-01-09 Thread Pablo Galindo Salgado



Change by Pablo Galindo Salgado :


--
Removed message: https://bugs.python.org/msg359687

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39276] type() cause segmentation fault in callback function called from C extension

2020-01-09 Thread Pablo Galindo Salgado



Pablo Galindo Salgado  added the comment:

static PyTypeObject InternalType should have

PyVarObject_HEAD_INIT(&PyType_Type, 0) 

instead of

PyVarObject_HEAD_INIT(NULL, 0)

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25172] Unix-only crypt should not be present on Windows.

2020-01-09 Thread Steve Dower



Steve Dower  added the comment:


New changeset ed367815eeb9329c48a86a8a7fa3186e27a10f2c by Steve Dower in branch 
'master':
bpo-25172: Reduce scope of crypt import tests (GH-17881)
https://github.com/python/cpython/commit/ed367815eeb9329c48a86a8a7fa3186e27a10f2c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25172] Unix-only crypt should not be present on Windows.



Change by miss-islington :


--
pull_requests: +17330
pull_request: https://github.com/python/cpython/pull/17922

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1681671] Python and Indeterminate Forms (Math)

2020-01-09 Thread Julien Palard



Change by Julien Palard :


--
nosy: +mdk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39150] See if PyToken_OneChar would be faster as a lookup table

2020-01-09 Thread Andy Lester



Andy Lester  added the comment:

Re: branch prediction.

The branch

if (c1>=37 && c1<=126)

could just as easily be

if (c1>=0 && c1<=126)

with no other changes to the code.  It could be just

if (c1<=126)

if c1 wasn't signed.

As far as branch prediction, we could make it something like

if (c1>=0 && c1<=255)

and expand the table lookup, depending on what the likely inputs are.  I could 
play around with that some if anyone was interested.

I'm not trying to push on this.  Just sharing some thoughts and research.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39150] See if PyToken_OneChar would be faster as a lookup table

2020-01-09 Thread Andy Lester



Andy Lester  added the comment:

Yes, I ran it multiple times on my 2013 Macbook Pro and got ~10% speedup.  I 
also ran it on my Linux VM (that I only use for development) and got a speedup 
but less so.

The code I used to run the tests is at: 
https://github.com/petdance/cpython/blob/Issue39150/timetests

LOG=timelog.txt
NRUNS=100
NFILES=100

if [ -x ./python.exe ] ; then
PYTHON=./python.exe
else
PYTHON=./python
fi

# Build a file list, but throw out some that have syntax errors.
# Use the 100 biggest files.
FILES=$(
find . -name '*.py' -type f -ls \
| grep -v Tools/test2to3/ \
| grep -v Lib/lib2to3/tests/ \
| grep -v Lib/test/badsyntax \
| grep -v Lib/test/bad_coding \
| sort -r -n -k7 \
| awk '{print $11}' \
| head -n $NFILES
)

echo "Compiling $NFILES files for $NRUNS iterations"

rm -f $LOG

for (( i=1; i<=$NRUNS; i++ )) do
echo "Run $i"
{ time $PYTHON -m py_compile $FILES; } >> $LOG 2>&1
done

perl -ne'if(/real\s+0m([\d.]+)s/){$t+=$1;$n++} END { printf "%d runs, average 
%0.4f\n", $n, $t/$n}' $LOG

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39277] _PyTime_FromDouble() fails to detect an integer overflow when converting a C double to a C int64_t

2020-01-09 Thread Thomas Grainger



Change by Thomas Grainger :


--
versions:  -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39277] _PyTime_FromDouble() fails to detect an integer overflow when converting a C double to a C int64_t



STINNER Victor  added the comment:

It's a bug in _PyTime_FromDouble() which fails to detect the integer overflow 
when casting a C double to a C _PyTime_T (int64_t): bug in 
_Py_InIntegralTypeRange(_PyTime_t, d) where d is a C double.

On my Fedora 31, double is a 64-bit IEEE 754 float, _PyTime_t is int64_t 
(64-bit signed integer).

_PyTime_t maximum is 9223372036854775807. But casted as C double, it becomes 
9223372036854775808:

>>> int(float(9223372036854775807))
9223372036854775808

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39277] _PyTime_FromDouble() fails to detect an integer overflow when converting a C double to a C int64_t

2020-01-09 Thread Thomas Grainger



New submission from Thomas Grainger :

_PyTime_FromDouble() fails to detect an integer overflow when converting a C 
double to a C int64_t

Python 3.7.5 (default, Nov 20 2019, 09:21:52) 
[GCC 9.2.1 20191008] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.sleep(9223372036.854777)
Traceback (most recent call last):
  File "", line 1, in 
ValueError: sleep length must be non-negative

--
messages: 359682
nosy: graingert, vstinner
priority: normal
severity: normal
status: open
title: _PyTime_FromDouble() fails to detect an integer overflow when converting 
a C double to a C int64_t
type: behavior
versions: Python 2.7, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39274] Conversion from fractions.Fraction to bool

2020-01-09 Thread Sebastian Berg



Sebastian Berg  added the comment:

Thanks for the quick responses.

@Victor Stinner, I suppose you could change `numbers.Complex.__bool__()` by 
adding the no-op bool to make it: `bool(self != 0)`.

But I am not sure I feel it is necessary. NumPy is a bit a strange in that it 
uses its own boolean scalar, and it is easy to override the slot for such 
objects.

--
nosy: +seberg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39276] type() cause segmentation fault in callback function called from C extension

2020-01-09 Thread Paweł Karczewski


New submission from Paweł Karczewski :

How to reproduce:

1. Create callback function, which may take any object and run type() on it
  def builtin_type_in_callback(obj):
  type(obj)

2. Create C extension with two types defined in it - Internal and External. 
  Eternal type should implement method (let's name it Call), which can get 
callback function

static PyObject *
Call(ExternalObject *self, PyObject* args) {
PyObject* python_callback;
if (!PyArg_ParseTuple(args, "O:set_callback",  
&python_callback)) {
return NULL;
}
callback_runner(python_callback);
if(PyErr_Occurred() != NULL)
return NULL;
Py_RETURN_NONE;
}  
  Inside this function create object of Internal type and pass it to callback 
function
void callback_runner(void* callback_function)  {
InternalObject *entry = PyObject_New(InternalObject, 
&InternalType);
PyObject_Init((PyObject*)entry, &InternalType);
PyObject *args = PyTuple_New(1);
if (args != NULL) {
if (PyTuple_SetItem(args, 0, (PyObject *)entry) == 0) {
PyObject *res = PyObject_CallObject((PyObject 
*) callback_function, args);
Py_XDECREF(res);
}
}

When type() is called on object of Internal type segmentation fault occur. 
However, if dir() was called
on such object before type(), type() works properly and returns type of 
Internal Object.

For more details please look into reproducer code.

--
components: C API
files: cpython_type_segfaulter.tgz
messages: 359680
nosy: karczex
priority: normal
severity: normal
status: open
title: type() cause segmentation fault  in callback function called from C 
extension
versions: Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48834/cpython_type_segfaulter.tgz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39274] Conversion from fractions.Fraction to bool



STINNER Victor  added the comment:

> ``return bool(a._numerator)`` does seem like the obvious way to do this.

Yes, it's a good fix. It's used on other places.

--

numbers.Complex.__bool__() also uses "self != 0". Is it an issue?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39274] Conversion from fractions.Fraction to bool

2020-01-09 Thread Mark Dickinson



Mark Dickinson  added the comment:

For completeness and to save people going to the NumPy tracker, here's an 
example of the problem:

Python 3.8.1 (default, Jan  5 2020, 21:32:35) 
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import fractions, numpy
>>> f = fractions.Fraction(numpy.int64(1), 2)
>>> not f
Traceback (most recent call last):
  File "", line 1, in 
TypeError: __bool__ should return bool, returned numpy.bool_

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39275] Traceback off by one line when

2020-01-09 Thread Facundo Batista



New submission from Facundo Batista :

When using pdb to debug, the traceback is off by one line.

For example, this simple script:

```
print("line 1")
import pdb;pdb.set_trace()
print("line 2")
print("line 3", broken)
print("line 4")
```

...when run produces the following traceback (after hitting 'n' in pdb, of 
course):

```
Traceback (most recent call last):
  File "/home/facundo/foo.py", line 3, in 
print("line 2")
NameError: name 'broken' is not defined
```

--
messages: 359678
nosy: facundobatista
priority: normal
severity: normal
status: open
title: Traceback off by one line when
versions: Python 3.6, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37539] CheckCommitCursorReset regression sqlite3 test fails with old sqlite3

2020-01-09 Thread Matej Cepl



Matej Cepl  added the comment:

Fix for this situations (and we are talking about sqlite3 3.6.4, which is truly 
archaeological excavation) skip particular test.

--
keywords: +patch
Added file: https://bugs.python.org/file48833/avoid-sqlite-race.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39274] Conversion from fractions.Fraction to bool

2020-01-09 Thread Mark Dickinson



Mark Dickinson  added the comment:

Agreed that __bool__ should return an actual bool.
``return bool(a._numerator)`` does seem like the obvious way to do this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39274] Conversion from fractions.Fraction to bool



Change by Karthikeyan Singaravelan :


--
nosy: +mark.dickinson, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39274] Conversion from fractions.Fraction to bool

2020-01-09 Thread François Durand


François Durand  added the comment:

As of now, fractions.Fraction.__bool__ is implemented as: ``return a._numerator 
!= 0``. However, this does not necessary return a bool (which would be 
desired). In particular, when the numerator is a numpy integer, this returns a 
numpy bool instead. Another solution would be to implement 
fractions.Fraction.__bool__ as: ``return bool(a._numerator)``. What do you 
think?

This message follows a thread here: https://github.com/numpy/numpy/issues/15277 
.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39274] Conversion from fractions.Fraction to bool

2020-01-09 Thread François Durand


New submission from François Durand :

As of now, fractions.Fraction.__bool__ is implemented as: ``return a._numerator 
!= 0``. However, this does not necessary return a bool (which would be 
desired). In particular, when the numerator is a numpy integer, this returns a 
numpy bool instead. Another solution would be to implement 
fractions.Fraction.__bool__ as: ``return bool(numerator)``. What do you think?

This message follows a thread here: https://github.com/numpy/numpy/issues/15277 
.

--
messages: 359673
nosy: francois-durand
priority: normal
severity: normal
status: open
title: Conversion from fractions.Fraction to bool
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

2020-01-09 Thread Gareth Rees



Gareth Rees  added the comment:

I'd like to push back on the idea that graphs with isolated vertices are 
"unusual cases" as suggested by Raymond.

A very common use case (possibly the most common) for topological sorting is 
job scheduling. In this use case you have a collection of jobs, some of which 
have dependencies on other jobs, and you want to output a schedule according to 
which the jobs can be executed so that each job is executed after all its 
dependencies.

In this use case, any job that has no dependencies, and is not itself a 
dependency of any other job, is an isolated vertex in the dependency graph. 
This means that the proposed interface (that is, the interface taking only 
pairs of vertices) will not be suitable for this use case. Any any programmer 
who tries to use it for this use case will be setting themselves up for failure.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39161] Py_NewInterpreter docs need updating for multi-phase initialization

2020-01-09 Thread Petr Viktorin



Change by Petr Viktorin :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39161] Py_NewInterpreter docs need updating for multi-phase initialization



miss-islington  added the comment:


New changeset 20c990229e98ad69c03e44fe61f8dce99b96cf9d by Miss Islington (bot) 
in branch '3.8':
bpo-39161: Document multi-phase init modules under Py_NewInterpreter() 
(GH-17896)
https://github.com/python/cpython/commit/20c990229e98ad69c03e44fe61f8dce99b96cf9d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39161] Py_NewInterpreter docs need updating for multi-phase initialization



Change by miss-islington :


--
pull_requests: +17329
pull_request: https://github.com/python/cpython/pull/17918

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39161] Py_NewInterpreter docs need updating for multi-phase initialization



miss-islington  added the comment:


New changeset 6c5d661342d12f6836580b0e75e3569c764527ae by Miss Islington (bot) 
(Petr Viktorin) in branch 'master':
bpo-39161: Document multi-phase init modules under Py_NewInterpreter() 
(GH-17896)
https://github.com/python/cpython/commit/6c5d661342d12f6836580b0e75e3569c764527ae


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39029] TestMaildir.test_clean fails randomly under parallel tests



STINNER Victor  added the comment:

> I was always under the assumption that -j 4 runs tests in parallel

If you run one test file, even with -j1000, only one test file will be run in 
parallel :-)

To run the same file in parallel, you have to ask to run it more than once: 
"-j2 test_mailbox test_mailbox" or "-j2 test_mailbox -F" for example.

-F runs the same tests in a loop until it fails. It's a convenient way to run 
the same test in parallel to detect race conditions ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39029] TestMaildir.test_clean fails randomly under parallel tests



Karthikeyan Singaravelan  added the comment:

I am able to reproduce it on my Mac and as below the values of os.getpid() are 
the same with the command. I will try to get to a Linux box to see if I can 
reproduce this. I was always under the assumption that -j 4 runs tests in 
parallel as it even says in the message "run tests in parallel using n child 
processes" . Maybe I am stumbling upon a mac os specific issue or understanding 
something wrong about the regrtest framework. I am using Macbook pro with 
Mojave.

./python.exe -Wall -m test -R 3:3 -j 20 test_mailbox -m test_clean -F
0:00:00 load avg: 2.31 Run tests in parallel using 20 child processes
0:00:03 load avg: 2.85 [  1/1] test_mailbox failed
os.getpid() = 37760
support.TESTFN = '@test_37760_tmp'
os.getpid() = 37760
support.TESTFN = '@test_37760_tmp'
beginning 6 repetitions
123456
.test test_mailbox failed -- Traceback (most recent call last):
  File "/Users/kasingar/stuff/python/cpython/Lib/test/test_mailbox.py", line 
743, in test_clean
self.assertFalse(os.path.exists(foo_path))
AssertionError: True is not false
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group

== Tests result: FAILURE ==

1 test failed:
test_mailbox

Total duration: 3.4 sec
Tests result: FAILURE

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39029] TestMaildir.test_clean fails randomly under parallel tests



STINNER Victor  added the comment:

I failed to reproduce the issue on my Fedora 31 laptop using 20 processes 
running on parallel (my CPU has 4 cores: 8 threads).

vstinner@apu$ ./python -Wall -m test -R 3:3 -j 20 test_mailbox -m test_clean -F
0:00:00 load avg: 14.86 Run tests in parallel using 20 child processes
(...)
0:01:12 load avg: 18.81 [321] test_mailbox passed
beginning 6 repetitions
123456
..
^C
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group
Kill  
process group

== Tests result: INTERRUPTED ==
Test suite interrupted by signal SIGINT.

321 tests OK.

Total duration: 1 min 12 sec
Tests result: INTERRUPTED

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39029] TestMaildir.test_clean fails randomly under parallel tests



STINNER Victor  added the comment:

./python.exe -Wall -m test -R 4:4 -j 4 test_mailbox -m test_clean

With this command: test_mailbox is run exactly once. It's not run in parallel. 
But the same test is run 8 times.

os.getpid() = 27901
support.TESTFN = '@test_27901_tmp'
os.getpid() = 27901
support.TESTFN = '@test_27901_tmp'

That sounds consistent: same pid means same TESTFN.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39029] TestMaildir.test_clean fails randomly under parallel tests



Karthikeyan Singaravelan  added the comment:

> You wrote "I guess the file is not really deleted in some cases." I think 
> that this should be investigated first.

Sorry, I wanted to mean that as the test deletes the file and goes on to assert 
it's not present the other test that runs in parallel creates the same file in 
setup code. Thus I wanted to imply that during assertion and parallel test's 
setup code the directory is again created.

I tried printing os.getpid() and support.TESTFN values in the setup and they 
are the same for all the tests.


./python.exe -Wall -m test -R 4:4 -j 4 test_mailbox -m test_clean
0:00:00 load avg: 2.91 Run tests in parallel using 4 child processes
0:00:00 load avg: 2.91 [1/1/1] test_mailbox failed
os.getpid() = 27901
support.TESTFN = '@test_27901_tmp'
os.getpid() = 27901
support.TESTFN = '@test_27901_tmp'
beginning 8 repetitions
12345678
.test test_mailbox failed -- Traceback (most recent call last):
  File "/Users/kasingar/stuff/python/cpython/Lib/test/test_mailbox.py", line 
740, in test_clean
self.assertFalse(os.path.exists(foo_path))
AssertionError: True is not false

== Tests result: FAILURE ==

1 test failed:
test_mailbox

Total duration: 789 ms
Tests result: FAILURE

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39150] See if PyToken_OneChar would be faster as a lookup table

2020-01-09 Thread Serhiy Storchaka



Serhiy Storchaka  added the comment:

The difference around 10% looks very strange. Tokenizing is just one of parts 
of the compiler, and unlikely it is a narrow way. There are input/output, many 
memory allocations, encoding and decoding, many iterations and recursions. Are 
you sure that you run benchmarks in the same environment, multiple times, and 
got stable results? Could you provide a script that reproduces this?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39150] See if PyToken_OneChar would be faster as a lookup table

2020-01-09 Thread Pablo Galindo Salgado



Pablo Galindo Salgado  added the comment:

I will take a look and try to reproduce the benchmarks result you mentioned, 
but if under PGO there is no substantial performance increase is much less 
appealing as the lookup table code is less clear and maintainable than the 
simple switch. Also you have a branch that is a good candidate to trigger the 
same branch mispredictions as the default of the switch (c1>=37 && c1<=126).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39029] TestMaildir.test_clean fails randomly under parallel tests

2020-01-09 Thread Serhiy Storchaka



Serhiy Storchaka  added the comment:

If TESTFN is the same in different processes, there is something wrong in the 
code for running parallel tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39154] "utf8" not always a synonym for "utf-8" in lib2to3



Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39029] TestMaildir.test_clean fails randomly under parallel tests



STINNER Victor  added the comment:

> yes, the test uses support.TESTFN which has the value of 
> "{}_{}_tmp".format(TESTFN, os.getpid()) in the setUp [0] to create tempdir. 
> Under parallel tests the pid value of TESTFN is initialised with a pid once 
> and used always in the setup calls. So setup acts under parallel tests with 
> same test folder. The tempdir can be dynamically generated but I guess this 
> pattern to use support.TESTFN is common and there could be other flaky tests.

I don't understand. Let's say that test_mailbox and test_os are run in 
parallel, process 1 runs test_mailbox and process 2 runs test_os. TESTFN should 
be different in each process but globally unique.

So I don't understand the difference between:

-self._path = support.TESTFN

and:

+self._tempdir = tempfile.TemporaryDirectory()
+self._path = self._tempdir.name

Is it because the test currently doesn't remove self._path directory in some 
cases?

You wrote "I guess the file is not really deleted in some cases." I think that 
this should be investigated first.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39259] poplib.POP3/POP3_SSL should reject timeout = 0 (non-blocking mode)



STINNER Victor  added the comment:

ftplib: 
https://github.com/python/cpython/blob/efa3b51fd060352cc6220b27a1026e4d4d5401bd/Lib/ftplib.py#L155
nntplib: 
https://github.com/python/cpython/blob/2e6a8efa837410327b593dc83c57492253b1201e/Lib/nntplib.py#L1049
smtplib: 
https://github.com/python/cpython/blob/3faf826e5879536d2272f1a51c58965a16827f81/Lib/smtplib.py#L386

It seems like these 3 modules use makefile() and have a timeout parameter. If 
makefile() doesn't support non-blocking mode, sure, it's a good idea to also 
reject explicitly timeout=0 in these modules.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39270] Dead assignment in config_init_module_search_paths



STINNER Victor  added the comment:


New changeset f3e5e9566989635225d1b9108074fc260817 by Victor Stinner (Alex 
Henrie) in branch 'master':
bpo-39270: Remove dead assignment from config_init_module_search_paths 
(GH-17914)
https://github.com/python/cpython/commit/f3e5e9566989635225d1b9108074fc260817


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39270] Dead assignment in config_init_module_search_paths



Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39272] Dead assignment in _ssl__SSLContext_load_verify_locations_impl



STINNER Victor  added the comment:


New changeset a1c1be24cb3ae25b5b53e9dc94d6327009626283 by Victor Stinner (Alex 
Henrie) in branch 'master':
bpo-39272: Remove dead assignment from 
_ssl__SSLContext_load_verify_locations_impl (GH-17916)
https://github.com/python/cpython/commit/a1c1be24cb3ae25b5b53e9dc94d6327009626283


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39272] Dead assignment in _ssl__SSLContext_load_verify_locations_impl