[issue34874] Python 3.6.3 command script wrapped in single quotes produces NameError: name 'A' is not defined

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I can reproduce this problem when use double quotes as both outer and internal 
quotes.

$ echo -n '{"A":"a"}' | python3 -c "import sys,json; j=json.load(sys.stdin); 
print(j["A"])"
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'A' is not defined

This has not relation to Python, this is how the quoting in the shell works. 
Perhaps something in your configuration makes single quotes be interpreted as 
double quotes.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34876] Python3.8 changes how decorators are traced

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

On other hand, consider the following example of multiline assignment:

$ rlwrap ./python -m dis
a = [
x,
y,
]

In 3.7:

  2   0 LOAD_NAME0 (x)

  3   2 LOAD_NAME1 (y)
  4 BUILD_LIST   2
  6 STORE_NAME   2 (a)
  8 LOAD_CONST   0 (None)
 10 RETURN_VALUE

In 3.8:

  2   0 LOAD_NAME0 (x)

  3   2 LOAD_NAME1 (y)

  1   4 BUILD_LIST   2
  6 STORE_NAME   2 (a)
  8 LOAD_CONST   0 (None)
 10 RETURN_VALUE

In 3.7 the line of the assignment "a = [" is not traced. In 3.8 it is traced.

These all are a consequences of the same change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34876] Python3.8 changes how decorators are traced

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

This is because the first line of the function definition was the line of the 
last decorator in 3.7, and it is the line of the first decorator in 3.8.

$ rlwrap ./python -m dis

@decorator
@decorator
@decorator
def func():
pass

In 3.7:

  2   0 LOAD_NAME0 (decorator)

  3   2 LOAD_NAME0 (decorator)

  4   4 LOAD_NAME0 (decorator)
  6 LOAD_CONST   0 (", line 2>)
  8 LOAD_CONST   1 ('func')
 10 MAKE_FUNCTION0
 12 CALL_FUNCTION1
 14 CALL_FUNCTION1
 16 CALL_FUNCTION1
 18 STORE_NAME   1 (func)
 20 LOAD_CONST   2 (None)
 22 RETURN_VALUE

In 3.8:

  2   0 LOAD_NAME0 (decorator)

  3   2 LOAD_NAME0 (decorator)

  4   4 LOAD_NAME0 (decorator)

  2   6 LOAD_CONST   0 (", line 2>)
  8 LOAD_CONST   1 ('func')
 10 MAKE_FUNCTION0
 12 CALL_FUNCTION1
 14 CALL_FUNCTION1
 16 CALL_FUNCTION1
 18 STORE_NAME   1 (func)
 20 LOAD_CONST   2 (None)
 22 RETURN_VALUE

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34874] Python 3.6.3 command script wrapped in single quotes produces NameError: name 'A' is not defined

[Benjamin Peterson]

Benjamin Peterson  added the comment:

I cannot reproduce this problem with exactly Python 3.6.3. This may be some 
strange situation specific to your configuration, which should be supported to 
RedHat.

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34879] bytesobject.c: Possible null pointer dereference due to formatfloat()

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
pull_requests: +9069
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34879] bytesobject.c: Possible null pointer dereference due to formatfloat()

[Zackery Spytz]

New submission from Zackery Spytz :

The PyBytes_FromStringAndSize() call in formatfloat() is not checked for 
failure.

--
components: Interpreter Core
messages: 326933
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: bytesobject.c: Possible null pointer dereference due to formatfloat()
type: crash
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34878] Lock Objects documentation bug

[Benjamin Peterson]

Benjamin Peterson  added the comment:

The current documentation is correct. While conceptually one may think of a 
lock as being held ("owned") by a particular thread, the lock internally has no 
idea what thread owns it—operations on a lock are influenced only by its 
current state not what thread is performing the operation. It's perfectly 
possible, if inadvisable, to release a lock on a thread different from the one 
it was acquired on. The description of ownership is meant to draw a distinction 
with recursive locks, which do have an internal notion of ownership.

--
nosy: +benjamin.peterson
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32174] nonASCII punctuation characters can not display in python363.chm.

[Sangbae Nam]

Sangbae Nam  added the comment:

This issue still persists in 3.6 and 3.7.

--
assignee:  -> docs@python
components: +Documentation
nosy: +Sangbae Nam, docs@python
versions: +Python 3.7
Added file: https://bugs.python.org/file47844/py37chm.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue

[Ezio Melotti]

Ezio Melotti  added the comment:

another test

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

> SeaHash seems to be designed for 64 bits.

Absolutely.

> I'm guessing that replacing the shifts by
>
> x ^= ((x >> 16) >> (x >> 29))
>
> would be what you'd do for a 32-bit hash.

My guess too.  But "the prime" is a puzzle.  As noted before, the 64-bit prime 
is specially constructed to satisfy specific dispersion properties.  While I 
haven't triple-checked this, I believe it fails at two specific bit positions:

- Bit 2**16 doesn't propagate into the leading 4 bits at all.

- Bit 2**17 does propagate into bit 2**60, but _only_ into that bit among the 
highest 4 bits.  One of the criteria is that the prime shouldn't propagate a 
lower-order bit into _only_ the least-significant of the 4 leading bits.

Repairing that would probably require adding a bit or two.  But the prime 
already has 35 bits set.  The greater the imbalance between 0 and 1 bits, the 
more multiply acts like a simpler sequence of shift-and-adds or 
shift-and-subtracts (for example, at an extreme, consider the multiplier (1 << 
63) - 1).

Anyway, it seems the density of 1 bits would have to be even higher for a 
32-bit multiplier to ensure all low-order bits directly propagated to at least 
one of the topmost 3 bits, but not to the third-most significant alone.

Or I could search for a 31-bit prime - or just an odd integer - that got as 
close as possible with 16 or 17 bits set.  Or ...

> Alternatively, we could always compute the hash with
> 64 bits (using uint64_t) and then truncate at the end
> if needed.

Provided we continue to like it at all ;-)  In which case I'd probably end it 
with

32_bit_result = (32_bit_result_type)(x ^ (x >> 32));

instead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34872] investigate task/future cancellation in asynciomodule.c

[Yury Selivanov]

Yury Selivanov  added the comment:

Ned, will we have 3.7.1rc2?  If so, would it be possible to include the fix for 
this one?

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34874] Python 3.6.3 command script wrapped in single quotes produces NameError: name 'A' is not defined

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34877] Inconsistent Behavior Of futures.ProcessPoolExecutor

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

FYI, this appears to be a good overview of what SMHasher is looking for:

https://github.com/aappleby/smhasher/wiki/SMHasher

Someg of everything: performance, correctness, statistical measures, and 
specific kinds of keysets that have proved problematic for other non-crypto 
hash functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34878] Lock Objects documentation bug

[ulrich.stern]

New submission from ulrich.stern :

The first sentence of the documentation for Lock Objects 
(https://docs.python.org/2/library/threading.html#lock-objects) seems 
incorrect.  It currently states "A primitive lock is a synchronization 
primitive that is not owned by a particular thread when locked."  The "not" 
should be deleted.  (Alternatively, one could change "locked" to "unlocked.")

--
assignee: docs@python
components: Documentation
messages: 326926
nosy: docs@python, ulrich.stern
priority: normal
severity: normal
status: open
title: Lock Objects documentation bug
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12458] Tracebacks should contain the first line of continuation lines

[Ned Batchelder]

Ned Batchelder  added the comment:

The other drawback is changing how the trace function is notified in a few 
scenarios.

--
nosy: +nedbat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34872] investigate task/future cancellation in asynciomodule.c

[Elvis Pranskevichus]

Change by Elvis Pranskevichus :


--
keywords: +patch
pull_requests: +9068
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34876] Python3.8 changes how decorators are traced

[Ammar Askar]

Ammar Askar  added the comment:

It looks like this is caused by 
https://github.com/python/cpython/commit/da8d72c953369b872a12c13f136ada77a786714a

Adding Serhiy to the nosy list.

--
nosy: +ammar2, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34875] Change .js mime to "test/javascript"

[Ammar Askar]

Ammar Askar  added the comment:

It should be noted that the HTML spec also says:

The term "JavaScript" is used to refer to ECMA-262, rather than the official 
term ECMAScript, since the term JavaScript is more widely known. Similarly, the 
MIME type used to refer to JavaScript in this specification is text/javascript, 
since that is the most commonly used type, despite it being an officially 
obsoleted type according to RFC 4329.

https://html.spec.whatwg.org/#dependencies:willful-violation

--
nosy: +ammar2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34877] Inconsistent Behavior Of futures.ProcessPoolExecutor

[Tom Ashley]

New submission from Tom Ashley :

Not sure if this goes in core or modules.

There is an inconsistency in the output of the attached script. From the docs I 
read it's supposed to have the behavior of:

"If something happens to one of the worker processes to cause it to exit 
unexpectedly, the ProcessPoolExecutor is considered “broken” and will no longer 
schedule tasks."

That script is supposed to exemplify that. Instead, if I run the code several 
times, I get the following output:

(bot-LP2ewIkY) ⋊> ~/w/p/b/bot on master ⨯ python brokenPool.py  

 18:54:55
getting the pid for one worker
killing process 4373
submitting another task
could not start new tasks: A process in the process pool was terminated 
abruptly while the future was running or pending.
(bot-LP2ewIkY) ⋊> ~/w/p/b/bot on master ⨯ python brokenPool.py  

 18:54:56
getting the pid for one worker
killing process 4443
submitting another task
could not start new tasks: A process in the process pool was terminated 
abruptly while the future was running or pending.
(bot-LP2ewIkY) ⋊> ~/w/p/b/bot on master ⨯ python brokenPool.py  

 18:54:57
getting the pid for one worker
killing process 4514
submitting another task  <- (No exception thrown after this)


The exception isn't always thrown. This seems problematic to me. Related stack 
post: 
https://stackoverflow.com/questions/52617558/python-inconsistent-behavior-of-futures-processpoolexecutor

--
components: Interpreter Core
files: brokenPool.py
messages: 326922
nosy: TensorTom
priority: normal
severity: normal
status: open
title: Inconsistent Behavior Of futures.ProcessPoolExecutor
versions: Python 3.7
Added file: https://bugs.python.org/file47843/brokenPool.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34876] Python3.8 changes how decorators are traced

[Ned Batchelder]

New submission from Ned Batchelder :

When decorating a function, the sequence of lines reported to the trace 
function is different in Python3.8 than with previous versions

$ cat -n decorator.py
 1  def decorator(f):
 2  return f
 3
 4  def f():
 5  @decorator
 6  @decorator
 7  @decorator
 8  def func():
 9  pass
10
11  import sys
12  def trace(frame, event, args):
13  print(frame.f_lineno, event)
14  return trace
15
16  sys.settrace(trace)
17  f()

$ python3.7 decorator.py
4 call
5 line
6 line
7 line
1 call
2 line
2 return
1 call
2 line
2 return
1 call
2 line
2 return
7 return

$ python3.8 decorator.py
4 call
5 line
6 line
7 line
5 line
1 call
2 line
2 return
1 call
2 line
2 return
1 call
2 line
2 return
5 return

Is this intentional? Will it be changed back before 3.8 ships?

People are testing their projects against 3.8-dev, and reporting problems with 
coverage.  The problems are due to these sorts of changes.

--
components: Interpreter Core
messages: 326921
nosy: nedbat
priority: normal
severity: normal
status: open
title: Python3.8 changes how decorators are traced
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34875] Change .js mime to "test/javascript"

[Myles Borins]

Change by Myles Borins :


--
keywords: +patch
pull_requests: +9066
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34875] Change .js mime to "test/javascript"

[Myles Borins]

New submission from Myles Borins :

I propose to change the mapping of file extension .js to mime type 
"text/javascript" from "application/javascript.

"text/javascript" is the currently documented best practice in the whatwg HTML 
spec.

https://html.spec.whatwg.org/multipage/scripting.html#scriptingLanguages:javascript-mime-type

--
components: Library (Lib)
messages: 326920
nosy: mylesborins
priority: normal
severity: normal
status: open
title: Change .js mime to "test/javascript"
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] Fix test_httpservers on AIX (trailingSlashOK)

[Martin Panter]

Martin Panter  added the comment:

Hi Michael, I agree with Victor that the best place to fix the problem is in 
the HTTP server module. In other words, the “medium fix” you mentioned in your 
original post.

Your recent proposal to just skip the test means that AIX will continue to 
suffer from the original bug.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

> So it seems that this SMHasher test suite doesn't
> catch the problem that we're seeing with negative integers.

Seems to be so, but I've never run SMHasher myself.  I believe it's focused on 
statistical properties, like avalanche and bit independence.


> However, we ideally want a hash that works well for
> all kinds of inputs. If the hash function is good,
> it shouldn't be possible to write a hash collision
> test function which has a significantly higher chance
> of failing than random chance.

I know of no such hash functions short of crypto-strength ones.  Nobody uses 
those for hash tables, though, because they're much slower and usually much 
more involved.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

> >>> from itertools import product
> >>> len(set(map(hash, product([0.5, 0.25], repeat=20
> 32

> Good catch! Would you like me to add this to the testsuite?

It's in mine already ;-)  I've added all the "bad examples" in all the messages 
here.  Sooner or later they'll get folded into Python's test suite.

BTW, there were no collisions in that under whatever 64-bit Python I last 
compiled.  That was a SeaHash variant.  I'm not certain, but I believe it had 
"t ^= t << 1" at the start and with the first multiply commented out.

Having learning _something_ about why SeaHash does what it does, I'm not 
convinced the first multiply is of much value.  As a standalone bit-scrambler 
for a single 64-bit input, it does matter.  But in the tuple hash context, 
we're running it in a loop.  Strictly alternating "propagate left" and 
"propagate right" seems to me almost as good - although that's just intuition.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34172] multiprocessing.Pool and ThreadPool leak resources after being deleted

[Antoine Pitrou]

Antoine Pitrou  added the comment:

(tzickel, sorry for mistyping your handle :-/)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34874] Python 3.6.3 command script wrapped in single quotes produces NameError: name 'A' is not defined

[Tim McDonough]

New submission from Tim McDonough :

I found an odd behavior that seems to be limited to Python 3.6.3. 
Python 3.6.3 command scripts seem to prefer wrapping in double quotes instead 
of single quotes.

Here is an example of the error.  

$ echo -n '{"A":"a"}' | python3 -c 'import sys,json; j=json.load(sys.stdin); 
print(j["A"])'
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'A' is not defined

# Swapping single and double quotes works as expected.
#
$ echo -n '{"A":"a"}' | python3 -c "import sys,json; j=json.load(sys.stdin); 
print(j['A'])"
a

# Executing the original, failing script against python 2.6.6 works.
#
$ echo -n '{"A":"a"}' | python2 -c 'import sys,json; j=json.load(sys.stdin); 
print(j["A"])'
a

The failing environment is:
$ bash --version
GNU bash, version 4.1.2(2)-release (x86_64-redhat-linux-gnu)
$ gcc --version
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23)

These python versions also seem to work fine: 2.6.6, 2.7.9, 3.5.1, and 3.5.2.
Assigning the script to an environment variable follows the single/double quote 
issue. For example, this fails:

$ export python_script='import sys,json; j=json.load(sys.stdin); print(j["A"])'
$ echo -n '{"A": "a"}' | python3 -c "${python_script}"

This would be a good unit test candidate if not already present.

--
components: Tests
messages: 326916
nosy: Tim McDonough
priority: normal
severity: normal
status: open
title: Python 3.6.3 command script wrapped in single quotes produces NameError: 
name 'A' is not defined
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34172] multiprocessing.Pool and ThreadPool leak resources after being deleted

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Thanks tzickler for the report and pull request, and sorry for the delay.

This is now fixed in all 3.x branches.  I will close this now as 
multiprocessing in 2.7 diverges quite a bit from 3.x.  If you want to fix the 
issue in 2.7 as well, please say so and I'll reopen.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34172] multiprocessing.Pool and ThreadPool leak resources after being deleted

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 07b96a95db78eff3557d1bfed1df9ebecc40815b by Antoine Pitrou (Miss 
Islington (bot)) in branch '3.6':
bpo-34172: multiprocessing.Pool leaks resources after being deleted (GH-8450) 
(GH-9677)
https://github.com/python/cpython/commit/07b96a95db78eff3557d1bfed1df9ebecc40815b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34873] re.finditer behaviour in re.MULTILINE mode fails to match first 7 characters

[Tom Dawes]

Tom Dawes  added the comment:

Please ignore, re.finditer and REGEX.finditer aren't the same. I was passing 
re.MULTILINE (= 8) to endPos.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34873] re.finditer behaviour in re.MULTILINE mode fails to match first 7 characters

[Tom Dawes]

New submission from Tom Dawes :

re.finditer appears to fail to match within the first 7 characters in a string 
when re.MULTILINE is used:

>>> REGEX = re.compile("y")
>>> [list(m.start() for m in REGEX.finditer("{}y".format("x"*i), re.MULTILINE)) 
>>> for i in range(10)]
[[], [], [], [], [], [], [], [], [8], [9]]

Without re.MULTILINE, this works fine:

>>> [list(m.start() for m in REGEX.finditer("{}y".format("x"*i))) for i in 
>>> range(10)]
[[0], [1], [2], [3], [4], [5], [6], [7], [8], [9]]

Passing re.MULTILINE to re.compile doesn't seem to have any effect.

--
components: Regular Expressions
messages: 326911
nosy: ezio.melotti, mrabarnett, tdawes
priority: normal
severity: normal
status: open
title: re.finditer behaviour in re.MULTILINE mode fails to match first 7 
characters
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

>> For that reason, I've only been looking at those that
>> scored 10 (best possible) on Appleby's SMHasher[1] test suite

> Do you have a list of such hash functions?

A world of searching.  The ones rated "Excellent" here (previously cited) for a 
start:

https://docs.rs/seahash/3.0.5/seahash/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34172] multiprocessing.Pool and ThreadPool leak resources after being deleted

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 97f998a4dfd6db6d867f446daa62445d0782bf39 by Antoine Pitrou (Miss 
Islington (bot)) in branch '3.7':
bpo-34172: multiprocessing.Pool leaks resources after being deleted (GH-8450) 
(GH-9676)
https://github.com/python/cpython/commit/97f998a4dfd6db6d867f446daa62445d0782bf39


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

>> I've noted before, e.g., that sticking to a prime
>> eliminates a world of regular bit patterns in the
>> multiplier.

> Why do you think this? 0x1fff is prime :-)

Point taken ;-)  But "a world of" is not the same as "the universe".  For 
example, sticking to a prime you'll never get 8 bytes the same.  Etc - "a world 
of" extremely regular patterns are eliminated.


> Having regular bit patterns and being prime are independent
> properties.
>
> To be clear: I don't have anything against picking a prime
> but we just shouldn't pretend that primes are important
> when they are not. That's all...

I don't like arguments from ignorance.  As I've said, I don't know why SeaHash 
uses a prime.  Neither do you.  Our collective ignorance doesn't imply the 
designer didn't have a good reason.

I can't think of a "good reason" for it either, but that's where we differ:  I 
think "so don't make gratuitous changes" while you think "therefore it can't 
possibly matter" ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> For that reason, I've only been looking at those that scored 10 (best 
> possible) on Appleby's SMHasher[1] test suite

Do you have a list of such hash functions?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34172] multiprocessing.Pool and ThreadPool leak resources after being deleted

[miss-islington]

Change by miss-islington :


--
pull_requests: +9064

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

>>> from itertools import product
>>> len(set(map(hash, product([0.5, 0.25], repeat=20
32

Good catch! Would you like me to add this to the testsuite?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> For that reason, I've only been looking at those that scored 10 (best 
> possible) on Appleby's SMHasher[1] test suite, which is used by everyone who 
> does recognized work in this field.

So it seems that this SMHasher test suite doesn't catch the problem that we're 
seeing with negative integers.

> I'm concerned that I've been putting way too much weight on "the new" tuple 
> test. [...] that's a minuscule region of the problem space.

I'll admit that it's a miniscule region of the problem space. However, we 
ideally want a hash that works well for all kinds of inputs. If the hash 
function is good, it shouldn't be possible to write a hash collision test 
function which has a significantly higher chance of failing than random chance.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34172] multiprocessing.Pool and ThreadPool leak resources after being deleted

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 97bfe8d3ebb0a54c8798f57555cb4152f9b2e1d0 by Antoine Pitrou 
(tzickel) in branch 'master':
bpo-34172: multiprocessing.Pool leaks resources after being deleted (GH-8450)
https://github.com/python/cpython/commit/97bfe8d3ebb0a54c8798f57555cb4152f9b2e1d0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34172] multiprocessing.Pool and ThreadPool leak resources after being deleted

[miss-islington]

Change by miss-islington :


--
pull_requests: +9065

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> I've noted before, e.g., that sticking to a prime eliminates a world of 
> regular bit patterns in the multiplier.

Why do you think this? 0x1fff is prime :-)

Having regular bit patterns and being prime are independent properties.

To be clear: I don't have anything against picking a prime but we just 
shouldn't pretend that primes are important when they are not. That's all...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34313] IDLE crashes with Tk-related error on macOS with ActiveTcl 8.6

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Current Mac installers do contain hopefully tested tcl/tk installers.  (This 
has always been true for the Windows installers.) We are not responsible if 
users use anything else with Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] Fix test_httpservers on AIX (trailingSlashOK)

[Eryk Sun]

Eryk Sun  added the comment:

> I confirm that in a junction point

This path-parsing bug that strips a trailing slash occurs when a traversed 
directory is a reparse point, such as a mount-point (junction) or directory 
symbolic link. It may be limited to just the NTFS and ReFS file-system drivers, 
which is all I tested. Or it may be a bug in the I/O manager itself, which 
actually implements the name grafting for Microsoft's mount-point (0xA003) 
and symlink (0xA00C) reparse tags.

I thought the suggested patch was ok because it followed the pattern of 
existing code that prevents opening directories on Unix. But I can see the 
distinction in that the latter is implementing behavior policy rather than 
working around a platform bug. Also, while _Py_open and _Py_fopen do allow 
opening a directory on a Linux box, in apparent violation of this policy, these 
are private, internal functions, and there's no use that I'm aware of in the 
CPython implementation for directory FDs or FILE streams opened this way.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34313] IDLE crashes with Tk-related error on macOS with ActiveTcl 8.6

[Vlad Tudorache]

Vlad Tudorache  added the comment:

@terry.reedy

Maybe "users are urged to install the most recent bug-fix releases" is a little 
bit misleading. As I wrote, Tcl/Tk 8.5.18 does not give crashes on macOS, when 
both 8.5.17 and 8.5.19 (bug-fix) do. Some times ago, if I remember well, the 
same thing happened 'round 8.6.3 something (there was a "bug-fix" release worse 
than the previous one). Should python.org installers provide only tested 
self-contained Tcl/Tk builds?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34872] investigate task/future cancellation in asynciomodule.c

[Vladimir Matveev]

Change by Vladimir Matveev :


--
nosy: +v2m

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] Fix test_httpservers on AIX (trailingSlashOK)

[Michael Felt]

Michael Felt  added the comment:

Changed the title to reflect the requested change is only in Tests.

--
components:  -IO
title: return ENOTDIR when open() accepts filenames with a trailing slash -> 
Fix test_httpservers on AIX (trailingSlashOK)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

> If we, e.g., tested tuples of little floats instead ...

Speaking of which:

>>> from itertools import product
>>> len(set(map(hash, product([0.5, 0.25], repeat=20
32

32 hash codes out of 1048576 distinct two-tuples isn't exactly 
confidence-inspiring either ;-)  No scheme that only "propagates to the left" 
is going to help that much, because the variation in those hashes is all in the 
high-order bits.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34872] investigate task/future cancellation in asynciomodule.c

[Yury Selivanov]

Yury Selivanov  added the comment:

Elvis, please take a look at this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34872] investigate task/future cancellation in asynciomodule.c

[Yury Selivanov]

New submission from Yury Selivanov :

Vladimir Matveev has discovered that C and Python implementation of 
asyncio.Task diverge:

* asynciomodule.c:
https://github.com/python/cpython/blob/9012a0fb4c4ec1afef9efb9fdb0964554ea17983/Modules/_asynciomodule.c#L2716

* tasks.py:
https://github.com/python/cpython/blob/9012a0fb4c4ec1afef9efb9fdb0964554ea17983/Lib/asyncio/tasks.py#L286

In tasks.py we call "fut_obj.cancel()", so either "Task.cancel()" or 
"Future.cancel()" can be called depending on what "fut_obj" is.

In asynciomodule.c we essentially always call "Future.cancel(fut_obj)".  This 
probably leads to unexpected behaviour if "fut_obj" is either a Task or a Task 
subclass with an overloaded "cancel()" method.

We need to first write a set of tests to understand the scope of this; then 
we'll need to patch asynciomodule.c and backport this to 3.7 & 3.6.

--
components: asyncio
messages: 326896
nosy: Elvis.Pranskevichus, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: investigate task/future cancellation in asynciomodule.c
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20504] cgi.FieldStorage, multipart, missing Content-Length

[Paul Goins (work)]

Paul Goins (work)  added the comment:

I'm just going to ping on this issue.  It looks like this has just slipped off 
the radar.  I've seen the last diff and the code review, but it seems that this 
just needs some final follow-up on the code review comments, no?

I could easily do the final cleanup myself, but would prefer to let someone 
already on this thread take it across the finish line since I didn't do any of 
the real work for this patch.  If really necessary, I may be able to clean this 
up if no one else can spare the time.

--
nosy: +pdgoins-work

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

> So my suggestion remains
>
> for y in INPUT:
>t = hash(y)
>t ^= t * SOME_LARGE_EVEN_NUMBER
>h ^= t
>h *= MULTIPLIER

On the face of it, I'd be amazed if that passed SMHasher, because it only 
propagates bits "to the left".  All hashes that score well on that test work to 
propagate "to the right" too.  SeaHash does that with its variable-count 
right-shift-xor.  Another popular choice in other hashes is a rotate.

As noted several times before, our tuple tests only use small integers, where 
there's virtually no variation in the high-order hash bits beyond "all 0" or 
"all 1".  Since all the variation is in a handful of low-order bits, "propagate 
right" _appears_ to be a waste of time.

If we, e.g., tested tuples of little floats instead, we may have "concluded" 
that it's "propagate left" that's a waste of time:

>>> for i in range(5):
... x = 1 / 2**i
... print(x, hex(hash(x)
...
1.0   0x1
0.50x1000
0.250x800
0.125   0x400
0.0625  0x200

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

>> the author wants this transformation to be easily
>> invertible, so a prime is necessary

> A multiplication by any odd number modulo 2**64 is
> invertible.

Right!  My mistake.


> As I argued before, the concept of primes is
> meaningless (except for the prime 2) when computing
> modulo 2**64.

I don't know why they're using a prime.  But that doesn't mean they don't have 
"a reason".  You seem quick to dismiss things if they don't make instant sense 
to you at first glance.  I've noted before, e.g., that sticking to a prime 
eliminates a world of regular bit patterns in the multiplier.

The original SeaHash used a different prime, and a fixed right shift of 32 (but 
twice in different places).

Switching to the current prime, and the variable shift, can be traced back to 
this long comment on Reddit:

https://www.reddit.com/r/rust/comments/5fdf8z/seahash_a_blazingly_fast_portable_hash_function/dakdii1/

The prime isn't "random":  it was constructed so that flipping a low-order bit 
to 1 would directly affect at least one of the topmost 4 bits, which in turn 
are used to select the shift count.  While that doesn't seem to matter for our 
tiny test suite, as the message shows it made huge improvements in other 
regions of the input space.

I haven't reported this, but doing "x ^= x >> 32" instead worked fine for our 
test suite too.  Well, big deal - we're testing almost nothing beyond two kinds 
of "oops!" cases (nested tuples, and mixed-sign tiny ints).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

A meta-comment:  a great deal of work has been done on non-crypto hashes in 
recent years.  I'm not interested in rolling our own if one of the "winners" 
from that process can be adapted.  For that reason, I've only been looking at 
those that scored 10 (best possible) on Appleby's SMHasher[1] test suite, which 
is used by everyone who does recognized work in this field.  SeaHash appears to 
be the fastest of those, with short & simple code.

I'm concerned that I've been putting way too much weight on "the new" tuple 
test.  That uses a grand total of 10 tiny integers in -5 .. 5 (omits -1).  
_All_ base-component variation is in the last 3 bits, while the top 61 bits are 
all 0 or all 1.  Great for testing nested tuples with tiny mixed-sign integers, 
but that's a minuscule region of the problem space.

[1] https://github.com/aappleby/smhasher

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] return ENOTDIR when open() accepts filenames with a trailing slash

[Michael Felt]

Change by Michael Felt :


--
pull_requests: +9063

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34728] deprecate *loop* argument for asyncio.sleep

[Yury Selivanov]

Yury Selivanov  added the comment:

[victor]
> Why does it not make sense to pass the loop to sleep? "it makes no sense 
> anymore" something changes?

[andrew]
`loop` argument passed to sleep should be always the same as returned from 
`get_running_loop()`.

What Andrew said.

Basically, it wasn't *ever* possible to pass a loop to sleep() that would be 
different from the loop that would run it, because sleep() is a *coroutine*.

In asyncio some APIs are functions and some are coroutines.

* asyncio.gather(), for example, is a function.  You can call it from top-level 
code (and pass an event loop to it) or from a coroutine.

* asyncio.sleep(), wait(), and wait_for() are *coroutines*; they can only be 
called from other coroutines or if you wrap them into a Task; in all cases, 
*loop* is always 100% defined for them.

Passing the loop isn't even a viable micro-optimization, it's just pointless.  
This extra argument just adds to the confusion and promotes bad patterns, so we 
want to eventually remove it.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34728] deprecate *loop* argument for asyncio.sleep

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 9012a0fb4c4ec1afef9efb9fdb0964554ea17983 by Yury Selivanov in 
branch 'master':
bpo-34728: Fix asyncio tests to run under "-Werror" (GH-9661)
https://github.com/python/cpython/commit/9012a0fb4c4ec1afef9efb9fdb0964554ea17983


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34871] test_site fails if run after test_inspect

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] return ENOTDIR when open() accepts filenames with a trailing slash

[Michael Felt]

Michael Felt  added the comment:

On 10/2/2018 7:36 PM, Michael Felt wrote:
> Python is designed as a thin wrapper to the operating system. IMHO Python 
> must not validate the filename itself.
To shorten the discussion, I'll kill the current PR and just modify the
test to skip the trailing slash test.

Assumption: if Python is not validating filenames, then a Module should
also not validate a filename.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] return ENOTDIR when open() accepts filenames with a trailing slash

[Michael Felt]

Michael Felt  added the comment:

Was not my intent. Firewall issues. After 4 more attempts gave up until now.

On 10/2/2018 3:17 PM, STINNER Victor wrote:
> STINNER Victor  added the comment:
>
>> 2018-10-02 11:02:32  Michael.Feltset files: + mime-attachment, 
>> encrypted.asc
> You replied with an encrypted message which isn't understood by the bug 
> tracker.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] return ENOTDIR when open() accepts filenames with a trailing slash

[Michael Felt]

Michael Felt  added the comment:

On 10/2/2018 10:36 AM, STINNER Victor wrote:
> STINNER Victor  added the comment:
>
> Jeremy Kloth: "This is also an issue on Windows when the target path resides 
> within a junction, paths outside of a junction respond (err, fail) as 
> expected."
> https://developercommunity.visualstudio.com/content/problem/272379/createfile-non-error-on-filename-with-trailing-bac.html
> I don't know the behavior on Windows. I tried to create a file name "a\" 
> (U+0061, U+005C): I get an OSError "invalid argument" (error 22).
>
> I confirm that in a junction point, I am able to:
>
> * open an existing file with an additional trailing antislash (U+005C): the 
> extra character is simply ignored and I am able to open the file
>
> * create a new file with an additional trailing antislash (U+005C): the 
> filename is created without the trailing antislash
>
> On the PR, I wrote:
>
>
>> There are much more functions which "open a file". Open Python/fileutils.c 
>> for a few mores. What about os.open()? What about all other functions which 
>> accept a filename and then call a third party library which calls open() 
>> directly?
> Ok, let me give some examples of function which directly open a file:
>
> * fileutils.c: _Py_open(), _Py_open_noraise(), _Py_wfopen(), _Py_fopen(), 
> _Py_fopen_obj()
> * os.open()
> * _io.FileIO, _pyio.FileIO (use os.open())
>
> Ok... But there are other functions to access files... stat()/fstat() 
> functions:
>
> * fileutils.c: _Py_fstat_noraise(), _Py_fstat(), _Py_stat()
> * Modules/getpath.c: _Py_wstat()
> * os.stat(), os.lstat(), os.fstat()
>
> To start to have a better ideas of how many functions accept filenames, open 
> also Lib/shutil.py. shutil.copyfile() uses os.stat(), but then it uses 
> os.symlink() and open()... So what about os.symlink()?
>
> Ok, here I only listen a *few* examples of functions which are "controlled" 
> by Python. But there are *many* wrappers to 3rd party libraries which accept 
> a filename. Examples:
>
> * _ssl.SSLContext.load_cert_chain()
> * sqlite3.connect()
> * etc.
>
> Where is the limit? How many functions must be patched in Python? How do we 
> patch OpenSSL and SQLite libraries?
>
> Python is designed as a thin wrapper to the operating system. IMHO Python 
> must not validate the filename itself.
>
> --
>
>> Going back to issue17234 - there has been a test to check that a URL with a 
>> trailing slash reports 404 status.
> IMHO you must fix a single place: the SimpleHTTPServer, not all code handling 
> the filesytem.
What I did not know: do you want the test to be okay as is, or set
SkipIf(condition) for operating systems that accept filename/ as a
filename when it is not directory.

IMHO: the "statement" of the test is that the name "test/" MUST be
rejected in all cases.

    # check for trailing "/" which should return 404. See Issue17324
    response = self.request(self.base_url + '/test/')
    self.check_status_and_reason(response, HTTPStatus.NOT_FOUND)

Title :
SimpleHTTPServer serves files even if the URL has a trailing slash

I can assume the test was adjusted to catch the 404 return code, and let
that pass on OS that will not open .../test.txt and .../text.txt/ as the
same object - while AIX (and windows does).

As to your comment about "other" things that can happen - the message in
the test implies that Python only wants "filenames" that are opened
without a slash. I guess there is not a PEP on this, so anyones guess is
likely to be wrong for someone.

In short, this PR took the road where Python says .../filename/ is wrong
- per the comment in the test.

So, what should be done: code in HTTPServer that checks for trailing
slash and the test can be left ASIS, or modify the test to skip this one
aspect on an OS that is known to accept ".../filename/" as a valid filename?

I am not "core" enough to make this choice. I can attempt to code any
decision made by "python-dev" aka core-devs.
> Same remark for AIX and Windows junctions.
>

> I suggest to reject this issue.
per above, I would rather not reject the issue - I would still like to
see the test fixed. But need help on what should actually be modified.
Test, Module, or internals.
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] return ENOTDIR when open() accepts filenames with a trailing slash

[Michael Felt]

Michael Felt  added the comment:

Final attempt to send as plain text

On 10/2/2018 1:07 AM, Benjamin Peterson wrote:
> On Mon, Oct 1, 2018, at 12:12, Michael Felt wrote:
>> Hi all,
>>
>> Before I submit a patch to increase the default MAXDATA setting for AIX
>> when in 32-bit mode - I want to know if I can put this LDFLAG setting in
>> LDLAST, or if I should introduce a new AC_SUBST() variable (e.g.,
>> LDMAXDATA).
> I think you should just put it in LDFLAGS.
I was wanting to avoid that, as LDFLAGS is an environmental variable.

At the surface, it appears Python is using PY_LDFLAGS (with
CONFIGURE_LDFLAGS coming from LDFLAGS during the ./configure moment.

A reason for a separate variable is that this particular option is only
relevant for the python EXE, and not for shared libraries and "other
things". IMHO, a reason for LDMAXDATA is because LDLAST is actually
already too widely used:

root@x066:[/data/prj/python/git/cpython-master]grep LDFLAGS *.in
Makefile.pre.in:CONFIGURE_LDFLAGS=  @LDFLAGS@
Makefile.pre.in:# Avoid assigning CFLAGS, LDFLAGS, etc. so users can use
them on the
Makefile.pre.in:# Both CPPFLAGS and LDFLAGS need to contain the shell's
value for setup.py to
Makefile.pre.in:PY_LDFLAGS= $(CONFIGURE_LDFLAGS) $(LDFLAGS)
Makefile.pre.in:LDSHARED=   @LDSHARED@ $(PY_LDFLAGS)
Makefile.pre.in:BLDSHARED=  @BLDSHARED@ $(PY_LDFLAGS)
Makefile.pre.in:OPENSSL_LDFLAGS=@OPENSSL_LDFLAGS@
Makefile.pre.in:    $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS)
$(PGO_PROF_GEN_FLAG)" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG)"
LIBS="$(LIBS)"
Makefile.pre.in:    $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS)
$(PGO_PROF_USE_FLAG)" LDFLAGS="$(LDFLAGS)"
Makefile.pre.in:    $(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@
Programs/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
Makefile.pre.in: $(CC) -dynamiclib -Wl,-single_module
$(PY_LDFLAGS) -undefined dynamic_lookup
-Wl,-install_name,$(prefix)/lib/libpython$(LDVERSION).dylib
-Wl,-compatibility_version,$(VERSION) -Wl,-current_version,$(VERSION) -o
$@ $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM) $(LDLAST); \
Makefile.pre.in:    $(CC) -o $(LDLIBRARY) $(PY_LDFLAGS) -dynamiclib \
Makefile.pre.in:    $(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@
Programs/_testembed.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
Makefile.pre.in:    $(LINKCC) $(PY_LDFLAGS) -o $@
Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LIBS)
$(MODLIBS) $(SYSLIBS) $(LDLAST)
Makefile.pre.in:    $(CC) $(OPT) $(PY_LDFLAGS) $(PGENOBJS)
$(LIBS) -o $(PGEN)

The ONLY line that needs $LDMAXDATA is:

Makefile.pre.in:    $(LINKCC) $(PY_LDFLAGS) -o $@
Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LIBS)
$(MODLIBS) $(SYSLIBS) $(LDLAST) $(LDMAXDATA)

or set $(LDLAST) at the end rather than append $(LDMAXDATA)
>> I have not looked yet, but I was thinking that MAYBE! LDLAST is intended
>> as a last resort variable that can be modified in Makefile.
> LDLAST looks vestigial from OSF/1 support and should probably be removed.

On 10/2/2018 1:02 PM, Michael Felt wrote:
> Change by Michael Felt :
>
>
> Added file: https://bugs.python.org/file47840/mime-attachment
> Added file: https://bugs.python.org/file47841/encrypted.asc
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34843] logging cookbook docs: remove 'recent' when referring to multiprocessing

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34871] test_site fails if run after test_inspect

[Chih-Hsuan Yen]

Chih-Hsuan Yen  added the comment:

Thanks very much for the test.bisect hint! After some more trials, I guess it's 
an issue in the `inspect` module rather than the newly-added test in test_site. 
Here's an example script:


import inspect
import sys
import _testcapi

builtin = _testcapi.docstring_with_signature_with_defaults
spec = inspect.getfullargspec(builtin)
print(type(sys.modules['__builtins__']))


After inspect.getfullargspec(), sys.modules['__builtins__'] is a dict. That's a 
little bit strange as every other item in sys.modules is a module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34784] Heap-allocated StructSequences

[Eddie Elizondo]

Change by Eddie Elizondo :


--
pull_requests: +9062

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34871] test_site fails if run after test_inspect

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

The failing test was added with d4c76d960b8b286b75c933780416ace9cda682fd


commit d4c76d960b8b286b75c933780416ace9cda682fd
Author: INADA Naoki 
Date:   Mon Oct 1 21:10:37 2018 +0900

bpo-30167: Add test for module.__cached__ is None (GH-7617)

diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py
index dc59e5917c..33a8f1a44c 100644
--- a/Lib/test/test_site.py
+++ b/Lib/test/test_site.py
@@ -389,6 +389,17 @@ class ImportSideEffectTests(unittest.TestCase):
 "expected absolute path, got {}"
 .format(os__cached__.decode('ascii')))

+def test_abs_paths_cached_None(self):
+"""Test for __cached__ is None.
+
+Regarding to PEP 3147, __cached__ can be None.
+
+See also: https://bugs.python.org/issue30167
+"""
+sys.modules['test'].__cached__ = None
+site.abs_paths()
+self.assertIsNone(sys.modules['test'].__cached__)
+


# Latest master

➜  cpython git:(master) ./python.exe -m test.regrtest test_inspect test_site
Run tests sequentially
0:00:00 load avg: 1.75 [1/2] test_inspect
0:00:02 load avg: 1.75 [2/2] test_site
test test_site failed -- Traceback (most recent call last):
  File 
"/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/test/test_site.py", 
line 400, in test_abs_paths_cached_None
site.abs_paths()
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/site.py", line 
101, in abs_paths
for m in set(sys.modules.values()):
TypeError: unhashable type: 'dict'

test_site failed

== Tests result: FAILURE ==

1 test OK.

1 test failed:
test_site

Total duration: 2 sec 868 ms
Tests result: FAILURE


➜  cpython git:(master) git checkout d4c76d960b8b286b75c933780416ace9cda682fd~1 
Lib/test/test_site.py
➜  cpython git:(master) ✗ ./python.exe -m test.regrtest test_inspect test_site
Run tests sequentially
0:00:00 load avg: 1.49 [1/2] test_inspect
0:00:02 load avg: 1.49 [2/2] test_site

== Tests result: SUCCESS ==

All 2 tests OK.

Total duration: 2 sec 848 ms
Tests result: SUCCESS

Adding INADA Naoki for thoughts.


Thanks

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34871] test_site fails if run after test_inspect

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> 100% pure SeaHash does x ^= t at the start first, instead of `t ^ (t << 1)` 
> on the RHS.

Indeed. Some initial testing shows that this kind of "input mangling" (applying 
such a permutation on the inputs) actually plays a much more important role to 
avoid collisions than the SeaHash operation x ^= ((x >> 16) >> (x >> 29)).

So my suggestion remains

for y in INPUT:
t = hash(y)
t ^= t * SOME_LARGE_EVEN_NUMBER
h ^= t
h *= MULTIPLIER

Adding in the additional SeaHash operations

x ^= ((x >> 16) >> (x >> 29))
x *= MULTIPLIER

does not increase the probability of the tests passing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34871] test_site fails if run after test_inspect

[Chih-Hsuan Yen]

New submission from Chih-Hsuan Yen :

$ ./python -m test.regrtest test_inspect test_site
Run tests sequentially
0:00:00 load avg: 0.03 [1/2] test_inspect
0:00:00 load avg: 0.03 [2/2] test_site
test test_site failed -- Traceback (most recent call last):
  File "/home/yen/Projects/cpython/Lib/test/test_site.py", line 400, in 
test_abs_paths_cached_None
site.abs_paths()
  File "/home/yen/Projects/cpython/Lib/site.py", line 101, in abs_paths
for m in set(sys.modules.values()):
TypeError: unhashable type: 'dict'

test_site failed


If I run test_inspect or test_site individually, both pass. Full log can be 
found in the attached file.

Environment: Arch Linux x86_64 latest. CPython commit 
81574b80e92554adf75c13fa42415beb8be383cb, configured with `./configure`

--
components: Tests
files: cpython-failure.txt
messages: 326881
nosy: yan12125
priority: normal
severity: normal
status: open
title: test_site fails if run after test_inspect
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file47842/cpython-failure.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34871] test_site fails if run after test_inspect

[STINNER Victor]

STINNER Victor  added the comment:

You can try to limit the number of tests needed to reproduce the bug using:

./python -m test.bisect -n 5 test_inspect test_site

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34867] Add mode to disable small integer and interned string caches

[Jakub Wilk]

Change by Jakub Wilk :


--
nosy: +jwilk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] return ENOTDIR when open() accepts filenames with a trailing slash

[STINNER Victor]

STINNER Victor  added the comment:

> 2018-10-02 11:02:32   Michael.Feltset files: + mime-attachment, 
> encrypted.asc

You replied with an encrypted message which isn't understood by the bug tracker.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34812] [EASY] support.args_from_interpreter_flags() doesn't inherit -I (isolated) flag

[Danish Prakash]

Danish Prakash  added the comment:

Thank you Karthikeyan, I'm going to take care of both of these issues.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

Correction: the FNV variant of SeaHash only fails the new testsuite, not the 
old one. The DJB variant of SeaHash fails both.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

SeaHash seems to be designed for 64 bits. I'm guessing that replacing the 
shifts by

x ^= ((x >> 16) >> (x >> 29))

would be what you'd do for a 32-bit hash. Alternatively, we could always 
compute the hash with 64 bits (using uint64_t) and then truncate at the end if 
needed.

However, when testing the hash function

for t in INPUT:
x ^= hash(t)
x *= MULTIPLIER
x ^= ((x >> 16) >> (x >> 29))
x *= MULTIPLIER

It fails horribly on the original and my new testsuite. I'm guessing that the 
problem is that the line x ^= ((x >> 16) >> (x >> 29)) ignores low-order bits 
of x, so it's too close to pure FNV which is known to have problems. When 
replacing the first line of the loop above by x += hash(t) (DJB-style), it 
becomes too close to pure DJB and it also fails horribly because of nested 
tuples.

So it doesn't seem that the line x ^= ((x >> 16) >> (x >> 29)) (which is what 
makes SeaHash special) really helps much to solve the known problems with DJB 
or FNV.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34812] [EASY] support.args_from_interpreter_flags() doesn't inherit -I (isolated) flag

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

> Thanks for bringing this up Karthikeyan, however, could there be another 
> reason why -I would be left out. Also, have you filed an issue for this?

I couldn't see any related issue for this though the table was changed in 3.7.0

> Also, Victor and Karthikeyan, since this issue has been categorized as an 
> easy issue, I would like to fix this if none of you have started working on 
> this.

I am not working on this. Feel free to pick it up.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34812] [EASY] support.args_from_interpreter_flags() doesn't inherit -I (isolated) flag

[Danish Prakash]

Danish Prakash  added the comment:

> With respect to documentation I was talking about '-I' not being documented 
> in the table at https://docs.python.org/3.7/library/sys.html#sys.flags though 
> it's present in the C code and in sys.flags.isolated.

Thanks for bringing this up Karthikeyan, however, could there be another reason 
why -I would be left out. Also, have you filed an issue for this?


Also, Victor and Karthikeyan, since this issue has been categorized as an easy 
issue, I would like to fix this if none of you have started working on this.

--
nosy: +danishprakash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] return ENOTDIR when open() accepts filenames with a trailing slash

[Michael Felt]

Change by Michael Felt :


Added file: https://bugs.python.org/file47840/mime-attachment
Added file: https://bugs.python.org/file47841/encrypted.asc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

This weekend I realized something important which I didn't realize before: some 
hash functions which I assumed to be good (i.e. small chance of collisions 
between any given two tuples) turned out to often fail the tests. This is 
because you don't just want to minimize collisions, you also want to minimize 
*correlations* between collisions.

More precisely: for a given hash function (considering the multiplier as 
parameter), it can happen that there are 4 tuples t, u, v, w such that whether 
or not hash(t) == hash(u) is correlated to whether or not hash(v) == hash(w). 
Such correlations increase the standard deviation of the number of collisions 
in the tests a lot (even if the average is unaffected), which leads to 
significant chances of failing the tests.

So with this in mind I stopped testing pairs of tuples but I ran the actual 
testsuites. The metric I'm using is now the probability that the testsuite 
passes for randomly chosen multipliers (3 mod 8). For example, the original 
tuple hash has a probability of around 97% of passing the original testsuite.

None of the hash functions that I tried (DJB or FNV with input mangling like t 
^= t << 7) achieved such a high probability of passing the original test. The 
*only* variation that I found which passes the original test and my new test 
(and a third "random" test which I haven't mentioned before) with a high enough 
probability was FNV with input mangling with a second multiplier:

h = 1
for y in INPUT:
t = hash(y)
t ^= t * SOME_LARGE_EVEN_NUMBER   # instead of t ^= t << SHIFT
h = (h ^ t) * MULTIPLIER

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32892] Remove specific constant AST types in favor of ast.Constant

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Submitted a fix for Pyflakes: https://github.com/PyCQA/pyflakes/pull/369. It 
had more problems due to changes in line and column numbers in SyntaxError.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34870] Core dump when Python VSCode debugger is attached

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> the author wants this transformation to be easily invertible, so a prime is 
> necessary

A multiplication by any odd number modulo 2**64 is invertible. As I argued 
before, the concept of primes is meaningless (except for the prime 2) when 
computing modulo 2**64.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34866] CGI DOS vulnerability via long post list

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34561] Replace list sorting merge_collapse()?

[Vincent Jugé]

Vincent Jugé  added the comment:

After having worked a little bit on improving AdaptiveShiversSort on few-run 
cases, I designed a new version of the algorithm, called shivers2 in the file 
runstack.py joined to this message

It looks more complicated than the original AdaptiveShiversSort but the spirit, 
inspired by your comments, is as follows:
- we allow the second (bottom-most) element of the stack to be moderately 
larger than the bottom-most one; if that 2nd element were way larger than the 
1st one, merging them, even if not optimal, would not be too painful anyways;
- we may force a merge between these 1st and 2nd elements only when the 2nd 
element is about to be merged with the 3rd one;
- similarly, we allow the top-most element of the stack to be moderately larger 
than the second top-most one;
- otherwise, we stick to the behaviour of AdaptiveShiversSort.

This variant's performance seems comparable with Powersort's.
Reasonable follow-up work that I plan to do in the coming weeks (when I have 
time to do so) is:
- prove that the new algorithm still runs in n H + O(n),
- investigate whether we can have guarantees such as "this new sort's merge 
cost is at most XXX times larger than the optimal merge cost",
- investigate improvements for Powersort.

Given its excellent overall performance, I think that trying to slightly tweak 
Powersort in cases it underperforms other sorts might be worth some effort.

Best,

Vincent

--
Added file: https://bugs.python.org/file47839/runstack.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23097] unittest can unnecessarily modify sys.path (and with the wrong case)

[Charlie Dyson]

Charlie Dyson  added the comment:

As an aside, should that be sys.path.insert(1, X)? As 0 has a special meaning 
(I've often thought this is a slightly odd convention).

Another aside: I noticed this because I was looking to write a module finder, 
and thought I could extract one out of this function. It would be nice to have 
a module-crawler function, and write discover() in terms of that.

--
nosy: +cdyson37

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34870] Core dump when Python VSCode debugger is attached

[Per Lundberg]

New submission from Per Lundberg :

My code has recently started triggering a core dump in the Python executable 
when the VSCode debugger is attached. This doesn't happen right away; it seems 
to happen more or less _after_ the program is done executing (I just placed a 
breakpoint and stepped it through).

The program in question is this: 
https://github.com/hiboxsystems/trac-to-gitlab/blob/master/migrate.py

To help in the debugging of this, I installed python2.7-dbg and gdb-python2 on 
my Debian machine, and re-ran the script using this version. Here is the GDB 
output when analyzing the backtrace:

$ gdb /usr/bin/python2.7-dbg core
GNU gdb (Debian 8.1-4+b1) 8.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/python2.7-dbg...done.
[New LWP 19749]
[New LWP 19744]
[New LWP 19747]
[New LWP 19754]
[New LWP 19751]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/python2.7-dbg -m ptvsd --host localhost --port 
43959 migrate.py --only'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  PyEval_EvalFrameEx (f=0x7f815c002310, throwflag=0) at ../Python/ceval.c:3347
3347if (tstate->frame->f_exc_type != NULL)
[Current thread is 1 (Thread 0x7f815bfff700 (LWP 19749))]


The python backtrace looks like this:

(gdb) py-bt
Traceback (most recent call first):
  File "/usr/lib/python2.7/threading.py", line 371, in wait
self._acquire_restore(saved_state)
  File "/usr/lib/python2.7/Queue.py", line 177, in get
self.not_empty.wait(remaining)
  File 
"/home/per/.vscode/extensions/ms-python.python-2018.8.0/pythonFiles/experimental/ptvsd/ptvsd/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py",
 line 458, in _on_run
cmd = self.cmdQueue.get(1, 0.1)
  File 
"/home/per/.vscode/extensions/ms-python.python-2018.8.0/pythonFiles/experimental/ptvsd/ptvsd/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py",
 line 319, in run
self._on_run()
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
  File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
self.__bootstrap_inner()


And the C-level backtrace:

(gdb) bt 
#0  PyEval_EvalFrameEx (f=Frame 0x7f815c002310, for file 
/usr/lib/python2.7/threading.py, line 371, in wait (), throwflag=0)
at ../Python/ceval.c:3347
#1  0x5624534af42c in PyEval_EvalCodeEx (co=0x7f816216e7d0, 
globals={'current_thread': None, '_BoundedSemaphore': None, 
'currentThread': None, '_Timer': None, '_format_exc': None, 'Semaphore': None, 
'_deque': None, 'activeCount': None, '_profile_hook': None, '_sleep': None, 
'_trace_hook': None, 'ThreadError': None, '_enumerate': None, 
'_start_new_thread': None, 'BoundedSemaphore': None, '_shutdown': None, 
'__all__': None, '_original_start_new_thread': None, '_Event': None, 
'active_count': None, '__package__': None, '_Condition': None, '_RLock': None, 
'_test': None, 'local': None, '__doc__': None, 'Condition': None, '_Verbose': 
None, '_DummyThread': None, 'Thread': None, 'warnings': None, '__builtins__': 
{'bytearray': None, 'IndexError': None, 'all': None, 'help': None, 'vars': 
None, 'SyntaxError': None, 'unicode': None, 'UnicodeDecodeError': None, 
'memoryview': None, 'isinstance': None, 'copyright': None, 'NameError': None, 
'BytesWarning': None, 'dict': None, 'input': None, 'oct': None, 'bin': None, 
'SystemExit': None, 'StandardError': No
 ne, 'format': None, 'repr': None, 'sor...(truncated), locals=0x0, 
args=0x562454463068, argcount=2, 
kws=0x562454463078, kwcount=0, defs=0x7f8162116408, defcount=1, 
closure=0x0) at ../Python/ceval.c:3604
#2  0x5624534b23a7 in fast_function (func=, pp_stack=0x7f815bffd3e8, n=2, na=2, nk=0)
at ../Python/ceval.c:4467
#3  0x5624534b1f8a in call_function (pp_stack=0x7f815bffd3e8, oparg=1) at 
../Python/ceval.c:4392
#4  0x5624534ac45d in PyEval_EvalFrameEx (
f=Frame 0x562454462eb0, for file /usr/lib/python2.7/Queue.py, line 177, in 
get (self=, maxsize=0, all_tasks_done=<_Condition(_Verbose__verbose=False, 
_Condition__lock=, acquire=, 
_Condition__waiters=[], release=) at remote 0x7f81610bc0d0>, mutex=, not_full=<_Condition(_Verbose__verbose=False, 
_Condition__lock=, acquire=, 
_Condition__waiters=[], release=) at remote 0x7f81610bc060>, 

[issue34430] Symmetrical chaining futures in asyncio.future.wrap_future

[Alfred Sawaya]

Alfred Sawaya  added the comment:

I use it to integrate a concurrent-based software (B) with an asyncio-based 
software (A).
(B) will undergo a massive refactoring to become asyncio-based in the future, 
but for now, I need to use it as-is.
I don't want to modify (A) to handle concurrent.futures.Future (it is 
developped by another team by the way) and I prefer to begin to implement 
asyncio into (B).

So (A) waits for a task request with an asyncio.Future, then processes it and 
populates the Future's result.
(B) has a waiting loop into a thread that handles multiples 
concurrent.futures.Future.

When (B) receives a request it creates a concurrent.futures.Future and push it 
into a queue for the waiting loop. If the request asks for a task implemented 
by (A), it wraps it into an asyncio.Future and call (A) with it.
(A) processes the task and populates the asyncio.Future's result. That's why I 
needed to implement the proposed behaviours.

Actually I will not need it for long, as (B) will become asyncio-based, but I 
wanted to integrate (A) and (B) with as less effort as possible, so I 
implemented my custom Future wrapper, but I think it is useful to refactor 
concurrent-based software to asyncio incrementally (and I will have to do it 
multiple times in the near future).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34711] return ENOTDIR when open() accepts filenames with a trailing slash

[STINNER Victor]

STINNER Victor  added the comment:

Jeremy Kloth: "This is also an issue on Windows when the target path resides 
within a junction, paths outside of a junction respond (err, fail) as expected."
https://developercommunity.visualstudio.com/content/problem/272379/createfile-non-error-on-filename-with-trailing-bac.html

I don't know the behavior on Windows. I tried to create a file name "a\" 
(U+0061, U+005C): I get an OSError "invalid argument" (error 22).

I confirm that in a junction point, I am able to:

* open an existing file with an additional trailing antislash (U+005C): the 
extra character is simply ignored and I am able to open the file

* create a new file with an additional trailing antislash (U+005C): the 
filename is created without the trailing antislash

On the PR, I wrote:


> There are much more functions which "open a file". Open Python/fileutils.c 
> for a few mores. What about os.open()? What about all other functions which 
> accept a filename and then call a third party library which calls open() 
> directly?

Ok, let me give some examples of function which directly open a file:

* fileutils.c: _Py_open(), _Py_open_noraise(), _Py_wfopen(), _Py_fopen(), 
_Py_fopen_obj()
* os.open()
* _io.FileIO, _pyio.FileIO (use os.open())

Ok... But there are other functions to access files... stat()/fstat() functions:

* fileutils.c: _Py_fstat_noraise(), _Py_fstat(), _Py_stat()
* Modules/getpath.c: _Py_wstat()
* os.stat(), os.lstat(), os.fstat()

To start to have a better ideas of how many functions accept filenames, open 
also Lib/shutil.py. shutil.copyfile() uses os.stat(), but then it uses 
os.symlink() and open()... So what about os.symlink()?

Ok, here I only listen a *few* examples of functions which are "controlled" by 
Python. But there are *many* wrappers to 3rd party libraries which accept a 
filename. Examples:

* _ssl.SSLContext.load_cert_chain()
* sqlite3.connect()
* etc.

Where is the limit? How many functions must be patched in Python? How do we 
patch OpenSSL and SQLite libraries?

Python is designed as a thin wrapper to the operating system. IMHO Python must 
not validate the filename itself.

--

> Going back to issue17234 - there has been a test to check that a URL with a 
> trailing slash reports 404 status.

IMHO you must fix a single place: the SimpleHTTPServer, not all code handling 
the filesytem.

Same remark for AIX and Windows junctions.

I suggest to reject this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34728] deprecate *loop* argument for asyncio.sleep

[Andrew Svetlov]

Andrew Svetlov  added the comment:

My understanding is:

`loop` argument passed to sleep should be always the same as returned from 
`get_running_loop()`.

Passing it explicitly can be considered as microoptimization but 
`get_running_loop()` is pretty fast now, no need for such micro-opts.

On another hand passing *non-current* loop is a serious error: nothing prevents 
to do it but the code just hangs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25300] Enable Intel MPX (Memory protection Extensions) feature

[STINNER Victor]

STINNER Victor  added the comment:

> I'm going to formally reject this, since GCC removed -fmpx support.

"The MPX extensions to the C and C++ languages have been deprecated and will be 
removed in a future release."
https://gcc.gnu.org/gcc-8/changes.html

Oh. It seems like the reason is to "reduce the maintenance burden".

Some links:

* https://www.kernel.org/doc/Documentation/x86/intel_mpx.txt
* https://en.wikipedia.org/wiki/Intel_MPX
* 
https://www.phoronix.com/scan.php?page=news_item=GCC-Possible-MPX-Deprecation

A patch has been proposed to remove MPX from GCC 9 (no merged?):
https://www.phoronix.com/scan.php?page=news_item=GCC-Patch-To-Drop-MPX

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34728] deprecate *loop* argument for asyncio.sleep

[STINNER Victor]

STINNER Victor  added the comment:

FYI if I recall correctly, in the past, we preferred to pass explicitly the 
loop to avoid to have to get the current loop which may add an overhead. But 
the current trend is to get rid of the explicit loop parameter.

> asyncio.sleep is a coroutine; passing a *loop* argument to it makes no sense 
> anymore.

sleep() requires the current event loop:

if loop is None:
loop = events.get_running_loop()
else:
warnings.warn("The loop argument is deprecated and scheduled for "
  "removal in Python 3.10.",
  DeprecationWarning, stacklevel=2)

future = loop.create_future()
h = loop.call_later(delay,
futures._set_result_unless_cancelled,
future, result)

Why does it not make sense to pass the loop to sleep? "it makes no sense 
anymore" something changes?

I'm not against the change, I'm just trying to understand the rationale for 
other changes :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34728] deprecate *loop* argument for asyncio.sleep

[STINNER Victor]

STINNER Victor  added the comment:

There is a new PR, so I change the issue resolution again.

--
resolution: fixed -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34728] deprecate *loop* argument for asyncio.sleep

[STINNER Victor]

STINNER Victor  added the comment:

> Victor, you're looking at an outdated comment on this issue.

No, I read the commit:

https://github.com/python/cpython/commit/558c49bcf3a8543d64a68de836b5d855efd56696


warnings.warn("The loop argument is deprecated and scheduled for"
  "removal in Python 4.0.",
  DeprecationWarning, stacklevel=2)

> This is what's in the master branch:

https://github.com/python/cpython/blob/d4c76d960b8b286b75c933780416ace9cda682fd/Lib/asyncio/tasks.py#L598-L599

Ah, you forgot to mention this bpo in your commit:

commit fad6af2744c0b022568f7f4a8afc93fed056d4db
Author: Yury Selivanov 
Date:   Tue Sep 25 17:44:52 2018 -0400

asyncio/docs: Replace Python 4.0 -> 3.10 (GH-9579)

Anyway, the current code is fine. Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34850] Emit a syntax warning for "is" with a literal

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> I like this solution of a syntax warning for `is ` 
> and I agree that `== None` should not be a warning.

+1 for me as well.  Besides catching potential bugs, it will be of instant 
benefit for teaching Python.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34867] Add mode to disable small integer and interned string caches

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I don't think this option will be of any value.  For it to work, the code would 
need to have this particular bug, have test cases that triggered those bugs, 
and a user sophisticated enough to run the tests but unsophisticated enough to 
make beginner mistakes regarding when to use identity tests versus equality 
tests (something I teach on day one of beginner Python courses).

Before this goes further, I would like to see some evidence that it would 
actually catch a real bug in the wild.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com