[issue41694] python3 futures.as_completed timeout broken if future contains undefined reference

[Josh Rosenberg]

Josh Rosenberg  added the comment:

The problem is a lot simpler than you're making it:

1. You submit a time.sleep(30) task. This begins running immediately
2. You try to submit another task, but a NameError is raised, bypassing the 
rest of the code (you never call as_completed, with or without a timeout)
3. The ThreadPoolExecutor's __exit__ is invoked, which implicitly invokes 
shutdown(wait=True). This does not return until the successfully submitted task 
(time.sleep(30)) finished.
4. At that point, the exception that was interrupted by with statement cleanup 
resumes bubbling

All of this is behaving exactly as documented, no bug is occurring.

--
nosy: +josh.r
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41694] python3 futures.as_completed timeout broken if future contains undefined reference

[Dallas Marlow]

Dallas Marlow  added the comment:

I apologize for all the messages, but the more I look into this issue the 
stranger it seems. The following code does the following:

- print the time
- execute 3 futures (2 w/ 30s sleeps)
- call as_completed
- catch the timeout exception
- print the time again
- then raise the timeout exception, but only after all futures complete.

the second time print executes after the as_completed timeout duration, but the 
exception was only raised after all futures completed. the exception message 
incorrectly states that it left 2/3 futures unfinished.

#

import concurrent.futures
import time

with concurrent.futures.ThreadPoolExecutor() as ex:
print(time.time())
futures = [
ex.submit(time.sleep, 30),
ex.submit(time.sleep, 30),
ex.submit(print, 'test'),
]
try:
for future in concurrent.futures.as_completed(
futures, timeout=1):
_ = future.result()
except Exception as e:
print(time.time())
raise e

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41694] python3 futures.as_completed timeout broken if future contains undefined reference

[Dallas Marlow]

Dallas Marlow  added the comment:

I also verified that this issue persists with blocking futures other than sleep 
calls:

import concurrent.futures
import subprocess

with concurrent.futures.ThreadPoolExecutor() as ex:
futures = [
ex.submit(subprocess.run, ['sleep', '30']),
ex.submit(print, 'test'),
]
for future in concurrent.futures.as_completed(futures, timeout=1):
_ = future.result()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41694] python3 futures.as_completed timeout broken if future contains undefined reference

[Dallas Marlow]

Dallas Marlow  added the comment:

After closer inspection it seems as though timeouts are not working as I would 
expect even when there are no undefined references. The following code runs for 
30s when I would expect a timeout exception thrown after 1 second. as_completed 
eventually does throw a timeout exception, but only after the sleep future 
completes.

###

import concurrent.futures
import time

with concurrent.futures.ThreadPoolExecutor() as ex:
futures = [
ex.submit(time.sleep, 30),
ex.submit(print, 'test'),
]
for future in concurrent.futures.as_completed(futures, timeout=1):
_ = future.result()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41694] python3 futures.as_completed timeout broken if future contains undefined reference

[Dallas Marlow]

New submission from Dallas Marlow :

I recently noticed that timeouts do not work with futures.as_completed if a 
future contains an undefined variable or method. The following code runs for 30 
seconds which I believe is a bug as I would expect the invalid print future to 
throw an exception immediately or the as_completed method throw an exception 
instead.

#

import concurrent.futures

with concurrent.futures.ThreadPoolExecutor() as ex:
futures = [
ex.submit(time.sleep, 30),
ex.submit(print, a),
]
for future in concurrent.futures.as_completed(futures, timeout=1):
_ = future.result()

--
components: Interpreter Core
messages: 376250
nosy: dallasmarlow
priority: normal
severity: normal
status: open
title: python3 futures.as_completed timeout broken if future contains undefined 
reference
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com