[issue21899] Futures are not marked as completed

[Sebastian Kreft]

Sebastian Kreft added the comment:

After more testing I finally found that in fact the process is not being 
killed. That means that there is no problem with the futures. But instead it is 
probably related with subprocess deadlocking, as the problematic process does 
not consume any CPU.

Sorry for the false report.

--
nosy:  -bquinlan
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21899] Futures are not marked as completed

[Ned Deily]

Changes by Ned Deily :


--
nosy: +bquinlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21899] Futures are not marked as completed

[Sebastian Kreft]

New submission from Sebastian Kreft:

With Python 3.4.1 compiled from source, I'm having an issue in which every now 
and then some Futures are not marked as completed even though the underlying 
workload is done.

My workload is launching two subprocess in parallel, and whenever one is ready, 
launches another one. In one of the runs, the whole process got stuck after 
launching about 3K subprocess, and the underlying processes had in fact 
finished.

To wait for the finished subprocesses, I'm using FIRST_COMPLETED. Below is the 
core of my workload:

for element in element_generator:
while len(running) >= max_tasks:
done, pending = concurrent.futures.wait(running, timeout=15.0, 
return_when=concurrent.futures.FIRST_COMPLETED)
process_results(done)
running = pending

running.add(executor.submit(exe_subprocess, element)) 
 

Replicating the issue takes time, but I've been able to successfully reproduce 
it with 2 and 3 processes in parallel.

Note: this was already posted in comments to http://bugs.python.org/issue20319, 
however it did not receive the proper atention as that issue is already closed.

As suggested there I printed the status of the never finished futures and this 
is the result:

State: RUNNING, Result: None, Exception: None, Waiters: 0, Cancelled: False, 
Running: True, Done: False

The information does not seem very relevant. However, I can attach a console 
and debug from there.

--
messages: 222034
nosy: Sebastian.Kreft.Deezer
priority: normal
severity: normal
status: open
title: Futures are not marked as completed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com