[issue26924] android: test_concurrent_futures fails

[Davin Potts]

Changes by Davin Potts :


--
nosy: +davin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

> #  undef sem_open

I meant of course:

#  undef HAVE_SEM_OPEN

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

Undefining the SEM_* macros in multiprocessing.h when __ANDROID__ is set, would 
allow for an easier update when those functions become implemented at a future 
API level. The __ANDROID_API__ macro can be used for that purpose. For example, 
if this happens at API 24, this could be a changed to:

#if defined(__ANDROID__)
# include 
# if __ANDROID_API__ < 24
#  undef sem_open
#  ...
# endif
#endif

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Berker Peksag]

Berker Peksag added the comment:

Thanks! Perhaps we should not set HAVE_SEM_* to 1 if we are on Android in the 
configure step.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

sem_open() is not implemented as well. Here is a gdb session with a breakpoint 
set at semlock_new() (SEM_FAILED is NULL on android):

Breakpoint 1, semlock_new (type=0xb6d070c0 <_PyMp_SemLockType>, 
args=(1, 1, 1, '/mp-dkzrq4ed', True), kwds=0x0)
at 
/home/xavier/src/packages/android/cpython/Modules/_multiprocessing/semaphore.c:420
420 {
(gdb) next
428 if (!PyArg_ParseTupleAndKeywords(args, kwds, "iiisi", kwlist,
(gdb) 
432 if (kind != RECURSIVE_MUTEX && kind != SEMAPHORE) {
(gdb) 
437 if (!unlink) {
(gdb) 
424 char *name, *name_copy = NULL;
(gdb) 
445 handle = SEM_CREATE(name, value, maxvalue);
(gdb) 
447 if (handle == SEM_FAILED || SEM_GET_LAST_ERROR() != 0)
(gdb) print handle
$5 = (SEM_HANDLE) 0x0
(gdb) next
460 if (handle != SEM_FAILED)
(gdb) 
462 PyMem_Free(name_copy);
(gdb) 
463 _PyMp_SetError(NULL, MP_STANDARD_ERROR);
(gdb) 
464 return NULL;
(gdb) 


This is run with sem_unlink_alt.diff modified to include the change to 'define 
SEM_UNLINK(name) 0' although I guess this does not change anything.

Sorry for misleading you about sem_unlink().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

There is a typo in sem_unlink_alt.diff, not in the patch itself but in the 
neighbouring lines. In Modules/_multiprocessing/semaphore.c, it should be, I 
think, after one does 's/define sem_unlink(name)/define SEM_UNLINK(name)/':

@@ -194,8 +194,8 @@
 #  define SEM_FAILED ((sem_t *)-1)
 #endif
 
-#ifndef HAVE_SEM_UNLINK
-#  define sem_unlink(name) 0
+#if !defined(HAVE_SEM_UNLINK) || defined(__ANDROID__)
+#  define SEM_UNLINK(name) 0
 #endif
 
 #ifndef HAVE_SEM_TIMEDWAIT


But applying this new patch, gives exactly the same results. Maybe it is not 
just sem_unlink() that is not implemented. I will run a gdb session to find out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

Attached the results with sem_unlink_alt.diff.

--
Added file: http://bugs.python.org/file42774/sem_unlink_alt_output.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Berker Peksag]

Berker Peksag added the comment:

Xavier, could you try this alternative patch?

--
Added file: http://bugs.python.org/file42773/sem_unlink_alt.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Roundup Robot]

Roundup Robot added the comment:

New changeset eee959fee5f5 by Berker Peksag in branch 'default':
Issue #26924: Fix Windows buildbots
https://hg.python.org/cpython/rev/eee959fee5f5

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Berker Peksag]

Berker Peksag added the comment:

Thanks, Xavier.

--
components: +Extension Modules -Cross-Build, Library (Lib)
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Roundup Robot]

Roundup Robot added the comment:

New changeset 1030aa8357a7 by Berker Peksag in branch 'default':
Issue #26924: Do not define _multiprocessing.sem_unlink under Android
https://hg.python.org/cpython/rev/1030aa8357a7

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

> Here is the result of the test:

I meant, results with the new patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

Here is the result of the test:

root@generic_x86:/data/local/tmp # python -m test -v test_concurrent_futures
== CPython 3.6.0a0 (default:811ccdee6f87+, May 7 2016, 18:51:28) [GCC 4.9 
20140827 (prerelease)]
==   Linux-3.4.67+-i686-with-libc little-endian
==   hash algorithm: fnv 32bit
==   /data/local/tmp/test_python_2024
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, 
dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, 
verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
Run tests sequentially
0:00:00 [1/1] test_concurrent_futures
test_concurrent_futures skipped -- This platform lacks a functioning sem_open 
implementation, therefore, the required synchronization primitives needed will 
not function, see issue 3770.
1 test skipped:
test_concurrent_futures
Total duration: 0:00:01

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Berker Peksag]

Berker Peksag added the comment:

Thanks for testing. Here is a new patch.

--
stage:  -> patch review
Added file: http://bugs.python.org/file42769/sem_unlink_v2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

Patch tested, it does not fix the problem.
This is because in pyconfig.h:
/* Define to 1 if you have the `sem_unlink' function. */
#define HAVE_SEM_UNLINK 1

Android declares the function but does not implement it. This is not the only 
case, for example issue #26857.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Berker Peksag]

Berker Peksag added the comment:

It normally should raise an ImportError if sem_unlink is not available. Could 
you try the attached patch please?

--
keywords: +patch
nosy: +berker.peksag
Added file: http://bugs.python.org/file42710/sem_unlink.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Xavier de Gaye]

Xavier de Gaye added the comment:

On android:

root@generic_x86:/data/local/tmp # python
Python 3.6.0a0 (default:811ccdee6f87+, May  4 2016, 10:31:14) 
[GCC 4.9 20140827 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from _multiprocessing import sem_unlink
>>> sem_unlink('foo')
Traceback (most recent call last):
  File "", line 1, in 
OSError: [Errno 38] Function not implemented
>>>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26924] android: test_concurrent_futures fails

[Xavier de Gaye]

New submission from Xavier de Gaye:

test_concurrent_futures fails on an android emulator running an x86 system 
image at API level 21.

There are multiple ERRORs, all resulting in the same traceback:

Traceback (most recent call last):
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.6/test/test_concurrent_futures.py", 
line 68, in setU
p
self.executor = self.executor_type(max_workers=self.worker_count)
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.6/concurrent/futures/process.py", line 
390, in __init__
EXTRA_QUEUED_CALLS)
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/multiprocessing/context.py", 
line 101, in Queue
return Queue(maxsize, ctx=self.get_context())
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/multiprocessing/queues.py", 
line 42, in __init__
self._rlock = ctx.Lock()
  File "/sdcard/org.bitbucket.pyona/lib/python3.6/multiprocessing/context.py", 
line 66, in Lock
return Lock(ctx=self.get_context())
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.6/multiprocessing/synchronize.py", 
line 163, in __in
it__
SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.6/multiprocessing/synchronize.py", 
line 60, in __ini
t__
unlink_now)
OSError: [Errno 38] Function not implemented

The ERRORs:
===

test_duplicate_futures 
(test.test_concurrent_futures.ProcessPoolAsCompletedTests) ... ERROR
test_no_timeout (test.test_concurrent_futures.ProcessPoolAsCompletedTests) ... 
ERROR
test_zero_timeout (test.test_concurrent_futures.ProcessPoolAsCompletedTests) 
... ERROR
test_killed_child (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 
ERROR
test_map (test.test_concurrent_futures.ProcessPoolExecutorTest) ... ERROR
test_map_chunksize (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 
ERROR
test_map_exception (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 
ERROR
test_map_timeout (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 
ERROR
test_max_workers_negative 
(test.test_concurrent_futures.ProcessPoolExecutorTest) ... ERROR
test_no_stale_references (test.test_concurrent_futures.ProcessPoolExecutorTest) 
... ERROR
test_shutdown_race_issue12456 
(test.test_concurrent_futures.ProcessPoolExecutorTest) ... ERROR
test_submit (test.test_concurrent_futures.ProcessPoolExecutorTest) ... ERROR
test_submit_keyword (test.test_concurrent_futures.ProcessPoolExecutorTest) ... 
ERROR
test_traceback (test.test_concurrent_futures.ProcessPoolExecutorTest) ... ERROR
test_context_manager_shutdown 
(test.test_concurrent_futures.ProcessPoolShutdownTest) ... ERROR
test_del_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 
ERROR
test_hang_issue12364 (test.test_concurrent_futures.ProcessPoolShutdownTest) ... 
ERROR
test_interpreter_shutdown 
(test.test_concurrent_futures.ProcessPoolShutdownTest) ... ERROR
test_processes_terminate (test.test_concurrent_futures.ProcessPoolShutdownTest) 
... ERROR
test_run_after_shutdown (test.test_concurrent_futures.ProcessPoolShutdownTest) 
... ERROR
test_all_completed (test.test_concurrent_futures.ProcessPoolWaitTests) ... ERROR
test_first_completed (test.test_concurrent_futures.ProcessPoolWaitTests) ... 
ERROR
test_first_completed_some_already_completed 
(test.test_concurrent_futures.ProcessPoolWaitTests) ...  ERROR
test_first_exception (test.test_concurrent_futures.ProcessPoolWaitTests) ... 
ERROR
test_first_exception_one_already_failed 
(test.test_concurrent_futures.ProcessPoolWaitTests) ... ERROR
test_first_exception_some_already_complete 
(test.test_concurrent_futures.ProcessPoolWaitTests) ... ERROR
test_timeout (test.test_concurrent_futures.ProcessPoolWaitTests) ... ERROR

--
components: Cross-Build, Library (Lib)
messages: 264721
nosy: Alex.Willmer, bquinlan, xdegaye
priority: normal
severity: normal
status: open
title: android: test_concurrent_futures fails
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com