Re: [Python-Dev] Issue #10348: concurrent.futures doesn't work on BSD
Le jeudi 30 décembre 2010 à 17:05 +0100, Martin v. Löwis a écrit : I really don't think it is our job to maintain a list of OS/versions which work and don't work. Of course not. I would propose a dynamic test: check how many POSIX semaphores the installation supports, and fail if it's less than 200 (say). I added informations about FreeBSD, NetBSD, Darwin and OpenBSD to the issue #10348: http://bugs.python.org/issue10348#msg125042 The maximum number of POSIX semaphores can be read with sysctl: - FreeBSD: p1003_1b.sem_nsems_max - NetBSD: kern.posix.semmax - Darwin: kern.posix.sem.max - OpenBSD: (no support) Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
Le mercredi 29 décembre 2010 à 14:20 +0100, Martin v. Löwis a écrit : Am 28.12.2010 18:08, schrieb Lukas Lueg: Also, the load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object That's not true; a debugger may change the value of x. That's why Python has the following option: -O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x I regulary recompile programs with gcc -O0 -g to debug them. It is very difficult to debug (with gdb) a program compiled with gcc -O2: many variables are stored in registers, and gdb doesn't support that correctly. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87603 - python/branches/py3k/Misc/NEWS
On Sat, 01 Jan 2011 16:07:02 -0500, Terry Reedy tjre...@udel.edu wrote: On 1/1/2011 5:07 AM, georg.brandl wrote: Author: georg.brandl Date: Sat Jan 1 11:07:30 2011 New Revision: 87603 Log: Fix issue references. (add '#' to issue numbers). Whoops, two of those are mine. I am still learning and will try to remember to include it in both log messages and NEWS entries. Heh. I think two of them were mine, and I'm supposed to know better by now. -- R. David Murray www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r87577 - in python/branches/py3k: Makefile.pre.in configure configure.in pyconfig.h.in
Am 30.12.2010 15:55, schrieb martin.v.loewis:
Author: martin.v.loewis
Date: Thu Dec 30 15:55:47 2010
New Revision: 87577
Log:
Build and install libpython3.so.
Modified: python/branches/py3k/configure.in
==
--- python/branches/py3k/configure.in (original)
+++ python/branches/py3k/configure.in Thu Dec 30 15:55:47 2010
@@ -737,6 +738,10 @@
BLDLIBRARY='-Wl,-R,$(LIBDIR) -L. -lpython$(LDVERSION)'
RUNSHARED=LD_LIBRARY_PATH=`pwd`:${LD_LIBRARY_PATH}
INSTSONAME=$LDLIBRARY.$SOVERSION
+ if test $with_pydebug == no
+ then
+ PY3LIBRARY=libpython3.so
+ fi
;;
Linux*|GNU*|NetBSD*|FreeBSD*|DragonFly*)
LDLIBRARY='libpython$(LDVERSION).so'
@@ -748,6 +753,11 @@
;;
esac
INSTSONAME=$LDLIBRARY.$SOVERSION
+ PY3LIBRARY=libpython3.so
+ if test $with_pydebug == no
+ then
+ PY3LIBRARY=libpython3.so
+ fi
;;
hp*|HP*)
case `uname -m` in
These changes do not work as written: if --with-pydebug is not given,
$with_pydebug is empty, not no. Also, in the second case the
unconditional PY3LIBRARY assignment should probably be deleted.
cheers,
Georg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #10348: concurrent.futures doesn't work on BSD
I added informations about FreeBSD, NetBSD, Darwin and OpenBSD to the issue #10348: http://bugs.python.org/issue10348#msg125042 The maximum number of POSIX semaphores can be read with sysctl: - FreeBSD: p1003_1b.sem_nsems_max - NetBSD: kern.posix.semmax - Darwin: kern.posix.sem.max - OpenBSD: (no support) I've been using os.sysconf(SC_SEM_NSEMS_MAX), which seems to have worked fine (it's also what POSIX mandates). See #10798. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
On 1/2/2011 8:17 AM, Victor Stinner wrote: Le mercredi 29 décembre 2010 à 14:20 +0100, Martin v. Löwis a écrit : Am 28.12.2010 18:08, schrieb Lukas Lueg: Also, the load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object That's not true; a debugger may change the value of x. That's why Python has the following option: -O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x I regulary recompile programs with gcc -O0 -g to debug them. It is very difficult to debug (with gdb) a program compiled with gcc -O2: many variables are stored in registers, and gdb doesn't support that correctly. Victor, you seem to be equating the gcc -O flag with the Python -O flag. They are described similarly, but can't be used the same way. In particular, there is no Python equivalent to gcc's -O0: there is no way to disable the Python peephole optimizer. --Ned. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
Cesare Di Mauro cesare.di.mauro at gmail.com writes: 2010/12/28 Lukas Lueg lukas.lueg at googlemail.com Consider the following code: def foobar(x): for i in range(5): x[i] = i The bytecode in python 2.7 is the following: 2 0 SETUP_LOOP 30 (to 33) 3 LOAD_GLOBAL 0 (range) 6 LOAD_CONST 1 (5) 9 CALL_FUNCTION 1 12 GET_ITER 13 FOR_ITER 16 (to 32) 16 STORE_FAST 1 (i) 3 19 LOAD_FAST 1 (i) 22 LOAD_FAST 0 (x) 25 LOAD_FAST 1 (i) 28 STORE_SUBSCR 29 JUMP_ABSOLUTE 13 32 POP_BLOCK 33 LOAD_CONST 0 (None) 36 RETURN_VALUE Can't we optimize the LOAD_FAST in lines 19 and 25 to a single load and put the reference twice on the stack? There is no way that the reference of i might change in between the two lines. Also, the load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object Yes, you can, but you need: - a better AST evaluator (to mark symbols/variables with proper attributes); - a better optimizer (usually located on compile.c) which has a global vision (not limited to single instructions and/or single expressions). It's not that simple, and the results aren't guaranteed to be good. Also, consider that Python, as a dynamic-and-not-statically-compiled language need to find a good trade-off between compilation time and execution. Just to be clear, a C program is usually compiled once, then executed, so you can spend even *hours* to better optimize the final binary code. With a dynamic language, usually the code is compiled and the executed as needed, in realtime. So it isn't practical neither desirable having to wait too much time before execution begins (the startup problem). Python stays in a gray area, because modules are usually compiled once (when they are first used), and executed many times, but it isn't the only case. You cannot assume that optimization techniques used on other (static) languages can be used/ported in Python. Cesare No, it's singularly impossible to prove that any global load will be any given value at compile time. Any optimization based on this premise is wrong. Alex ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
On Sun, Jan 2, 2011 at 5:50 PM, Alex Gaynor alex.gay...@gmail.com wrote: No, it's singularly impossible to prove that any global load will be any given value at compile time. Any optimization based on this premise is wrong. True. My proposed way out of this conundrum has been to change the language semantics slightly so that global names which (a) coincide with a builtin, and (b) have no explicit assignment to them in the current module, would be fair game for such optimizations, with the understanding that the presence of e.g. len = len anywhere in the module (even in dead code!) would be sufficient to disable the optimization. But barring someone interested in implementing something based on this rule, the proposal has languished for many years. FWIW, this is reminiscent of Fortran's rules for intrinsics (its name for builtins), which have a similar optimization behavior (except there the potential overrides that the compiler doesn't need to take into account are load-time definitions). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
On 1/2/2011 10:18 PM, Guido van Rossum wrote:
My proposed way out of this conundrum has been to change the language
semantics slightly so that global names which (a) coincide with a
builtin, and (b) have no explicit assignment to them in the current
module, would be fair game for such optimizations, with the
understanding that the presence of e.g. len = len anywhere in the
module (even in dead code!) would be sufficient to disable the
optimization.
I believe this amounts to saying
1) Python code executes in three scopes (rather than two): global
builtin, modular (misleadingly call global), and local. This much is a
possible viewpoint today.
2) A name that is not an assignment target anywhere -- and that matches
a builtin name -- is treated as a builtin. This is the new part, and it
amounts to a rule for entire modules that is much like the current rule
for separating local and global names within a function. The difference
from the global/local rule would be that unassigned non-builtin names
would be left to runtime resolution in globals.
It would seem that this new rule would simplify the lookup of module
('global') names since if xxx in not in globals, there is no need to
look in builtins. This is assuming that following 'len=len' with 'del
len' cannot 'unmodularize' the name.
For the rule to work 'retroactively' within a module as it does within
functions would require a similar preliminary pass. So it could not work
interactively. Should batch mode main modules work the same as when
imported?
Interactive mode could work as it does at present or with slight
modification, which would be that builtin names within functions, if not
yet overriden, also get resolved when the function is compiled.
--
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
2011/1/1 Ned Batchelder n...@nedbatchelder.com On 12/31/2010 12:51 PM, Cesare Di Mauro wrote: Aggressive optimizations can be enabled with explicit options, in order to leave normal debugger-prone code. I wish the Python compiler would adopt a strategy of being able to disable optimizations. I wrote a bug about a leaky abstraction optimization messing up coverage testing 2.5 years ago, and it was closed as won't fix: http://bugs.python.org/issue2506. The debate there centered around, but that line isn't executed, because it's been optimized away. It's common in sophisticated compilers (as in, any C compiler) to be able to choose whether you want optimizations for speed, or disabling optimizations for debugging and reasoning about the code. Python would benefit from the same choice. --Ned. Command line parameters and/or environment variables are suitable for this, but they aren't immediate and, also, have global effect. I wish an explicit (Explicit is better than implicit) and a finer control over optimizations, with a per-module usage: from __compiler__ import disable_peepholer, strict_syntax, static_builtins, globals_as_fasts Cesare ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
2011/1/3 Alex Gaynor alex.gay...@gmail.com No, it's singularly impossible to prove that any global load will be any given value at compile time. Any optimization based on this premise is wrong. Alex That's your opinion, but I have very different ideas. Of course we can't leave the problem only on the compiler shoulders, but I think that can be ways to threat builtins as static variables, and globals like local (fast) variables too, taking into account changes on the builtins' and modules dictionaries. But it doesn't make sense to invest time in these things: JITs are becoming a good alternative, and may be they will be ready soon to take the CPython place as the mainstream implementation. Cesare ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible optimization for LOAD_FAST ?
On Mon, Jan 3, 2011 at 3:36 PM, Terry Reedy tjre...@udel.edu wrote:
I believe this amounts to saying
1) Python code executes in three scopes (rather than two): global builtin,
modular (misleadingly call global), and local. This much is a possible
viewpoint today.
2) A name that is not an assignment target anywhere -- and that matches a
builtin name -- is treated as a builtin. This is the new part, and it
amounts to a rule for entire modules that is much like the current rule for
separating local and global names within a function. The difference from the
global/local rule would be that unassigned non-builtin names would be left
to runtime resolution in globals.
It would seem that this new rule would simplify the lookup of module
('global') names since if xxx in not in globals, there is no need to look in
builtins. This is assuming that following 'len=len' with 'del len' cannot
'unmodularize' the name.
For the rule to work 'retroactively' within a module as it does within
functions would require a similar preliminary pass. So it could not work
interactively. Should batch mode main modules work the same as when
imported?
Interactive mode could work as it does at present or with slight
modification, which would be that builtin names within functions, if not yet
overriden, also get resolved when the function is compiled.
This could potentially be handled by having the exec mode in
compile() assume it can see all the global assignments (and hence
assume builtin names refer to the builtins), while single would
assume this was not the case (and hence skip the optimisation). It may
also need an additional parameter to tell the compiler which names are
known to be visible in the current locals and globals (e.g. to allow
exec() to do the right thing)
This kind of issue is the reason Guido pointed out the idea really
needs someone else to pick it up and run with it - not just to figure
out the implementation details, but also to ferret out the full
implications for the language semantics and backwards compatibility.
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
