Re: [Python-Dev] [Python-ideas] Remove GIL with CAS instructions?

[Kristján Valur Jónsson]

> -Original Message-
> From: python-ideas-bounces+kristjan=ccpgames@python.org
> [mailto:python-ideas-bounces+kristjan=ccpgames@python.org] On
> Behalf Of Sturla Molden
> Sent: 20. október 2009 22:13
> To: python-id...@python.org
> Subject: Re: [Python-ideas] Remove GIL with CAS instructions?
> 
> 
> - The GIL has consequences on multicore CPUs that are overlooked:
> thread
> switches are usually missed at check intervals. This could be fixed
> without removing the GIL: For example, there could be a wait-queue for
> the GIL; a thread that request the GIL puts itself in the back.
> 


This depends entirely on the platform and primitives used to implement the GIL.
I'm interested in windows.  There, I found this article:
http://fonp.blogspot.com/2007/10/fairness-in-win32-lock-objects.html
So, you may be on to something.  Perhaps a simple C test is in order then?

I did that.  I found, on my dual-core vista machine, that running "release", 
that both Mutexes and CriticalSections behaved as you describe, with no 
"fairness".  Using a "semaphore" seems to retain fairness, however.
"fairness" was retained in debug builds too, strangely enough.

Now, Python uses none of these.  On windows, it uses an "Event" object coupled 
with an atomically updated counter.  This also behaves fairly.

The test application is attached.


I think that you ought to sustantiate your claims better, maybe with a specific 
platform and using some test like the above.

On the other hand, it shows that we must be careful what we use.  There has 
been some talk of using CriticalSections for the GIL on windows.  This test 
ought to show the danger of that.  The GIL is different than a regular lock.  
It is a reverse-lock, really, and therefore may need to be implemented in its 
own special way, if we want very fast mutexes for the rest of the system (cc to 
python-dev)

Cheers,

Kristján
// giltest.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include 
#include 

#include 
#include 
#include 

#define EVENT

#ifdef EVENT
LONG gilcount;
HANDLE gil;
void CreateGIL(){
gilcount = 0;
gil = CreateEvent(0, FALSE, FALSE, 0) ;
}

void ClaimGIL() {
if (InterlockedIncrement(&gilcount) > 1) {
DWORD r = WaitForSingleObject(gil, INFINITE);
assert(r == WAIT_OBJECT_0);
}
}

void ReleaseGIL() {
if (InterlockedDecrement(&gilcount) > 0) {
BOOL ok = SetEvent(gil);
assert(ok);
}
}
#endif 



#ifdef MUTEX
HANDLE gil = 0;
void CreateGIL(){
gil = CreateMutex(0, FALSE, 0);
}

void ClaimGIL() {
DWORD r = WaitForSingleObject(gil, INFINITE);
assert(r == WAIT_OBJECT_0);
}

void ReleaseGIL() {
BOOL ok = ReleaseMutex(gil);
assert(ok);
}
#endif

#ifdef SEMAPHORE
HANDLE gil = 0;
void CreateGIL(){
gil = CreateSemaphore(0,1, 1, 0);
}

void ClaimGIL() {
DWORD r = WaitForSingleObject(gil, INFINITE);
assert(r == WAIT_OBJECT_0);
}

void ReleaseGIL() {
BOOL ok = ReleaseSemaphore(gil, 1, 0);
assert(ok);
}
#endif

#ifdef CRIT
CRITICAL_SECTION gil;
void CreateGIL(){
InitializeCriticalSection(&gil);
}

void ClaimGIL() {
EnterCriticalSection(&gil);
}

void ReleaseGIL() {
LeaveCriticalSection(&gil);
}
#endif


void FlashGIL() {
DWORD id = GetCurrentThreadId();
printf("thread %10d flashing GIL\n", id);
ReleaseGIL();
ClaimGIL();
printf("thread %10d reclaims GIL\n", id);
}



void DoWork(int rounds)
{
DWORD id = GetCurrentThreadId();
while (rounds--) {
int sum = 0;
int n = rand();
n += RAND_MAX;
printf("thread %10d working %d units\n", id, n);
for (int i=0; i threads;
for (int i = 0; i___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SIGCHECK() in longobject.c

[Nick Coghlan]

Brett Cannon wrote:
> On Tue, Oct 20, 2009 at 07:57, Mark Dickinson  > wrote:
> The Deccoeff type is very simple, though.  It would be easy to create
> a pure Python version of it, and then do something like:
> 
> try:
>from _decimal import Deccoeff  # try to get C version
> except ImportError:
>from deccoeff import Deccoeff  # else use Python fallback code.
> 
> 
> And this is why you shouldn't have to worry. As long as the tests
> exercise both the pure Python and C versions then the other VMs are covered.

We need to worry at least a little bit, as the pure Python version
doesn't exist at this point in time.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Interest in integrating C decimal module into Python?

[Nick Coghlan]

Maciej Fijalkowski wrote:
> For example other python implementations might decide to use python
> version as long as builtin version does not appear. Python versions are
> usually also better targets for jit than mixed versions. C level versions also
> usually have more bugs (just statistics), so some people might want to
> choose pure-python version.
> 
> In general - some people have some reasons.

Although nobody has broken "sys.modules['_decimal'] = 0", so
deliberately turning off optimisations is pretty easy if you really
don't want them.

There's a reason we moved to implicit import of optimised versions in
Py3k - we're unlikely to revert to the old way of doing things.

As far as decimal.py in particular goes, there are significant
maintenance gains in keeping a lot of the non-performance critical
context management code in pure Python. So we're likely to wait and see
how much speed Mark can wring out of a simple C decimal coefficient
object (that other implementations can also fairly easily provide
natively) before looking seriously at a wholesale replacement of the module.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
---
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Antoine Pitrou]

Hello Kristjan,

> This depends entirely on the platform and primitives used to implement the 
> GIL.
> I'm interested in windows.

Could you try ccbench (*) under Windows? The only Windows system I have here is
a qemu virtual machine and it wouldn't be very realistic to do concurrency
measurements on it.

(*) http://svn.python.org/view/sandbox/trunk/ccbench/

Thanks

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Scott Dial]

Antoine Pitrou wrote:
> Could you try ccbench (*) under Windows? The only Windows system I have here 
> is
> a qemu virtual machine and it wouldn't be very realistic to do concurrency
> measurements on it.
> 
> (*) http://svn.python.org/view/sandbox/trunk/ccbench/
> 

I don't really know how this test works, so I won't claim to understand
the results either. However, here you go:

C:\>systeminfo
OS Name:   Microsoft Windows XP Professional
OS Version:5.1.2600 Service Pack 3 Build 2600

C:\>c:\Python26\python.exe
Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit
(Intel)] on win32

C:\>start /B /HIGH c:\Python26\python.exe c:\ccbench.py
--- Throughput ---

Pi calculation (Python)

threads=1: 377 iterations/s.
threads=2: 376 ( 99 %)
threads=3: 380 ( 100 %)
threads=4: 376 ( 99 %)

regular expression (C)

threads=1: 222 iterations/s.
threads=2: 213 ( 95 %)
threads=3: 223 ( 100 %)
threads=4: 218 ( 97 %)

bz2 compression (C)

threads=1: 324 iterations/s.
threads=2: 324 ( 99 %)
threads=3: 327 ( 100 %)
threads=4: 324 ( 100 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

-- 
Scott Dial
sc...@scottdial.com
scod...@cs.indiana.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Antoine Pitrou]

> I don't really know how this test works, so I won't claim to understand
> the results either. However, here you go:

Thanks.

Interesting results. I wonder what they would be like on a multi-core
machine. The GIL seems to behave perfectly on your setup (no performance
degradation due to concurrency, and zero latencies).

For a quick explanation of what the benchmark does:

- the "throughput" part launches N computational (CPU-bound) threads and
measures the total work done per second, and then compares the result to
the 1-thread result. It does so with three different workloads, which
have different impacts on the GIL. 100% is the most you can get on a
single-core machine. On a multi-core machine, you can get more than 100%
with the workload that explicitly releases the GIL before taxing the CPU
(bz2 compression).

- the "latency" part launches N computational threads in the background,
and the main thread listens for periodic ping messages on an UDP socket
(the ping messages themselves are emitted from a separate Python
process, so as to decouple it from the process under test). The
latencies are the measured delay between the emission of the UDP message
and the moment at which the recv() call returns in the main thread. This
aims at reproducing the situation where a thread handles IO operations
while one or several other threads perform heavy computations.

Regards

Antoine.


> C:\>systeminfo
> OS Name:   Microsoft Windows XP Professional
> OS Version:5.1.2600 Service Pack 3 Build 2600
> 
> C:\>c:\Python26\python.exe
> Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit
> (Intel)] on win32
> 
> C:\>start /B /HIGH c:\Python26\python.exe c:\ccbench.py
> --- Throughput ---
> 
> Pi calculation (Python)
> 
> threads=1: 377 iterations/s.
> threads=2: 376 ( 99 %)
> threads=3: 380 ( 100 %)
> threads=4: 376 ( 99 %)
> 
> regular expression (C)
> 
> threads=1: 222 iterations/s.
> threads=2: 213 ( 95 %)
> threads=3: 223 ( 100 %)
> threads=4: 218 ( 97 %)
> 
> bz2 compression (C)
> 
> threads=1: 324 iterations/s.
> threads=2: 324 ( 99 %)
> threads=3: 327 ( 100 %)
> threads=4: 324 ( 100 %)
> 
> --- Latency ---
> 
> Background CPU task: Pi calculation (Python)
> 
> CPU threads=0: 0 ms. (std dev: 0 ms.)
> CPU threads=1: 0 ms. (std dev: 0 ms.)
> CPU threads=2: 0 ms. (std dev: 0 ms.)
> CPU threads=3: 0 ms. (std dev: 0 ms.)
> CPU threads=4: 0 ms. (std dev: 0 ms.)
> 
> Background CPU task: regular expression (C)
> 
> CPU threads=0: 0 ms. (std dev: 0 ms.)
> CPU threads=1: 0 ms. (std dev: 0 ms.)
> CPU threads=2: 0 ms. (std dev: 0 ms.)
> CPU threads=3: 0 ms. (std dev: 0 ms.)
> CPU threads=4: 0 ms. (std dev: 0 ms.)
> 
> Background CPU task: bz2 compression (C)
> 
> CPU threads=0: 0 ms. (std dev: 0 ms.)
> CPU threads=1: 0 ms. (std dev: 0 ms.)
> CPU threads=2: 0 ms. (std dev: 0 ms.)
> CPU threads=3: 0 ms. (std dev: 0 ms.)
> CPU threads=4: 0 ms. (std dev: 0 ms.)
> 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Scott Dial]

Antoine Pitrou wrote:
> Interesting results. I wonder what they would be like on a multi-core
> machine. The GIL seems to behave perfectly on your setup (no performance
> degradation due to concurrency, and zero latencies).

You are correct, my machine is a single-core system. I don't have any
multi-core systems around to test it on, I'm still in the stone age.

-- 
Scott Dial
sc...@scottdial.com
scod...@cs.indiana.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Kristján Valur Jónsson]

> -Original Message-
> From: python-dev-bounces+kristjan=ccpgames@python.org
> [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf
> Of Antoine Pitrou
> Sent: 21. október 2009 10:52
> To: python-dev@python.org
> Subject: Re: [Python-Dev] GIL behaviour under Windows
> 
> 
> Hello Kristjan,
> 
> > This depends entirely on the platform and primitives used to
> implement the GIL.
> > I'm interested in windows.
> 
> Could you try ccbench (*) under Windows? The only Windows system I have
> here is
> a qemu virtual machine and it wouldn't be very realistic to do
> concurrency
> measurements on it.
> 

Hi, I just want to stress, that according to my test, the current GIL 
implementation works as intended on windows.  But if we were to reimplement it, 
say using a CriticalSection, then yielding the GIL as we do in 
sys.checkinterval wouldn't work as intended anymore.  Just something to keep in 
mind in case anyone is thinking along those lines.

K

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Kristján Valur Jónsson]

> -Original Message-
> Could you try ccbench (*) under Windows? The only Windows system I have
> here is
> a qemu virtual machine and it wouldn't be very realistic to do
> concurrency
> measurements on it.
> 
> (*) http://svn.python.org/view/sandbox/trunk/ccbench/

I´ve run it twice on my dual core machine.  It hangs every time, but not in the 
same place:
D:\pydev\python\trunk\PCbuild>python.exe \tmp\ccbench.py
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
--- Throughput ---

Pi calculation (Python)

threads=1: 514 iterations/s.
threads=2: 403 ( 78 %)
threads=3: 392 ( 76 %)
threads=4: 364 ( 70 %)

regular expression (C)

threads=1: 443 iterations/s.
threads=2: 474 ( 106 %)
threads=3: 461 ( 104 %)
threads=4: 466 ( 105 %)

SHA1 hashing (C)

threads=1: 983 iterations/s.
threads=2: 1026 ( 104 %)
^C
D:\pydev\python\trunk\PCbuild>python.exe \tmp\ccbench.py
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
malloc 262144 gave 262144, diff 0
--- Throughput ---

Pi calculation (Python)

threads=1: 506 iterations/s.
threads=2: 405 ( 80 %)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Antoine Pitrou]

> > (*) http://svn.python.org/view/sandbox/trunk/ccbench/
> 
> I´ve run it twice on my dual core machine.  It hangs every time, but not in 
> the same place:
> D:\pydev\python\trunk\PCbuild>python.exe \tmp\ccbench.py

Ah, you should report a bug then. ccbench is pure Python (and not
particularly evil Python), it shouldn't be able to crash the
interpreter.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Interest in integrating C decimal module into Python?

[Mark Dickinson]

On Tue, Oct 20, 2009 at 2:15 PM, Stefan Krah  wrote:
> 1. Are you generally in favour of a C decimal module in Python?

I'm certainly interested in the general idea;  whether I'd be in favour
of replacing decimal.py with a particular C version would depend on
a lot of factors, with code quality, interchangeability with the current
decimal module, and maintainability by Python core developers
high on the list.  There's also the question of what IronPython
and Jython would do.

> 2. Would fastdec - after achieving full decimal.py compatibility - be
>   a serious candidate?

Definitely.  As far as I know it's the only real candidate for a full
C version of decimal right now.  Other possibilities that I'm aware of:

* I think Raymond Hettinger is working on a C version of decimal.
I'm not sure what its current status is.  Raymond?

* There's a partially complete rewrite of decimal in C in the sandbox,
dating from the Need for Speed sprint in 2006:

* Wrapping the decNumber library directly would make some sense,
but I think licensing issues rule this out.

http://svn.python.org/view/sandbox/trunk/decimal-c/

Last time I looked at this it wasn't up to date with the decimal
specification:  I'm not sure that functions like exp, log and log10
are currently working.  Georg Brandl might know better than I do.

> 3. Could I use this list to settle a couple of questions, or would perhaps
>   a Python developer be willing to work with me to make it compatible? I'm
>   asking this to avoid doing work that would not find acceptance afterwards.

I don't see why not.  Working closely with one of the core developers
on this sounds like a good idea, as well:  if the code were to go into
Python core, at least one (and preferably more than one) core dev
should be familiar enough with it for maintenance.  (BTW, I think that
*if* fastdec went in it's likely you'd be granted commit privileges at
some point.)

It's difficult to gauge the likelihood of eventual acceptance in advance,
though.  Maybe writing a PEP would be an efficient use of time in this
regard?  There are certainly some open issues (e.g., what to do with
the existing Python module;  what should other Python implementations
do).

I think my biggest concern is maintenance:  we'd be replacing
8500 lines of Python code in a single file, that several of the
current core developers understand well, with 3 (Stefan, is
that about accurate?) lines of C in several files, that presumably
none of the current core devs is familiar with right now.  What
happens when (e.g.,) the number-theoretic transform needs
updating for one reason or another?  Stefan, do you see yourself
having time to spend on maintenance of this code for the
forseeable future?

BTW, does anyone know the current SLOC count for py3k?

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Interest in integrating C decimal module into Python?

[Mark Dickinson]

On Wed, Oct 21, 2009 at 11:37 AM, Nick Coghlan  wrote:
> As far as decimal.py in particular goes, there are significant
> maintenance gains in keeping a lot of the non-performance critical
> context management code in pure Python. So we're likely to wait and see
> how much speed Mark can wring out of a simple C decimal coefficient
> object

No need to wait for this. :-)  I don't really expect to get much speed gain
at all in normal, low-precision (= precisions less than 100 digits, say)
use.  Even doubling the speed would be way too much to hope for here.

The only real gain from a decimal integer coefficient would be fixing
the asymptotics for high-precision calculations.

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Interest in integrating C decimal module i nto Python?

[Antoine Pitrou]

Mark Dickinson  gmail.com> writes:
> 
> There are certainly some open issues (e.g., what to do with
> the existing Python module;  what should other Python implementations
> do).

The existing module could be kept as a fallback. Also, the test suite should be
careful to test both implementations (like what is currently done for the io
module).

> BTW, does anyone know the current SLOC count for py3k?

Here you are, generated using David A. Wheeler's 'SLOCCount':

SLOCDirectory   SLOC-by-Language (Sorted)
261496  Lib python=261451,sh=45
186769  Modules ansic=173279,asm=9561,sh=3929
53258   Objects ansic=53258
40257   Python  ansic=40229,python=28
27220   Tools   python=27117,ansic=67,sh=36
18892   Demopython=18511,ansic=377,sh=4
9168PC  ansic=8465,python=703
5840Include ansic=5840
5799Parser  ansic=3914,python=1885
3485Misclisp=2948,python=242,sh=185,ansic=110
3101Doc python=2306,ansic=795
3030Mac python=2138,objc=775,sh=109,ansic=8
1666top_dir python=1140,ansic=286,sh=240
349 PCbuild python=279,ansic=70
337 build   ansic=295,python=42
0   Grammar (none)

Totals grouped by language (dominant language first):
python:  315842 (50.89%)
ansic:   286993 (46.24%)
asm:   9561 (1.54%)
sh:4548 (0.73%)
lisp:  2948 (0.47%)
objc:   775 (0.12%)

Total Physical Source Lines of Code (SLOC)= 620,667


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[John Arbash Meinel]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Antoine Pitrou wrote:
>> I don't really know how this test works, so I won't claim to understand
>> the results either. However, here you go:
> 
> Thanks.
> 
> Interesting results. I wonder what they would be like on a multi-core
> machine. The GIL seems to behave perfectly on your setup (no performance
> degradation due to concurrency, and zero latencies).
> 

C:\downloads>C:\Python26\python.exe ccbench.py
- --- Throughput ---

Pi calculation (Python)

threads=1: 691 iterations/s.
threads=2: 400 ( 57 %)
threads=3: 453 ( 65 %)
threads=4: 467 ( 67 %)

^- seems to have some contention



regular expression (C)

threads=1: 592 iterations/s.
threads=2: 598 ( 101 %)
threads=3: 587 ( 99 %)
threads=4: 586 ( 99 %)

bz2 compression (C)

threads=1: 536 iterations/s.
threads=2: 1056 ( 196 %)
threads=3: 1040 ( 193 %)
threads=4: 1060 ( 197 %)


^- seems to properly show that I have 2 cores here.


- --- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 38 ms. (std dev: 18 ms.)
CPU threads=2: 173 ms. (std dev: 77 ms.)
CPU threads=3: 518 ms. (std dev: 264 ms.)
CPU threads=4: 661 ms. (std dev: 343 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)


John
=;->
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkrfJawACgkQJdeBCYSNAANQlgCgwx0TCLh7YhLSJxkfOuMi1/YF
XhkAoIONtdP0rR1YW0nDza+wpKpAlInd
=L4WZ
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-ideas] Remove GIL with CAS instructions?

[John Arbash Meinel]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Kristján Valur Jónsson wrote:
...

> This depends entirely on the platform and primitives used to implement the 
> GIL.
> I'm interested in windows.  There, I found this article:
> http://fonp.blogspot.com/2007/10/fairness-in-win32-lock-objects.html
> So, you may be on to something.  Perhaps a simple C test is in order then?
> 
> I did that.  I found, on my dual-core vista machine, that running "release", 
> that both Mutexes and CriticalSections behaved as you describe, with no 
> "fairness".  Using a "semaphore" seems to retain fairness, however.
> "fairness" was retained in debug builds too, strangely enough.
> 
> Now, Python uses none of these.  On windows, it uses an "Event" object 
> coupled with an atomically updated counter.  This also behaves fairly.
> 
> The test application is attached.
> 
> 
> I think that you ought to sustantiate your claims better, maybe with a 
> specific platform and using some test like the above.
> 
> On the other hand, it shows that we must be careful what we use.  There has 
> been some talk of using CriticalSections for the GIL on windows.  This test 
> ought to show the danger of that.  The GIL is different than a regular lock.  
> It is a reverse-lock, really, and therefore may need to be implemented in its 
> own special way, if we want very fast mutexes for the rest of the system (cc 
> to python-dev)
> 
> Cheers,
> 
> Kristján

I can compile and run this, but I'm not sure I know how to interpret the
results. If I understand it correctly, then everything but "Critical
Sections" are fair on my Windows Vista machine.

To run, I changed the line "#define EVENT" to EVENT, MUTEX, SEMAPHORE
and CRIT. I then built and ran in "Release" environment (using VS 2008
Express)

For all but CRIT, I saw things like:
thread   5532 reclaims GIL
thread   5532 working 51234 units
thread   5532 worked 51234 units: 1312435761
thread   5532 flashing GIL
thread   5876 reclaims GIL
thread   5876 working 51234 units
thread   5876 worked 51234 units: 1312435761
thread   5876 flashing GIL

Where there would be 4 lines for one thread, then 4 lines for the other
thread.

for CRIT, I saw something more like 50 lines for one thread, and then 50
lines for the other thread.

This is Vista Home Basic, and VS 2008 Express Edition, with a 2-core
machine.

John
=:->
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkrfKFAACgkQJdeBCYSNAANbuQCgudU0IChylofTwvUk/JglChBd
9gsAoIJHj63/CagKpduUsd68HV8pP3QX
=CuUj
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Interest in integrating C decimal module into Python?

[Mark Dickinson]

On Wed, Oct 21, 2009 at 4:05 PM, Antoine Pitrou  wrote:
> Mark Dickinson  gmail.com> writes:
>> BTW, does anyone know the current SLOC count for py3k?
>
> Here you are, generated using David A. Wheeler's 'SLOCCount':
> [...]

Thanks, Antoine!  With SLOCCount I can revise my earlier numbers, as well:
Here's Stefan Krah's mpdecimal, version 0.80:

  SLOC  Directory   SLOC-by-Language (Sorted)
  21445   top_dir ansic=21267,sh=105,python=55,asm=18
  6238python  python=6177,java=43,sh=18
  1403tests   ansic=1356,sh=47
  476 literature  lisp=476
  274 cmd ansic=274
  11  tools   sh=11
  0   doc (none)

  Totals grouped by language (dominant language first):
  ansic:22897 (76.71%)
  python:6232 (20.88%)
  lisp:   476 (1.59%)
  sh: 181 (0.61%)
  java:43 (0.14%)
  asm: 18 (0.06%)


Lib/decimal.py:

  SLOC  Directory   SLOC-by-Language (Sorted)
  2636tmp python=2636

  Totals grouped by language (dominant language first):
  python:2636 (100.00%)

So it looks like 2636 lines of Python versus 21000-ish
lines of C.

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Sturla Molden]

Antoine Pitrou skrev:

(*) http://svn.python.org/view/sandbox/trunk/ccbench/
  

I´ve run it twice on my dual core machine.  It hangs every time, but not in the 
same place:
D:\pydev\python\trunk\PCbuild>python.exe \tmp\ccbench.py



Ah, you should report a bug then. ccbench is pure Python (and not
particularly evil Python), it shouldn't be able to crash the
interpreter.

  

It does not crash the interpreter, but it seems it can deadlock.

Here is what I get con a quadcore (Vista, Python 2.6.3).




D:\>ccbench.py
--- Throughput ---

Pi calculation (Python)

threads=1: 568 iterations/s.
threads=2: 253 ( 44 %)
threads=3: 274 ( 48 %)
threads=4: 283 ( 49 %)

regular expression (C)

threads=1: 510 iterations/s.
threads=2: 508 ( 99 %)
threads=3: 503 ( 98 %)
threads=4: 502 ( 98 %)

bz2 compression (C)

threads=1: 456 iterations/s.
threads=2: 892 ( 195 %)
threads=3: 1320 ( 289 %)
threads=4: 1743 ( 382 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 37 ms. (std dev: 21 ms.)
CPU threads=2: 379 ms. (std dev: 175 ms.)
CPU threads=3: 625 ms. (std dev: 310 ms.)
CPU threads=4: 718 ms. (std dev: 381 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 1 ms. (std dev: 3 ms.)




D:\>ccbench.py
--- Throughput ---

Pi calculation (Python)

threads=1: 554 iterations/s.
threads=2: 400 ( 72 %)
threads=3: 273 ( 49 %)
threads=4: 231 ( 41 %)

regular expression (C)

threads=1: 508 iterations/s.
threads=2: 509 ( 100 %)
threads=3: 509 ( 100 %)
threads=4: 509 ( 100 %)

bz2 compression (C)

threads=1: 456 iterations/s.
threads=2: 897 ( 196 %)
threads=3: 1316 ( 288 %)

DEADLOCK





D:\>ccbench.py
--- Throughput ---

Pi calculation (Python)

threads=1: 559 iterations/s.
threads=2: 397 ( 71 %)
threads=3: 274 ( 49 %)
threads=4: 238 ( 42 %)

regular expression (C)

threads=1: 507 iterations/s.
threads=2: 499 ( 98 %)
threads=3: 505 ( 99 %)
threads=4: 495 ( 97 %)

bz2 compression (C)

threads=1: 455 iterations/s.
threads=2: 896 ( 196 %)
threads=3: 1320 ( 290 %)
threads=4: 1736 ( 381 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 34 ms. (std dev: 21 ms.)
CPU threads=2: 358 ms. (std dev: 174 ms.)
CPU threads=3: 619 ms. (std dev: 312 ms.)
CPU threads=4: 742 ms. (std dev: 382 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 6 ms. (std dev: 13 ms.)









___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Sturla Molden]

Sturla Molden skrev:

does not crash the interpreter, but it seems it can deadlock.

Here is what I get con a quadcore (Vista, Python 2.6.3).



This what I get with affinity set to CPU  3.

There are deadlocks happening at random locations in ccbench.py. It gets 
worse with affinity set to one processor.


Sturla






D:\>start /AFFINITY 3 /B ccbench.py

D:\>--- Throughput ---

Pi calculation (Python)

threads=1: 554 iterations/s.
threads=2: 257 ( 46 %)
threads=3: 272 ( 49 %)
threads=4: 280 ( 50 %)

regular expression (C)

threads=1: 501 iterations/s.
threads=2: 505 ( 100 %)
threads=3: 493 ( 98 %)
threads=4: 507 ( 101 %)

bz2 compression (C)

threads=1: 455 iterations/s.
threads=2: 889 ( 195 %)
threads=3: 1309 ( 287 %)
threads=4: 1710 ( 375 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 0 ms.)
CPU threads=4: 0 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 40 ms. (std dev: 22 ms.)
CPU threads=2: 384 ms. (std dev: 179 ms.)
CPU threads=3: 618 ms. (std dev: 314 ms.)
CPU threads=4: 713 ms. (std dev: 379 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 0 ms. (std dev: 0 ms.)
CPU threads=1: 0 ms. (std dev: 0 ms.)
CPU threads=2: 0 ms. (std dev: 0 ms.)
CPU threads=3: 0 ms. (std dev: 3 ms.)
CPU threads=4: 0 ms. (std dev: 1 ms.)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Antoine Pitrou]

Sturla Molden  molden.no> writes:
> 
> It does not crash the interpreter, but it seems it can deadlock.

Kristján sent me a patch which I applied and is supposed to fix this.
Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero
latency with the Pi calculation in the background) than under Linux, although it
may be caused by the limited resolution of time.time() under Windows.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Interest in integrating C decimal module into Python?

[Georg Brandl]

Mark Dickinson schrieb:

> * There's a partially complete rewrite of decimal in C in the sandbox,
> dating from the Need for Speed sprint in 2006:
> 
> http://svn.python.org/view/sandbox/trunk/decimal-c/
> 
> Last time I looked at this it wasn't up to date with the decimal
> specification:  I'm not sure that functions like exp, log and log10
> are currently working.  Georg Brandl might know better than I do.

I haven't touched that code since the sprint, but a student named
Mateusz Rukowicz worked on it for a past Summer of Code, I think.
I never heard of the outcome of that particular project.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[John Arbash Meinel]

Antoine Pitrou wrote:
> Sturla Molden  molden.no> writes:
>> It does not crash the interpreter, but it seems it can deadlock.
> 
> Kristján sent me a patch which I applied and is supposed to fix this.
> Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero
> latency with the Pi calculation in the background) than under Linux, although 
> it
> may be caused by the limited resolution of time.time() under Windows.
> 
> Regards
> 
> Antoine.

You can use time.clock() instead to get <15ms resolution. Changing all
instances of 'time.time' to 'time.clock' gives me this result:

(2-core machine, python 2.6.2)

$ py ccbench.py
--- Throughput ---

Pi calculation (Python)

threads=1: 675 iterations/s.
threads=2: 388 ( 57 %)
threads=3: 374 ( 55 %)
threads=4: 445 ( 65 %)

regular expression (C)

threads=1: 588 iterations/s.
threads=2: 519 ( 88 %)
threads=3: 511 ( 86 %)
threads=4: 513 ( 87 %)

bz2 compression (C)

threads=1: 536 iterations/s.
threads=2: 949 ( 176 %)
threads=3: 900 ( 167 %)
threads=4: 927 ( 172 %)

--- Latency ---

Background CPU task: Pi calculation (Python)

CPU threads=0: 24727 ms. (std dev: 0 ms.)
CPU threads=1: 27930 ms. (std dev: 0 ms.)
CPU threads=2: 31029 ms. (std dev: 0 ms.)
CPU threads=3: 34170 ms. (std dev: 0 ms.)
CPU threads=4: 37292 ms. (std dev: 0 ms.)

Background CPU task: regular expression (C)

CPU threads=0: 40454 ms. (std dev: 0 ms.)
CPU threads=1: 43674 ms. (std dev: 21 ms.)
CPU threads=2: 47100 ms. (std dev: 165 ms.)
CPU threads=3: 50441 ms. (std dev: 304 ms.)
CPU threads=4: 53707 ms. (std dev: 377 ms.)

Background CPU task: bz2 compression (C)

CPU threads=0: 56138 ms. (std dev: 0 ms.)
CPU threads=1: 59332 ms. (std dev: 0 ms.)
CPU threads=2: 62436 ms. (std dev: 0 ms.)
CPU threads=3: 66130 ms. (std dev: 0 ms.)
CPU threads=4: 69859 ms. (std dev: 0 ms.)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Antoine Pitrou]

Le mercredi 21 octobre 2009 à 12:42 -0500, John Arbash Meinel a écrit :
> 
> You can use time.clock() instead to get <15ms resolution. Changing all
> instances of 'time.time' to 'time.clock' gives me this result:
[snip]
> 
> --- Latency ---
> 
> Background CPU task: Pi calculation (Python)
> 
> CPU threads=0: 24727 ms. (std dev: 0 ms.)
> CPU threads=1: 27930 ms. (std dev: 0 ms.)
> CPU threads=2: 31029 ms. (std dev: 0 ms.)
> CPU threads=3: 34170 ms. (std dev: 0 ms.)
> CPU threads=4: 37292 ms. (std dev: 0 ms.)

Well apparently time.clock() has a per-process time reference, which
makes it unusable for this benchmark :-(
(the numbers above are obviously incorrect)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[John Arbash Meinel]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Antoine Pitrou wrote:
> Le mercredi 21 octobre 2009 à 12:42 -0500, John Arbash Meinel a écrit :
>> You can use time.clock() instead to get <15ms resolution. Changing all
>> instances of 'time.time' to 'time.clock' gives me this result:
> [snip]
>> --- Latency ---
>>
>> Background CPU task: Pi calculation (Python)
>>
>> CPU threads=0: 24727 ms. (std dev: 0 ms.)
>> CPU threads=1: 27930 ms. (std dev: 0 ms.)
>> CPU threads=2: 31029 ms. (std dev: 0 ms.)
>> CPU threads=3: 34170 ms. (std dev: 0 ms.)
>> CPU threads=4: 37292 ms. (std dev: 0 ms.)
> 
> Well apparently time.clock() has a per-process time reference, which
> makes it unusable for this benchmark :-(
> (the numbers above are obviously incorrect)
> 
> Regards
> 
> Antoine.

I believe that 'time.count()' is measured as seconds since the start of
the process. So yeah, I think spawning a background process will reset
this counter back to 0.

John
=:->

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkrfTFYACgkQJdeBCYSNAAObWQCfRJsRENbcp6kuo2x1k+HvhYGZ
ftsAn2PNnNHNj6D4esNBMhlSdH4IjeMA
=1KWG
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Proposal: Moratorium on Python language changes

[Guido van Rossum]

In the python-ideas list I've proposed a moratorium on language
changes. It seems to be gaining momentum; if you want to have a say,
come over. You can watch the discussion in the archives starting here:
http://mail.python.org/pipermail/python-ideas/2009-October/006305.html
. (Eventually I'll move the proposal over to python-dev, but I thought
it only fair to give early warning to those who don't read
python-ideas.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] time.clock() on windows

[Kristján Valur Jónsson]

You are right, on windows time.clock() is based relative to its first call in 
the process.  There is no such promise made on unix.
QueryPerformanceCounter() (what time.clock uses()) is a robust high resolution 
timer that is processor/core independent.  It should be possible to use it 
across different processes too, if this annoying rebasing wasn't there.

I wonder if we should consider this a bug?

If so, I see three remedies:
1) simply using the absolute value and stop creating this arbitrary zero point. 
 this should be ok since the same is done on unix, but it would be a break from 
the documented behavior.  Never the less, the absolute value of this timer is 
irrelevant, it is the deltas that matter.
2) Add a flag to time.clock() for it to return absolute value
3) Create yet another api, either something like time.rclock() returning the 
absolute value or something like time.clockbase() returning the base of the 
zeroed clock timer.

If you just want to patch locally for your timing pleasure, change line 184 of 
timemodule.c to:
diff = (double)(now.QuadPart);

K

> -Original Message-
> From: python-dev-bounces+kristjan=ccpgames@python.org
> [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf
> Of Antoine Pitrou
>
 ---
> >
> > Background CPU task: Pi calculation (Python)
> >
> > CPU threads=0: 24727 ms. (std dev: 0 ms.)
> > CPU threads=1: 27930 ms. (std dev: 0 ms.)
> > CPU threads=2: 31029 ms. (std dev: 0 ms.)
> > CPU threads=3: 34170 ms. (std dev: 0 ms.)
> > CPU threads=4: 37292 ms. (std dev: 0 ms.)
> 
> Well apparently time.clock() has a per-process time reference, which
> makes it unusable for this benchmark :-(
> (the numbers above are obviously incorrect)
> 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] time.clock() on windows

[Antoine Pitrou]

Kristján Valur Jónsson  ccpgames.com> writes:
> 
> You are right, on windows time.clock() is based relative to its first call in
the process.  There is no such
> promise made on unix.
> QueryPerformanceCounter() (what time.clock uses()) is a robust high resolution
timer that is
> processor/core independent.  It should be possible to use it across different
processes too, if this
> annoying rebasing wasn't there.

Well, could we simply have a high-resolution time.time()?
Or is Windows just too limited to provide this?

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] time.clock() on windows

[Curt Hagenlocher]

On Wed, Oct 21, 2009 at 1:51 PM, Antoine Pitrou  wrote:
>
> Kristján Valur Jónsson  ccpgames.com> writes:
>>
>> You are right, on windows time.clock() is based relative to its first call
>> in the process.  There is no such promise made on unix.
>> QueryPerformanceCounter() (what time.clock uses()) is a robust high
>> resolution timer that is processor/core independent.  It should be
>> possible to use it across different processes too, if this
>> annoying rebasing wasn't there.
>
> Well, could we simply have a high-resolution time.time()?
> Or is Windows just too limited to provide this?

Presumably you could fake something like this by combining output from
an initial time(), an initial QueryPerformanceCounter() and the
current QueryPerformanceCounter(). But it makes more sense to
understand why someone chose to implement time.clock() on Windows the
way they did -- this seems very broken to me, and I think it should be
changed.

Of course, there are no doubt people relying on the broken behavior...

--
Curt Hagenlocher
c...@hagenlocher.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] time.clock() on windows

[Scott Dial]

Curt Hagenlocher wrote:
> But it makes more sense to
> understand why someone chose to implement time.clock() on Windows the
> way they did -- this seems very broken to me, and I think it should be
> changed.

Some SVN detective work takes this to all the way back to r7713
(1997-04-02). The original implementation checked by Guido and
attributed to Mark Hammond. So, we should ask Mark why he did that.

Can anyone honestly use it, as it is, without already having normalize
it across platforms themselves?

I don't know how much of an impact it is, but the current implementation
of clock() does not require argument parsing, so the proposal to add a
"absolute" boolean-flag argument is perhaps bad. This is generally a
function used for performance timing and that proposal adds some amount
of latency to the query. The proposal to add a clockbase() function is
perhaps better because of this, you need only call it once, and you can
cache the result for the life of your process.

-- 
Scott Dial
sc...@scottdial.com
scod...@cs.indiana.edu
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.6.4rc2

[Zooko O'Whielacronx]

Barry:

Do you know anything about this alleged regression in 2.6.3 with
regard to the __doc__ property?

https://bugs.edge.launchpad.net/ubuntu/+source/boost1.38/+bug/457688

Regards,

Zooko
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] time.clock() on windows

[Kristján Valur Jónsson]

> Presumably you could fake something like this by combining output from
> an initial time(), an initial QueryPerformanceCounter() and the
> current QueryPerformanceCounter(). But it makes more sense to
> understand why someone chose to implement time.clock() on Windows the
> way they did -- this seems very broken to me, and I think it should be
> changed.

Yes.  The problem with QPC is that although it has very high resolution, it is 
not precise in the long term.  And GetSystemTimeAsFileTime() is high precision 
in the long term but only updated evey 20ms or so.
In EVE Online we use a combination of the two for high resolution, long term 
precision.  But I'm not happy with the way I'm doing it.  It needs some sort of 
smoothing of course.  I've even played with using Kalman filtering to do it...
The idea is to use the low frequency timer to apply correction coefficients to 
the high frequency timer, yet keep the flow of time smooth (no backwards jumps 
because of corrections.).  An optimal solution has so far eluded me.

Cheers,

K
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.6.4rc2

[Barry Warsaw]

On Oct 21, 2009, at 7:00 PM, Zooko O'Whielacronx wrote:


Do you know anything about this alleged regression in 2.6.3 with
regard to the __doc__ property?

https://bugs.edge.launchpad.net/ubuntu/+source/boost1.38/+bug/457688


This is the first I've heard of it, but my guess is that it's caused  
by the fix for bug 5890.


http://bugs.python.org/issue5890

It's a change in Python 2.6.3, but I don't believe it's a regression  
(as in, currently broken).


-Barry



PGP.sig
Description: This is a digitally signed message part
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python Package Management Roadmap in Python Releases

[David Lyon]

Hi All,

I started out some time ago and wrote a Python Package Manager
with wxpython. It was an interesting learning experience for
me. I was new to python.

Some have pointed out that using wx was not a good idea for
a tool to go into all the python distributions. Because it
required the external dependency of wx.

So, given that, I'm ready to have another go.

Some have suggested using Tk. But I noticed python Tk isn't
standard on ubuntu.

A console mode tool is possible. That would be better than
the nothing that we have under windows today.

Vote [1] - console mode tool for this

Alternatively, a web server based tool along the lines
of the old python docs system might be possible.

Vote [2] - web tool for this

I have come to understand that python-dev is full of
people that know the packages that they like, know how
to install them by hand, and probably only ever go
looking for upgrades.

However, for new users, it is so much harder.

It also seems to me that we have a lot of reasonable
and talented people, who are committed and dedicated.

To me, a relative outsider, it's not clear if this
is a python central issue, or a distutils issue. It
could be either. Advise me please.

By June next year, some people including myself, want
to go to europe for holidays - sorry conferences.. and
we want to take some work with us to talk about.

We can do the work..

But we need concessions...

What can we work on that might make a difference ?

If the current roadmap for distutils package management
on windows for the next 8 months is nothing, I can
live with that.

But seriously, lets get this discussion going again.

If a command line tool is all that windows users will
allowed to have, then it would be better than the
current plan which is for them to have nothing.

It's not just nothing for now, it's nothing for years.

That doesn't seem fair or nice. It's not just one
person saying that, it is a handful who don't use
windows on a regular basis. 

Why can't we have an advocate from each major 
platform? bring their issues, and try to come 
to a consensus.

Even if that tool would simply allow them to choose:
 - PIP
 - Distribute
 - Easy Install
 - Python Package Manager

>From there, users could explore each offer on it's
own merits.

I'm interested in getting all types of opinions and feedback. 

David




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 384

[Andrew Svetlov]

Is there some public branch for this PEP?
I like the idea and glad to look on implementation.
Thanks.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 384

[Brett Cannon]

On Wed, Oct 21, 2009 at 17:44, Andrew Svetlov wrote:

> Is there some public branch for this PEP?
> I like the idea and glad to look on implementation.
> Thanks.
>

If there was such a thing the PEP would mention it. But since the PEP is
about not changing ABIs between releases I don't see why some branch would
be needed for it.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Package Management Roadmap in Python Releases

[Brett Cannon]

On Wed, Oct 21, 2009 at 17:34, David Lyon  wrote:

>
> Hi All,
>
> I started out some time ago and wrote a Python Package Manager
> with wxpython. It was an interesting learning experience for
> me. I was new to python.
>
> Some have pointed out that using wx was not a good idea for
> a tool to go into all the python distributions. Because it
> required the external dependency of wx.
>
> So, given that, I'm ready to have another go.
>
>
I honestly don't see what this has to do with python-dev or the
distutils-sig. If you want to write a package management tool that's great,
but I don't see why python-dev should have input on that sort of thing.
Asking what kind of GUI to use is a comp.lang.python question and not a
python-dev question as I don't see anything like this going into the stdlib.

If you want distutils to expose something to make it easier to write your
tool then that belong on the distutils-sig. But otherwise this seems
off-topic for python-dev.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Package Management Roadmap in Python Releases

[David Lyon]

On Wed, 21 Oct 2009 17:56:57 -0700, Brett Cannon  wrote:

> but I don't see why python-dev should have input on that sort of thing.

python-dev is the only place where we could get a change to the
installation
binary release. We'd need a change and the addition of a program
shortcut.

> If you want distutils to expose something to make it easier to write your
> tool then that belong on the distutils-sig. But otherwise this seems
> off-topic for python-dev.

I didn't ask for that - because I know that getting that assistance on the
distutils side is certainly possible.

Distutils is simply just one of the many libraries within python. It
doesn't
have an external interface.

The roadmap for distutils for windows doesn't include getting a shortcut
or utility for windows so that's why I'm asking about it here.

Surely logic says that if it's 'python' and 'development' and it's
not in distutils then some discussion of it should be allowed here.

What I am really talking about is the menu shortcuts in the cpython
distribution for windows. And how they can be improved to help
windows users. This is the only place that I can think to discuss
that.

Best Regards

David





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Package Management Roadmap in Python Releases

[Brett Cannon]

On Wed, Oct 21, 2009 at 18:17, David Lyon  wrote:

> On Wed, 21 Oct 2009 17:56:57 -0700, Brett Cannon  wrote:
>
> > but I don't see why python-dev should have input on that sort of thing.
>
> python-dev is the only place where we could get a change to the
> installation
> binary release. We'd need a change and the addition of a program
> shortcut.
>
>
But that assumes you can get your tool into the stdlib. It would have been
better to phrase the question as "is there interest in having a package
manager tool included with Python" rather than ask us out of the blue what
GUI you should use.


> > If you want distutils to expose something to make it easier to write your
> > tool then that belong on the distutils-sig. But otherwise this seems
> > off-topic for python-dev.
>
> I didn't ask for that - because I know that getting that assistance on the
> distutils side is certainly possible.
>
> Distutils is simply just one of the many libraries within python. It
> doesn't
> have an external interface.
>
>
And that's on purpose.


> The roadmap for distutils for windows doesn't include getting a shortcut
> or utility for windows so that's why I'm asking about it here.
>
> Surely logic says that if it's 'python' and 'development' and it's
> not in distutils then some discussion of it should be allowed here.
>
>
No it surely does not. We do not need to discuss design decisions of pip,
setuptools, virtualenv, buildout, or various other tools that involve the
terms "python" and "development" and are not in distutils.


> What I am really talking about is the menu shortcuts in the cpython
> distribution for windows. And how they can be improved to help
> windows users. This is the only place that I can think to discuss
> that.
>

David, you are making a huge leap here thinking that we even want a package
manager in the stdlib. You did not ask about menu shortcuts but whether a
package manager should be written using Tk or a web front-end. Then you
start discussing about wanting to add some UI to package management by
default on Windows or add some tool that sounds like what the EU is going to
have MS stick in front of users to get new user by browsers. This extends
beyond adding some shortcut the Windows installer adds to someone's machine.

I realize you are trying to help, David, but you are going in the wrong
direction here and pushing rather hard. At the language summit we discussed
opening up some APIs in distutils about making it easier for people to write
package management tools, but we don't have a burning desire to get into the
tool business. We make a language here. Distutils exists as a bootstrap
mechanism for the package story and for our own building needs of the
stdlib. But I doubt I am the only core developer who has no interest to be
in charge of a package management tool when there already exists several
good ones out there that people seem to find on their own without issue.

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] nonlocal keyword in 2.x?

[Mike Krell]

Is there any possibility of backporting support for the nonlocal keyword
into a  2.x release?  I see it's not in 2.6, but I don't know if that was an
intentional design choice or due to a lack of demand / round tuits.  I'm
also not sure if this would fall under the scope of the proposed moratorium
on new language features (although my first impression was that it could be
allowed since it already exists in python 3.

One of my motivations for asking is a recent blog post by Fernando Perez of
IPython fame that describes an interesting decorator-based idiom inspired by
Apple's Grand Central Dispatch which would allow many interesting
possibilities for expressing parallelization and other manipulations of
execution context for blocks of python code.  Unfortunately, using the
technique to its fullest extent requires the nonlocal keyword.

The blog post is here:
https://cirl.berkeley.edu/fperez/py4science/decorators.html

   Mike
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Package Management Roadmap in Python Releases

[David Lyon]

On Wed, 21 Oct 2009 18:38:26 -0700, Brett Cannon  wrote:
> But that assumes you can get your tool into the stdlib. 

No I'm not assuming that I can. I am actually assuming that I cannot..

So lets move forward..

> It would have been
> better to phrase the question as "is there interest in having a package
> manager tool included with Python" rather than ask us out of the blue
what
> GUI you should use.

ok - but I think I know the answer to that.. you answer it next.

> David, you are making a huge leap here thinking that we even want a
package
> manager in the stdlib. 

Well - who is 'we'? If it's python-dev people I can accept and respect
that.

If it's regular developers out there in developer land, I'm not so sure
about your assertion. I'd even politely have to disagree from my
experience.

> You did not ask about menu shortcuts but whether a
> package manager should be written using Tk or a web front-end. 

I was thinking about the issue on the fly...

Menu shortcuts that link off to a a standard community web page
would be an excellent compromise - in the case where some tool
could not be added.

That would be a tremendous improvement for windows users over
what they are given at the moment.

> Then you
> start discussing about wanting to add some UI to package management by
> default on Windows or add some tool that sounds like what the EU is going
> to
> have MS stick in front of users to get new user by browsers. This extends
> beyond adding some shortcut the Windows installer adds to someone's
> machine.

That's going further than what I'm saying..

> I realize you are trying to help, David, but you are going in the wrong
> direction here and pushing rather hard. 

On the counter side, others are pushing rather hard for 0 improvement
for the windows platform for the user experience. While everything else
on windows rushes ahead..

My direction is improving the developer experience for windows users. I
can't do compiler writing. I'm not clever enough.

> At the language summit we discussed
> opening up some APIs in distutils about making it easier for people to
> write
> package management tools, but we don't have a burning desire to get into
> the
> tool business. 

ok - but nothing happened there...

I'm not in the tools business either. I'm not doing it for money but
that shouldn't be the point.

> We make a language here. Distutils exists as a bootstrap
> mechanism for the package story and for our own building needs of the
> stdlib. 

I accept that it's a tool for building stdlib. No debate.

> But I doubt I am the only core developer who has no interest to be
> in charge of a package management tool when there already exists several
> good ones out there that people seem to find on their own without issue.

umm.. I disagree with the 'without issue' statement. I'm sure if I tralled
the mailing lists I could find more than one.. 

Enough from me for now. Thanks Brett.

David


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Sturla Molden]

Antoine Pitrou skrev:

Kristján sent me a patch which I applied and is supposed to fix this.
Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero
latency with the Pi calculation in the background) than under Linux, although it
may be caused by the limited resolution of time.time() under Windows.

  


My critisism of the GIL on python-ideas was partly motivated by this:

http://blip.tv/file/2232410

However, David Beazley is not talking about Windows. Since the GIL is 
apparently not a mutex on Windows, it could behave differently. So I 
wrote a small script that contructs a GIL battle, and record how often a 
check-interval results in a thread-switch or not. For monitoring check 
intervals, I used a small C extension to read _Py_Ticker from ceval.c. 
It is not declared static so I could easily hack into it.


With two threads and a check interval og 100, only 61 of 10 check 
intervals failed to produce a thread-switch in the interpreter. I'd call 
that rather fair. :-)


And in case someone asks, the nthreads=1 case is just for verification.

S.M.




D:\>test.py
check interval = 1
nthreads=1, swiched=0, missed=10
nthreads=2, swiched=57809, missed=42191
nthreads=3, swiched=91535, missed=8465
nthreads=4, swiched=99751, missed=249
nthreads=5, swiched=95839, missed=4161
nthreads=6, swiched=10, missed=0

D:\>test.py
check interval = 10
nthreads=1, swiched=0, missed=10
nthreads=2, swiched=99858, missed=142
nthreads=3, swiched=2, missed=8
nthreads=4, swiched=10, missed=0
nthreads=5, swiched=10, missed=0
nthreads=6, swiched=10, missed=0

D:\>test.py
check interval = 100
nthreads=1, swiched=0, missed=10
nthreads=2, swiched=99939, missed=61
nthreads=3, swiched=10, missed=0
nthreads=4, swiched=10, missed=0
nthreads=5, swiched=10, missed=0
nthreads=6, swiched=10, missed=0

D:\>test.py
check interval = 1000
nthreads=1, swiched=0, missed=10
nthreads=2, swiched=9, missed=1
nthreads=3, swiched=10, missed=0
nthreads=4, swiched=10, missed=0
nthreads=5, swiched=10, missed=0
nthreads=6, swiched=10, missed=0
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GIL behaviour under Windows

[Sturla Molden]

Sturla Molden skrev:


However, David Beazley is not talking about Windows. Since the GIL is 
apparently not a mutex on Windows, it could behave differently. So I 
wrote a small script that contructs a GIL battle, and record how often 
a check-interval results in a thread-switch or not. For monitoring 
check intervals, I used a small C extension to read _Py_Ticker from 
ceval.c. It is not declared static so I could easily hack into it.


Anyway, if anyone wants to run a GIL battle, here is the code I used.

If it turns out the GIL is far worse with pthreads, as it is implemented 
with a mutex, it might be a good idea to reimplement it with an event 
object as it is on Windows.


Sturla Molden



In python:

from giltest import *
from time import clock
import threading
import sys


def thread(rank, battle, start):

   while not start.isSet():
   if rank == 0:
   start.set()

   try:
   while 1:
   battle.record(rank)
   except:
   pass   



if __name__ == '__main__':

   sys.setcheckinterval(1000)

   print "check interval = %d" % sys.getcheckinterval()

   for nthreads  in range(1,7):
  
   start = threading.Event()

   battle = GIL_Battle(10)
   threads = [threading.Thread(target=thread, args=(i,battle,start))
   for i in range(1,nthreads)]
   for t in threads:
   t.setDaemon(True)
   t.start()

   thread(0, battle, start)
   for t in threads: t.join()
  
   s,m = battle.report()
  
   print "nthreads=%d, swiched=%d, missed=%d" % (nthreads, s, m)



In Cython or Pyrex:


from exceptions import Exception

cdef extern from *:
   ctypedef int vint "volatile int"
   vint _Py_Ticker

class StopBattle(Exception):
   pass

cdef class GIL_Battle:

   """ tests the fairness of the GIL """

   cdef vint prev_tick, prev_rank, switched, missed
   cdef int trials
  
   def __cinit__(GIL_Battle self, int trials=10):

   self.prev_tick = _Py_Ticker
   self.prev_rank = -1
   self.missed = 0
   self.switched = 0
   self.trials = trials

   def record(GIL_Battle self, int rank):
   if self.trials == self.switched + self.missed:
   raise StopBattle   
   if self.prev_rank == -1:

   self.prev_tick = _Py_Ticker
   self.prev_rank = rank
   else:
   if _Py_Ticker > self.prev_tick:
   if self.prev_rank == rank:
   self.missed += 1
   else:
   self.switched += 1
   self.prev_tick = _Py_Ticker
   self.prev_rank = rank
   else:
   self.prev_tick = _Py_Ticker
  
   def report(GIL_Battle self):

   return int(self.switched), int(self.missed)








___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] time.clock() on windows

[Mark Hammond]

On 22/10/2009 8:52 AM, Scott Dial wrote:

Curt Hagenlocher wrote:

But it makes more sense to
understand why someone chose to implement time.clock() on Windows the
way they did -- this seems very broken to me, and I think it should be
changed.


Some SVN detective work takes this to all the way back to r7713
(1997-04-02). The original implementation checked by Guido and
attributed to Mark Hammond. So, we should ask Mark why he did that.


The thread seems to be at 
http://groups.google.com/group/comp.lang.python/browse_frm/thread/be32478a4b8e77b6/816d6228119a3474 
(although I do seem to recall more discussion of the patch which I 
currently can't find). I'd be very surprised if any applications rely on 
the fact that each process starts counting at zero, so if someone can 
come up with a high-res counter which avoids that artifact I'd expect it 
could be used.


Cheers,

Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] nonlocal keyword in 2.x?

[Neal Norwitz]

On Wed, Oct 21, 2009 at 6:56 PM, Mike Krell  wrote:
> Is there any possibility of backporting support for the nonlocal keyword
> into a  2.x release?  I see it's not in 2.6, but I don't know if that was an
> intentional design choice or due to a lack of demand / round tuits.  I'm
> also not sure if this would fall under the scope of the proposed moratorium
> on new language features (although my first impression was that it could be
> allowed since it already exists in python 3.

IIRC, Jeremy Hylton tried to backport it during a Pycon sprint a few
years back.  I think it was difficult and he dropped it.  I don't know
if there were any showstoppers or if Jeremy saved his work...or if my
memory is even correct. :-)

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] time.clock() on windows

[Robert Collins]

On Thu, 2009-10-22 at 15:21 +1100, Mark Hammond wrote:

>  I'd be very surprised if any applications rely on 
> the fact that each process starts counting at zero, so if someone can 
> come up with a high-res counter which avoids that artifact I'd expect it 
> could be used.

Could you offset it by the system time on the first call?

-Rob


signature.asc
Description: This is a digitally signed message part
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Package Management Roadmap in Python Releases

[Glenn Linderman]

On approximately 10/21/2009 7:13 PM, came the following characters from 
the keyboard of David Lyon:

On Wed, 21 Oct 2009 18:38:26 -0700, Brett Cannon  wrote:



We make a language here. Distutils exists as a bootstrap
mechanism for the package story and for our own building needs of the
stdlib. 



Maybe what David is missing is that since python-dev is uninterested in 
the package management issue, the only remaining way to include package 
management in a convenient, single installation, is to


1) Create the package management tool
2) Create a bundled installer that will install both Python, the package 
management tool, and any dependencies of the package management tool

3) Advertise the availability of the bundle, and its usefulness*
4) See how many users try it out, and as new versions of Python are 
created, how many users keep coming back, in preference to the package 
management tool free python.org distribution.


*There does seem to be precedent for mentioning/linking to other 
distributions of Python from http://www.python.org/download/ so that 
would surely be an avenue that is open to David, although I have no idea 
what criteria must be met to obtain a listing on that page.



--
Glenn -- http://nevcal.com/
===
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Package Management Roadmap in Python Releases

[Stephen J. Turnbull]

David Lyon writes:
 > On Wed, 21 Oct 2009 18:38:26 -0700, Brett Cannon  wrote:

 > > David, you are making a huge leap here thinking that we even want
 > > a package manager in the stdlib.
 > 
 > Well - who is 'we'? If it's python-dev people I can accept and
 > respect that.

Yes.  The stdlib is a product of python-dev.

 > If it's regular developers out there in developer land,

They are not participants in this decision.  python-dev people *do*
care about them.  Nevertheless python-dev is manifestly uninterested
in dealing with *this* issue on behalf of the "regular developers out
there".  (With good reasons IMHO, but you can find those reasons
elsewhere.)

To make progress here on python-dev, you need to get those regular
developers involved in python-dev.  I can think of two ways to do this
that will help.  (1) Turn them into committers via whatever aspect of
the language they are interested in.  This will take a while, and
substantial contribution (ie, effort) on their part.  Once that
happens, they will be in a much stronger position to advocate the
package manager and credibly promise to maintain it (or find a new,
credible maintainer) indefinitely.

(2) Put the package manager on PyPI and get a lot of people on the
bandwagon.  If it's a good product and recommended, the movers and
shakers of python-dev will be very likely to try it and more likely to
second your motion in the future.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com