[Python-Dev] Alternative Python VM
It could be interesting. - pyvm is * 2 times * faster than Python 2.4. In the source code there is a collection of benchmarks which includes 65 python scripts collected from the internet. At average if Python 2.4 needs 24 hours to do some job, pyvm can do it in 12 hours. - pyvm is a virtual machine that can run Python 2.3/2.4 bytecode. There is a compiler written in python (the 'pyc' program) which is executed by the vm to compile source code to bytecode. It is very easy to do advanced optimizations in python and the compiler produces bytecode of high quality (speed and size). http://students.ceid.upatras.gr/~sxanth/pyvm/ Sokolov Yura (funny_falcon) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] str.dedent
/ msg = textwrap.dedent('''\
// IDLE's subprocess can't connect to %s:%d. This may be due \
// to your personal firewall configuration. It is safe to \
// allow this internal connection because no data is visible on \
// external ports.''' % address)
//
/
Unfortunately, it won't help, since the 'dedent' method won't treat
those spaces as indentation.
So that it would be usefull to implicit parser dedent on string with 'd' prefix
/ msg = d'''\
// IDLE's subprocess can't connect to %s:%d. This may be due \
// to your personal firewall configuration. It is safe to \
// allow this internal connection because no data is visible on \
// external ports.''' % address/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] (no subject)
Mixing Decimal and float is nearly ALWAYS a user error. Doing it correctly requires significant expertise in the peculiarities of floating point representations. So that I think user should declare floats explicitly (###.###f) - he will fall into float space only if he wish it. So Python protects the user by throwing exceptions when attempts are made to mix Decimal and floats. I hate it. I want to get float when I wish to get float. In that case i would like to write #f. I want to stay with decimals by default. (and I want decimals written in C) But it just an opinion of young inexperienced/unpractised man. Excuse my English. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Unifying decimal numbers.
Excuse my English I think, we could just segregate tokens for decimal and real float and make them interoperable. Motivation: Most of us works with business databases - all floats are really decimals, algebraic operations should work without float inconsistency and those operations rare so speed is not important. But some of us use floats for speed in scientific and multimedia programs. with from __future__ import Decimal we could: a) interpret regular float constants as decimal b) interpret float constants with suffix 'f' as float (like1.5f 345.2e-5f etc) c) result of operation with decimal operands should be decimal 1.0/3.0 0.3 d) result of operation with float operands should be float 1.0f/3.0f 0.1f e) result of operation with decimal and float should be float (decimal converts into float and operation perfomed) 1.0f/3.0 0.1f 1.0/3.0f 0.1f ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 3000 and exec
Agree. i=1 def a(): i=2 def b(): print i return b a()() 2 def a(): i=2 def b(): exec print i return b a()() 1 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pythonic concurrency - offtopic
Offtopic: Microsoft Windows [Version 5.2.3790] (C) Copyright 1985-2003 Microsoft Corp. G:\Working\1c:\Python24\python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. from os import fork Traceback (most recent call last): File stdin, line 1, in ? ImportError: cannot import name fork ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Autoloading? (Making Queue.Queue easier to use)
May be allow modules to define __getattr__ ? def __getattr__(thing): try: return __some_standart_way__(thing) except AttributeError: if thing==Queue: import sys from Queue import Queue setattr(sys.modules[__name__],Queue,Queue) return Queue raise ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding a conditional expression in Py3.0
New keyword is so expensive? And why special case for 'then' is better than special case for 'take'? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding a conditional expression in Py3.0
Sorry for looking in every hole. Just a suggestion. A= condition and first or second problem is in case when first in (None,0,[],). May be invent new operator 'take'. take - returns right operator when left evals to True and stops computing condidtional expression. Then we could write: A = condition take first or second. A = x==y take w or s A = z is not None and q!=12 take [] or allowable(z,q) take [(z,q)] or Impossible Ok, it might looks ugly. But may be not. - Excuse my english. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] GIL, Python 3, and MP vs. UP
Ok. While windows already prefers threads, while linux-2.6 improves thread support and speed, while process startup expensive on time and memory, while we ought to dublicate our data and/or use obscure shared memory, while it is much simpler to make threaded program with care just about locks than multiprocessing with inventing interchange protocol (which abuses both CPU - sender and reciever), MULTIPROCESSING RULES!!! And on Opterons: you will win with multiprocessing ONLY if OS will bind process and processor/memory place pair. That would be difficult. Guido van Rossum wrote: That's an understatement. I expect that *most* problems (even most problems that we will be programming 10-20 years from now) get little benefit out of MP. They are allready here. Servers they are. It is so simple to write application server in Python. It is so difficult to make it scallable in CPython. Hardware is cheap, development time is expensive. To make scallable CPython-backed server one needs too much time. And he goes to Java, .NET and others (and rare to Jython, IronPython (in the future)). CPython will not be wide popular without real multithreading. --I want my Apache, written in Python ! :-) --I want my MySQL, written in Python ! ;-) It is a joke. But every joke has a peace of truth (as russian sayes). (Excuse my English) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Skiping searching throw dictionaries of mro() members.
Phillip J. Eby wrote: At 09:12 PM 9/14/2005 +0400, Sokolov Yura wrote: We could cash looks by this way: Store with a class its version. Every time after creation when we change a class (add,remove or chage class member, including __base__, __bases__ and mro) , we increase the version number. Lets call it VERSION FYI, there is already a mechanism for new-style classes that notifies subclasses of changes to the base class; this mechanism could be used to push cache flushes downward, rather than doing version checking upward. I didn't know it. So, if this thing works, it should be used. Another think. I do not really know, how emplemented scopes, but could this caching be applied to a scope realisation? So we derive current scope from his outer and cache all lookups, that goes out of this scope. Top scope is derived from a module and module it self derived from builtins. I think, there not many different deep upper lookups in current programs, but many reapeating, and not often we change outer scope, so cache will be fine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Variant of removing GIL.
Excuse my English. I think I know how to remove GIL Obviously I am an idiot. First about Py_INCREF and Py_DECREF. We should not remove GIL at all. We should change it. It must be one writer-many reader in a following semantic: Lock has a read-counter and a write-counter. Initially both are 0. When reader tries to acquire lock for read it sleeps until write-counter is 0. When he reader acquires lock, he increase read-counter. When reader releases lock, he decreases read-counter. One reader will not block other, since he not increases write-counter. Reader will sleep, if there is any waiting writers, since they are increase write-counter. When writer tries to acquire lock for write, he increase write-counter and sleeps until read-counter happens 0. For writers lock for write - simple lock. when writer release lock, he decrease write-counter. When there is no waiting writers, readers arise. Excuse me for telling obviouse things. I am really reinvent wheel in my head, since I was a bad studient. I think this kind of lock is native for linux (i saw it in a kernel source, but do not know is waiting writer locks new readers or not?). Now, every thread keep an queue of objects to decref. It can be implemented as array, cause it will be freed at once. Initially, every object acquires GIL for read. Py_INCREF works as usually, Py_DECREF places a ref into a queue. When queue has became full or 100 instruction left ( :-) , it usefull), thread releases GIL for read and acquires for write, when he acquire it, he decrefs all objects stored in a queue and clear queue. After all he acquires GIL for read. But what could we do with changing objects (dicts,lists and another)? There should be a secondary one-writer-many-reader public-write GIL - PWGIL. SGIL ought to be more complicated, since it should work in RLOCK semantic for write lock. Lets call this lock ROWMR(reentreed one writer - many reader) So semantic for ROWMR can be: When a thread acquires ROWMR lock, it acquires it at a read level. Lets name it write-level=0. While threads write-level=0 it is a reader. Thread can increase write-level. When he turns write-level from 0 to 1, he becomes writer. while write-level0, thread is writer. Thread can decrease write-level. When write-level turns from 1 to 0, thread becomes reader. With PWGIL : We can mark every _mutable_ object with a creator thread number. If mark match current thread number, object is private for the thread. If mark is 0 (or another imposible thread number) object is public. If mark !=0 and !=current thread number, object is alien. When we access _mutable_ object, we check is it private? If it is, we can do anything without locking. If it is not and we access for read, we check is it public. If yes (read of public), then we can read it without locking. If no, we increase write-level, if object is alien, make it public, if we need to change object, change it, decrease write-level. Of couse, when we append object to public collection, we chould make it public, write-level is already increased so we do not make many separate locks, but when we then will access thouse object for read, we will not lock for make it public. I don't know, how nested scopes are implemented, but i think it should be considered as a mutable object. So there is a small overhead for a single threaded application ( only for comparing 2 numbers) and in a big part of multithreaded, since we are locking only writting on _mutable_ _public_ objects. Most part of public objects is not accessed to write often: they are numbers, classes and mostly-read collections. And one can optimize a program by accumulating results in a private collection and then flush it to public one. Also, there may be a statement for explicit increasing write-level around big update of public object and decreasing after it. PWGIL also must be released and reacquired with every 100 instructions left,but only if write-level=0, it conforms to current GIL semantic. I think, it must be not released with flushing decref queue, since it can happen while we are in C code. And there must be strong think about blocking IO. Mostly awful situation (at my point of view): object O is private for a thread A. thread B accesses O and try to mark it public, so it locks in attempt of increasing write-level thread A starts to change O (it is in write-level 0), and in a C code it releases PWGIL (around blocking IO, for example). thread B becomes writer, changes object to public, becomes reader and starts to read O, returning thread A continue to change O , remaining in a write-level=0. But, I think, well written C code should not attemt to make blocking IO inside of changing non-local objects (and it does not attempt at the moment, as I guess. Am I mistaken?). Or/and, when it returns and continues to change O, it must check, is it private or it isn't? I think, big part of checks
Re: [Python-Dev] Variant of removing GIL.
Corrections: Now, every thread keeps one n queue of objects to incref and second queue to decref. Both can be implemented as array, cause they will be freed at once. Initially, every thread acquires GIL for read. Py_INCREF places a ref into a incref queue of a thread, Py_DECREF places a ref into a decref queue of a thread. When queue has became full or 100 instruction left ( :-) , it usefull), thread releases GIL for read and acquires for write, when first process acquire it he: walk throw all incref queues in all threads, incref all thouse refs, and clear queues. then walk throw all decref queues in all threads, decref all thouse refs, and clear queues After all he acquires GIL for read. Other processes could stupidly repeat it founding clear queues. Since there only one thread works as a garbage collector, we will not loose any incref and decref. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Skiping searching throw dictionaries of mro() members.
Excuse my english. I have a complex Idea. It can be not worth, but just look at. It touches the classes and inheritance. Base point: We change classes not too often. But every look at a member leads to search throw a __dict__ of all bases in ierarhy till member found. We could cache this search in class.__dict__ itself and check only bases versions. Lets see: we have some classes: __metaclass__=object class a: def meth_a(self): pass class b: def meth_b(self): pass class c(a,b): pass Every class has mro. c.mro() [class '__main__.c', class '__main__.a', class '__main__.b', type 'object'] if we call methon from a base class, we look at c.__dict__ and at dict of every inherited class till found the method itself. so c.meth_a() leads to search throw c.__dict__ and a.__dict__ c.meth_b() leads to search throw c.__dict__ , a.__dict__ and b.__dict__ We could cash looks by this way: Store with a class its version. Every time after creation when we change a class (add,remove or chage class member, including __base__, __bases__ and mro) , we increase the version number. Lets call it VERSION Also store a tuple MRO_VERSION=tuple(base.VERSION for base in self.mro()). It changes, when we touches __base__, __bases__ or self.mro) Store with every class member a tuple : MEMBER_VERSION=(self.mro version,number in mro) When we add member to (or change member of) class __dict__ directly, we store in this tuple (MRO_VERSION,class.mro().index(class)). When we search a class member: a) and have not found it in class.__dict__, we search it throw mro() by a classic way. If we've found it (in a class _BASE_), we store it in class.__dict__ with a tuple MEMBER_VERSION=(MRO_VERSION, class.mro().index(_BASE_)). Also we check all of seen base.VERSION to match their cached in MRO_VERSION values. If some is not match, we should update MRO_VERSION (berfore storing founded member in a class.__dict__) and adjust VERSION. If it was not found, we could store a (MRO_VERSION, class.mro().index(class)) with an internal value NotFound or do not store anything, and report that we have not found anything (so we search in an object itself). b) When we've found a class member in class.__dict__ we check versions if MEMBER_VERSION[0]==class.MRO_VERSION (the same object) then we just compare MRO_VERSION[i]==class.mro()[i].VERSION for i in range(MEMBER_VERSION[1]+1) and not search throw mro()[i].__dict__ what we are doing now. If all versions concur, we return found member (or, if we've found previously stored NotFound, we report that we have not found anything) if version of MRO_VERSION[j]!=class.mro()[j].VERSION and class!=class.mro()[j] (here can be a bug, one must think about it more) then we revert to a) point, but search throw [base.__dict__ for base in class.mro()[j:]], (and, of cause, update MRO_VERSION) if MEMBER_VERSION[0]!=class.MRO_VERSION, we reverts to a) point. That's all. PS. We can subclass module from a __builtin__, so we leave one dict lookup. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
