Re: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?

2008-07-21 Thread Jesus Cea

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Barry Scott wrote:
| See http://code.google.com/p/python-incompatibility/source/checkout

Thanks.

I'm *VERY* interested in 2.6->3.0 migration guide for C module
extensions. 3.0 is around the corner and the API is changing almost
daily :-p.

- --
Jesus Cea Avion _/_/  _/_/_/_/_/_/
[EMAIL PROTECTED] - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:[EMAIL PROTECTED] _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIRRpZlgi5GaxT1NAQLUlwP9GDwp2zKFfHQWmRnhOyGfTwa0GqWGW1rZ
IrHrPXgFweh1mnbhGIjVkvlMJxHnhP2yGKStkN4641zYJ5KgdRJBaIdfrqoOkPTb
xzzHJ7DqaAGmRSKRoCZ8jzjCuC4BKeuLalii3V7ofYBimDgsHL42lb0MWxZBVNK5
SnafyX+rqGU=
=re/E
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?

2008-07-21 Thread Georg Brandl

Jesus Cea schrieb:

Barry Scott wrote:
| See http://code.google.com/p/python-incompatibility/source/checkout

Thanks.

I'm *VERY* interested in 2.6->3.0 migration guide for C module
extensions. 3.0 is around the corner and the API is changing almost
daily :-p.


So it's good that nobody has written a migration guide yet; he'd have
to rewrite it daily.

Georg

--
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?

2008-07-21 Thread Lennart Regebro
On Mon, Jul 21, 2008 at 11:06, Jesus Cea <[EMAIL PROTECTED]> wrote:
> I'm *VERY* interested in 2.6->3.0 migration guide for C module
> extensions. 3.0 is around the corner and the API is changing almost
> daily :-p.

It would be great if python-incompatibility would have examples of the
C-api changes as well. That would really help in migrating and writing
a migration guide. It would be great if you could help with this!

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Python-3000] Removing bsddb module from py3k (was Re: No beta2 tonight)

2008-07-21 Thread Jesus Cea

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Josiah Carlson wrote:
| I'm still curious as to what deep features people are using in bsddb.
| Anyone have any pointers to open source software?

I'm using replication and distributed transactions. Database encryption
and page integrity checks. Abusing the master election BDB
infrastructure for some evil but fun purposes. Managing databases in the
200 terabyte range. Locking & logging infrastructure to implement
application logic integrated with database operation. MVCC everywhere.

Nothing I can show you without killing you after, though.

Independently of current bsddb usage profile, current 2.6 code support
for distributed transactions and replication enables new application
uses that sqlite can't match.

Since I'm taking full responsability over bsddb both in 2.6 and 3.0, I
don't really see what the issue is. Past mistakes and maintenance
nightmares WILL NOT be repeated.

- --
Jesus Cea Avion _/_/  _/_/_/_/_/_/
[EMAIL PROTECTED] - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:[EMAIL PROTECTED] _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIRWSplgi5GaxT1NAQL2CgQAnbv7hrxb4I5KzdZ1T05A/pGzzuX4NATm
SN5TBeKsesl1LlEwI1+5xaBv9uxuJFfnPYMFLlW7NZdGumIdNCo7QXMhPmEdvuNx
2BxcmlS2Zuag8piAbmgq31Ov2RnnskOCf5ZxmuK4smk5Y0H2L+hUGG96Rk8dz9BU
s+AVkiaorKY=
=szyU
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?

2008-07-21 Thread Jesus Cea

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Georg Brandl wrote:
| So it's good that nobody has written a migration guide yet; he'd have
| to rewrite it daily.

Yes. I was delaying battling the 3.0 bsddb migration until RC to avoid
redoing the same work 15 times XDD

I'm not lazy, but it is so sunny outside...

- --
Jesus Cea Avion _/_/  _/_/_/_/_/_/
[EMAIL PROTECTED] - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:[EMAIL PROTECTED] _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIRYHplgi5GaxT1NAQL4hwP+LUSeAZxP/6q2K8GBs9ptqMnZUPh6ct0j
mhJqnAGDrI3XgcyHa3DeWwAltIfbLj9KC+UcgBNSvVQvqqMaPxKhc+tsbXqmsude
+o2e9IhZCgWg91FyJv64LPJr/tgMG4g7QBxHR1G9GF76YYrjCQCzi1f06lNkplQh
o/KIbRBmwRc=
=Q6o6
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?

2008-07-21 Thread Jesus Cea

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Lennart Regebro wrote:
| On Mon, Jul 21, 2008 at 11:06, Jesus Cea <[EMAIL PROTECTED]> wrote:
|> I'm *VERY* interested in 2.6->3.0 migration guide for C module
|> extensions. 3.0 is around the corner and the API is changing almost
|> daily :-p.
|
| It would be great if python-incompatibility would have examples of the
| C-api changes as well. That would really help in migrating and writing
| a migration guide. It would be great if you could help with this!

I can comment about some issues I had this weekend.

Remember that my intention is to keep a single C codebase for 2.6 and 3.0.

* Int/Long integration. In Python 3.0 "PyInt_*" has vanished. But using
"PyLong_*" in Python 2.x surfaces so many issues that I have decided to
define "NUMBER_*" macros to be conditionally expanded to
"PyInt"/"PyLong" when compiling to 2.x/3.0. Not nice, but I can't see a
better way.

* The module initialization has changed completely. In fact I am
fighting this, still. Some help needed.

* "Py_FindMember" was eliminated ?last week?. I workaround this, but I
don't know really how to write a "smart" getattr function now. Any pointer?

At least my current code compiles under 3.0, but it still bombs when
importing it.

Life sucks.

PS: I'm learning the hard way, doing "diff" between 2.6 and 3.0 module
sourcecode. It must be a better way!.

- --
Jesus Cea Avion _/_/  _/_/_/_/_/_/
[EMAIL PROTECTED] - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:[EMAIL PROTECTED] _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIRbTplgi5GaxT1NAQJTBAP/cHndbYfnpNF+UoJP0vswoJrMtrNRu6F4
TAAMGbv4sfE5a308sfjoXxFuvUA8kXAyqdJCMXlNWDfX6CEsozQwoS06d6HNkkZn
k9iGEbjwiWVdcw0y6gXvt9rL+klK004Rg4pUHMYtO3yUbaFSqvCI1kox6Rf1Q4dB
3CywJ2/fAu4=
=V42Q
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?

2008-07-21 Thread Lennart Regebro
On Mon, Jul 21, 2008 at 11:48, Jesus Cea <[EMAIL PROTECTED]> wrote:
> I can comment about some issues I had this weekend.

I don't do C development myself, so comments aren't that useful for
me, but code examples are, so we can stick them into
python-incompatibility.

> Remember that my intention is to keep a single C codebase for 2.6 and 3.0.

Cool.

> * Int/Long integration. In Python 3.0 "PyInt_*" has vanished. But using
> "PyLong_*" in Python 2.x surfaces so many issues that I have decided to
> define "NUMBER_*" macros to be conditionally expanded to
> "PyInt"/"PyLong" when compiling to 2.x/3.0. Not nice, but I can't see a
> better way.

Seems resonable.

> PS: I'm learning the hard way, doing "diff" between 2.6 and 3.0 module
> sourcecode. It must be a better way!.

Yeah, these changes should be properly documented in the CHANGES.txt.
I've seen some C-API chnges mentiones at least.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Subversion 1.5 and better merge support

2008-07-21 Thread Jesus Cea

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Subversion 1.5 solves the "repeated merge" problem. At last!!.

Somebody is considering upgrading python svn to 1.5, and allowing only
commits coming from 1.5 clients?.

- --
Jesus Cea Avion _/_/  _/_/_/_/_/_/
[EMAIL PROTECTED] - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:[EMAIL PROTECTED] _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
"Things are not so easy"  _/_/  _/_/_/_/  _/_/_/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/_/_/_/  _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSIRfxZlgi5GaxT1NAQLvggQAj8+RlRxL6Vtntr9GhMIcQ4WnyY7FezKn
XyoMwN7mMzOPOxcyiKhakk/bQFLwezBW2tYa5Z40tI5oD8t/xJx6+1gJ2e3aMQAm
NttgAke0ix8Q79kOI8iak4Kp60AeO/eD/80VIY6yOkaEAygOPT2WqzPofMO5bkCK
E/MwQaqwwK4=
=hyn0
-END PGP SIGNATURE-
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Syntax error in python2.6

2008-07-21 Thread Bristow Thankachan
Hi everybody,

During the porting of Zope2 to Python2.6, I am stuck with a syntax error in
the module AccessControl, which is given below.

def reorder(s, with=None, without=()):
   ^
SyntaxError: invalid syntax

in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py.
The same code when run in python2.4 and python2.5 didn't give any syntax
errors. Can anybody suggest  the reason for this syntax error in python2.6.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Syntax error in python2.6

2008-07-21 Thread Jesse Noller
Because as of 2.6 the 'with' word is now part of the with statement,  
see pep 343.



On Jul 21, 2008, at 7:13 AM, "Bristow Thankachan" <[EMAIL PROTECTED] 
> wrote:



Hi everybody,

During the porting of Zope2 to Python2.6, I am stuck with a syntax  
error in the module AccessControl, which is given below.


def reorder(s, with=None, without=()):
   ^
SyntaxError: invalid syntax

in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/ 
Utilities.py.
The same code when run in python2.4 and python2.5 didn't give any  
syntax errors. Can anybody suggest  the reason for this syntax error  
in python2.6.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Zope-dev] Syntax error in python2.6

2008-07-21 Thread Bristow Thankachan
Thank you for the quick response. I made the change and got the syntax error
corrected.

with regards

Bristow.

On Mon, Jul 21, 2008 at 4:57 PM, Tino Wildenhain <[EMAIL PROTECTED]> wrote:

> Bristow Thankachan wrote:
>
>> Hi everybody,
>>
>> During the porting of Zope2 to Python2.6, I am stuck with a syntax error
>> in the module AccessControl, which is given below.
>>
>> def reorder(s, with=None, without=()):
>>   ^
>> SyntaxError: invalid syntax
>>
>> in line 56 of
>> /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py.
>> The same code when run in python2.4 and python2.5 didn't give any syntax
>> errors. Can anybody suggest  the reason for this syntax error in python2.6.
>>
>
> I'd say, "with" is now a keyword.
>
> http://docs.python.org/ref/keywords.html
>
> btw, shouldn't this already give a warning in 2.5?
>
> Cheers
> Tino
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Fuzzing bugs: most bugs are closed

2008-07-21 Thread M.-A. Lemburg

On 2008-07-20 22:45, Victor Stinner wrote:

Le Saturday 19 July 2008 21:52:09 A.M. Kuchling, vous avez écrit :

Excellent work!  Another fruitful area for fuzzing might be the
miniature virtual machine used by the re module.  It's possible to
import _sre and call the compile() function directly (see the end of
Lib/sre_compile.py for how it's invoked); I wonder how the regex VM
copes with random strings of bytecode.


Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted 
_sre.compile() in my fuzzer.


For information, it's also very easy to crash CPython with fuzzed .pyc file.

It's hard to check bytecode without execute it. It's maybe better to add 
checks directly in the VM.


I don't see that as a big problem: if you execute untrusted byte code,
you are on your own anyway... whether that's byte code for the re
engine or ceval.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jul 21 2008)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


 Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Zope-dev] Syntax error in python2.6

2008-07-21 Thread Tino Wildenhain

Bristow Thankachan wrote:

Hi everybody,

During the porting of Zope2 to Python2.6, I am stuck with a syntax error 
in the module AccessControl, which is given below.


def reorder(s, with=None, without=()):
   ^
SyntaxError: invalid syntax

in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py.
The same code when run in python2.4 and python2.5 didn't give any syntax 
errors. Can anybody suggest  the reason for this syntax error in python2.6.


I'd say, "with" is now a keyword.

http://docs.python.org/ref/keywords.html

btw, shouldn't this already give a warning in 2.5?

Cheers
Tino


smime.p7s
Description: S/MIME Cryptographic Signature
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Syntax error in python2.6

2008-07-21 Thread Bristow Thankachan
hi all,

 I changed the attribute 'with' in
/home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py
to 'wit' and I ran the tests in python2.4 and I got the following result.


Running tests at level 1
Running unit tests:
  Running:

  Ran 285 tests with 0 failures and 0 errors in 6.587 seconds.
Running Testing.ZopeTestCase.layer.ZopeLite tests:
  Set up Testing.ZopeTestCase.layer.ZopeLite in 0.000 seconds.
  Running:

  Ran 1 tests with 0 failures and 0 errors in 0.064 seconds.
Tearing down left over layers:
  Tear down Testing.ZopeTestCase.layer.ZopeLite in 0.000 seconds.
Total: 286 tests, 0 failures, 0 errors in 1.044 seconds.

Can anybody tell me whether this implies it is backward compatible in
python2.4?

with regards

Bristow

On Mon, Jul 21, 2008 at 4:43 PM, Bristow Thankachan <
[EMAIL PROTECTED]> wrote:

> Hi everybody,
>
>
> During the porting of Zope2 to Python2.6, I am stuck with a syntax error in
> the module AccessControl, which is given below.
>
> def reorder(s, with=None, without=()):
>^
> SyntaxError: invalid syntax
>
> in line 56 of /home/zope/ztrunk26/lib/python/RestrictedPython/Utilities.py.
> The same code when run in python2.4 and python2.5 didn't give any syntax
> errors. Can anybody suggest  the reason for this syntax error in python2.6.
>
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] import error in python2.6

2008-07-21 Thread Bristow Thankachan
Hi all,

During the porting of Zope2, I am stuck with import errors in many modules.
The same code works well in python2.5 and 2.4. Can anybody give the details
of this import error in python2.6 and how to get the error corrected?

with regards

Bristow
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] import error in python2.6

2008-07-21 Thread Quentin Gallet-Gilles
Hi Bristow,

You didn't provide any broken code that could help us give an explanation.
Also this kind of question is best answered on the python-users mailing
list. Python-dev is reserved for discussion about the evolution of Python,
not its use.

Cheers,
Quentin

On Mon, Jul 21, 2008 at 2:12 PM, Bristow Thankachan <
[EMAIL PROTECTED]> wrote:

> Hi all,
>
> During the porting of Zope2, I am stuck with import errors in many modules.
> The same code works well in python2.5 and 2.4. Can anybody give the details
> of this import error in python2.6 and how to get the error corrected?
>
> with regards
>
> Bristow
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/qgallet%40gmail.com
>
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Syntax error in python2.6

2008-07-21 Thread Lennart Regebro
On Mon, Jul 21, 2008 at 14:09, Bristow Thankachan
<[EMAIL PROTECTED]> wrote:
> Can anybody tell me whether this implies it is backward compatible in
> python2.4?

If you change the API it isn't backwards compatible. The question is
if this is a problem or not, if anything outside Zope itself is using
this call, which is hard to say.

Options here are
1. Deprecating the "with" parameter for "with_" and supporting both in
2.12 but not supporting Python2.6.
2. Using **kw in the argument and looking for noth "with" and "with_",
that way, which will be backwards compatible.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Syntax error in python2.6

2008-07-21 Thread Malthe Borch

Lennart Regebro wrote:

2. Using **kw in the argument and looking for noth "with" and "with_",
that way, which will be backwards compatible.


+1

\malthe

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Subversion 1.5 and better merge support

2008-07-21 Thread Benjamin Peterson
On Mon, Jul 21, 2008 at 5:07 AM, Jesus Cea <[EMAIL PROTECTED]> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Subversion 1.5 solves the "repeated merge" problem. At last!!.
>
> Somebody is considering upgrading python svn to 1.5, and allowing only
> commits coming from 1.5 clients?.

While Subversion 1.5 has merge tracking, it doesn't support blocking
revisions to be merged as well as svnmerge.py. (This is important for
merging trunk -> py3k). Basically, I think we can get more from
svnmerge.py than builtin merge tracking.


-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] import error in python2.6

2008-07-21 Thread Benjamin Peterson
On Mon, Jul 21, 2008 at 7:20 AM, Quentin Gallet-Gilles
<[EMAIL PROTECTED]> wrote:
> Hi Bristow,
>
> You didn't provide any broken code that could help us give an explanation.
> Also this kind of question is best answered on the python-users mailing
> list. Python-dev is reserved for discussion about the evolution of Python,
> not its use.

Since this is prelease software, it's probably ok to talk about issues
with it. You could file a bug next time. However, AFAIK, Zope hasn't
even been ported to 2.5.



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Fuzzing bugs: most bugs are closed

2008-07-21 Thread A.M. Kuchling
On Sun, Jul 20, 2008 at 10:45:39PM +0200, Victor Stinner wrote:
> Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted 
> _sre.compile() in my fuzzer.

We should certainly try to fix those issues, then; people usually
assume the re module is safe for use inside a sandbox and probably
aren't careful enough to block importing of the _sre module.

--amk
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] import error in python2.6

2008-07-21 Thread Fred Drake

On Jul 21, 2008, at 9:20 AM, Benjamin Peterson wrote:

Since this is prelease software, it's probably ok to talk about issues
with it. You could file a bug next time. However, AFAIK, Zope hasn't
even been ported to 2.5.



Many people are using Zope 3 with Python 2.5 without problems, though  
Python 2.5 isn't "officially" supported (whatever that means).


I think current versions of Zope 2 work with Python 2.5 as well.


  -Fred

--
Fred Drake   

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Fuzzing bugs: most bugs are closed

2008-07-21 Thread Victor Stinner
Le Monday 21 July 2008 15:33:19 A.M. Kuchling, vous avez écrit :
> On Sun, Jul 20, 2008 at 10:45:39PM +0200, Victor Stinner wrote:
> > Hum... how can I say it? It's trivial to crash _sre :-) So I blacklisted
> > _sre.compile() in my fuzzer.
>
> We should certainly try to fix those issues, then; people usually
> assume the re module is safe for use inside a sandbox and probably
> aren't careful enough to block importing of the _sre module.

Why is this function public? Is it used by re module? Only _sre module should 
be allowed to generated "regex bytecode".

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] import error in python2.6

2008-07-21 Thread Bristow Thankachan
Hi all,

I asked about the import error in python2.6 while trying to port zope2. The
error message is given below.

 File "/home/zope/ztrunk26/lib/python/AccessControl/ImplC.py", line 30, in

from ImplPython import RestrictedDTML, SecurityManager, ZopeSecurityPolicy
ImportError: No module named ImplPython

It works well in python2.4 and 2.5. Please give your suggestions.

with regards

Bristow
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Zope-dev] import error in python2.6

2008-07-21 Thread Lennart Regebro
On Mon, Jul 21, 2008 at 15:56, Bristow Thankachan
<[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I asked about the import error in python2.6 while trying to port zope2. The
> error message is given below.
>
>  File "/home/zope/ztrunk26/lib/python/AccessControl/ImplC.py", line 30, in
> 
> from ImplPython import RestrictedDTML, SecurityManager, ZopeSecurityPolicy
> ImportError: No module named ImplPython
>
> It works well in python2.4 and 2.5. Please give your suggestions.

ImplPython is a python module in Zope. It likely has some sort of
syntax error under 2.6 that is not a syntaxerror under 2.5, so the
import fails, or the C-modules that it uses fail it's compile during
setup, so import of them failed.

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Syntax error in python2.6

2008-07-21 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Malthe Borch wrote:
> Lennart Regebro wrote:
>> 2. Using **kw in the argument and looking for noth "with" and "with_",
>> that way, which will be backwards compatible.
> 
> +1

The implementation of this function is already so obscure that using
keywords should not be a problem.

OTOH, the reorder function in RestrictedPython was ported from a version
originally designed for doing ordered select lists in DTML:

 $ cvs log lib/python/DocumentTemplate/DT_Util.py
 ...
 revision 1.52
 date: 1999/03/22 17:28:37;  author: jim;  state: Exp;  lines: +31 -2
 Added a new "builtin" function reorder:

   reorder(s, [with, without]]) --

  Reorder the items in s according to the order given in with
  and with items mentioned in without removed.  Items from s
  not mentioned in with are removed.

  s, with, and without are all either sequences if strings
  or sequences of key-value tuples, with ordering done on the
  keys.

 This function is useful for constructing ordered select lists.
 ...

The only use of the method in Zope's own DTML is in the addZClass.dtml
template.  I'm willing to bet that *nobody* who is still using it would
be surprised if they had to recode it when upgrading to a recent Zope,
so maybe we can just change the trunk without worrying about it.


Tres.
- --
===
Tres Seaver  +1 540-429-0999  [EMAIL PROTECTED]
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIhJ7A+gerLs4ltQ4RAk6lAJ0T6JJC/4nr6kA7jzSQqKSNZUGP7ACeIO2R
H7k2D46BKQ+DL0girE8EGOk=
=HLxo
-END PGP SIGNATURE-

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Deprecate parser's "ast" functions?

2008-07-21 Thread Benjamin Peterson
On Sat, Jun 7, 2008 at 3:13 PM, Georg Brandl <[EMAIL PROTECTED]> wrote:
> The parser module exports each function and type twice, once with "AST" in
> the name, once with "ST".  Since "AST" now has a different meaning for
> Python code compilation, I propose to deprecate the "ast" variants in 2.6
> and remove them in Python 3000.

+1 This personally has confused me!

>
> (Also, all keyword arguments are called "ast", that cannot change in 2.6
> but should in 3.0.)
>
> I'm at the moment changing the documentation of parser to only refer to
> the "st" variants anymore.
>
> Georg

-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Fuzzing bugs: most bugs are closed

2008-07-21 Thread Antoine Pitrou
Victor Stinner  haypocalc.com> writes:
> 
> Le Monday 21 July 2008 15:33:19 A.M. Kuchling, vous avez écrit :
> > On Sun, Jul 20, 2008 at 10:45:39PM +0200, Victor Stinner wrote:
> > > Hum... how can I say it? It's trivial to crash _sre  So I blacklisted
> > > _sre.compile() in my fuzzer.
> >
> > We should certainly try to fix those issues, then; people usually
> > assume the re module is safe for use inside a sandbox and probably
> > aren't careful enough to block importing of the _sre module.
> 
> Why is this function public? Is it used by re module? Only _sre module should 
> be allowed to generated "regex bytecode".

The underscore at the beginning of _sre clearly indicates that the module is 
not recommended for direct consumption, IMO. Even the functions that don't 
themselves start with an underscore...


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Python VM

2008-07-21 Thread Jakob Sievers
Hi,
I've been reading the Python VM sources over the last few afternoons and
I took some notes, which I thought I'd share (and if anyone familiar
with the VM internals could have a quick look at them, I'd really
appreciate it).

Cheers,
-jakob


Unless otherwise noted, the source file in question is Python/ceval.c.

Control Flow

The calling sequence is:
main() (in python.c) -> Py_Main() (main.c) -> PyRun_FooFlags() (pythonrun.c) ->
run_bar() (pythonrun.c) -> PyEval_EvalCode() (ceval.c) -> PyEval_EvalCodeEx()
(ceval.c) -> PyEval_EvalFrameEx() (ceval.c).

EvalCodeEx() does some initialization (creating a new execution frame,
argument processing, and some generator-specific stuff I haven't looked at yet)
before calling EvalFrameEx() which contains the main interpreter loop.


Threads
===
PyEval_InitThreads() initializes the GIL (interpreter_lock) and sets
main_thread to the (threading package dependent) ID of the current thread.
Thread switching is done using PyThreadState_Swap(), which sets
_PyThreadState_Current (both defined in pystate.c) and PyThreadState_GET()
(an alias for _PyThreadState_Current) (pystate.h).


Async Callbacks
===
Asynchronous callbacks can be registered by adding the function to be called
to pendingcalls[] (see Py_AddPendingCall()). The state of this queue is
communicated to the main loop via things_to_do.


State
=
The global state is recorded in a (per-process?) PyInterpreterState struct and
a per-thread PyThreadState struct.
Each execution frame's state is contained in that frame's PyFrameObject
(which includes the instruction stream, the environment (globals, locals,
builtins, etc.), the value stack and so forth).
EvalFrameEx()'s local variables are initialized from this frame object.


Instruction Stream
==
The instruction stream looks as follows (c.f. assemble_emit() in compile.c):
A byte stream where each instruction consists of either
1) a single byte opcode: OP
2) a single byte opcode plus a two-byte immediate argument: OP LO HI
3) a special opcode followed by the real opcode followed by a four byte
   argument: EXTENDED_ARG OP BYTE2 BYTE1 BYTE4 BYTE3


Opcode Prediction
=
One nice trick used to speed up opcode dispatch is the following:
Using the macros PREDICT() and PREDICTED() it is sometimes possible
to jump directly to the code implementing the next instruction
rather than having to go through the whole loop preamble, e.g.

case FOO:
  // ...
  PREDICT(BAR);
  continue;

PREDICTED(BAR);
case BAR:
 // ...

expands to

case FOO:
  // ...
  if (*next_instr == BAR) goto PRED_BAR;
  continue;

PRED_BAR: next_instr++;
case BAR:
  // ...


Main Loop
=

Variables and macros used in EvalFrameEx()
--
The value stack:
  PyObject **stack_pointer;
The instruction stream:
  unsigned char *next_instr;
NEXTOP(), NEXTARG(), PEEKARG(), JUMPTO(), and JUMPBY() simply fiddle
with next_instr. Likewise for TOP(), SET_SECOND(), PUSH(), POP(),
etc. and stack_pointer.

Current opcode plus argument:
  int opcode;
  int oparg;

Error status:
  enum why_code why; // no, exn, exn re-raised, return, break, continue, yield
  int err;   // non-zero is error

The environment:
  PyObject *names;
  PyObject *consts;
and
  PyObject **fastlocals;
which is accessed via GETLOCAL() and SETLOCAL().

Finally, there are some more PyObject *'s (v, w, u, and so forth, used
as temporary variables) as well as
  PyObject *retval;


Basic structure
---
EvalFrameEx() {
why = WHY_NOT;
err = 0;

for (;;) {<--+---+
// do periodic tasks |   |
 |   |
fast_next_opcode:|   |
opcode = NEXTOP();   |   |
if (HAS_ARG(opcode)) |   |
oparg = NEXTARG();   |   |
 |   |
dispatch_opcode: |   |
switch(opcode) { |   |
 |   |
continue; ---+   |
 |
break; --+   |
 |   |
// Also, opcode prediction   |   |
// jumps around inside the   |   |
// switch statement  |   |
 |   |
}<---+   |
 |
on_error:|
// no error: continue ---+
// otherwise why == WHY_EXCEPTION after this

fast_block_end:
// unwind stacks if there was an error
}

// more unwinding

fast_yield:
// reset current thread's exception info
exit_eval_frame:
// set thread's execution frame to previous execution frame
return retval;
}

Periodic Tasks
--
By checking and decrementing _Py_Ticker, the main loop 

Re: [Python-Dev] Fuzzing bugs: most bugs are closed

2008-07-21 Thread A.M. Kuchling
yOn Mon, Jul 21, 2008 at 03:53:18PM +, Antoine Pitrou wrote:
> The underscore at the beginning of _sre clearly indicates that the module is 
> not recommended for direct consumption, IMO. Even the functions that don't 
> themselves start with an underscore...

Sure, but if someone is trying to break in or DoS your application
server, they don't care if the module starts with an underscore or
not.

To answer Victor's original question: the parser & compiler that turn
a regex into bytecode is written in Python.  I can't think of a way to
prevent other Python modules from importing _sre or accessing the
compile() function; if nothing else, code could always do 'import re ;
re.sre_compile._sre.compile(...)'.

--amk
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?

2008-07-21 Thread Brett Cannon
On Mon, Jul 21, 2008 at 2:34 AM, Jesus Cea <[EMAIL PROTECTED]> wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Georg Brandl wrote:
> | So it's good that nobody has written a migration guide yet; he'd have
> | to rewrite it daily.
>
> Yes. I was delaying battling the 3.0 bsddb migration until RC to avoid
> redoing the same work 15 times XDD
>

But waiting until all the betas have gone out totally defeats the
purpose of the betas! It has already been stated that new code changes
that are even remotely shaky or anything not small needs a code review
at this point since we only have one beta left. Barry can correct me
if I am wrong, but dropping a major rewrite of bsddb into 3.0 between
b3 and rc1 is not acceptable. I wouldn't be surprised if all code
after b3 requires a code review since at that point we should just be
squashing bugs, not rewriting code. And if you are migrating all of
bsddb3 at once then that is going to be a massive code review that
should be happening in prep for rc1.

I really hate feeling like I am coming off as a jerk harping on this,
Jesus, as I don't want to discourage your contributions to pybsddb,
but it just doesn't feel reasonable to me to be doing this so late in
the release cycle.

-Brett
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Subversion 1.5 and better merge support

2008-07-21 Thread Brett Cannon
On Mon, Jul 21, 2008 at 6:16 AM, Benjamin Peterson
<[EMAIL PROTECTED]> wrote:
> On Mon, Jul 21, 2008 at 5:07 AM, Jesus Cea <[EMAIL PROTECTED]> wrote:
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>
>> Subversion 1.5 solves the "repeated merge" problem. At last!!.
>>
>> Somebody is considering upgrading python svn to 1.5, and allowing only
>> commits coming from 1.5 clients?.
>
> While Subversion 1.5 has merge tracking, it doesn't support blocking
> revisions to be merged as well as svnmerge.py. (This is important for
> merging trunk -> py3k). Basically, I think we can get more from
> svnmerge.py than builtin merge tracking.
>

This has also been discussed by the infrastructure committee and the
person in charge of maintaining the svn install does not want to
upgrade until 1.5 lands in debian-stable. So as of right now there are
no immediate plans to upgrade.

-Brett
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?

2008-07-21 Thread Benjamin Peterson
On Mon, Jul 21, 2008 at 1:16 PM, Brett Cannon <[EMAIL PROTECTED]> wrote:

> On Mon, Jul 21, 2008 at 2:34 AM, Jesus Cea <[EMAIL PROTECTED]> wrote:
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA1
> >
> > Georg Brandl wrote:
> > | So it's good that nobody has written a migration guide yet; he'd have
> > | to rewrite it daily.
> >
> > Yes. I was delaying battling the 3.0 bsddb migration until RC to avoid
> > redoing the same work 15 times XDD
> >
>
> But waiting until all the betas have gone out totally defeats the
> purpose of the betas! It has already been stated that new code changes
> that are even remotely shaky or anything not small needs a code review
> at this point since we only have one beta left. Barry can correct me
> if I am wrong, but dropping a major rewrite of bsddb into 3.0 between
> b3 and rc1 is not acceptable. I wouldn't be surprised if all code
> after b3 requires a code review since at that point we should just be
> squashing bugs, not rewriting code. And if you are migrating all of
> bsddb3 at once then that is going to be a massive code review that
> should be happening in prep for rc1.


I have a sickening feeling that this isn't the only major change that may
hit the betas. PEP 3118 is only about 2/3's implemented (with hardly any
tests, I might add), and Travis has said the earliest he can get to it is
August...



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] fileobj.read(float): warning or error?

2008-07-21 Thread Victor Stinner
Hi,

Since Python 2.4 (maybe 2.2 or older), fileobj.read(4.2) displays an error and 
works as fileobj.read(4).

>>> i=open('/etc/issue')
>>> i.read(4.2)
__main__:1: DeprecationWarning: integer argument expected, got float

It should raises an error instead of a warning, it has no sense to read a 
partial byte :-) But that should breaks some applications?

Well, the real problem is os.urandom(4.2) which goes to an unlimited loop:

  while len(bytes) < n:
  bytes += read(_urandomfd, n - len(bytes))

because read(0.2) works as read(0) :-/

Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fileobj.read(float): warning or error?

2008-07-21 Thread Benjamin Peterson
On Mon, Jul 21, 2008 at 2:17 PM, Victor Stinner <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> Since Python 2.4 (maybe 2.2 or older), fileobj.read(4.2) displays an error
> and
> works as fileobj.read(4).
>
> >>> i=open('/etc/issue')
> >>> i.read(4.2)
> __main__:1: DeprecationWarning: integer argument expected, got float


This warning is actually given by the argument parser when "i" gets a Python
non-integer.

>
>
> It should raises an error instead of a warning, it has no sense to read a
> partial byte :-) But that should breaks some applications?


This doesn't come into effect until 3.0.

>
>
> Well, the real problem is os.urandom(4.2) which goes to an unlimited loop:
>
>  while len(bytes) < n:
>  bytes += read(_urandomfd, n - len(bytes))
>
> because read(0.2) works as read(0) :-/
>
> Victor
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com
>



-- 
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [Zope-dev] Syntax error in python2.6

2008-07-21 Thread Stephan Richter
On Monday 21 July 2008, Tino Wildenhain wrote:
> btw, shouldn't this already give a warning in 2.5?

It does. I see it.

Regards,
Stephan
-- 
Stephan Richter
Web Software Design, Development and Training
Google me. "Zope Stephan Richter"
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] import error in python2.6

2008-07-21 Thread Martin Aspeli

Fred Drake wrote:

On Jul 21, 2008, at 9:20 AM, Benjamin Peterson wrote:

Since this is prelease software, it's probably ok to talk about issues
with it. You could file a bug next time. However, AFAIK, Zope hasn't
even been ported to 2.5.



Many people are using Zope 3 with Python 2.5 without problems, though  
Python 2.5 isn't "officially" supported (whatever that means).


I think current versions of Zope 2 work with Python 2.5 as well.


No, they don't. Work is ongoing to port it, though.

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python VM

2008-07-21 Thread Terry Reedy



Jakob Sievers wrote:

Hi,
I've been reading the Python VM sources over the last few afternoons and
I took some notes, which I thought I'd share (and if anyone familiar
with the VM internals could have a quick look at them, I'd really
appreciate it).


This is the CPython VM.  Other implementations do what they do.  Perhaps 
similar, but certainly different in details.


There have been various requests over the years on 
Python-list/comp.lang.python for documents describing the C-VM.  The 
usual answer has been 'read the source' as you did.  So I encourage you 
to announce it there and perhaps consider adding a PyWiki page (I assume 
there is not one now).


tjr

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Any PEP about 2.6 -> 3000 code transition?

2008-07-21 Thread Lennart Regebro
On Mon, Jul 21, 2008 at 20:16, Brett Cannon <[EMAIL PROTECTED]> wrote:
> But waiting until all the betas have gone out totally defeats the
> purpose of the betas!

I agree. Writing an actual *guide* can wait, but documenting the
differences with code examples is a work that can start now, and I
agree that it would be great if this would start now.

But writing a guide might not be a good idea until we know what the
changes are, and if the API is changing quickly now we don't. :-)

-- 
Lennart Regebro: Zope and Plone consulting.
http://www.colliberty.com/
+33 661 58 14 64
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python VM

2008-07-21 Thread Martin v. Löwis
Jakob,

This looks fairly correct. A few comments below.

> Control Flow
> 
> The calling sequence is:
> main() (in python.c) -> Py_Main() (main.c) -> PyRun_FooFlags() (pythonrun.c) 
> ->
> run_bar() (pythonrun.c) -> PyEval_EvalCode() (ceval.c) -> PyEval_EvalCodeEx()
> (ceval.c) -> PyEval_EvalFrameEx() (ceval.c).

What this misses is the compiler stuff, i.e. PyParser_ASTFromFoo and
PyAST_Compile, which precedes the call to PyEval_ (atleast, no byte code
file is available).

> Threads
> ===
> PyEval_InitThreads() initializes the GIL (interpreter_lock) and sets
> main_thread to the (threading package dependent) ID of the current thread.
> Thread switching is done using PyThreadState_Swap(), which sets
> _PyThreadState_Current (both defined in pystate.c) and PyThreadState_GET()
> (an alias for _PyThreadState_Current) (pystate.h).

True, however, in most cases, this is triggered through
Py_BEGIN_ALLOW_THREADS, which passes NULL for the new thread. The actual
*switching* occurs by releasing the GIL, not by ThreadState_Swap.

Actually, Python doesn't dispatch threads at all. It just releases the
GIL, giving the operating system permission to wake up a different
thread - which the operating system may or may not chose to do. After
some time, the original thread will try to reacquire the GIL. Assuming
the OS applies fairness, it will not get it back if a different thread
was also waiting for it, so our thread will block - and *then* the OS
will dispatch (at latest).

> State
> =
> The global state is recorded in a (per-process?) PyInterpreterState struct and
> a per-thread PyThreadState struct.

Yes and no. In principle, multiple interpreter states are supported per
process (and the current interpreter is identified by thread). However,
there are many limitations and quirks in the multiple-interpreter code.

> Each execution frame's state is contained in that frame's PyFrameObject
> (which includes the instruction stream, the environment (globals, locals,
> builtins, etc.), the value stack and so forth).
> EvalFrameEx()'s local variables are initialized from this frame object.

Not only. A lot of stuff also lives on the regular C stack, which exists
in parallel to the frame object stack (the latter being a spaghetti
stack).

> The instruction stream looks as follows (c.f. assemble_emit() in compile.c):

See also dis.py for the inverse operation.

> Basic structure
> ---
> EvalFrameEx() {

Somewhere you need to merge the thread-switching for threads that
have been executing a lot of instructions.

>   - Objects are transferred onto the value stack by GETITEM()'ing them from
> consts or names, or by GETLOCAL()'ing them using oparg as an offset into
> fastlocals (c.f. LOAD_* instructions).

Or, of course, as the result from some operation or function call, or
load from a global variable, or import, or ...

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python VM

2008-07-21 Thread Thomas Lee

Martin v. Löwis wrote:

Jakob,

This looks fairly correct. A few comments below.

  

Control Flow

The calling sequence is:
main() (in python.c) -> Py_Main() (main.c) -> PyRun_FooFlags() (pythonrun.c) ->
run_bar() (pythonrun.c) -> PyEval_EvalCode() (ceval.c) -> PyEval_EvalCodeEx()
(ceval.c) -> PyEval_EvalFrameEx() (ceval.c).



What this misses is the compiler stuff, i.e. PyParser_ASTFromFoo and
PyAST_Compile, which precedes the call to PyEval_ (atleast, no byte code
file is available).

  
Further, if I have my way with the AST optimization code, the symtable 
construction will be an explicit step in between these >:)


In any case, this is awesome work Jakob. It'd be great for this stuff to 
be documented in such detail -- I sure wish I had something like this to 
go by when I first started hacking on the source -- but the details seem 
to change quite often. Still, seeing the detail distilled once in a 
while is sort of nice, and great for anybody looking to get their teeth 
into the code.


Thanks for doing the hard yards. :)

Cheers,
T

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fileobj.read(float): warning or error?

2008-07-21 Thread Martin v. Löwis
> Well, the real problem is os.urandom(4.2) which goes to an unlimited loop:
> 
>   while len(bytes) < n:
>   bytes += read(_urandomfd, n - len(bytes))
> 
> because read(0.2) works as read(0) :-/

I can't quite accept that as a bug in the library. If you give invalid
parameters, Python should not crash, but it may start to behave in a
nonsensical way.

Of course, it would be possible to move the conversion warning one layer
up, into os.urandom; if the argument is float, raise a warning, and then
truncate.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fileobj.read(float): warning or error?

2008-07-21 Thread Alex Martelli
I thought that's what we had __index__ for -- reject arguments that
don't SMOOTHLY turn into integers when an integer is actually
required!

Alex


On Mon, Jul 21, 2008 at 10:01 PM, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
>> Well, the real problem is os.urandom(4.2) which goes to an unlimited loop:
>>
>>   while len(bytes) < n:
>>   bytes += read(_urandomfd, n - len(bytes))
>>
>> because read(0.2) works as read(0) :-/
>
> I can't quite accept that as a bug in the library. If you give invalid
> parameters, Python should not crash, but it may start to behave in a
> nonsensical way.
>
> Of course, it would be possible to move the conversion warning one layer
> up, into os.urandom; if the argument is float, raise a warning, and then
> truncate.
>
> Regards,
> Martin
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fileobj.read(float): warning or error?

2008-07-21 Thread Martin v. Löwis
Alex Martelli wrote:
> I thought that's what we had __index__ for -- reject arguments that
> don't SMOOTHLY turn into integers when an integer is actually
> required!

Sure. However, using that would create an incompatibility, that's why
you only get a warning when it falls back to not using __index__.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fileobj.read(float): warning or error?

2008-07-21 Thread Cameron Simpson
On 21Jul2008 21:17, Victor Stinner <[EMAIL PROTECTED]> wrote:
| Well, the real problem is os.urandom(4.2) which goes to an unlimited loop:
|   while len(bytes) < n:
|   bytes += read(_urandomfd, n - len(bytes))
| because read(0.2) works as read(0) :-/

Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an
exception if asked for < 1 bytes? Or is there a legitimate use for read(0)
with which I was not previously aware?
-- 
Cameron Simpson <[EMAIL PROTECTED]> DoD#743
http://www.cskk.ezoshosting.com/cs/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] fileobj.read(float): warning or error?

2008-07-21 Thread Leif Walsh
On Tue, 22 Jul 2008, Cameron Simpson wrote:
> Leaving aside the 0.2 => 0 converstion, shouldn't read() raise an
> exception if asked for < 1 bytes? Or is there a legitimate use for
> read(0) with which I was not previously aware?

I think read(0) should be a no-op, just like it is in libc.  This lets
you write 'read(bytes)' without worrying about checking bytes, and
also lets you silently stop reading when you have no more space, like
in the following:

buf = f.read(max(bytes_left, page_size))
while buf:
  process(buf)  # updates bytes_left
  buf = f.read(max(bytes_left, page_size))

-- 
Cheers,
Leif
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com