Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
Problem: if you implement a frozendict type inheriting from dict in Python, it is still possible to call dict methods (e.g. dict.__setitem__()). To fix this issue, pysandbox removes all dict methods modifying the dict: __setitem__, __delitem__, pop, etc. This is a problem because untrusted

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Giampaolo Rodolà
Il 01 marzo 2012 02:45, Raymond Hettinger raymond.hettin...@gmail.com ha scritto: On Feb 29, 2012, at 4:23 PM, Victor Stinner wrote: One of my colleagues implemented recently its own frozendict class (which the frozendict name ;-) I write new collection classes all the time. That doesn't

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread André Malo
On Wednesday 29 February 2012 20:17:05 Raymond Hettinger wrote: On Feb 27, 2012, at 10:53 AM, Victor Stinner wrote: A frozendict type is a common request from users and there are various implementations. ISTM, this request is never from someone who has a use case. Instead, it almost always

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
The main idea of pysandbox is to reuse most of CPython but hide dangerous functions and run untrusted code in a separated namespace. The problem is to create the sandbox and ensure that it is not possible to escape from this sandbox. pysandbox is still a proof-of-concept, even if it works

Re: [Python-Dev] State of PEP-3118 (memoryview part)

2012-03-01 Thread Stefan Krah
Terry Reedy tjre...@udel.edu wrote: Options 2) and 3) would ideally entail one backwards incompatible bugfix: In 2.7 and 3.2 assignment to a memoryview with format 'B' rejects integers but accepts byte objects, but according to the struct syntax mandated by the PEP it should be the other way

Re: [Python-Dev] Spreading the Python 3 religion

2012-03-01 Thread Stefan Krah
Brett Cannon br...@python.org wrote: Changes to http://docs.python.org/howto/pyporting.html are welcome. I tried to make sure it exposed all possibilities with tips on how to support as far back as Python 2.5. I'd like to add a section that highlights the advantages of separate branches.

Re: [Python-Dev] PEP 416: Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
Rationale = A frozendict mapping cannot be changed, but its values can be mutable (not hashable). A frozendict is hashable and so immutable if all values are hashable (immutable). The wording of the above seems very unclear to me. Do you mean A frozendict has a constant set of

Re: [Python-Dev] Spreading the Python 3 religion

2012-03-01 Thread Merlijn van Deen
On 1 March 2012 12:11, Stefan Krah ste...@bytereef.org wrote: Advantages of separate branches: Even though I agree on most of your points, I disagree with 2) Neither version is a second class citizen. In my experience, this is only true if you have a very strict discipline, or if both

Re: [Python-Dev] PEP 416: Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
An immutable mapping can be implemented using frozendict::     class immutabledict(frozendict):         def __new__(cls, *args, **kw):             # ensure that all values are immutable             for key, value in itertools.chain(args, kw.items()):                 if not isinstance(value,

Re: [Python-Dev] PEP 416: Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
Rationale = A frozendict mapping cannot be changed, but its values can be mutable (not hashable). A frozendict is hashable and so immutable if all values are hashable (immutable). The wording of the above seems very unclear to me. Do you mean A frozendict has a constant set of

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Yury Selivanov
Actually I find fronzendict concept quite useful. We also have an implementation in our framework, and we use it, for instance, in http request object, for parsed arguments and parsed forms, which values shouldn't be ever modified once parsed. Of course everybody can live without it, but given

Re: [Python-Dev] Spreading the Python 3 religion

2012-03-01 Thread Lennart Regebro
I also don't agree with the claim that a py3 version using 2to3 is a second class citizen. You need to adopt the Python 2 code to Python 3 in that case too, and none of the overrules the other. //Lennart ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
A builtin frozendict type compatible with the PyDict C API is very convinient for pysandbox because using this type for core features like builtins requires very few modification. For example, use frozendict for __builtins__ only requires to modify 3 lines in frameobject.c. See the

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
Here are my real-world use cases. Not for security, but for safety and performance reasons (I've built by own RODict and ROList modeled after dictproxy): - Global, but immutable containers, e.g. as class members I attached type_final.patch to the issue #14162 to demonstrate how frozendict

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread André Malo
On Thursday 01 March 2012 14:07:10 Victor Stinner wrote: Here are my real-world use cases. Not for security, but for safety and performance reasons (I've built by own RODict and ROList modeled after dictproxy): - Global, but immutable containers, e.g. as class members I attached

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Nick Coghlan
On Thu, Mar 1, 2012 at 7:29 PM, André Malo n...@perlig.de wrote: - Caching. My data container objects (say, resultsets from a db or something)  usually inherit from list or dict (sometimes also set) and are cached  heavily. In order to ensure that they are not modified (accidentially), I  have

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Serhiy Storchaka
01.03.12 11:11, Victor Stinner написав(ла): You can redefine dict.__setitem__. Ah? It doesn't work here. dict.__setitem__=lambda key, value: None Traceback (most recent call last): File stdin, line 1, inmodule TypeError: can't set attributes of built-in/extension type 'dict' Hmm, yes,

Re: [Python-Dev] PEP 416: Add a frozendict builtin type

2012-03-01 Thread Paul Moore
On 1 March 2012 12:08, Victor Stinner victor.stin...@gmail.com wrote: New try: A frozendict is a read-only mapping: a key cannot be added nor removed, and a key is always mapped to the same value. However, frozendict values can be mutable (not hashable). A frozendict is hashable and so

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Paul Moore
On 1 March 2012 12:37, Yury Selivanov yselivanov...@gmail.com wrote: Actually I find fronzendict concept quite useful.  We also have an implementation in our framework, and we use it, for instance, in http request object, for parsed arguments and parsed forms, which values shouldn't be ever

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Serhiy Storchaka
01.03.12 11:29, André Malo написав(ла): - Caching. My data container objects (say, resultsets from a db or something) usually inherit from list or dict (sometimes also set) and are cached heavily. In order to ensure that they are not modified (accidentially), I have to choices: deepcopy

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread André Malo
On Thursday 01 March 2012 15:17:35 Serhiy Storchaka wrote: 01.03.12 11:29, André Malo написав(ла): - Caching. My data container objects (say, resultsets from a db or something) usually inherit from list or dict (sometimes also set) and are cached heavily. In order to ensure that they are

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
But I will try to suggest another approach. `frozendict` inherits from `dict`, but data is not stored in the parent, but in the internal dictionary. And even if dict.__setitem__ is used, it will have no visible effect.  class frozendict(dict):      def __init__(self, values={}):          

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
Here are my real-world use cases. Not for security, but for safety and performance reasons (I've built by own RODict and ROList modeled after dictproxy): - Global, but immutable containers, e.g. as class members I attached type_final.patch to the issue #14162 to demonstrate how

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread André Malo
On Thursday 01 March 2012 15:54:01 Victor Stinner wrote: I'm not sure about your final types. I'm using __slots__ = () for such things You can still replace an attribute value if a class defines __slots__: class A: ... __slots__=('x',) ... x = 1 ... A.x=2 A.x 2 Ah, ok, I

[Python-Dev] EuroPython 2012: Call for Proposal is Open! [Please spread the word]

2012-03-01 Thread Francesco Pallanti
Hi guys, I'm Francesco and I am writing on behalf of EuroPython Staff (www.europython.eu). We are happy to announce that the Call for Proposals is now officially open! DEADLINE FOR PROPOSALS: MARCH 18TH, 23:59:59 CET For those who have never been at EuroPython (or similar conferences) before,

Re: [Python-Dev] Spreading the Python 3 religion

2012-03-01 Thread Stefan Krah
Merlijn van Deen valhall...@arctus.nl wrote: Another cause for this is the painful merging in most version control systems. I'm guessing you all know the pain of 'svn merge' - and there are a lot of projects still using SVN or even CVS. As such, you need to impose the discipline to always

Re: [Python-Dev] Spreading the Python 3 religion

2012-03-01 Thread Antoine Pitrou
On Thu, 1 Mar 2012 16:31:14 +0100 Stefan Krah ste...@bytereef.org wrote: As an example for a pretty large project, it looks like Antoine is making good progress with Twisted: https://bitbucket.org/pitrou/t3k/wiki/Home Well, to be honest, making good progress currently means bored and not

Re: [Python-Dev] Spreading the Python 3 religion

2012-03-01 Thread Stefan Krah
Lennart Regebro rege...@gmail.com wrote: I also don't agree with the claim that a py3 version using 2to3 is a second class citizen. You need to adopt the Python 2 code to Python 3 in that case too, and none of the overrules the other. That's a fair point. Then of course *both* versions do not

Re: [Python-Dev] Spreading the Python 3 religion

2012-03-01 Thread Barry Warsaw
On Mar 01, 2012, at 04:42 PM, Antoine Pitrou wrote: Well, to be honest, making good progress currently means bored and not progressing at all :-) But that's not due to the strategy I adopted, only to the sheer amount of small changes needed, and lack of immediate motivation to continue this work.

Re: [Python-Dev] Spreading the Python 3 religion

2012-03-01 Thread Antoine Pitrou
On Thu, 1 Mar 2012 11:24:19 -0500 Barry Warsaw ba...@python.org wrote: I really do think that to the extent that you can do that kind of thing, you may end up with essentially Python 3 support without even realizing it. :) That's unlikely. Twisted processes bytes data a lot, and the bytes

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Guido van Rossum
On Thu, Mar 1, 2012 at 2:01 AM, Victor Stinner victor.stin...@haypocalc.com wrote: The main idea of pysandbox is to reuse most of CPython but hide dangerous functions and run untrusted code in a separated namespace. The problem is to create the sandbox and ensure that it is not possible to

Re: [Python-Dev] Spreading the Python 3 religion

2012-03-01 Thread R. David Murray
On Thu, 01 Mar 2012 17:24:31 +0100, Antoine Pitrou solip...@pitrou.net wrote: On Thu, 1 Mar 2012 11:24:19 -0500 Barry Warsaw ba...@python.org wrote: I really do think that to the extent that you can do that kind of thing, you may end up with essentially Python 3 support without even

Re: [Python-Dev] PEP 414

2012-03-01 Thread Guido van Rossum
I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
In App Engine's case, an attacker who broke out of the sandbox would have access to the inside of Google's datacenter, which would obviously be bad -- that's why Google has developed its own sandboxing technologies. This is not specific to Google: if an attacker breaks a sandbox, he/she has

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Serhiy Storchaka
01.03.12 16:47, André Malo написав(ла): On Thursday 01 March 2012 15:17:35 Serhiy Storchaka wrote: This is the first rational use of frozendict that I see. However, a deep copy is still necessary to create the frozendict. For this case, I believe, would be better to freeze dict inplace and

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Paul Moore
On 1 March 2012 17:44, Victor Stinner victor.stin...@gmail.com wrote: I challenge anyone to try to break pysandbox! Can you explain precisely how a frozendict will help pysandbox? Then I'll be able to beat this challenge :-) Paul. ___ Python-Dev

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Guido van Rossum
On Thu, Mar 1, 2012 at 9:44 AM, Victor Stinner victor.stin...@gmail.com wrote: frozendict would help pysandbox but also any security Python module, not security, but also (many) other use cases ;-) Well, let's focus on the other use cases, because to me the sandbox use case is too controversial

[Python-Dev] Sandboxing Python

2012-03-01 Thread Victor Stinner
Hi, The frozendict discussion switched somewhere to sandboxing, and so I prefer to start a new thread. There are various ways to implement a sandbox, but I would like to expose here how I implemented pysandbox to have your opinion. pysandbox is written to execute quickly a short untrusted

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
I challenge anyone to try to break pysandbox! Can you explain precisely how a frozendict will help pysandbox? Then I'll be able to beat this challenge :-) See this email: http://mail.python.org/pipermail/python-dev/2012-February/117011.html The issue #14162 has also two patches: one to make

Re: [Python-Dev] PEP 414

2012-03-01 Thread Barry Warsaw
On Mar 01, 2012, at 09:42 AM, Guido van Rossum wrote: I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction? Not yet, but I believe Nick volunteered to do a rewrite. -Barry

[Python-Dev] Compiling Python on Linux with Intel's icc

2012-03-01 Thread Alex Leach
Dear Python Devs, I've been attempting to compile a fully functional version of Python 2.7 using Intel's C compiler, having built supposedly optimal versions of numpy and scipy, using Intel Composer XE and Intel's Math Kernel Library. I can build a working Python binary, but I'd really

Re: [Python-Dev] PEP 414

2012-03-01 Thread Vinay Sajip
Guido van Rossum guido at python.org writes: I noticed there were some complaints about unnecessarily offensive language in PEP 414. Have those passages been edited to everyone's satisfaction? I'm not sure if Nick has finished his updates, but I for one would like to see some improvements in

Re: [Python-Dev] Compiling Python on Linux with Intel's icc

2012-03-01 Thread Stefan Krah
Hi, Alex Leach albl...@york.ac.uk wrote: I've managed to compile everything in the python distribution except for Modules/_ctypes/libffi/src/x86/ffi64.c. There is an issue for this: http://bugs.python.org/issue4130 After compilation, there's a few tests that are consistently failing,

Re: [Python-Dev] PEP 414

2012-03-01 Thread Yury Selivanov
Vinay, Thank you for the comprehensive summary. Big +1. I really do hope that Nick and Armin will rectify the PEP. Otherwise, many of its points are moot, and we need to raise a question of rejecting it somehow. On 2012-03-01, at 2:00 PM, Vinay Sajip wrote: Guido van Rossum guido at

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread André Malo
* Serhiy Storchaka wrote: 01.03.12 16:47, André Malo написав(ла): On Thursday 01 March 2012 15:17:35 Serhiy Storchaka wrote: This is the first rational use of frozendict that I see. However, a deep copy is still necessary to create the frozendict. For this case, I believe, would be

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread André Malo
* Guido van Rossum wrote: On Thu, Mar 1, 2012 at 9:44 AM, Victor Stinner victor.stin...@gmail.com wrote: frozendict would help pysandbox but also any security Python module, not security, but also (many) other use cases ;-) Well, let's focus on the other use cases, because to me the

Re: [Python-Dev] PEP 414

2012-03-01 Thread Armin Ronacher
Hi, On 2/29/12 12:30 PM, Yury Selivanov wrote: I see you've (or somebody) changed: Yes, I reworded that. Could you just remove the statement completely? I will let Nick handle the PEP wording. I don't think that PEPs are the right place to put such polemic and biased statements. Why call

Re: [Python-Dev] PEP 414

2012-03-01 Thread R. David Murray
On Thu, 01 Mar 2012 21:12:48 +, Armin Ronacher armin.ronac...@active-4.com wrote: Hi, On 2/29/12 12:30 PM, Yury Selivanov wrote: I see you've (or somebody) changed: Yes, I reworded that. Could you just remove the statement completely? I will let Nick handle the PEP wording. I

Re: [Python-Dev] Sandboxing Python

2012-03-01 Thread Victor Stinner
I challenge anymore to break pysandbox! I would be happy if anyone breaks it because it would make it more stronger. Hum, I should give some rules for such contest: - the C module (_sandbox) must be used - you have to get access to a object outside the sandbox, like a real module, or get

Re: [Python-Dev] PEP 414

2012-03-01 Thread Barry Warsaw
Hopefully, I can say the following in a constructive way. I certainly don't mean to attack anyone personally for their closely held beliefs, though I might disagree with them. And you have the right to those beliefs and to express them in a respectful and constructive manner on this mailing

Re: [Python-Dev] PEP 414

2012-03-01 Thread Yury Selivanov
Hi Armin, Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. But if you'd just stuck to the point, without touching very controversial topics of what version of python is a good choice and what is a bad, with full review of all

Re: [Python-Dev] PEP 414

2012-03-01 Thread Armin Ronacher
Hi, On 3/1/12 10:38 PM, Yury Selivanov wrote: Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. Sorry if I sound cranky but I got that impression from the responses here (which are greatly different from the responses I got on other

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Guido van Rossum
On Thu, Mar 1, 2012 at 12:35 PM, André Malo n...@perlig.de wrote: * Guido van Rossum wrote: On Thu, Mar 1, 2012 at 9:44 AM, Victor Stinner victor.stin...@gmail.com wrote: frozendict would help pysandbox but also any security Python module, not security, but also (many) other use cases ;-)

Re: [Python-Dev] PEP 414

2012-03-01 Thread Yury Selivanov
On 2012-03-01, at 5:52 PM, Armin Ronacher wrote: Hi, On 3/1/12 10:38 PM, Yury Selivanov wrote: Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. Sorry if I sound cranky but I got that impression from the responses here (which

Re: [Python-Dev] PEP 414

2012-03-01 Thread Vinay Sajip
Armin Ronacher armin.ronacher at active-4.com writes: I tried my best but obviously it was not good enough to please everybody. In all honesty I did not expect that such a small change would spawn such a great discussion. After all what we're discussing here is the introduction of one

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Mark Janssen
On Thu, Mar 1, 2012 at 10:00 AM, Guido van Rossum gu...@python.org wrote: I do know that I don't feel comfortable having a sandbox in the Python standard library or even recommending a 3rd party sandboxing solution -- if someone uses the sandbox to protect a critical resource, and a hacker

Re: [Python-Dev] PEP 414

2012-03-01 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/01/2012 05:52 PM, Armin Ronacher wrote: Hi, On 3/1/12 10:38 PM, Yury Selivanov wrote: Sorry if I sounded like 'attacking' you. I certainly had no such intention, as I believe nobody on this list. Sorry if I sound cranky but I got that

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Victor Stinner
Le 01/03/2012 19:07, Guido van Rossum a écrit : What other use cases are there? frozendict could be used to implement read-only types: it is not possible to add or remove an attribute or set an attribute value, but attribute value can be a mutable object. Example of an enum with my

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Guido van Rossum
On Thu, Mar 1, 2012 at 4:39 PM, Victor Stinner victor.stin...@gmail.com wrote: Le 01/03/2012 19:07, Guido van Rossum a écrit : What other use cases are there? frozendict could be used to implement read-only types: it is not possible to add or remove an attribute or set an attribute value,

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread R. David Murray
On Thu, 01 Mar 2012 16:50:06 -0800, Guido van Rossum gu...@python.org wrote: On Thu, Mar 1, 2012 at 4:39 PM, Victor Stinner victor.stin...@gmail.com wrote: frozendict could be used to implement read-only types: it is not possible to add or remove an attribute or set an attribute value, but

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Nick Coghlan
On Fri, Mar 2, 2012 at 11:50 AM, R. David Murray rdmur...@bitdance.com wrote: +1.  Except in very limited circumstances (such as a security sandbox) I would *much* rather have the code I'm interacting with use advisory means rather than preventing me from being a consenting adult.  (Having to

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Yury Selivanov
On 2012-03-01, at 7:50 PM, Guido van Rossum wrote: I think you should provide stronger arguments in each case why the data needs to be truly immutable or read-only, rather than just using a convention or an advisory API (like __private can be circumvented but clearly indicates intent to the

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Guido van Rossum
On Thu, Mar 1, 2012 at 6:13 PM, Yury Selivanov yselivanov...@gmail.com wrote: On 2012-03-01, at 7:50 PM, Guido van Rossum wrote: I think you should provide stronger arguments in each case why the data needs to be truly immutable or read-only, rather than just using a convention or an advisory

Re: [Python-Dev] Add a frozendict builtin type

2012-03-01 Thread Yury Selivanov
On 2012-03-01, at 9:31 PM, Guido van Rossum wrote: That sounds *very* far-fetched. You're pretty much designing a new language variant. It's not an argument for burdening the original Yeah, that's what we do ;) You should be able to prototype what you want using an advisory subclass (if you

Re: [Python-Dev] PEP 414

2012-03-01 Thread Éric Araujo
Hello, Le 02/03/2012 00:15, Yury Selivanov a écrit : And that's just the beginning of such questions. And when this PEP was suddenly approved, many of us felt that all those questions are not answered and were not even discussed. Let me comment on that “suddenly”. We joke about Guido being

Re: [Python-Dev] PEP 414

2012-03-01 Thread Nick Coghlan
On Fri, Mar 2, 2012 at 2:08 PM, Éric Araujo mer...@netwok.org wrote: I can’t read Guido’s mind, but I think that here he pronounced somewhat quickly because he was convinced by the arguments in the PEP, while choosing to ignore the problems therein, knowing that they could be fixed later.