Re: [Python-Dev] dict.setdefault(object, object) instead of "sys.intern()" (was Re: sys.intern should work on bytes)

2013-09-21 Thread Stefan Behnel 
Jesus Cea, 20.09.2013 15:46:
> On 20/09/13 15:33, Benjamin Peterson wrote:
>> Well, the pickler should memoize bytes objects if you have lots of
>> the same one in a pickle...
> 
> Only if they are the very same object. Not diferent bytes objects with
> the same value. Pickle doesn't do "a==b" but "id(a)==id(b)".
> 
> Yes, I know that "a==b" would break mutable objects. It is just an
> example.
> 
> I don't want to pursue that path. Performance of pickle is already
> appallingly slow.
> 
> In my project, I will do the redundancy removal on my own way, as
> explained in ither message on this thread.
> 
> Example:
> 
> * Original pickle: 14416284 bytes
> 
> * Pickle with "interned" strings: 3004880 bytes
> (quite an improvement, but this is particular to my case, I have a lot
> of string duplications here. The pickle also loads a bit faster)
> 
> * Pickle including an extra dictionary of "interned" strings, created
> using the "interned.setdefault(object,object)" pattern: 5126587 bytes.
> Sniff.
> 
> Could I do this more compactly?.

ISTM that what you are looking for is a compression-like pattern that
efficiently encodes repeated literals (i.e. constants of safe types) in the
pickle. That could be achieved by extending the pickle protocol to include
backreferences to earlier objects, I guess (I'm not all that familiar with
the internals of the pickle format). Any of the well known compression
algorithms that are capable of handling streaming data would apply here.
Assuming you don't want to simply send the pickle output through gzip &
friends, that is...

It also seems to me that python-dev isn't the right place to discuss this.
python-ideas seems more appropriate for now.

Stefan

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use an empty def as a lambda

2013-09-21 Thread Ryan 
Change def to func? That's the worst idea I've heard yet. Def is already there; 
why break all existing code just for a word?

"Westley Martínez"  wrote:

>'def' is no more ambiguous than 'lambda', and is in fact more
>ambiguous,
>for 'def' doesn't lend itself to anything other than the word define,
>whilst 'lambda' can only mean lambda function.  Calling def explicit is
>silly.  It only makes sense because def arbitrarily means a function in
>Python (I'm proposing def become func or proc in Python 4000).
>
>To call lambda too 'computer sciencey' is equally ridiculous, for pro-
>gramming is a key spawn of computer science.  A programmer needs to
>have
>some knowledge of computer science to program, just like a physicist
>needs knowledge of calculus to understand mechanics.
>
>> -Original Message-
>> From: Python-Dev
>[mailto:[email protected]] On
>> Behalf Of Ben Gift
>> Sent: Thursday, September 19, 2013 1:54 PM
>> To: [email protected]
>> Subject: [Python-Dev] Use an empty def as a lambda
>> 
>> I think the lambda keyword is difficult to understand for many
>people. It
>> would be more pythonic to use an empty def call instead.
>> 
>> 
>> For instance this:
>> 
>> 
>> words.sort(key = lambda x: x[2])
>> 
>> 
>> could look like this:
>> 
>> words.sort(key = def (x): x[2])
>> 
>> 
>> It's obvious and explicit that we're creating an unnamed, anonymous
>function
>> this way.
>
>
>___
>Python-Dev mailing list
>[email protected]
>https://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe:
>https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use an empty def as a lambda

2013-09-21 Thread R. David Murray 
On Sat, 21 Sep 2013 12:55:20 -0500, Ryan  wrote:
> "Westley Martínez"  wrote:
> >'def' is no more ambiguous than 'lambda', and is in fact more
> >ambiguous,
> >for 'def' doesn't lend itself to anything other than the word define,
> >whilst 'lambda' can only mean lambda function.  Calling def explicit is
> >silly.  It only makes sense because def arbitrarily means a function in
> >Python (I'm proposing def become func or proc in Python 4000).
>
> Change def to func? That's the worst idea I've heard yet. Def is
> already there; why break all existing code just for a word?

Note that whatever happens with Python4, breaking of backward
compatibility (by anything other than the deletion of cruft) is very
unlikely to be part of it.

--David
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use an empty def as a lambda

2013-09-21 Thread Westley Martínez 
When I say Python 4000, I don't actually mean Python 4.x, I mean the
next backwards-incompatible revision to Python (if it comes to pass).
My reasoning is that we use class to make classes, lambda to make
lambda functions, and def to make--well not defs--functions, which
doesn't really make sense to me.

Changing lambda to def would break compatibility as well and restrict
the grammar further.  In addition, it provides little foreseeable
benefit since the term def has no real meaning outside of Python,
whilst lambda has a clearly defined meaning.  Had the original C pro-
grammers chosen def in lieu of typedef or #def in lieu of #define, I
doubt that def would have been chosen as the keyword for functions in
Python.

> -Original Message-
> From: Ryan [mailto:[email protected]]
> Sent: Saturday, September 21, 2013 10:55 AM
> To: Westley Martínez; 'Ben Gift'; [email protected]
> Subject: Re: [Python-Dev] Use an empty def as a lambda
> 
> Change def to func? That's the worst idea I've heard yet. Def is already
> there; why break all existing code just for a word?
> 
> 
> "Westley Martínez"  wrote:
> 
>   'def' is no more ambiguous than 'lambda', and is in fact more ambiguous,
>   for 'def' doesn't lend itself to anything other than the word define,
>   whilst 'lambda' can only mean lambda function.  Calling def explicit is
>   silly.  It only makes sense because def arbitrarily means a function in
>   Python (I'm proposing def become func or proc in Python 4000).
> 
>   To call lambda too 'computer sciencey' is equally ridiculous, for pro-
>   gramming is a key spawn of computer science.  A programmer needs to have
>   some knowledge of computer science to program, just like a physicist
>   needs knowledge of calculus to understand mechanics.
> 
> 
>   -Original Message-
>   From: Python-Dev [mailto:python-dev-
> [email protected]] On
>   Behalf Of Ben Gift
>   Sent: Thursday, September 19, 2013 1:54 PM
>   To: [email protected]
>   Subject: [Python-Dev] Use an empty def as a lambda
> 
>   I think the lambda keyword is difficult to understand for many
> people. It
>   would be more pythonic to use an empty def call instead.
> 
> 
>   For instance this:
> 
> 
>   words.sort(key = lambda x: x[2])
> 
> 
>   could look like this:
> 
>   words.sort(key = def (x): x[2])
> 
> 
>   It's obvious and explicit that we're creating an unnamed,
> anonymous function
>   this way.
> 
> 
> 
> 
> 
> 
>   Python-Dev mailing list
>   [email protected]
>   https://mail.python.org/mailman/listinfo/python-dev
>   Unsubscribe: https://mail.python.org/mailman/options/python-
> dev/rymg19%40gmail.com
> 
> 
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use an empty def as a lambda

2013-09-21 Thread Ben Finney 
Westley Martínez  writes:

> My reasoning is that we use class to make classes, lambda to make
> lambda functions, and def to make--well not defs--functions, which
> doesn't really make sense to me.

Your reasoning is flawed. There is no such thing in Python as a “lambda
function”.

Python has functions. It doesn't matter whether you use a ‘lambda’ or
‘def’ statement to create it, there's no resulting difference in the
type of the object. It is a function.

So: you make a class with a ‘class’ statement; you make a function using
either a ‘def’ statement or a ‘lambda’ expression. There is no third
type of object being discussed here.

-- 
 \ “Faith may be defined briefly as an illogical belief in the |
  `\  occurrence of the improbable.” —Henry L. Mencken |
_o__)  |
Ben Finney

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Revert #12085 fix for __del__ attribute error message

2013-09-21 Thread Terry Reedy 
When an AttributeError is raised in a __del__ method, it is caught and 
ignored, except that it is not completely ignored but is replaced by a 
warning message sent to stderr. Example:

>>> class C():
def __del__(self): raise AttributeError

>>> c=C()
>>> del c
Exception AttributeError: AttributeError() in <__main__.C object at 0x0351A198>> ignored


The AttributeError is ignored, I presume, because one may be expected if 
__new__ runs but __init__ does not, because of an exception. 
AttributeErrors can also be expected during exit cleanup.


The message seems not to be a Warning. So it seems that is cannot be 
suppressed. The OP of http://bugs.python.org/issue12085 encountered this 
message with an unnecessary use of subprocesses.Popen. (There is better 
code that avoids the problem.)  He considered the message a problem as 
it interfered with his output. He suggested the fix of changing 
'self.attr' to "getattr(self. 'attr', False)".


Viktor Stinner applied this fix. I believe it should be reverted for 
three reasons:


1. It results in TypeError on exit when gettattr is set to None before 
__delete__ is called.  http://bugs.python.org/issue19021.  Even is that 
is fixed with an alternate patch (creating a 'backup' class attribute), 
I think it would still be wrong.


2. If we do not agree with the OP that putting output on stderr is bad, 
then no fix is needed and there should be none.


3. If we do agree that the unconditional output is a problem, then there 
should be a general fix. It is, in a sense, just happenstance that the 
OP encountered this with Popen. Some possibilities:


a. Decide the the message is not needed; just silently ignore the 
__del__ AttributeError. I am not sure what its intended use is.


b. Make the message a Warning that can be blocked, perhaps a 
RuntimeWarning. Does this warning message predate the warnings module, 
new in 2.1?


c. If the __del__ AttributeError occurs while processing the __init__ 
error, just chain it normally. But this would, I presume interfere with 
catching the __init__ exception.


d. Add a missing attribute in cleanup message to end of the __init__ 
exception message.


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Revert #12085 fix for __del__ attribute error message

2013-09-21 Thread R. David Murray 
On Sat, 21 Sep 2013 17:16:41 -0400, Terry Reedy  wrote:
> When an AttributeError is raised in a __del__ method, it is caught and 
> ignored, except that it is not completely ignored but is replaced by a 
> warning message sent to stderr. Example:
>  >>> class C():
>   def __del__(self): raise AttributeError
>   
>  >>> c=C()
>  >>> del c
> Exception AttributeError: AttributeError() in  <__main__.C object at 0x0351A198>> ignored

This is a replacement for a traceback.  In later Python versions, the
full traceback is printed.  In the general case it represents a bug in
the code that should be fixed.  Most such errors arise from the vagaries
of module finalization (such as your issue 19021), but not all of them
do: the rest represent real bugs in __del__ methods (which are executed
asynchronously in the general case).

So the question is, is the bug in the user code, or the stdlib code?
>From the issue, it sounds like it could be considered either (or most
likely, both).

--David
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Revert #12085 fix for __del__ attribute error message

2013-09-21 Thread Terry Reedy 

On 9/21/2013 6:15 PM, R. David Murray wrote:

On Sat, 21 Sep 2013 17:16:41 -0400, Terry Reedy  wrote:

When an AttributeError is raised in a __del__ method, it is caught and
ignored, except that it is not completely ignored but is replaced by a
warning message sent to stderr. Example:
  >>> class C():
def __del__(self): raise AttributeError

  >>> c=C()
  >>> del c
Exception AttributeError: AttributeError() in > ignored


This is a replacement for a traceback.  In later Python versions, the
full traceback is printed.


The above is 3.3.2. In 3.4.0a2, the traceback of the ignored exception 
is indeed printed.


Exception ignored in: 0x039946D8>>

Traceback (most recent call last):
  File "", line 2, in __del__
AttributeError:


In the general case it represents a bug in
the code that should be fixed.  Most such errors arise from the vagaries
of module finalization (such as your issue 19021),


Lets call that a buglet ;-). Not really harmful, but annoying. Accepting 
that even such buglets 'should' be fixed in the stdllib, so that the 
message does not appear, is there any reason *not* to make it a 
RuntimeWarning so that users who care about clean output can filter it 
out while waiting for us to fix it?


This would be a separate issue from #12085.

> but not all of them

do: the rest represent real bugs in __del__ methods (which are executed
asynchronously in the general case).


Which is why the message should be printed, so the developer can decide.

--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use an empty def as a lambda

2013-09-21 Thread Westley Martínez 
> -Original Message-
> From: Python-Dev [mailto:[email protected]] On
> Behalf Of Ben Finney
> Sent: Saturday, September 21, 2013 12:56 PM
> To: [email protected]
> Subject: Re: [Python-Dev] Use an empty def as a lambda
> 
> Westley Martínez  writes:
> 
> > My reasoning is that we use class to make classes, lambda to make
> > lambda functions, and def to make--well not defs--functions, which
> > doesn't really make sense to me.
> 
> Your reasoning is flawed. There is no such thing in Python as a “lambda
> function”.
> 
> Python has functions. It doesn't matter whether you use a ‘lambda’ or
> ‘def’ statement to create it, there's no resulting difference in the
> type of the object. It is a function.
> 
> So: you make a class with a ‘class’ statement; you make a function using
> either a ‘def’ statement or a ‘lambda’ expression. There is no third
> type of object being discussed here.

This is true.  The final object the Python creates is, to my knowledge,
the same regardless of whether it uses define or lambda.  But it's
irrelevant if it's the same or not, since one could argue that every-
thing is going to end up as a series of 0s and 1s.  In that sense a
function is nothing more than a glorified integer.  Lambda function is
just a term, like method or procedure instead function, although it is
certainly more specific in the sense that it is anonymous.  In my list I
was citing general concepts, not specific Python objects.

class is obvious, lambda is obvious, def is not as obvious, though
certainly more obvious than C-style declarations.  If lambda should
change I think anon or expr would be more suitable, or even func (so
long as def not become func).

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Revert #12085 fix for __del__ attribute error message

2013-09-21 Thread Guido van Rossum 
Exceptions in __del__ point to bugs (sometimes in the stdlib) that should
be fixed, period. The only reason they do not result in exceptions that are
properly bubbled up and catchable is because __del__ is called from a
DECREF macro which has no return value. Also, IMO writing to stderr is fair
game -- reporting errors is what it is for.


As to making them warnings, I don't know that the warnings machinery is
easily adaptable for this purpose. Warnings can be suppressed but they can
also be turned into full exceptions; the latter doesn't really apply here
(see previous paragraph). But I would not object if someone found a way to
do this, though I'd prefer the default behavior to remain what it is in 3.4
(print a full traceback).


On Sat, Sep 21, 2013 at 4:55 PM, Terry Reedy  wrote:

> On 9/21/2013 6:15 PM, R. David Murray wrote:
>
>> On Sat, 21 Sep 2013 17:16:41 -0400, Terry Reedy  wrote:
>>
>>> When an AttributeError is raised in a __del__ method, it is caught and
>>> ignored, except that it is not completely ignored but is replaced by a
>>> warning message sent to stderr. Example:
>>>   >>> class C():
>>> def __del__(self): raise AttributeError
>>>
>>>   >>> c=C()
>>>   >>> del c
>>> Exception AttributeError: AttributeError() in >> <__main__.C object at 0x0351A198>> ignored
>>>
>>
>> This is a replacement for a traceback.  In later Python versions, the
>> full traceback is printed.
>>
>
> The above is 3.3.2. In 3.4.0a2, the traceback of the ignored exception is
> indeed printed.
>
> Exception ignored in:  0x039946D8>>
> Traceback (most recent call last):
>   File "", line 2, in __del__
> AttributeError:
>
>
>  In the general case it represents a bug in
>> the code that should be fixed.  Most such errors arise from the vagaries
>> of module finalization (such as your issue 19021),
>>
>
> Lets call that a buglet ;-). Not really harmful, but annoying. Accepting
> that even such buglets 'should' be fixed in the stdllib, so that the
> message does not appear, is there any reason *not* to make it a
> RuntimeWarning so that users who care about clean output can filter it out
> while waiting for us to fix it?
>
> This would be a separate issue from #12085.
>
>
> > but not all of them
>
>> do: the rest represent real bugs in __del__ methods (which are executed
>> asynchronously in the general case).
>>
>
> Which is why the message should be printed, so the developer can decide.
>
> --
> Terry Jan Reedy
>
>
> __**_
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/**mailman/options/python-dev/**
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Use an empty def as a lambda

2013-09-21 Thread Steven D'Aprano 
Guys, this thread is not Python-Dev territory. It should have gone to
Python-Ideas. I repeat what I posted two days ago:

Proposals for changes to syntax and functionality are normally 
expected to gather feedback on python-ideas before coming to 
python-dev for final approval or rejection.

https://mail.python.org/mailman/listinfo/python-ideas

http://docs.python.org/devguide/communication.html


Please drop the discussion here.


-- 
Steven
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Best practice for documentation for std lib

2013-09-21 Thread Steven D'Aprano 
Hi all,

I have a question about how I should manage documentation for the 
statistics module for Python 3.4. At the moment, I have extensive 
docstrings in the module itself. I don't believe anyone has flagged that 
as "too much information" in a code review, so I'm going to assume that 
large docstrings will be acceptable.

However, I have been asked to ensure that there is a separate 
statistics.rst file for documentation.

I don't want to have to maintain the documentation in two places, both 
in the .py module and in .rst file. Can anyone give me some pointers as 
to best practice in this situation? Is there a "How To Write Docs For 
The Standard Library" document somewhere? Perhaps I missed it, but my 
searches found nothing useful. I have read this:

http://docs.python.org/devguide/documenting.html

but it didn't shed any light on my situation.


Thanks in advance,



-- 
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Best practice for documentation for std lib

2013-09-21 Thread Benjamin Peterson 
There's not really much to do but maintain them separately. Truncate
the docstrings if it makes life easier.

2013/9/21 Steven D'Aprano :
> Hi all,
>
> I have a question about how I should manage documentation for the
> statistics module for Python 3.4. At the moment, I have extensive
> docstrings in the module itself. I don't believe anyone has flagged that
> as "too much information" in a code review, so I'm going to assume that
> large docstrings will be acceptable.
>
> However, I have been asked to ensure that there is a separate
> statistics.rst file for documentation.
>
> I don't want to have to maintain the documentation in two places, both
> in the .py module and in .rst file. Can anyone give me some pointers as
> to best practice in this situation? Is there a "How To Write Docs For
> The Standard Library" document somewhere? Perhaps I missed it, but my
> searches found nothing useful. I have read this:
>
> http://docs.python.org/devguide/documenting.html
>
> but it didn't shed any light on my situation.
>
>
> Thanks in advance,
>
>
>
> --
> Steve
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/benjamin%40python.org



-- 
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com