Re: [Python-Dev] terminology for "free variables" in Python

[Eli Bendersky]

>> def some_func(myparam):

> > def internalfunc():
> > return cc * myparam
> >
> > CPython infers that in 'internalfunc', while 'myparam' is free, 'cc' is
>
> What exactly do you mean by "infers" ? How do you know that it infers
> that? How does it matter for your understanding of the code?
>

The easiest way I found to see what CPython thinks is use the 'symtable'
module. With its help, it's clear that in the function above, myparam is
considered free while cc is considered global. When querying symtable about
the symbol myparam, the is_free method returns True while the is_global
method returns False, and vice versa for cc.

Of course it can also be seen in the code of symtable.c in function
analyze_name, and as Nick showed in his message it also affects the way
bytecode is generated for the two symbols.

My intention in this post was to clarify whether I'm misunderstanding
something or the term 'free' is indeed used for different things in
different places. If this is the latter, IMHO it's an inconsistency, even if
a small one. When I read the code I saw  'free' I went to the docs only to
read that 'free' is something else. This was somewhat confusing.

Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] terminology for "free variables" in Python

[Eli Bendersky]

> There's a slight inconsistency. The names a code object explicitly
> calls out as free variables (i.e. references to cells in outer scopes)
> are only a subset of the full set of free variables (every referenced
> name that isn't a local variable or an attribute).
>
> >>> from dis import show_code
> >>> def outer():
> ...   x, y = 1, 2
> ...   def inner():
> ... print (x, y, a, b, c.e)
> ...   return inner
> ...
> >>> f = outer()
> >>> show_code(f)
>

Nick, did you know that dis.show_code is neither exported by default from
the dis module, nor it's documented in its help() or .rst documentation?
Neither is code_info(), which is used by show_code(). I wonder if this is
intentional.

Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] terminology for "free variables" in Python

[Nick Coghlan]

On Fri, Sep 10, 2010 at 5:06 PM, Eli Bendersky  wrote:
> Nick, did you know that dis.show_code is neither exported by default from
> the dis module, nor it's documented in its help() or .rst documentation?
> Neither is code_info(), which is used by show_code(). I wonder if this is
> intentional.

code_info is in the normal documentation. I even remembered the
versionadded tag without Georg reminding me ;)

The omission from __all__ (and hence the module help text) was
accidental and is now fixed.

The omission of show_code from the documentation was deliberate, and
I've now added a comment to that effect (the history is that
dis.show_code has been around, but undocumented, for a while. The fact
that it printed directly to stdout rather than producing a formatted
string was mildly irritating, so I refactored the formatting part out
into code_info, leaving just a single print call in show_code. Since I
only kept show_code around for backwards compatibility reasons, I
don't see any point in advertising its existence - better for people
to just call code_info and print the result themselves.

Although it *is* somewhat handy for quick introspection at the
interpreter prompt... maybe I should document it after all. Thoughts?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] terminology for "free variables" in Python

[Eli Bendersky]

On Fri, Sep 10, 2010 at 15:41, Nick Coghlan  wrote:

> On Fri, Sep 10, 2010 at 5:06 PM, Eli Bendersky  wrote:
> > Nick, did you know that dis.show_code is neither exported by default from
> > the dis module, nor it's documented in its help() or .rst documentation?
> > Neither is code_info(), which is used by show_code(). I wonder if this is
> > intentional.
>
> code_info is in the normal documentation. I even remembered the
> versionadded tag without Georg reminding me ;)
>

When you say "is in the normal documentation", do you mean you added it
recently ? Although I see it here:
http://docs.python.org/dev/py3k/library/dis.html, it's neither in the docs
of 3.1.2 (http://docs.python.org/py3k/library/dis.html), nor in 2.7, nor in
a build of 3.2 I have lying around from a couple of weeks ago.

Although it *is* somewhat handy for quick introspection at the
> interpreter prompt... maybe I should document it after all. Thoughts?
>
>
I mostly use the dis module for quick-n-dirty exploration of the results of
compilation into bytecode, and I'm sure many people use for the same effect.
Thus show_code seems like a convenient shortcut, although not a necessary
one. The string returned by code_info isn't interactive-shell friendly, and
show_code saves the print(...).

Personally I think that if it's there, it should be documented. If it's
better not to use it, it should be removed or at least marked deprecated in
the documentation/docstring.

Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] terminology for "free variables" in Python

[Nick Coghlan]

On Fri, Sep 10, 2010 at 11:23 PM, Eli Bendersky  wrote:
> When you say "is in the normal documentation", do you mean you added it
> recently ? Although I see it here:
> http://docs.python.org/dev/py3k/library/dis.html, it's neither in the docs
> of 3.1.2 (http://docs.python.org/py3k/library/dis.html), nor in 2.7, nor in
> a build of 3.2 I have lying around from a couple of weeks ago.

The module and docs changes both went in on August 17 as part of the
same commit (r84133), so I'm not sure how you could have a local
checkout with the module changes but not the doc changes. A checkout
from early August wouldn't have either, of course.

> I mostly use the dis module for quick-n-dirty exploration of the results of
> compilation into bytecode, and I'm sure many people use for the same effect.
> Thus show_code seems like a convenient shortcut, although not a necessary
> one. The string returned by code_info isn't interactive-shell friendly, and
> show_code saves the print(...).
>
> Personally I think that if it's there, it should be documented. If it's
> better not to use it, it should be removed or at least marked deprecated in
> the documentation/docstring.

Yeah, I changed my mind and have now documented it properly. The 3.2
versionadded tag on show_code is currently a little questionable
though. Guido actually checked in the original (undocumented) version
of show_code before 3.0 was released. The only thing new about it in
3.2 is it being mentioned in the documentation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] terminology for "free variables" in Python

[Eli Bendersky]

> > I mostly use the dis module for quick-n-dirty exploration of the results
> of
> > compilation into bytecode, and I'm sure many people use for the same
> effect.
> > Thus show_code seems like a convenient shortcut, although not a necessary
> > one. The string returned by code_info isn't interactive-shell friendly,
> and
> > show_code saves the print(...).
> >
> > Personally I think that if it's there, it should be documented. If it's
> > better not to use it, it should be removed or at least marked deprecated
> in
> > the documentation/docstring.
>
> Yeah, I changed my mind and have now documented it properly. The 3.2
> versionadded tag on show_code is currently a little questionable
> though. Guido actually checked in the original (undocumented) version
> of show_code before 3.0 was released. The only thing new about it in
> 3.2 is it being mentioned in the documentation.
>

Looks good to me.

Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] terminology for "free variables" in Python

[Guido van Rossum]

On Fri, Sep 10, 2010 at 12:00 AM, Eli Bendersky  wrote:
>>> def some_func(myparam):
>>
>> >     def internalfunc():
>> >     return cc * myparam
>> >
>> > CPython infers that in 'internalfunc', while 'myparam' is free, 'cc' is
>>
>> What exactly do you mean by "infers" ? How do you know that it infers
>> that? How does it matter for your understanding of the code?
>
> The easiest way I found to see what CPython thinks is use the 'symtable'
> module. With its help, it's clear that in the function above, myparam is
> considered free while cc is considered global. When querying symtable about
> the symbol myparam, the is_free method returns True while the is_global
> method returns False, and vice versa for cc.
>
> Of course it can also be seen in the code of symtable.c in function
> analyze_name, and as Nick showed in his message it also affects the way
> bytecode is generated for the two symbols.
>
> My intention in this post was to clarify whether I'm misunderstanding
> something or the term 'free' is indeed used for different things in
> different places. If this is the latter, IMHO it's an inconsistency, even if
> a small one. When I read the code I saw  'free' I went to the docs only to
> read that 'free' is something else. This was somewhat confusing.

I'm still not clear if my explanation that globals are a subset of
free variables got rid of the confusion. The full name for what
CPython marks as "free" would be "free but not global" but that's too
much of a mouthful.

Also you're digging awfully deep into the implementation here -- AFAIC
CPython could have called them "type A" and "type B" and there would
not have been any problem for compliance with the langage reference.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] terminology for "free variables" in Python

[Eli Bendersky]

>
> > My intention in this post was to clarify whether I'm misunderstanding
> > something or the term 'free' is indeed used for different things in
> > different places. If this is the latter, IMHO it's an inconsistency, even
> if
> > a small one. When I read the code I saw  'free' I went to the docs only
> to
> > read that 'free' is something else. This was somewhat confusing.
>
> I'm still not clear if my explanation that globals are a subset of
> free variables got rid of the confusion. The full name for what
> CPython marks as "free" would be "free but not global" but that's too
> much of a mouthful.
>

Yes, I understand it now. The source code of symtable.c has a long comment
above the SET_SCOPE macro which says, among other things: "An implicit
global is a free variable for which the compiler has found no binding in an
enclosing function scope", which is in tune with what you said.


> Also you're digging awfully deep into the implementation here --
>

Indeed, it all started when I set to understand how symbol tables are
implemented in CPython. The inconsistency in the usage of "free" confused
me, so I consulted pydev for clarification. I'm no longer confused :-)

Regards,
Eli
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] terminology for "free variables" in Python

[Éric Araujo]

> Yeah, I changed my mind and have now documented it properly. The 3.2
> versionadded tag on show_code is currently a little questionable
> though. Guido actually checked in the original (undocumented) version
> of show_code before 3.0 was released. The only thing new about it in
> 3.2 is it being mentioned in the documentation.

versionadded marks the addition of a feature (see
docs.python.org/documenting), so it should be removed here.

Regards

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2010-09-03 - 2010-09-10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues stats:
  open2554 (+46)
  closed 19050 (+69)
  total  21604 (+65)

Open issues with patches: 1078 


Issues opened (46)
==

#8420: Objects/setobject.c contains unsafe code
http://bugs.python.org/issue8420  reopened by eric.araujo

#9129: DoS smtpd module vulnerability
http://bugs.python.org/issue9129  reopened by giampaolo.rodola

#9261: include higher (../../) dirs fails
http://bugs.python.org/issue9261  reopened by eric.araujo

#9759: GzipFile object should raise ValueError on .read() after .clos
http://bugs.python.org/issue9759  opened by ack

#9761: stale .pyc files aren't cleaned out
http://bugs.python.org/issue9761  opened by Steve.Thompson

#9762: build failures
http://bugs.python.org/issue9762  opened by pitrou

#9763: Crashes upon run after syntax error encountered in OSX 10.5.8
http://bugs.python.org/issue9763  opened by william.barr

#9764: Tools/buildbot/external.bat should download and built tix
http://bugs.python.org/issue9764  opened by stutzbach

#9765: tcl-8 vs tcl8
http://bugs.python.org/issue9765  opened by stutzbach

#9769: PyUnicode_FromFormatV() doesn't handle non-ascii text correctl
http://bugs.python.org/issue9769  opened by haypo

#9770: curses.isblank function doesn't match ctype.h
http://bugs.python.org/issue9770  opened by kevinpt

#9771: add an optional "default" argument to tokenize.detect_encoding
http://bugs.python.org/issue9771  opened by flox

#9772: test_pep277 failure on AMD64 debian parallel buildbot
http://bugs.python.org/issue9772  opened by flox

#9773: test_tarfile fails because of inaccurate mtime on AMD64 debian
http://bugs.python.org/issue9773  opened by flox

#9774: test_smtpnet fails with "[110] Connection timed out" on AMD64 
http://bugs.python.org/issue9774  opened by flox

#9775: Multiprocessing, logging and atexit play not well together
http://bugs.python.org/issue9775  opened by aronacher

#9778: Make hash values the same width as   a pointer   (or Py_ssize_t)
http://bugs.python.org/issue9778  opened by pitrou

#9779: argparse.ArgumentParser not support unicode in print help
http://bugs.python.org/issue9779  opened by gkraser

#9782: _multiprocessing.c warnings under 64-bit Windows
http://bugs.python.org/issue9782  opened by pitrou

#9783: _elementtree.c warnings under 64-bit Windows
http://bugs.python.org/issue9783  opened by pitrou

#9784: _msi.c warnings under 64-bit Windows
http://bugs.python.org/issue9784  opened by pitrou

#9786: Native TLS support for pthreads
http://bugs.python.org/issue9786  opened by krisvale

#9787: Release the TLS lock during allocations
http://bugs.python.org/issue9787  opened by krisvale

#9788: atexit and execution order
http://bugs.python.org/issue9788  opened by giampaolo.rodola

#9790: ntpath contains imports inside functions
http://bugs.python.org/issue9790  opened by brian.curtin

#9791: nntplib.py lacks a test suite
http://bugs.python.org/issue9791  opened by giampaolo.rodola

#9795: nntplib.NNTP should support the context protocol
http://bugs.python.org/issue9795  opened by pitrou

#9796: Add summary tables for unittest API
http://bugs.python.org/issue9796  opened by eric.araujo

#9800: Fast path for small int-indexing of lists and tuples
http://bugs.python.org/issue9800  opened by pitrou

#9801: Can not use append/extend to lists in a multiprocessing manage
http://bugs.python.org/issue9801  opened by Jimbofbx

#9802: Document 'stability' of builtin min() and max()
http://bugs.python.org/issue9802  opened by mattheww

#9803: IDLE closes with save while breakpoint open
http://bugs.python.org/issue9803  opened by Jimbofbx

#9807: deriving configuration information for different builds  with t
http://bugs.python.org/issue9807  opened by doko

#9808: Implement os.getlogin on Windows
http://bugs.python.org/issue9808  opened by janglin

#9809: Wrong Registery Entries on win64
http://bugs.python.org/issue9809  opened by GreYFoX

#9810: bzip2 build sometimes fails (VS8.0)
http://bugs.python.org/issue9810  opened by ocean-city

#9811: strftime strips '%' from unknown format codes on OS X
http://bugs.python.org/issue9811  opened by sverrejoh

#9812: cPickle segfault with nested dicts in threaded env
http://bugs.python.org/issue9812  opened by kdombrowski

#9814: subprocess isn't friendly to other Python implementations with
http://bugs.python.org/issue9814  opened by dino.viehland

#9815: test_tarfile sometimes ends with error "Cannot remoe dir"
http://bugs.python.org/issue9815  opened by ocean-city

#9817: expat copyright/license file is missing
http://bugs.python.org/issue9817  opened by doko

#9818: build files to build Lib/distutils/command/wininst-9.0* are mi
http://bugs.python.org/issue9818  opened by doko

#9820: Windows : os.listdir(b'.') doesn't raise an error for unencoda
http://bugs.python.or

[Python-Dev] Garbage announcement printed on interpreter shutdown

[Georg Brandl]

Hey #python-dev,

I'd like to ask your opinion on this change; I think it should be reverted
or at least made silent by default.  Basically, it prints a warning like

  gc: 2 uncollectable objects at shutdown:
  Use gc.set_debug(gc.DEBUG_UNCOLLECTABLE) to list them.

at interpreter shutdown if gc.garbage is nonempty.

IMO this runs contrary to the decision we made when DeprecationWarnings were
made silent by default: it spews messages not only at developers, but also at
users, who don't need it and probably are going to be quite confused by it,
assuming it came from their console application (imagine Mercurial printing
this).

Opinions?

Georg


Am 09.08.2010 00:18, schrieb antoine.pitrou:
> Author: antoine.pitrou
> Date: Mon Aug  9 00:18:46 2010
> New Revision: 83861
> 
> Log:
> Issue #477863: Print a warning at shutdown if gc.garbage is not empty.
> 
> 
> 
> Modified:
>python/branches/py3k/Doc/library/gc.rst
>python/branches/py3k/Doc/whatsnew/3.2.rst
>python/branches/py3k/Include/pythonrun.h
>python/branches/py3k/Lib/test/test_gc.py
>python/branches/py3k/Misc/NEWS
>python/branches/py3k/Modules/gcmodule.c
>python/branches/py3k/Python/pythonrun.c
> 
> Modified: python/branches/py3k/Doc/library/gc.rst
> ==
> --- python/branches/py3k/Doc/library/gc.rst   (original)
> +++ python/branches/py3k/Doc/library/gc.rst   Mon Aug  9 00:18:46 2010
> @@ -177,6 +177,15 @@
> If :const:`DEBUG_SAVEALL` is set, then all unreachable objects will be 
> added to
> this list rather than freed.
>  
> +   .. versionchanged:: 3.2
> +  If this list is non-empty at interpreter shutdown, a warning message
> +  gets printed:
> +
> +   ::
> +
> +  gc: 2 uncollectable objects at shutdown:
> +  Use gc.set_debug(gc.DEBUG_UNCOLLECTABLE) to list them.
> +
>  The following constants are provided for use with :func:`set_debug`:
>  
>  
> @@ -197,6 +206,9 @@
> reachable but cannot be freed by the collector).  These objects will be 
> added to
> the ``garbage`` list.
>  
> +   .. versionchanged:: 3.2
> +  Also print the contents of the :data:`garbage` list at interpreter
> +  shutdown (rather than just its length), if it isn't empty.
>  
>  .. data:: DEBUG_SAVEALL
>  
> 
> Modified: python/branches/py3k/Doc/whatsnew/3.2.rst
> ==
> --- python/branches/py3k/Doc/whatsnew/3.2.rst (original)
> +++ python/branches/py3k/Doc/whatsnew/3.2.rst Mon Aug  9 00:18:46 2010
> @@ -119,6 +119,11 @@
>  * The :class:`ftplib.FTP` class now supports the context manager protocol
>(Contributed by Tarek Ziadé and Giampaolo Rodolà; :issue:`4972`.)
>  
> +* A warning message will now get printed at interpreter shutdown if
> +  the :data:`gc.garbage` list isn't empty.  This is meant to make the
> +  programmer aware that his code contains object finalization issues.
> +  (Added by Antoine Pitrou; :issue:`477863`.)
> +
>  * The :func:`shutil.copytree` function has two new options:
>  
>* *ignore_dangling_symlinks*: when ``symlinks=False`` (meaning that the
> 
> Modified: python/branches/py3k/Include/pythonrun.h
> ==
> --- python/branches/py3k/Include/pythonrun.h  (original)
> +++ python/branches/py3k/Include/pythonrun.h  Mon Aug  9 00:18:46 2010
> @@ -148,6 +148,7 @@
>  PyAPI_FUNC(void) PyByteArray_Fini(void);
>  PyAPI_FUNC(void) PyFloat_Fini(void);
>  PyAPI_FUNC(void) PyOS_FiniInterrupts(void);
> +PyAPI_FUNC(void) _PyGC_Fini(void);
>  
>  /* Stuff with no proper home (yet) */
>  PyAPI_FUNC(char *) PyOS_Readline(FILE *, FILE *, char *);
> 
> Modified: python/branches/py3k/Lib/test/test_gc.py
> ==
> --- python/branches/py3k/Lib/test/test_gc.py  (original)
> +++ python/branches/py3k/Lib/test/test_gc.py  Mon Aug  9 00:18:46 2010
> @@ -1,5 +1,5 @@
>  import unittest
> -from test.support import verbose, run_unittest
> +from test.support import verbose, run_unittest, strip_python_stderr
>  import sys
>  import gc
>  import weakref
> @@ -466,6 +466,42 @@
>  # would be damaged, with an empty __dict__.
>  self.assertEqual(x, None)
>  
> +def test_garbage_at_shutdown(self):
> +import subprocess
> +code = """if 1:
> +import gc
> +class X:
> +def __init__(self, name):
> +self.name = name
> +def __repr__(self):
> +return "" %% self.name
> +def __del__(self):
> +pass
> +
> +x = X('first')
> +x.x = x
> +x.y = X('second')
> +del x
> +if %d:
> +gc.set_debug(gc.DEBUG_UNCOLLECTABLE)
> +"""
> +def run_command(code):
> +p = subproc

Re: [Python-Dev] terminology for "free variables" in Python

[Georg Brandl]

Am 10.09.2010 14:41, schrieb Nick Coghlan:
> On Fri, Sep 10, 2010 at 5:06 PM, Eli Bendersky  wrote:
>> Nick, did you know that dis.show_code is neither exported by default from
>> the dis module, nor it's documented in its help() or .rst documentation?
>> Neither is code_info(), which is used by show_code(). I wonder if this is
>> intentional.
> 
> code_info is in the normal documentation. I even remembered the
> versionadded tag without Georg reminding me ;)
> 
> The omission from __all__ (and hence the module help text) was
> accidental and is now fixed.
> 
> The omission of show_code from the documentation was deliberate, and
> I've now added a comment to that effect (the history is that
> dis.show_code has been around, but undocumented, for a while. The fact
> that it printed directly to stdout rather than producing a formatted
> string was mildly irritating, so I refactored the formatting part out
> into code_info, leaving just a single print call in show_code. Since I
> only kept show_code around for backwards compatibility reasons, I
> don't see any point in advertising its existence - better for people
> to just call code_info and print the result themselves.
> 
> Although it *is* somewhat handy for quick introspection at the
> interpreter prompt... maybe I should document it after all. Thoughts?

IMO show_code() is not a good name, because the only thing it doesn't
do is to -- show the code.

I'd rather call it "codeinfo" (which also is more in line with current
dis module function names).

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Garbage announcement printed on interpreter shutdown

[Fred Drake]

On Fri, Sep 10, 2010 at 4:32 PM, Georg Brandl  wrote:
> IMO this runs contrary to the decision we made when DeprecationWarnings were
> made silent by default: it spews messages not only at developers, but also at
> users, who don't need it and probably are going to be quite confused by it,

Agreed; this should be silent by default.


  -Fred

--
Fred L. Drake, Jr.    
"A storm broke loose in my mind."  --Albert Einstein
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Garbage announcement printed on interpreter shutdown

[Amaury Forgeot d'Arc]

2010/9/10 Fred Drake :
> On Fri, Sep 10, 2010 at 4:32 PM, Georg Brandl  wrote:
>> IMO this runs contrary to the decision we made when DeprecationWarnings were
>> made silent by default: it spews messages not only at developers, but also at
>> users, who don't need it and probably are going to be quite confused by it,
>
> Agreed; this should be silent by default.

+1. I suggest to enable it only when Py_DEBUG (or Py_TRACE_REFS or
Py_REF_DEBUG?) is defined.

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Garbage announcement printed on interpreter shutdown

[Łukasz Langa]

Georg Brandl writes:

> it prints a warning (...) at interpreter shutdown if gc.garbage is nonempty.
> 
> IMO this runs contrary to the decision we made when DeprecationWarnings were
> made silent by default
> 
> Opinions?

Agreed, this should be reverted for the reasons you give but DO LEAVE THIS on 
by default for regrtest (or maybe unittest in general) :) It has already proved 
useful for me. Is that doable?

-- 
Best regards,
Łukasz Langa
tel. +48 791 080 144
WWW http://lukasz.langa.pl/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Garbage announcement printed on interpreter shutdown

[Daniel Stutzbach]

On Fri, Sep 10, 2010 at 3:32 PM, Georg Brandl  wrote:

> IMO this runs contrary to the decision we made when DeprecationWarnings
> were
> made silent by default: it spews messages not only at developers, but also
> at
> users, who don't need it and probably are going to be quite confused by it,
> assuming it came from their console application (imagine Mercurial printing
> this).
>

A non-empty gc.garbage often indicates that there is a bug in the program
and that it is probably leaking memory [1].  That's a little different from
a DeprecationWarning which doesn't indicate a bug; it just indicates that
the program might not run correctly using a future version of Python.  I
think a better comparison would be with exceptions throw from a __del__,
which (as far as I know) are still printed to the console.

+1 on adding a way to enable/disable the feature.
-1 on removing the feature
-0 on making it disabled by default

[1] I know that some large, long-running programs periodically check
gc.garbage and carefully choose where to break cycles, but those are the
exception and not the rule.
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python needs a standard asynchronous return object

[James Yonan]

I'd like to propose that the Python community standardize on a 
"deferred" object for asynchronous return values, modeled after the 
well-thought-out Twisted Deferred class.


With more and more Python libraries implementing asynchronicity (for 
example Futures -- PEP 3148), it's crucial to have a standard deferred 
object in place so that code using a single asynchronous reactor can 
interoperate with different asynchronous libraries.


I think a lot of people don't realize how much cooler and more elegant 
it is to return a deferred object from an asynchronous function rather 
than using a generic callback approach (where you pass a function 
argument to the asynchronous function telling it where to call when the 
asynchronous operation completes).


While asynchronous systems have been shown to have excellent scalability 
properties, the callback-based programming style often used in 
asynchronous programming has been criticized for breaking up the 
sequential readability of program logic.


This problem is elegantly addressed by using Deferred Generators.  Since 
Python 2.5 added enhanced generators (i.e. the capability for "yield" to 
return a value), the infrastructure is now in place to allow an 
asynchronous function to be written in a sequential style, without the 
use of explicit callbacks.


See the following blog article for a nice write-up on the capability:

http://blog.mekk.waw.pl/archives/14-Twisted-inlineCallbacks-and-deferredGenerator.html

Mekk's Twisted Deferred example:

@defer.inlineCallbacks
def someFunction():
a = 1
b = yield deferredReturningFunction(a)
c = yield anotherDeferredReturningFunction(a, b)
defer.returnValue(c)

What's cool about this is that between the two yield statements, the 
Twisted reactor is in control meaning that other pending asynchronous 
tasks can be attended to or the thread's remaining time slice can be 
yielded to the kernel, yet this is all accomplished without the use of 
multi-threading.  Another interesting aspect of this approach is that 
since it leverages on Python's enhanced generators, an exception thrown 
inside either of the deferred-returning functions will be propagated 
through to someFunction() where it can be handled with try/except.


Think about what this means -- this sort of emulates the "stackless" 
design pattern you would expect in Erlang or Stackless Python without 
leaving standard Python.  And it's made possible under the hood by 
Python Enhanced Generators.


Needless to say, it would be great to see this coolness be part of the 
standard Python library, instead of having every Python asynchronous 
library implement its own ad-hoc callback system.


James Yonan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Garbage announcement printed on interpreter shutdown

[Glyph Lefkowitz]

On Sep 10, 2010, at 5:10 PM, Amaury Forgeot d'Arc wrote:

> 2010/9/10 Fred Drake :
>> On Fri, Sep 10, 2010 at 4:32 PM, Georg Brandl  wrote:
>>> IMO this runs contrary to the decision we made when DeprecationWarnings were
>>> made silent by default: it spews messages not only at developers, but also 
>>> at
>>> users, who don't need it and probably are going to be quite confused by it,
>> 
>> Agreed; this should be silent by default.
> 
> +1. I suggest to enable it only when Py_DEBUG (or Py_TRACE_REFS or
> Py_REF_DEBUG?) is defined.

Would it be possible to treat it the same way as a deprecation warning, and 
show it under the same conditions?  It would be nice to know if my Python 
program is leaking uncollectable objects without rebuilding the interpreter.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] terminology for "free variables" in Python

[Nick Coghlan]

On Sat, Sep 11, 2010 at 6:46 AM, Georg Brandl  wrote:
> [me]
>> Although it *is* somewhat handy for quick introspection at the
>> interpreter prompt... maybe I should document it after all. Thoughts?
>
> IMO show_code() is not a good name, because the only thing it doesn't
> do is to -- show the code.
>
> I'd rather call it "codeinfo" (which also is more in line with current
> dis module function names).

And, indeed, the variant I added that just returns the formatted
string instead of printing it directly to stdout is called
dis.code_info.

dis.show_code is the existing helper that Guido added way back in
2007. As the checkin comment from back then put it, it shows you
everything the interpreter knows about the code object except the
details of the bytecode (which is already covered by dis.dis).

So while I agree the name isn't great, I also don't think it is wrong
enough to bother changing.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python needs a standard asynchronous return object

[Guido van Rossum]

Moving to python-ideas.

Have you seen http://www.python.org/dev/peps/pep-3148/ ? That seems
exactly what you want.

--Guido

On Fri, Sep 10, 2010 at 4:00 PM, James Yonan  wrote:
> I'd like to propose that the Python community standardize on a "deferred"
> object for asynchronous return values, modeled after the well-thought-out
> Twisted Deferred class.
>
> With more and more Python libraries implementing asynchronicity (for example
> Futures -- PEP 3148), it's crucial to have a standard deferred object in
> place so that code using a single asynchronous reactor can interoperate with
> different asynchronous libraries.
>
> I think a lot of people don't realize how much cooler and more elegant it is
> to return a deferred object from an asynchronous function rather than using
> a generic callback approach (where you pass a function argument to the
> asynchronous function telling it where to call when the asynchronous
> operation completes).
>
> While asynchronous systems have been shown to have excellent scalability
> properties, the callback-based programming style often used in asynchronous
> programming has been criticized for breaking up the sequential readability
> of program logic.
>
> This problem is elegantly addressed by using Deferred Generators.  Since
> Python 2.5 added enhanced generators (i.e. the capability for "yield" to
> return a value), the infrastructure is now in place to allow an asynchronous
> function to be written in a sequential style, without the use of explicit
> callbacks.
>
> See the following blog article for a nice write-up on the capability:
>
> http://blog.mekk.waw.pl/archives/14-Twisted-inlineCallbacks-and-deferredGenerator.html
>
> Mekk's Twisted Deferred example:
>
> @defer.inlineCallbacks
> def someFunction():
>    a = 1
>    b = yield deferredReturningFunction(a)
>    c = yield anotherDeferredReturningFunction(a, b)
>    defer.returnValue(c)
>
> What's cool about this is that between the two yield statements, the Twisted
> reactor is in control meaning that other pending asynchronous tasks can be
> attended to or the thread's remaining time slice can be yielded to the
> kernel, yet this is all accomplished without the use of multi-threading.
>  Another interesting aspect of this approach is that since it leverages on
> Python's enhanced generators, an exception thrown inside either of the
> deferred-returning functions will be propagated through to someFunction()
> where it can be handled with try/except.
>
> Think about what this means -- this sort of emulates the "stackless" design
> pattern you would expect in Erlang or Stackless Python without leaving
> standard Python.  And it's made possible under the hood by Python Enhanced
> Generators.
>
> Needless to say, it would be great to see this coolness be part of the
> standard Python library, instead of having every Python asynchronous library
> implement its own ad-hoc callback system.
>
> James Yonan
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Garbage announcement printed on interpreter shutdown

[Nick Coghlan]

On Sat, Sep 11, 2010 at 9:42 AM, Glyph Lefkowitz
 wrote:
>
> On Sep 10, 2010, at 5:10 PM, Amaury Forgeot d'Arc wrote:
>
>> 2010/9/10 Fred Drake :
>>> On Fri, Sep 10, 2010 at 4:32 PM, Georg Brandl  wrote:
 IMO this runs contrary to the decision we made when DeprecationWarnings 
 were
 made silent by default: it spews messages not only at developers, but also 
 at
 users, who don't need it and probably are going to be quite confused by it,
>>>
>>> Agreed; this should be silent by default.
>>
>> +1. I suggest to enable it only when Py_DEBUG (or Py_TRACE_REFS or
>> Py_REF_DEBUG?) is defined.
>
> Would it be possible to treat it the same way as a deprecation warning, and 
> show it under the same conditions?  It would be nice to know if my Python 
> program is leaking uncollectable objects without rebuilding the interpreter.

My suggestion:

1. Add a gc.WARN_UNCOLLECTABLE flag on gc.set_debug that enables the
warning message.
2. Have regrtest explicitly set this for our own test suite

As far as automatically turning it on for third party test suites
goes, we could either:
- require them to turn it on explicitly via gc.set_debug
- have gc.WARN_UNCOLLECTABLE default to true for non-optimised runs
(__debug__ == True) and false for runs with -O or -OO (__debug__ ==
False)
- set it by looking at the -W arguments passed in at interpreter
startup (e.g. enable it when all warnings are enabled, leave it
disabled by default otherwise)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r84685 - in python/branches/py3k: Doc/library/dis.rst Doc/reference/simple_stmts.rst Doc/whatsnew/3.2.rst Include/opcode.h Lib/opcode.py Lib/test/test_exceptions.py

[Nick Coghlan]

On Sat, Sep 11, 2010 at 7:39 AM, amaury.forgeotdarc
 wrote:
> There is no need to bump the PYC magic number: the new opcode is used
> for code that did not compile before.

If the magic number doesn't change for 3.2, how will 3.1 know it can't
run pyc and pyo files containing these opcodes?

The magic number needs a bump or this change may cause SystemErrors
when older versions attempt to run affected pyc files.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r84685 - in python/branches/py3k: Doc/library/dis.rst Doc/reference/simple_stmts.rst Doc/whatsnew/3.2.rst Include/opcode.h Lib/opcode.py Lib/test/test_exceptions.py

[Benjamin Peterson]

2010/9/10 Nick Coghlan :
> On Sat, Sep 11, 2010 at 7:39 AM, amaury.forgeotdarc
>  wrote:
>> There is no need to bump the PYC magic number: the new opcode is used
>> for code that did not compile before.
>
> If the magic number doesn't change for 3.2, how will 3.1 know it can't
> run pyc and pyo files containing these opcodes?

The magic number is already bumped since 3.1. However, it's true that
the number should be bumped anyway for good measure.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] r84685 - in python/branches/py3k: Doc/library/dis.rst Doc/reference/simple_stmts.rst Doc/whatsnew/3.2.rst Include/opcode.h Lib/opcode.py Lib/test/test_exceptions.py

[Nick Coghlan]

On Sat, Sep 11, 2010 at 10:33 AM, Benjamin Peterson  wrote:
> 2010/9/10 Nick Coghlan :
>> On Sat, Sep 11, 2010 at 7:39 AM, amaury.forgeotdarc
>>  wrote:
>>> There is no need to bump the PYC magic number: the new opcode is used
>>> for code that did not compile before.
>>
>> If the magic number doesn't change for 3.2, how will 3.1 know it can't
>> run pyc and pyo files containing these opcodes?
>
> The magic number is already bumped since 3.1. However, it's true that
> the number should be bumped anyway for good measure.

Yeah, I saw your subsequent checkin. I've updated the comment just
above MAGIC and TAG to make it clearer when they should be changed.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com