Re: [Python-Dev] Issue #21205: add __qualname__ to generators
2014-06-11 18:17 GMT+02:00 Antoine Pitrou : > Le 11/06/2014 10:28, Victor Stinner a écrit : >> (...) >> Issues describing the problem, I attached a patch implementing my ideas: >> http://bugs.python.org/issue21205 >> >> Would you be ok with these (minor) incompatible changes? > > +1 from me. > > Regards > Antoine. I asked myself if this change can cause issues with serialization. The marshal and pickle modules cannot serialize a generator. Marshal only supports a few types. For pickle, I found this explanation: http://peadrop.com/blog/2009/12/29/why-you-cannot-pickle-generators/ So I consider that my change is safe. It changes the representation of a generator, but repr() is usually only checked in unit tests, tests can be fixed. It also changes the value of the __name__ attribute if the name of the function was changed, but I don't think that anyone relies on it. If you really want the original name of the code object, you can still get gen.gi_code.co_name. Another recent change in the Python API was the __wrapped__ attribute set by functools.wraps(). It is now chain wrapper functions, and I'm not aware of anyone complaining of such change. So I'm confident in my change :) Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] close() questions
11.06.14 05:28, Antoine Pitrou написав(ла): close() should indeed be idempotent on all bundled IO class implementations (otherwise it's a bug), and so should it preferably on third-party IO class implementations. There are some questions about close(). 1. If object owns several resources, should close() try to clean up all them if error is happened during cleaning up some resource. E.g. should BufferedRWPair.close() close reader if closing writer failed? 2. If close() raises an exception, should repeated call of close() raise an exception or do nothing? E.g. if GzipFile.close() fails during writing gzip tail (CRC and size), should repeated call of it try to write this tail again? 3. If close() raises an exception, should the closed attribute (if exists) be True or False? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #21205: add __qualname__ to generators
Hello Victor, On 2014-06-11, 10:28 AM, Victor Stinner wrote: Hi, I'm working on asyncio and it's difficult to debug code because @asyncio.coroutine decorator removes the name of the function if the function is not a generator (if it doesn't use yield from). I propose to add new gi_name and gi_qualname fields to the C structure PyGenObject, add a new __qualname__ (= gi_qualname) attribute to the Python API of generator, and change how the default value of __name__ (= gi_name) of generators. Instead of getting the name from the code object, I propose to get the name from the function (if the generator was created from a function). So if the function name was modified, you get the new name instead of getting the name from the code object (as done in Python 3.4). I also propose to display the qualified name in repr(generator) instead of the name. All these changes should make my life easier to debug asyncio, but it should help any project using generators. Issues describing the problem, I attached a patch implementing my ideas: http://bugs.python.org/issue21205 Would you be ok with these (minor) incompatible changes? I'm +1 for your proposal. This change will indeed make debugging asyncio (and any generator-heavy code) easier. I wouldn't worry too much about compatibility, as the change is fairly minimal, and the feature will only land in 3.5, where people expect new things and are generally OK with slightly updated behaviors. Yury By the way, it looks like generator attributes were never documented :-( My patch also adds a basic documentation (at least, it lists all attributes in the documentation of the inspect module). Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Backwards Incompatibility in logging module in 3.4?
Hi there,
I just started testing a project of mine on Python 3.4.0b1. I ran into a
change that broke compatibility with the logging module in 3.3.
The basic test is:
$ py34/bin/python -c 'import logging;
print(logging.getLevelName("debug".upper()))'
Level DEBUG
$ py33/bin/python -c 'import logging;
print(logging.getLevelName("debug".upper()))'
10
I quickly stumbled upon this webpage:
http://aazza.github.io/2014/05/31/testing-on-multiple-versions-of-Python/
Which led me to this ticket regarding the change:
http://bugs.python.org/issue18046
Is this a bug or an intentional break? If it's the latter, shouldn't this
at least be mentioned in the "What's new in Python 3.4" document? If it's
the former, should I file a bug?
Thanks,
Don
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backwards Incompatibility in logging module in 3.4?
On 13 Jun 2014 08:59, "Don Spaulding" wrote:
>
> Hi there,
>
> I just started testing a project of mine on Python 3.4.0b1. I ran into a
change that broke compatibility with the logging module in 3.3.
>
> The basic test is:
>
> $ py34/bin/python -c 'import logging;
print(logging.getLevelName("debug".upper()))'
> Level DEBUG
>
> $ py33/bin/python -c 'import logging;
print(logging.getLevelName("debug".upper()))'
> 10
>
> I quickly stumbled upon this webpage:
>
> http://aazza.github.io/2014/05/31/testing-on-multiple-versions-of-Python/
>
> Which led me to this ticket regarding the change:
>
> http://bugs.python.org/issue18046
>
> Is this a bug or an intentional break? If it's the latter, shouldn't
this at least be mentioned in the "What's new in Python 3.4" document? If
it's the former, should I file a bug?
Yes, it sounds like a bug to me - there's no indication of an intent to
change behaviour with that cleanup patch.
Cheers,
Nick.
>
> Thanks,
> Don
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backwards Incompatibility in logging module in 3.4?
Hi,
2014-06-13 0:38 GMT+02:00 Don Spaulding :
> Is this a bug or an intentional break? If it's the latter, shouldn't this
> at least be mentioned in the "What's new in Python 3.4" document?
IMO the change is intentional. The previous behaviour was not really expected.
Python 3.3 documentation is explicit: the result is a string and the
input paramter is an integer. logging.getLevelName("DEBUG") was more
an implementation
https://docs.python.org/3.3/library/logging.html#logging.getLevelName
"Returns the textual representation of logging level lvl. If the level
is one of the predefined levels CRITICAL, ERROR, WARNING, INFO or
DEBUG then you get the corresponding string. If you have associated
levels with names using addLevelName() then the name you have
associated with lvl is returned. If a numeric value corresponding to
one of the defined levels is passed in, the corresponding string
representation is returned. Otherwise, the string ‘Level %s’ % lvl is
returned."
If your code uses something like
logger.setLevel(logging.getLevelName("DEBUG")), use directly
logger.setLevel("DEBUG").
This issue was fixed in OpenStack with this change:
https://review.openstack.org/#/c/94028/6/openstack/common/log.py,cm
https://review.openstack.org/#/c/94028/6
Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
SHELLS ARE NOT CROSS-PLATFORM Seriously, there are going to be
differences. If you really must:
escape = lambda s: s.replace('^', '^^') if os.name == 'nt' else s
Viola.
On Wed, Jun 11, 2014 at 5:53 PM, anatoly techtonik
wrote:
> On Thu, Jun 12, 2014 at 1:30 AM, Chris Angelico wrote:
>
>> On Thu, Jun 12, 2014 at 7:58 AM, Ryan wrote:
>> > In all seriousness, to me this is obvious. When you pass a command to
>> the
>> > shell, naturally, certain details are shell-specific.
>>
>
> On Windows cmd.exe is used by default:
> http://hg.python.org/cpython/file/38a325c84564/Lib/subprocess.py#l1108
> so it makes sense to make default behavior cross-platform.
>
>
>> > -1. Bad idea. Very bad idea. If you want the ^ to be escaped, do it
>> > yourself. Or better yet, don't pass shell=True.
>>
>> Definitely the latter. Why pass shell=True when executing a single
>> command? I don't get it.
>>
>
> This is a complete use case using Rietveld upload script:
>
> http://techtonik.rainforce.org/2013/07/code-review-with-rietveld-and-mercurial.html
>
> I am interested to know how to modify upload script without kludges:
> https://code.google.com/p/rietveld/source/browse/upload.py#1056
> I expect many people are facing with the same problem trying to wrap
> Git and HG with Python scripts.
> --
> anatoly t.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>
>
--
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does IOBase.__del__ call .close?
Benjamin Peterson writes:
> On Wed, Jun 11, 2014, at 17:11, Nikolaus Rath wrote:
>> MRAB writes:
>> > On 2014-06-11 02:30, Nikolaus Rath wrote:
>> >> Hello,
>> >>
>> >> I recently noticed (after some rather protacted debugging) that the
>> >> io.IOBase class comes with a destructor that calls self.close():
>> >>
>> >> [0] nikratio@vostro:~/tmp$ cat test.py
>> >> import io
>> >> class Foo(io.IOBase):
>> >> def close(self):
>> >> print('close called')
>> >> r = Foo()
>> >> del r
>> >> [0] nikratio@vostro:~/tmp$ python3 test.py
>> >> close called
>> >>
>> >> To me, this came as quite a surprise, and the best "documentation" of
>> >> this feature seems to be the following note (from the io library
>> >> reference):
>> >>
>> >> "The abstract base classes also provide default implementations of some
>> >> methods in order to help implementation of concrete stream classes. For
>> >> example, BufferedIOBase provides unoptimized implementations of
>> >> readinto() and readline()."
>> >>
>> >> For me, having __del__ call close() does not qualify as a reasonable
>> >> default implementation unless close() is required to be idempotent
>> >> (which one could deduce from the documentation if one tries to, but it's
>> >> far from clear).
>> >>
>> >> Is this behavior an accident, or was that a deliberate decision?
>> >>
>> > To me, it makes sense. You want to make sure that it's closed, releasing
>> > any resources it might be holding, even if you haven't done so
>> > explicitly.
>>
>> I agree with your intentions, but I come to the opposite conclusion:
>> automatically calling close() in the destructor will hide that there's a
>> problem in the code. Without that automatic cleanup, there's at least a
>> good chance that a ResourceWarning will be emitted so the problem gets
>> noticed. "Silently work around bugs in caller's code" doesn't seem like
>> a very useful default to me...
>
> Things which actually hold system resources (like FileIO) give
> ResourceWarning if they close in __del__, so I don't understand your
> point.
Consider this simple example:
$ cat test.py
import io
import warnings
class StridedStream(io.IOBase):
def __init__(self, name, stride=2):
super().__init__()
self.fh = open(name, 'rb')
self.stride = stride
def read(self, len_):
return self.fh.read(self.stride*len_)[::self.stride]
def close(self):
self.fh.close()
class FixedStridedStream(StridedStream):
def __del__(self):
# Prevent IOBase.__del__ frombeing called.
pass
warnings.resetwarnings()
warnings.simplefilter('error')
print('Creating & loosing StridedStream..')
r = StridedStream('/dev/zero')
del r
print('Creating & loosing FixedStridedStream..')
r = FixedStridedStream('/dev/zero')
del r
$ python3 test.py
Creating & loosing StridedStream..
Creating & loosing FixedStridedStream..
Exception ignored in: <_io.FileIO name='/dev/zero' mode='rb'>
ResourceWarning: unclosed file <_io.BufferedReader name='/dev/zero'>
In the first case, the destructor inherited from IOBase actually
prevents the ResourceWarning from being emitted.
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
"R. David Murray" writes:
> Also notice that using a list with shell=True is using the API
> incorrectly. It wouldn't even work on Linux, so that torpedoes
> the cross-platform concern already :)
>
> This kind of confusion is why I opened http://bugs.python.org/issue7839.
Can someone describe an use case where shell=True actually makes sense
at all?
It seems to me that whenever you need a shell, the argument's that you
pass to it will be shell specific. So instead of e.g.
Popen('for i in `seq 42`; do echo $i; done', shell=True)
you almost certainly want to do
Popen(['/bin/sh', 'for i in `seq 42`; do echo $i; done'], shell=False)
because if your shell happens to be tcsh or cmd.exe, things are going to
break.
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
On Fri, Jun 13, 2014 at 12:11 PM, Nikolaus Rath wrote:
> Can someone describe an use case where shell=True actually makes sense
> at all?
>
> It seems to me that whenever you need a shell, the argument's that you
> pass to it will be shell specific. So instead of e.g.
>
> Popen('for i in `seq 42`; do echo $i; done', shell=True)
>
> you almost certainly want to do
>
> Popen(['/bin/sh', 'for i in `seq 42`; do echo $i; done'], shell=False)
>
> because if your shell happens to be tcsh or cmd.exe, things are going to
> break.
Some features, while technically shell-specific, are supported across
a lot of shells. You should be able to pipe output from one command
into another in most shells, for instance.
But yes, I generally don't use it.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
On 13 Jun 2014 12:12, "Nikolaus Rath" wrote: > > "R. David Murray" writes: > > Also notice that using a list with shell=True is using the API > > incorrectly. It wouldn't even work on Linux, so that torpedoes > > the cross-platform concern already :) > > > > This kind of confusion is why I opened http://bugs.python.org/issue7839. > > Can someone describe an use case where shell=True actually makes sense > at all? When you're writing platform specific code, it's occasionally useful. It's generally best avoided, though. Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
* Nikolaus Rath [2014-06-12 19:11:07 -0700]:
> "R. David Murray" writes:
> > Also notice that using a list with shell=True is using the API
> > incorrectly. It wouldn't even work on Linux, so that torpedoes
> > the cross-platform concern already :)
> >
> > This kind of confusion is why I opened http://bugs.python.org/issue7839.
>
> Can someone describe an use case where shell=True actually makes sense
> at all?
>
> It seems to me that whenever you need a shell, the argument's that you
> pass to it will be shell specific. So instead of e.g.
>
> Popen('for i in `seq 42`; do echo $i; done', shell=True)
>
> you almost certainly want to do
>
> Popen(['/bin/sh', 'for i in `seq 42`; do echo $i; done'], shell=False)
>
> because if your shell happens to be tcsh or cmd.exe, things are going to
> break.
My usecase is a spawn-command in a GUI application, which the user can
use to spawn an executable. I want the user to be able to use the
usual shell features from there. However, I also pass an argument to
that command, and that should be escaped.
Florian
--
http://www.the-compiler.org | m...@the-compiler.org (Mail/XMPP)
GPG 0xFD55A072 | http://the-compiler.org/pubkey.asc
I love long mails! | http://email.is-not-s.ms/
pgpsnlpNbDtTn.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
Nikolaus Rath wrote: you almost certainly want to do Popen(['/bin/sh', 'for i in `seq 42`; do echo $i; done'], shell=False) because if your shell happens to be tcsh or cmd.exe, things are going to break. On Unix, the C library's system() and popen() functions always use /bin/sh, NOT the user's current login shell, for this very reason. I would hope that the Python versions of these, and also the new subprocess stuff, do the same. That still leaves differences between Unix and Windows, but explicitly naming the shell won't help with that. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does IOBase.__del__ call .close?
On Thu, Jun 12, 2014, at 18:06, Nikolaus Rath wrote:
> Consider this simple example:
>
> $ cat test.py
> import io
> import warnings
>
> class StridedStream(io.IOBase):
> def __init__(self, name, stride=2):
> super().__init__()
> self.fh = open(name, 'rb')
> self.stride = stride
>
> def read(self, len_):
> return self.fh.read(self.stride*len_)[::self.stride]
>
> def close(self):
> self.fh.close()
>
> class FixedStridedStream(StridedStream):
> def __del__(self):
> # Prevent IOBase.__del__ frombeing called.
> pass
>
> warnings.resetwarnings()
> warnings.simplefilter('error')
>
> print('Creating & loosing StridedStream..')
> r = StridedStream('/dev/zero')
> del r
>
> print('Creating & loosing FixedStridedStream..')
> r = FixedStridedStream('/dev/zero')
> del r
>
> $ python3 test.py
> Creating & loosing StridedStream..
> Creating & loosing FixedStridedStream..
> Exception ignored in: <_io.FileIO name='/dev/zero' mode='rb'>
> ResourceWarning: unclosed file <_io.BufferedReader name='/dev/zero'>
>
> In the first case, the destructor inherited from IOBase actually
> prevents the ResourceWarning from being emitted.
Ah, I see. I don't see any good ways to fix it, though, besides setting
some flag if close() is called from __del__.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
