Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
On 12 June 2014 05:34, Florian Bruhin m...@the-compiler.org wrote: Do the lookup in PATH yourself, it's not like that's rocket science. Am I missing something here? I routinely do subprocess.check_call(['hg', 'update']) or whatever, and it finds the hg executable fine. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #21205: add __qualname__ to generators
2014-06-11 18:17 GMT+02:00 Antoine Pitrou anto...@python.org: Le 11/06/2014 10:28, Victor Stinner a écrit : (...) Issues describing the problem, I attached a patch implementing my ideas: http://bugs.python.org/issue21205 Would you be ok with these (minor) incompatible changes? +1 from me. Regards Antoine. I asked myself if this change can cause issues with serialization. The marshal and pickle modules cannot serialize a generator. Marshal only supports a few types. For pickle, I found this explanation: http://peadrop.com/blog/2009/12/29/why-you-cannot-pickle-generators/ So I consider that my change is safe. It changes the representation of a generator, but repr() is usually only checked in unit tests, tests can be fixed. It also changes the value of the __name__ attribute if the name of the function was changed, but I don't think that anyone relies on it. If you really want the original name of the code object, you can still get gen.gi_code.co_name. Another recent change in the Python API was the __wrapped__ attribute set by functools.wraps(). It is now chain wrapper functions, and I'm not aware of anyone complaining of such change. So I'm confident in my change :) Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] close() questions
11.06.14 05:28, Antoine Pitrou написав(ла): close() should indeed be idempotent on all bundled IO class implementations (otherwise it's a bug), and so should it preferably on third-party IO class implementations. There are some questions about close(). 1. If object owns several resources, should close() try to clean up all them if error is happened during cleaning up some resource. E.g. should BufferedRWPair.close() close reader if closing writer failed? 2. If close() raises an exception, should repeated call of close() raise an exception or do nothing? E.g. if GzipFile.close() fails during writing gzip tail (CRC and size), should repeated call of it try to write this tail again? 3. If close() raises an exception, should the closed attribute (if exists) be True or False? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue #21205: add __qualname__ to generators
Hello Victor, On 2014-06-11, 10:28 AM, Victor Stinner wrote: Hi, I'm working on asyncio and it's difficult to debug code because @asyncio.coroutine decorator removes the name of the function if the function is not a generator (if it doesn't use yield from). I propose to add new gi_name and gi_qualname fields to the C structure PyGenObject, add a new __qualname__ (= gi_qualname) attribute to the Python API of generator, and change how the default value of __name__ (= gi_name) of generators. Instead of getting the name from the code object, I propose to get the name from the function (if the generator was created from a function). So if the function name was modified, you get the new name instead of getting the name from the code object (as done in Python 3.4). I also propose to display the qualified name in repr(generator) instead of the name. All these changes should make my life easier to debug asyncio, but it should help any project using generators. Issues describing the problem, I attached a patch implementing my ideas: http://bugs.python.org/issue21205 Would you be ok with these (minor) incompatible changes? I'm +1 for your proposal. This change will indeed make debugging asyncio (and any generator-heavy code) easier. I wouldn't worry too much about compatibility, as the change is fairly minimal, and the feature will only land in 3.5, where people expect new things and are generally OK with slightly updated behaviors. Yury By the way, it looks like generator attributes were never documented :-( My patch also adds a basic documentation (at least, it lists all attributes in the documentation of the inspect module). Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Backwards Incompatibility in logging module in 3.4?
Hi there, I just started testing a project of mine on Python 3.4.0b1. I ran into a change that broke compatibility with the logging module in 3.3. The basic test is: $ py34/bin/python -c 'import logging; print(logging.getLevelName(debug.upper()))' Level DEBUG $ py33/bin/python -c 'import logging; print(logging.getLevelName(debug.upper()))' 10 I quickly stumbled upon this webpage: http://aazza.github.io/2014/05/31/testing-on-multiple-versions-of-Python/ Which led me to this ticket regarding the change: http://bugs.python.org/issue18046 Is this a bug or an intentional break? If it's the latter, shouldn't this at least be mentioned in the What's new in Python 3.4 document? If it's the former, should I file a bug? Thanks, Don ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backwards Incompatibility in logging module in 3.4?
On 13 Jun 2014 08:59, Don Spaulding donspauldin...@gmail.com wrote: Hi there, I just started testing a project of mine on Python 3.4.0b1. I ran into a change that broke compatibility with the logging module in 3.3. The basic test is: $ py34/bin/python -c 'import logging; print(logging.getLevelName(debug.upper()))' Level DEBUG $ py33/bin/python -c 'import logging; print(logging.getLevelName(debug.upper()))' 10 I quickly stumbled upon this webpage: http://aazza.github.io/2014/05/31/testing-on-multiple-versions-of-Python/ Which led me to this ticket regarding the change: http://bugs.python.org/issue18046 Is this a bug or an intentional break? If it's the latter, shouldn't this at least be mentioned in the What's new in Python 3.4 document? If it's the former, should I file a bug? Yes, it sounds like a bug to me - there's no indication of an intent to change behaviour with that cleanup patch. Cheers, Nick. Thanks, Don ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backwards Incompatibility in logging module in 3.4?
Hi, 2014-06-13 0:38 GMT+02:00 Don Spaulding donspauldin...@gmail.com: Is this a bug or an intentional break? If it's the latter, shouldn't this at least be mentioned in the What's new in Python 3.4 document? IMO the change is intentional. The previous behaviour was not really expected. Python 3.3 documentation is explicit: the result is a string and the input paramter is an integer. logging.getLevelName(DEBUG) was more an implementation https://docs.python.org/3.3/library/logging.html#logging.getLevelName Returns the textual representation of logging level lvl. If the level is one of the predefined levels CRITICAL, ERROR, WARNING, INFO or DEBUG then you get the corresponding string. If you have associated levels with names using addLevelName() then the name you have associated with lvl is returned. If a numeric value corresponding to one of the defined levels is passed in, the corresponding string representation is returned. Otherwise, the string ‘Level %s’ % lvl is returned. If your code uses something like logger.setLevel(logging.getLevelName(DEBUG)), use directly logger.setLevel(DEBUG). This issue was fixed in OpenStack with this change: https://review.openstack.org/#/c/94028/6/openstack/common/log.py,cm https://review.openstack.org/#/c/94028/6 Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
SHELLS ARE NOT CROSS-PLATFORM Seriously, there are going to be
differences. If you really must:
escape = lambda s: s.replace('^', '^^') if os.name == 'nt' else s
Viola.
On Wed, Jun 11, 2014 at 5:53 PM, anatoly techtonik techto...@gmail.com
wrote:
On Thu, Jun 12, 2014 at 1:30 AM, Chris Angelico ros...@gmail.com wrote:
On Thu, Jun 12, 2014 at 7:58 AM, Ryan rym...@gmail.com wrote:
In all seriousness, to me this is obvious. When you pass a command to
the
shell, naturally, certain details are shell-specific.
On Windows cmd.exe is used by default:
http://hg.python.org/cpython/file/38a325c84564/Lib/subprocess.py#l1108
so it makes sense to make default behavior cross-platform.
-1. Bad idea. Very bad idea. If you want the ^ to be escaped, do it
yourself. Or better yet, don't pass shell=True.
Definitely the latter. Why pass shell=True when executing a single
command? I don't get it.
This is a complete use case using Rietveld upload script:
http://techtonik.rainforce.org/2013/07/code-review-with-rietveld-and-mercurial.html
I am interested to know how to modify upload script without kludges:
https://code.google.com/p/rietveld/source/browse/upload.py#1056
I expect many people are facing with the same problem trying to wrap
Git and HG with Python scripts.
--
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
--
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does IOBase.__del__ call .close?
Benjamin Peterson benja...@python.org writes:
On Wed, Jun 11, 2014, at 17:11, Nikolaus Rath wrote:
MRAB pyt...@mrabarnett.plus.com writes:
On 2014-06-11 02:30, Nikolaus Rath wrote:
Hello,
I recently noticed (after some rather protacted debugging) that the
io.IOBase class comes with a destructor that calls self.close():
[0] nikratio@vostro:~/tmp$ cat test.py
import io
class Foo(io.IOBase):
def close(self):
print('close called')
r = Foo()
del r
[0] nikratio@vostro:~/tmp$ python3 test.py
close called
To me, this came as quite a surprise, and the best documentation of
this feature seems to be the following note (from the io library
reference):
The abstract base classes also provide default implementations of some
methods in order to help implementation of concrete stream classes. For
example, BufferedIOBase provides unoptimized implementations of
readinto() and readline().
For me, having __del__ call close() does not qualify as a reasonable
default implementation unless close() is required to be idempotent
(which one could deduce from the documentation if one tries to, but it's
far from clear).
Is this behavior an accident, or was that a deliberate decision?
To me, it makes sense. You want to make sure that it's closed, releasing
any resources it might be holding, even if you haven't done so
explicitly.
I agree with your intentions, but I come to the opposite conclusion:
automatically calling close() in the destructor will hide that there's a
problem in the code. Without that automatic cleanup, there's at least a
good chance that a ResourceWarning will be emitted so the problem gets
noticed. Silently work around bugs in caller's code doesn't seem like
a very useful default to me...
Things which actually hold system resources (like FileIO) give
ResourceWarning if they close in __del__, so I don't understand your
point.
Consider this simple example:
$ cat test.py
import io
import warnings
class StridedStream(io.IOBase):
def __init__(self, name, stride=2):
super().__init__()
self.fh = open(name, 'rb')
self.stride = stride
def read(self, len_):
return self.fh.read(self.stride*len_)[::self.stride]
def close(self):
self.fh.close()
class FixedStridedStream(StridedStream):
def __del__(self):
# Prevent IOBase.__del__ frombeing called.
pass
warnings.resetwarnings()
warnings.simplefilter('error')
print('Creating loosing StridedStream..')
r = StridedStream('/dev/zero')
del r
print('Creating loosing FixedStridedStream..')
r = FixedStridedStream('/dev/zero')
del r
$ python3 test.py
Creating loosing StridedStream..
Creating loosing FixedStridedStream..
Exception ignored in: _io.FileIO name='/dev/zero' mode='rb'
ResourceWarning: unclosed file _io.BufferedReader name='/dev/zero'
In the first case, the destructor inherited from IOBase actually
prevents the ResourceWarning from being emitted.
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
R. David Murray rdmur...@bitdance.com writes:
Also notice that using a list with shell=True is using the API
incorrectly. It wouldn't even work on Linux, so that torpedoes
the cross-platform concern already :)
This kind of confusion is why I opened http://bugs.python.org/issue7839.
Can someone describe an use case where shell=True actually makes sense
at all?
It seems to me that whenever you need a shell, the argument's that you
pass to it will be shell specific. So instead of e.g.
Popen('for i in `seq 42`; do echo $i; done', shell=True)
you almost certainly want to do
Popen(['/bin/sh', 'for i in `seq 42`; do echo $i; done'], shell=False)
because if your shell happens to be tcsh or cmd.exe, things are going to
break.
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
On Fri, Jun 13, 2014 at 12:11 PM, Nikolaus Rath nikol...@rath.org wrote:
Can someone describe an use case where shell=True actually makes sense
at all?
It seems to me that whenever you need a shell, the argument's that you
pass to it will be shell specific. So instead of e.g.
Popen('for i in `seq 42`; do echo $i; done', shell=True)
you almost certainly want to do
Popen(['/bin/sh', 'for i in `seq 42`; do echo $i; done'], shell=False)
because if your shell happens to be tcsh or cmd.exe, things are going to
break.
Some features, while technically shell-specific, are supported across
a lot of shells. You should be able to pipe output from one command
into another in most shells, for instance.
But yes, I generally don't use it.
ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
On 13 Jun 2014 12:12, Nikolaus Rath nikol...@rath.org wrote: R. David Murray rdmur...@bitdance.com writes: Also notice that using a list with shell=True is using the API incorrectly. It wouldn't even work on Linux, so that torpedoes the cross-platform concern already :) This kind of confusion is why I opened http://bugs.python.org/issue7839. Can someone describe an use case where shell=True actually makes sense at all? When you're writing platform specific code, it's occasionally useful. It's generally best avoided, though. Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
* Nikolaus Rath nikol...@rath.org [2014-06-12 19:11:07 -0700]:
R. David Murray rdmur...@bitdance.com writes:
Also notice that using a list with shell=True is using the API
incorrectly. It wouldn't even work on Linux, so that torpedoes
the cross-platform concern already :)
This kind of confusion is why I opened http://bugs.python.org/issue7839.
Can someone describe an use case where shell=True actually makes sense
at all?
It seems to me that whenever you need a shell, the argument's that you
pass to it will be shell specific. So instead of e.g.
Popen('for i in `seq 42`; do echo $i; done', shell=True)
you almost certainly want to do
Popen(['/bin/sh', 'for i in `seq 42`; do echo $i; done'], shell=False)
because if your shell happens to be tcsh or cmd.exe, things are going to
break.
My usecase is a spawn-command in a GUI application, which the user can
use to spawn an executable. I want the user to be able to use the
usual shell features from there. However, I also pass an argument to
that command, and that should be escaped.
Florian
--
http://www.the-compiler.org | m...@the-compiler.org (Mail/XMPP)
GPG 0xFD55A072 | http://the-compiler.org/pubkey.asc
I love long mails! | http://email.is-not-s.ms/
pgpsnlpNbDtTn.pgp
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess shell=True on Windows doesn't escape ^ character
Nikolaus Rath wrote: you almost certainly want to do Popen(['/bin/sh', 'for i in `seq 42`; do echo $i; done'], shell=False) because if your shell happens to be tcsh or cmd.exe, things are going to break. On Unix, the C library's system() and popen() functions always use /bin/sh, NOT the user's current login shell, for this very reason. I would hope that the Python versions of these, and also the new subprocess stuff, do the same. That still leaves differences between Unix and Windows, but explicitly naming the shell won't help with that. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does IOBase.__del__ call .close?
On Thu, Jun 12, 2014, at 18:06, Nikolaus Rath wrote:
Consider this simple example:
$ cat test.py
import io
import warnings
class StridedStream(io.IOBase):
def __init__(self, name, stride=2):
super().__init__()
self.fh = open(name, 'rb')
self.stride = stride
def read(self, len_):
return self.fh.read(self.stride*len_)[::self.stride]
def close(self):
self.fh.close()
class FixedStridedStream(StridedStream):
def __del__(self):
# Prevent IOBase.__del__ frombeing called.
pass
warnings.resetwarnings()
warnings.simplefilter('error')
print('Creating loosing StridedStream..')
r = StridedStream('/dev/zero')
del r
print('Creating loosing FixedStridedStream..')
r = FixedStridedStream('/dev/zero')
del r
$ python3 test.py
Creating loosing StridedStream..
Creating loosing FixedStridedStream..
Exception ignored in: _io.FileIO name='/dev/zero' mode='rb'
ResourceWarning: unclosed file _io.BufferedReader name='/dev/zero'
In the first case, the destructor inherited from IOBase actually
prevents the ResourceWarning from being emitted.
Ah, I see. I don't see any good ways to fix it, though, besides setting
some flag if close() is called from __del__.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
