Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
2014-02-24 3:45 GMT+01:00 Nick Coghlan ncogh...@gmail.com:
Would leaving %a out destroy the utility of the PEP?
Usually, debug code is not even commited. So writing b'var=%s' %
ascii(var).encode() is not hard.
Or maybe: b'var=%s' % repr(var).encode('ascii', 'backslashreplace')
which is the same but longer :-)
Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 02/23/2014 02:54 PM, Nick Coghlan wrote:
It's a harm containment tactic, based on the assumption people *will*
want to include the output of ascii() in binary protocols containing
ASCII segments, regardless of whether or not we consider their reasons
for doing so to be particularly good.
One possible problem with %a -- it becomes the bytes equivalent of %s in Python 2 strings, with the minor exception of
how unicode strings are handled (quote marks are added). In other words, instead of %d, one could use %a.
On the other hand, %a is so much more user-friendly than b'%s' % ('%d' %
123).encode('ascii', errors='backslashreplace').
--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On Mon, 24 Feb 2014 09:15:29 -0800
Ethan Furman et...@stoneleaf.us wrote:
On 02/23/2014 02:54 PM, Nick Coghlan wrote:
It's a harm containment tactic, based on the assumption people *will*
want to include the output of ascii() in binary protocols containing
ASCII segments, regardless of whether or not we consider their reasons
for doing so to be particularly good.
One possible problem with %a -- it becomes the bytes equivalent of %s in
Python 2 strings, with the minor exception of
how unicode strings are handled (quote marks are added). In other words,
instead of %d, one could use %a.
On the other hand, %a is so much more user-friendly than b'%s' % ('%d' %
123).encode('ascii', errors='backslashreplace').
But why not b'%d' % 123 ?
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On 22/02/2014 21:26, Tim Delaney wrote: On 23 February 2014 02:29, Nick Coghlan ncogh...@gmail.com mailto:ncogh...@gmail.com wrote: On 22 Feb 2014 22:15, Stephen J. Turnbull step...@xemacs.org mailto:step...@xemacs.org wrote: Antoine Pitrou writes: Chris Angelico ros...@gmail.com mailto:ros...@gmail.com wrote: hasattr(x,y) - (x.y or True except AttributeError: False) But it's not the same. hasattr() returns a boolean, not an arbitrary value. I think he meant hasattr(x,y) - (x.y and True except AttributeError: False) With PEP 463, the explicit equivalent of hasattr() would be something like : hasattr(x,y) - (bool(x.y) or True except AttributeError: False) That would work, but I think I'd prefer: hasattr(x,y) - bool(x.y or True except AttributeError: False) Makes it clearer IMO that the entire expression will always return a boolean. This also works: hasattr(x,y) - (lambda v: True)(x.y) except AttributeError: False Which is less obscure is a matter of judgement. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 02/24/2014 09:43 AM, Antoine Pitrou wrote:
On Mon, 24 Feb 2014 09:15:29 -0800
Ethan Furman et...@stoneleaf.us wrote:
On 02/23/2014 02:54 PM, Nick Coghlan wrote:
It's a harm containment tactic, based on the assumption people *will*
want to include the output of ascii() in binary protocols containing
ASCII segments, regardless of whether or not we consider their reasons
for doing so to be particularly good.
One possible problem with %a -- it becomes the bytes equivalent of %s in Python
2 strings, with the minor exception of
how unicode strings are handled (quote marks are added). In other words,
instead of %d, one could use %a.
On the other hand, %a is so much more user-friendly than b'%s' % ('%d' %
123).encode('ascii', errors='backslashreplace').
But why not b'%d' % 123 ?
I was just using 123 as an example of the user-unfriendliness of the rest of
that line.
--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On Mon, 24 Feb 2014 09:58:30 -0800
Ethan Furman et...@stoneleaf.us wrote:
On 02/24/2014 09:43 AM, Antoine Pitrou wrote:
On Mon, 24 Feb 2014 09:15:29 -0800
Ethan Furman et...@stoneleaf.us wrote:
On 02/23/2014 02:54 PM, Nick Coghlan wrote:
It's a harm containment tactic, based on the assumption people *will*
want to include the output of ascii() in binary protocols containing
ASCII segments, regardless of whether or not we consider their reasons
for doing so to be particularly good.
One possible problem with %a -- it becomes the bytes equivalent of %s in
Python 2 strings, with the minor exception of
how unicode strings are handled (quote marks are added). In other words,
instead of %d, one could use %a.
On the other hand, %a is so much more user-friendly than b'%s' % ('%d' %
123).encode('ascii', errors='backslashreplace').
But why not b'%d' % 123 ?
I was just using 123 as an example of the user-unfriendliness of the rest of
that line.
The thing is, we don't have any believable example of a data type for
which '%a' would be useful. IME, most formatting happens with basic
data types such as str, int, etc., and '%a' can't be useful for those.
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
Some of your points have been answered by others, I'll try to avoid
repetition.
On 21/02/2014 19:04, Yury Selivanov wrote:
[snip]
Inconvenience of dict[] raising KeyError was solved by
introducing the dict.get() method. And I think that
dct.get('a', 'b')
is 1000 times better than
dct['a'] except KeyError: 'b'
Do you? I don't. Explicit is better than implicit. I think this may
be partly a matter of familiarity.
Translate numbers to names, falling back on the numbers::
g = grp.getgrnam(tarinfo.gname)[2] except KeyError:
tarinfo.gid
u = pwd.getpwnam(tarinfo.uname)[2] except KeyError:
tarinfo.uid
# Lib/tarfile.py:2198:
try:
g = grp.getgrnam(tarinfo.gname)[2]
except KeyError:
g = tarinfo.gid
try:
u = pwd.getpwnam(tarinfo.uname)[2]
except KeyError:
u = tarinfo.uid
This one is a valid example, but totally unparseable by
humans.
Again, I find the more concise version easier to read than the
original. It is dense, yes. It takes time to read and absorb, sure -
but so does the original 8-line version.
And it makes the repetition of the same code structure much more obvious.
I think all of the above more readable with try statement.
Retrieving a message from either a cache or the internet, with auth
check::
logging.info(Message shown to user: %s,((cache[k]
except LookupError:
(backend.read(k) except OSError: 'Resource not available')
)
if check_permission(k) else 'Access denied'
) except BaseException: This is like a bare except clause)
try:
if check_permission(k):
try:
_ = cache[k]
except LookupError:
try:
_ = backend.read(k)
except OSError:
_ = 'Resource not available'
else:
_ = 'Access denied'
except BaseException:
_ = This is like a bare except clause
logging.info(Message shown to user: %s, _)
I think there is a consensus that this was a poor example, unless
intended to show how the new construction (like any other) could be
abused to produce hard-to-understand code.
If you replace '_' with a 'msg' (why did you use '_'??)
then try statements are *much* more readable.
[snip]
Lib/ipaddress.py:343::
try:
ips.append(ip.ip)
except AttributeError:
ips.append(ip.network_address)
Becomes::
ips.append(ip.ip except AttributeError: ip.network_address)
or it may become:
ips.append(getattr(ip, 'ip', ip.network_address))
or
address = getattr(ip, 'ip', ip.network_address)
ips.append(address)
Please note that any of these is an improvement on the original, in that
they don't trap an AttributeError evaluating ips.append. (The old try
... except syntax can be used incorrectly, just as any new syntax can.)
---
All in all, your proposal scares me. I doesn't make python
code readable,
Again, a personal judgement.
it doesn't solve the problem of overbroad
exceptions handling (you have couple examples of overbroad
handling in your PEP examples section).
Yes, some examples look neat. But your syntax is much easier
to abuse, than 'if..else' expression,
Why? Any syntax can be abused. I can easily abuse if..else if I want to:
ErrorMessage = (None if eggs else No eggs) if ham else No ham if
eggs else No ham or eggs # Tested
Suppose you were learning Python.
Which is easier, to learn lots of special methods (dict.get, getattr
etc.), or to learn ONE self-explanatory form of syntax that covers them all:
Dict[Key] except KeyError: default
List[Index] except IndexError: default
x.y except AttributeError: default
(Of course, I'm not saying Don't use getattr. Just that you could get
by if you've never heard of it.)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
Okay, types corrected, most comments taken into account. %b is right out, %a is still suffering scrutiny. The arguments seem to boil down to: We don't need it. vs Somebody might, and it's better than having them inappropriately add a __bytes__ method if we don't have it. We don't need it doesn't really need any further explanation. Does anybody have any examples where %a could be useful? Considering the work-arounds are either wrong or painful, it wouldn't take much to sway me towards keeping it, but at the moment it seems to be a YAGNI, plus we could always add it later if it turns out to be useful. (For that matter, we could implement the main portion of the PEP now, and maybe a %a use-case will show up before 3.5 is released and we could add it then.) So, any last thoughts about %a? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 2/24/2014 10:40 AM, Ethan Furman wrote: Somebody might, and it's better than having them inappropriately add a __bytes__ method if we don't have it. I'll admit my first thought on reading the initial discussions about adding bytes % formatting was Oh, if I want to display custom objects in a byte stream, just add a __bytes__ method. I don't believe there is any verbiage in the PEP (that might get transferred to the documentation) that explains why that would be a bad idea. Should there be? Whether or not %a is implemented sooner or later? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On Mon, 24 Feb 2014 10:40:46 -0800 Ethan Furman et...@stoneleaf.us wrote: Okay, types corrected, most comments taken into account. %b is right out, %a is still suffering scrutiny. The arguments seem to boil down to: We don't need it. vs Somebody might, and it's better than having them inappropriately add a __bytes__ method if we don't have it. Don't forget that Python is a language for consenting adults. Adding a near-useless feature for fear that otherwise people might shoot themselves in the foot by using another feature must be one of the worst arguments I've ever heard :-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 24/02/2014 18:40, Ethan Furman wrote: Okay, types corrected, most comments taken into account. %b is right out, %a is still suffering scrutiny. The arguments seem to boil down to: We don't need it. vs Somebody might, and it's better than having them inappropriately add a __bytes__ method if we don't have it. We don't need it doesn't really need any further explanation. Does anybody have any examples where %a could be useful? Considering the work-arounds are either wrong or painful, it wouldn't take much to sway me towards keeping it, but at the moment it seems to be a YAGNI, plus we could always add it later if it turns out to be useful. (For that matter, we could implement the main portion of the PEP now, and maybe a %a use-case will show up before 3.5 is released and we could add it then.) So, any last thoughts about %a? I placed it under your nose https://mail.python.org/pipermail/python-dev/2014-January/131636.html but personally I wouldn't lose any sleep whether it stays or goes. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 02/24/2014 11:54 AM, Mark Lawrence wrote: On 24/02/2014 18:40, Ethan Furman wrote: So, any last thoughts about %a? I placed it under your nose https://mail.python.org/pipermail/python-dev/2014-January/131636.html but personally I wouldn't lose any sleep whether it stays or goes. So you did, sorry I forgot about it. So the argument, then, is that %a should be included because it is present in str? Note that %r, while it works for str, is rejected from this proposal (primarily because of the possibility of having non-ASCII characters); while %a doesn't suffer from that possibility (obviously ;) , I think the case needs to be made that %a is useful for including ... in a mixed binary/ASCII format, but so far nobody has filled in the ... . -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On 21/02/2014 23:36, Ethan Furman wrote: On 02/21/2014 02:26 PM, Eric V. Smith wrote: On 2/21/2014 5:06 PM, Greg Ewing wrote: On 21 February 2014 13:15, Chris Angelico wrote: Generator expressions require parentheses, unless they would be strictly redundant. Ambiguities with except expressions could be resolved in the same way, forcing nested except-in-except trees to be correctly parenthesized There would be no ambiguity if only nested excepts are allowed. If one wants to catch multiple exceptions from one expression, /and do something different for each one/, use the statement form as it's going to be clearer. For example: try: value = 1/x except ZeroDivisionError: try: value = 1/default['denominator'] except KeyError: value = NaN is much cleaner as: value = 1/x except ZeroDivisionError: 1/default['denominator'] except KeyError: NaN However, this: try: result = Parse(some_stuff) except MissingOperator: result = ... except InvalidOperand: result = ... except SomethingElse: result = ... would not benefit from being condensed into a single expression -- ~Ethan~ Funny, my feeling was exactly the reverse. :-) Probably because the latter seems to me to be a more natural thing to want to do (I find it easier to imagine use cases for it). And also because there is no way of getting exactly the same effect with a parenthesized except-expression: (expr except ValueError: ValueErrrorMessage) except NameError:NameErrorMessage # doesn't quite do what I want (here if expr raises a ValueError, evaluating ValueErrrorMessage which is mis-spelt will raise a NameError which will be misleadingly caught). But I guess the truth is that any except-expression which gets too long and complicated should be written some other way. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On 22/02/2014 02:08, Glenn Linderman wrote: On 2/21/2014 5:06 PM, Jan Kaliszewski wrote: Or even (still being my favorite): msg = seq[i] except (IndexError: nothing) This syntax actually has a benefit: the parenthesized syntax after except could become a list, to allow handling different exceptions from the tried expression with different results: msg = seq[dictionary[i]] except (IndexError: nothing, KeyError: serious problems) It shouldn't be a true list. We need lazy evaluation of the default values. And if an unlisted exception is raised, we don't want any of the defaults evaluated. Rob Cliffe And still allows nesting: msg = seq[i] except (IndexError: dictionary[i] except (KeyError: no fallback data for %s % i)) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/rob.cliffe%40btinternet.com No virus found in this message. Checked by AVG - www.avg.com http://www.avg.com Version: 2012.0.2247 / Virus Database: 3705/6616 - Release Date: 02/22/14 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
Victor Stinner wrote: Will ascii() ever emit an antislash representation? Try ascii(chr(0x1f)). In which version? I get: ValueError: chr() arg not in range(0x11) How do you plan to use this output? Write it into a socket or a file? When I debug, I use print logging which both expect text string. So I think that b'%a' is useless. Sad Use Case 1: There is not yet a working implementation of the file or wire format. Either I am still writing it, or the file I need to parse is coming from a partner who configured rather than wrote the original program. I write (or request that they write) something recognizable to the actual stream, as a landmark. Case 1a: I want a repr of the same object that is supposedly being represented in the official format, so I can see whether the problem is bad data or bad serialization. Use Case 2: Fallback for some sort of serialization format; I expect not to ever use the fallback in production, but better something ugly than a failure, let alone a crash. Use Case 3: Shortcut for serialization of objects whose repr is good enough. (My first instinct would probably be to implement the __bytes__ special method, but if I thought that was supposed to expose the real data, as opposed to a serialized copy, then I would go for %a.) -jJ -- If there are still threading problems with my replies, please email me with details, so that I can try to resolve them. -jJ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
2014-02-24 22:08 GMT+01:00 Jim J. Jewett jimjjew...@gmail.com: Will ascii() ever emit an antislash representation? Sorry, it's chr(0x10): print(ascii(chr(0x10))) '\U0010' Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 25 Feb 2014 05:44, Antoine Pitrou solip...@pitrou.net wrote:
On Mon, 24 Feb 2014 10:40:46 -0800
Ethan Furman et...@stoneleaf.us wrote:
Okay, types corrected, most comments taken into account.
%b is right out, %a is still suffering scrutiny.
The arguments seem to boil down to:
We don't need it.
vs
Somebody might, and it's better than having them inappropriately add a
__bytes__ method if we don't have it.
Don't forget that Python is a language for consenting adults. Adding a
near-useless feature for fear that otherwise people might shoot
themselves in the foot by using another feature must be one of the
worst arguments I've ever heard :-)
That's not quite the argument. The argument is that __bytes__ is expected
to work in arbitrary binary contexts and hence should *never* assume ASCII
compatibility (the PEP should probably propose a new addition to PEP 8 to
that effect), so the question is then OK, since it isn't defining
__bytes__, what is the preferred spelling for getting the ASCII compatible
representation of an object as a byte sequence?.
If we do nothing, then that spelling is ascii(obj).encode('ascii').
If %a is allowed as part of a binary interpolation pattern, then it becomes
b'%a' % obj
Allowing %a also improves the consistency with text interpolation. In the
case of %r, the inconsistency is based on needing to disallow arbitrary
Unicode code points in the result and not wanting to redefine %r as a
second way to spell %a. There's no corresponding reason to disallow %a -
the result is guaranteed to be ASCII compatible, so there's no risk of data
driven encoding errors, and no difference between doing the binary
interpolation directly, or doing text interpolation and then encoding the
result as ASCII.
As far as use cases go, as someone else mentioned, the main one is likely
to be binary logging and error reporting formats, as it becomes a quick and
easy way to embed a backslash escaped string. However, my interest is more
in providing an obvious way to do it and in minimising the differences
between text and binary interpolation.
Cheers,
Nick.
Regards
Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On Tue, 25 Feb 2014 08:33:53 +1000 Nick Coghlan ncogh...@gmail.com wrote: As far as use cases go, as someone else mentioned, the main one is likely to be binary logging and error reporting formats, as it becomes a quick and easy way to embed a backslash escaped string. That's a fringe use case, though. Also, your binary logging format probably has a well-defined character set that's not necessarily ASCII (perhaps UTF-8), so using the proposed %a is sub-optimal and potentially confusing (if lots of non-ASCII characters get escaped as \u). However, my interest is more in providing an obvious way to do it and in minimising the differences between text and binary interpolation. That sounds very theoretical. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 02/24/2014 02:33 PM, Nick Coghlan wrote:
Allowing %a also improves the consistency with text interpolation. In the case
of %r, the inconsistency is based on
needing to disallow arbitrary Unicode code points in the result and not wanting
to redefine %r as a second way to spell
%a. There's no corresponding reason to disallow %a - the result is guaranteed
to be ASCII compatible, so there's no risk
of data driven encoding errors, and no difference between doing the binary
interpolation directly, or doing text
interpolation and then encoding the result as ASCII.
As far as use cases go, as someone else mentioned, the main one is likely to be
binary logging and error reporting
formats, as it becomes a quick and easy way to embed a backslash escaped
string. However, my interest is more in
providing an obvious way to do it and in minimising the differences between
text and binary interpolation.
Jim Jewett had some use-cases that I'm happy to run with. (Thanks jJ!)
So final question for %a:
%a can only be used in Python 3 (3.2+, I believe) -- do we want to be able to
use %a as a short way of including text?
In Python2/3 code bases it will need to be '%s' % 'a string'.encode('ascii').
In Python 3 only code bases that could be shortened to '%a' % 'a string':
pro: much easier
if mojibake ( \x and \u sequences ) sneak in, the original data can
still be retrieved
cons: has surrounding quotes (would need to have bytes.__mod__ remove them)
--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
Yury Selivanov wrote:
I think the Motivation section is pretty weak.
I have normally wished for this when I was (semi-
interactively) exploring a weakly structured dataset.
Often, I start with a string, split it into something
hopefully like records, and then start applying filters
and transforms. I would prefer to write a comprehension
instead of a for loop. Alas, without pre-editing, I
can be fairly confident that the data is dirty.
Sometimes I can solve it with a filter (assuming
that I remember and don't mind the out-of-order
evaluation):
# The if value happens first,
# so the 1/value turns out to be safe.
[1/value for value in working_list if value]
Note that this means dropping the bad data, so that
items in this list will have different indices than
those in the parent working_list.
I would rather have written:
[1/value except (TypeError, ZeroDivisionError): None]
which would keep the matching indices, and clearly
indicate where I now had missing/invalid data.
Sometimes I solve it with a clumsy workaround:
sum((e.weight if hasattr(e, 'weight') else 1.0)
for e in working_list)
But the hasattr implies that I am doing some sort of
classification based on whether or not the element has
a weight.
The true intent was to recognize that while every element
does have a weight, the representation that I'm starting
from didn't always bother to store it -- so I am repairing
that before processing.
sum(e.weight except AttributeError: 1)
Often I give up, and create a junky helper function, or several.
But to avoid polluting the namespace, I may leave it outside
the class, or give it a truly bad name:
def __only_n2(worklist):
results = []
for line in worklist:
line=line.strip()
if not line: # or maybe just edit the input file...
continue
split1=line.split(, )
if 7 != len(split1):
continue
if n2 == split1[3]:
results.append(split1)
return results
worklist_n2 = __only_n2(worklist7)
In real life code, even after hand-editing the input data
to fix a few cases, I recently ended up with:
class VoteMark:
...
@classmethod
def from_property(cls, voteline):
# print (voteline)
count, _junk, prefs = voteline.partition(: )
return cls(count, prefs)
... # module level scope
def make_votes(vs=votestring):
return [VoteMark.from_property(e) for e in vs.splitlines()]
vs=make_votes()
You can correctly point out that I was being sloppy, and that I
*should* have gone back to clean it up. But I wouldn't have had
to clean up either the code or the data (well, not as much), if
I had been able to just keep the step-at-a-time transformations
I was building up during development:
vs=[(VoteMark(*e.strip().split(: ))
except (TypeError, ValueError): None)
for e in votestring.splitlines()]
Yes, the first line is still doing too much, and might be
worth a helper function during cleanup.
But it is already better than an alternate constructor that
exists only to simplify a single (outside the class) function
that is only called once.
Which in turn is better than the first draft that was so
ugly that I actually did fix it during that same work session.
Inconvenience of dict[] raising KeyError was solved by
introducing the dict.get() method. And I think that
dct.get('a', 'b')
is 1000 times better than
dct['a'] except KeyError: 'b'
I don't.
dct.get('a', default='b')
would be considerably better, but it would still imply
that missing values are normal. So even after argclinic
is fully integrated, there will still be times when I
prefer to make it explicit that I consider this an
abnormal case. (And, as others have pointed out, .get
isn't a good solution when the default is expensive to
compute.)
Consider this example of a two-level cache::
for key in sequence:
x = (lvl1[key] except KeyError: (lvl2[key] except KeyError: f(key)))
I'm sorry, it took me a minute to understand what your
example is doing. I would rather see two try..except blocks
than this.
Agreed -- like my semi-interactive code above, it does too much
on one line. I don't object as much to:
for key in sequence:
x = (lvl1[key]
except KeyError: (lvl2[key]
except KeyError: f(key)))
Retrieve an argument, defaulting to None::
cond = args[1] except IndexError: None
# Lib/pdb.py:803:
try:
cond = args[1]
except IndexError:
cond = None
cond = None if (len(args) 2) else args[1]
This is an area where tastes will differ.
I view the first as saying that not having a cond
would be unusual, or at least a different kind of
call.
I view your version as a warning that argument
parsing will be complex, and that
Re: [Python-Dev] PEP 463: Exception-catching expressions
Greg Ewing suggested: This version might be more readable: value = lst[2] except No value if IndexError Ethan Furman asked: It does read nicely, and is fine for the single, non-nested, case (which is probably the vast majority), but how would it handle nested exceptions? With parentheses. Sometimes, the parentheses will make a complex expression ugly. Sometimes, a complex expression should really be factored into pieces anyway. Hopefully, these times are highly correlated. The above syntax does lend itself somewhat naturally to multiple *short* except clauses: value = (lst[2] except No value if IndexError except Bad Input if TypeError) and nested exception expressions are at least possible, but deservedly ugly: value = (lvl1[name] except (lvl2[name] except (compute_new_answer(name) except None if AppValueError) if KeyError) if KeyError) This also makes me wonder whether the cost of a subscope (for exception capture) could be limited to when an exception actually occurs, and whether that might lower the cost enough to make the it a good tradeoff. def myfunc1(a, b, e): assert main scope e value == e e = main scope e value value = (myfunc1(val1, val2, e) except e.reason if AppError as e) assert main scope e value == e -jJ -- If there are still threading problems with my replies, please email me with details, so that I can try to resolve them. -jJ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 463: Exception-catching expressions
On Tue, Feb 25, 2014 at 11:27 AM, Jim J. Jewett jimjjew...@gmail.com wrote: This also makes me wonder whether the cost of a subscope (for exception capture) could be limited to when an exception actually occurs, and whether that might lower the cost enough to make the it a good tradeoff. def myfunc1(a, b, e): assert main scope e value == e e = main scope e value value = (myfunc1(val1, val2, e) except e.reason if AppError as e) assert main scope e value == e I'm sure it could. But there aren't many use-cases. Look at the one example I was able to find in the stdlib: http://legacy.python.org/dev/peps/pep-0463/#capturing-the-exception-object It's hardly a shining example of the value of the proposal. Got any really awesome demo that requires 'as'? Most of the supporting examples use something like KeyError where it's simply was an exception thrown or not. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python Remote Code Execution in socket.recvfrom_into()
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, this looks pretty serious -- and it caught me off guard, too. :( https://www.trustedsec.com/february-2014/python-remote-code-execution-socket-recvfrom_into/ Next time please inform the Python Security Response Team about any and all issues that are related to buffer overflows or similar bugs. In fact please drop a note about anything that even remotely look like an exploitable issue. Even public bug reports should be forwarded to PSRT. I have requested a CVE number. How about security releases? The upcoming 3.3 and 3.4 release should contain the fix (not verified yet). Python 2.7 to 3.2 will need a security release, though. Regards Christian -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCgAGBQJTDEi1AAoJEMeIxMHUVQ1FdAwP/j36bioIzz+kFvX9AEo2Bxtq H+JsvRxiWHJrXLG0YUf1AolW+s92/2dRAYLq86DQa7PK2rvrqR4bQUOP+fLi9hdT 5b9YF4mHhBtte9lTDwESYw4IXtoOz4gbhXpY/dGGLjiEeWYNgRl40xSZYXf6cZfR okRRE0c6EZ9WnAYWl1vW1oUzPjua0KOpVOhvabog/YNSPL3SW8shWANpu0fg/n+G guBYTP90pgUEz7Jc20xeVAB9BeZoC/jjDPv1QRMu+PWjyFeaI4iLdNe3loRXBYy1 xmzHxQACzQR45lxAzCoBwBopC49JIF7o7pnTBrY9id9j0yRMAPC/N1uQCceLO1yc GoKardxzUT9IX++yfLTOYwdGnpXDQeXUbHAImcWNGMN8QfsWUFBezPmqKM7tfTNR I/khqTaLPewr58z4d/erfJ5wSEHVdyWASmUWGniS9jjfFNVBDNA2pSPBCP9TJhK5 30BOnKB+MMNG+LCe5chiQOyKje/pbfwrEwwrdiJYCOSXK+w/hbPClNBBq4w9XXVk sIIk5xO1IZ4rMG/YLkg9vaWzn7Yi6O0GJXOmQWp+22kYwaQK+3l6qqSSo2laMVN8 c6sLFng3loO1v3SDO0AOTTU2VcdsS0SdYLkEXMwHK/tRWeXWrwC5HrYnFS0Hu0iI EQlaImb433lu8mHrvPEx =K5ZL -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 2
On 23 February 2014 08:56, Ethan Furman et...@stoneleaf.us wrote: ``%a`` will call :func:``ascii()`` on the interpolated value's :func:``repr()``. This is intended as a debugging aid, rather than something that should be used in production. Non-ascii values will be encoded to either ``\xnn`` or ``\u`` representation. So we use %a for exactly the same purposes that we used to use %r. Unsupported codes - ``%r`` (which calls ``__repr__`` and returns a :class:`str`) is not supported. But you propose changing the code. I think there would have been a lot less discussion if you just defined %r to do what you propose for %a, as everything would work as people expected. -- Stuart Bishop stu...@stuartbishop.net http://www.stuartbishop.net/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Remote Code Execution in socket.recvfrom_into()
On 25 February 2014 17:39, Christian Heimes christ...@python.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, this looks pretty serious -- and it caught me off guard, too. :( https://www.trustedsec.com/february-2014/python-remote-code-execution-socket-recvfrom_into/ Next time please inform the Python Security Response Team about any and all issues that are related to buffer overflows or similar bugs. In fact please drop a note about anything that even remotely look like an exploitable issue. Even public bug reports should be forwarded to PSRT. I have requested a CVE number. How about security releases? The upcoming 3.3 and 3.4 release should contain the fix (not verified yet). I've checked these, and noted the relevant hg.python.org links on the tracker issue at http://bugs.python.org/issue20246 Python 2.7 to 3.2 will need a security release, though. Agreed. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
