Re: Bizarre behavior with mutable default arguments
On Dec 30 2007, 12:27 am, Steven D'Aprano [EMAIL PROTECTED] cybersource.com.au wrote: In the absence of a better solution, I'm very comfortable with keeping the behaviour as is. Unfortunately, there's no good solution in Python to providing functions with local storage that persists across calls to the function: ... (4) Instances handle this pretty well, just s/functions/methods/. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Jan 1, 5:26 am, NickC [EMAIL PROTECTED] wrote:
On Jan 1, 3:22 am, Arnaud Delobelle [EMAIL PROTECTED] wrote:
On Dec 31, 10:58 am, Odalrick [EMAIL PROTECTED] wrote:
I'm surprised noone has said anything about the why of default
mutables. I think it is becasue it isn't easy to do it an other way.
[...]
There is an easy enough way: evaluate default values when the function
is called rather than when it is defined. This behaviour comes with
its own caveats as well I imagine, and it's not 'as easy' to implement
as the current one.
As Odalrick notes, there is no way to give different calls to a
function their own copies of mutable default arguments without re-
evaluating the defaults every time the function is called. The
horrendous performance implications mean that that simply isn't going
to happen. So the status quo, where the defaults are calculated once
when the function is defined and the result cached in the function
object is unlikely to change.
I'm in no way advocating a change, in fact I wouldn't like things to
change. I was just saying that it was not difficult (technically) to
alter the behaviour, but that this change wouldn't be desirable
because it would make code more difficult to reason on. OTOH a very
common idiom in python is
def foo(x, y, z=None):
if z is None: z = ['a', 'mutable', 'object']
# stuff that foo does
This the current way to say I want the default value of z to be
reevaluated each time it is used. I use this much more often than
def bar(x, y, z=ExpensiveImmutableCreation())
So I'm not so convinced with the performance argument at face value
(though it's probably pertinent:)
What's good about the current behaviour is that it is easy to reason
with (once you know what happens), even though you almost have to get
bitten once. But using this to have static variable is extremely ugly
IMHO.
The only thing it doesn't give you is a static variable that isn't
visible to the caller. Py3k's keyword-only arguments (PEP 3102) will
make those cases a little tidier, since it won't be possible to
accidentally replace the static variables by providing too many
positional arguments.
I was always a bit puzzled by this PEP. If this is one of the
underlying reasons for it, then I am even more puzzled.
I believe the suggestion of permitting static variables after the **
entry in a function's parameter list was raised during the PEP 3102
discussions, but never gained much traction over a '_cache={}' keyword-
only argument approach (and the latter has the distinct advantage of
being *much* easier to test, since you can override the cache from the
test code to ensure it is being handled correctly).
Well I'm glad that didn't go through, argument lists in function
definitions are complicated enough already!
--
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 30 2007, 11:01 pm, Steven D'Aprano [EMAIL PROTECTED] cybersource.com.au wrote: On Sun, 30 Dec 2007 20:00:14 -0800, bukzor wrote: I think you struck at the heart of the matter earlier when you noted that this is the simplest way to declare a static variable in python. Using the 'global' keyword is the other way, and is much more explicit, and much more widely used. I also see this as the main use of the 'notlocal' keyword to be introduced in py3k (it also fixes the example given by Istvan above). There doesn't appear to be any reference to a notlocal keyword in Python 3 that I can find. Have I missed something? It sounds like an April Fool's gag to me. Do you have a reference to a PEP or other official announcement? -- Steven I got it slightly wrong. It's 'nonlocal': http://www.python.org/dev/peps/pep-3104/ --Buck -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 31 2007, 1:30 pm, Chris Mellon [EMAIL PROTECTED] wrote: On Dec 31, 2007 2:08 PM, Odalrick [EMAIL PROTECTED] wrote: On 31 Dec, 18:22, Arnaud Delobelle [EMAIL PROTECTED] wrote: On Dec 31, 10:58 am, Odalrick [EMAIL PROTECTED] wrote: On 30 Dec, 17:26, George Sakkis [EMAIL PROTECTED] wrote: On Dec 29, 9:14 pm, bukzor [EMAIL PROTECTED] wrote: Here's the answer to the question:http://www.python.org/doc/faq/general/#why-are-default-values-shared-... It looks like Guido disagrees with me, so the discussion is closed. Note that the FAQ mainly explains *what* happens, not *why* was this decision taken. Although it shows an example where this feature can be useful, it's neither the only way to do it nor is memoization as common as wanting fresh default arguments on every call. I'm surprised noone has said anything about the why of default mutables. I think it is becasue it isn't easy to do it an other way. [...] There is an easy enough way: evaluate default values when the function is called rather than when it is defined. This behaviour comes with its own caveats as well I imagine, and it's not 'as easy' to implement as the current one. Adding overhead to *all* function calls, even the ones without mutable defaults. That doesn't sound like an attractive tradeoff. And also removing the only way you can currently do early binding in Python. I agree that it's a gotcha, but unless someone comes up with an answer to the following questions, I'll stick with the status quo (Note that this is not blind Python group-think as a previous poster implied, but a pragmatic decision that this is the most practical solution): a) If we don't evaluate default arguments at function compilation, when do we do it? b) If you do it at call time, how do you implement early binding? c) If you want to introduce new syntax for the current behavior, what is it and can you justify it? d) What are the performance implications of your proposal versus the current behavior? Note that the desired behavior can be implemented under the current behavior, at the expense of verbosity - using factories and sentinel values as the default arguments, and then expanding them in the function. It's not possible to implement the current behavior of early-bound arguments if default arguments are evaluated with every call. This alone is a good reason to keep the current behavior until someone actually has a good alternative that covers the current use cases and isn't just upset by the behavior. I'm confused by what you mean by 'early binding'. Can you give a quick- n-dirty example? Thanks, --Buck -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Jan 1, 2008 12:26 AM, NickC [EMAIL PROTECTED] wrote: On Jan 1, 3:22 am, Arnaud Delobelle [EMAIL PROTECTED] wrote: On Dec 31, 10:58 am, Odalrick [EMAIL PROTECTED] wrote: I'm surprised noone has said anything about the why of default mutables. I think it is becasue it isn't easy to do it an other way. [...] There is an easy enough way: evaluate default values when the function is called rather than when it is defined. This behaviour comes with its own caveats as well I imagine, and it's not 'as easy' to implement as the current one. As Odalrick notes, there is no way to give different calls to a function their own copies of mutable default arguments without re- evaluating the defaults every time the function is called. There's a policy solution: stipulate that default arguments must be immutable (or hashable, as Python prefers to call it). The horrendous performance implications mean that that simply isn't going to happen. Are the performance implications truly horrendous, though? Considering that the poorly performing solution seems to be the functionality that the uninformed expect, perhaps they *want* it to perform that way? -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Jan 1, 9:00 am, bukzor [EMAIL PROTECTED] wrote: On Dec 31 2007, 1:30 pm, Chris Mellon [EMAIL PROTECTED] wrote: On Dec 31, 2007 2:08 PM, Odalrick [EMAIL PROTECTED] wrote: On 31 Dec, 18:22, Arnaud Delobelle [EMAIL PROTECTED] wrote: On Dec 31, 10:58 am, Odalrick [EMAIL PROTECTED] wrote: On 30 Dec, 17:26, George Sakkis [EMAIL PROTECTED] wrote: On Dec 29, 9:14 pm, bukzor [EMAIL PROTECTED] wrote: Here's the answer to the question:http://www.python.org/doc/faq/general/#why-are-default-values-shared-... It looks like Guido disagrees with me, so the discussion is closed. Note that the FAQ mainly explains *what* happens, not *why* was this decision taken. Although it shows an example where this feature can be useful, it's neither the only way to do it nor is memoization as common as wanting fresh default arguments on every call. I'm surprised noone has said anything about the why of default mutables. I think it is becasue it isn't easy to do it an other way. [...] There is an easy enough way: evaluate default values when the function is called rather than when it is defined. This behaviour comes with its own caveats as well I imagine, and it's not 'as easy' to implement as the current one. Adding overhead to *all* function calls, even the ones without mutable defaults. That doesn't sound like an attractive tradeoff. And also removing the only way you can currently do early binding in Python. I agree that it's a gotcha, but unless someone comes up with an answer to the following questions, I'll stick with the status quo (Note that this is not blind Python group-think as a previous poster implied, but a pragmatic decision that this is the most practical solution): a) If we don't evaluate default arguments at function compilation, when do we do it? b) If you do it at call time, how do you implement early binding? c) If you want to introduce new syntax for the current behavior, what is it and can you justify it? d) What are the performance implications of your proposal versus the current behavior? Note that the desired behavior can be implemented under the current behavior, at the expense of verbosity - using factories and sentinel values as the default arguments, and then expanding them in the function. It's not possible to implement the current behavior of early-bound arguments if default arguments are evaluated with every call. This alone is a good reason to keep the current behavior until someone actually has a good alternative that covers the current use cases and isn't just upset by the behavior. I'm confused by what you mean by 'early binding'. Can you give a quick- n-dirty example? Thanks, --Buck Is an 'early bound' variable synonymous with a 'static' variable (in C)? -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
En Tue, 01 Jan 2008 15:45:00 -0200, bukzor [EMAIL PROTECTED] escribi�: On Jan 1, 9:00 am, bukzor [EMAIL PROTECTED] wrote: On Dec 31 2007, 1:30 pm, Chris Mellon [EMAIL PROTECTED] wrote: And also removing the only way you can currently do early binding in Python. I agree that it's a gotcha, but unless someone comes up with an answer to the following questions, I'll stick with the status quo (Note that this is not blind Python group-think as a previous poster implied, but a pragmatic decision that this is the most practical solution): a) If we don't evaluate default arguments at function compilation, when do we do it? b) If you do it at call time, how do you implement early binding? I'm confused by what you mean by 'early binding'. Can you give a quick- n-dirty example? Is an 'early bound' variable synonymous with a 'static' variable (in C)? No. It means, in which moment the name gets its value assigned. Usually Python does late binding, that is, names are resolved at the time the code is executed, not when it's compiled or defined. Consider this example: z = 1 def foo(a) print a+z foo(3) # prints 4 z = 20 foo(3) # prints 23 The second time it prints 23, not 4, because the value for z is searched when the code is executed, so the relevant value for z is 20. Note that if you later assign a non-numeric value to z, foo(3) will fail. If you want to achieve the effect of early binding, that is, you want to freeze z to be always what it was at the time the function was defined, you can do that using a default argument: z = 1 def foo(a, z=z) print a+z z = None foo(3) # prints 4 This way, foo(3) will always print 4, independently of the current value of z. Moreover, you can `del z` and foo will continue to work. This is what I think Chris Mellon was refering to. This specific default argument semantics allows one to achieve the effect of early binding in a language which is mostly late binding. If someone changes this, he has to come with another way of faking early binding semantics at least as simple as this, else we're solving an [inexistant for me] problem but creating another one. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Jan 1, 6:48 pm, Gabriel Genellina [EMAIL PROTECTED] wrote: En Tue, 01 Jan 2008 15:45:00 -0200, bukzor [EMAIL PROTECTED] escribi�: [...] I'm confused by what you mean by 'early binding'. Can you give a quick- n-dirty example? Is an 'early bound' variable synonymous with a 'static' variable (in C)? No. It means, in which moment the name gets its value assigned. Usually Python does late binding, that is, names are resolved at the time the code is executed, not when it's compiled or defined. Consider this example: z = 1 def foo(a) print a+z foo(3) # prints 4 z = 20 foo(3) # prints 23 The second time it prints 23, not 4, because the value for z is searched when the code is executed, so the relevant value for z is 20. Note that if you later assign a non-numeric value to z, foo(3) will fail. If you want to achieve the effect of early binding, that is, you want to freeze z to be always what it was at the time the function was defined, you can do that using a default argument: z = 1 def foo(a, z=z) print a+z z = None foo(3) # prints 4 This way, foo(3) will always print 4, independently of the current value of z. Moreover, you can `del z` and foo will continue to work. This is what I think Chris Mellon was refering to. This specific default argument semantics allows one to achieve the effect of early binding in a language which is mostly late binding. If someone changes this, he has to come with another way of faking early binding semantics at least as simple as this, else we're solving an [inexistant for me] problem but creating another one. -- Gabriel Genellina Let me say again that I believe the current behaviour to be the correct one. But I don't think this 'early binding' is critical for this sort of example. There are lots of ways to solve the problem of having persistent state across function calls, for example: * using classes * using modules * or simply nested functions: def getfoo(z): def foo(a): print a + z return foo z = 1 foo = getfoo(z) z = None foo(3) 4 And with nonlocal, we could even modify z inside foo and this change would persist across calls. This will be a much cleaner solution than the current def bar(x, y, _hidden=[startvalue]). Also, note that it's easy to implement default arguments in pure python-without-default-arguments using a decorator: def default(**defaults): defaults = defaults.items() def decorator(f): def decorated(*args, **kwargs): for name, val in defaults: kwargs.setdefault(name, val) return f(*args, **kwargs) return decorated return decorator Here is your example: z=1 @default(z=z) ... def foo(a, z): ... print a + z ... z=None foo(3) 4 Another example, using mutables: @default(history=[]) ... def bar(x, history): ... history.append(x) ... return list(history) ... map(bar, 'spam') [['s'], ['s', 'p'], ['s', 'p', 'a'], ['s', 'p', 'a', 'm']] -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On 30 Dec, 17:26, George Sakkis [EMAIL PROTECTED] wrote:
On Dec 29, 9:14 pm, bukzor [EMAIL PROTECTED] wrote:
Here's the answer to the
question:http://www.python.org/doc/faq/general/#why-are-default-values-shared-...
It looks like Guido disagrees with me, so the discussion is closed.
Note that the FAQ mainly explains *what* happens, not *why* was this
decision taken. Although it shows an example where this feature can
be useful, it's neither the only way to do it nor is memoization as
common as wanting fresh default arguments on every call.
I'm surprised noone has said anything about the why of default
mutables. I think it is becasue it isn't easy to do it an other way.
def some_function( an_integer=1,pointless_list=[],
random_fuction_value=random_function()):
pass
To you and me it is obvious that this is an integer, a list and a
function call, but to python it is just 3 objects. Python'd have to
check each argument carefully to determine if it is mutable or not. Or
always copy each object, adding additional overhead to function calls,
and making passing arguments to functions expensive.
Even if these problems were solved, it would only make the problem
less common, not extinct.
# hypothetical
def another_function( still_alive=([],) ):
still_alive[0].append('spam')
print still_alive
another_function()
(['spam'],)
another_function()
(['spam', 'spam'],)
(Could of course be solved by always making deep copies of all
arguments.)
While I would welcome making mutable defaults work differently, I
don't see any way to make such a change without making unacceptable
tradeoffs.
--
Incidentally, I wrote a program a while back, with a bug caused by
mutable defaults. Never bothered to change it, it was the behaviour I
wanted, just not the one I thought I had implemented. -- Python, so
good even the bugs make the program better.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 31, 10:58 am, Odalrick [EMAIL PROTECTED] wrote: On 30 Dec, 17:26, George Sakkis [EMAIL PROTECTED] wrote: On Dec 29, 9:14 pm, bukzor [EMAIL PROTECTED] wrote: Here's the answer to the question:http://www.python.org/doc/faq/general/#why-are-default-values-shared-... It looks like Guido disagrees with me, so the discussion is closed. Note that the FAQ mainly explains *what* happens, not *why* was this decision taken. Although it shows an example where this feature can be useful, it's neither the only way to do it nor is memoization as common as wanting fresh default arguments on every call. I'm surprised noone has said anything about the why of default mutables. I think it is becasue it isn't easy to do it an other way. [...] There is an easy enough way: evaluate default values when the function is called rather than when it is defined. This behaviour comes with its own caveats as well I imagine, and it's not 'as easy' to implement as the current one. What's good about the current behaviour is that it is easy to reason with (once you know what happens), even though you almost have to get bitten once. But using this to have static variable is extremely ugly IMHO. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On 31 Dec, 18:22, Arnaud Delobelle [EMAIL PROTECTED] wrote: On Dec 31, 10:58 am, Odalrick [EMAIL PROTECTED] wrote: On 30 Dec, 17:26, George Sakkis [EMAIL PROTECTED] wrote: On Dec 29, 9:14 pm, bukzor [EMAIL PROTECTED] wrote: Here's the answer to the question:http://www.python.org/doc/faq/general/#why-are-default-values-shared-... It looks like Guido disagrees with me, so the discussion is closed. Note that the FAQ mainly explains *what* happens, not *why* was this decision taken. Although it shows an example where this feature can be useful, it's neither the only way to do it nor is memoization as common as wanting fresh default arguments on every call. I'm surprised noone has said anything about the why of default mutables. I think it is becasue it isn't easy to do it an other way. [...] There is an easy enough way: evaluate default values when the function is called rather than when it is defined. This behaviour comes with its own caveats as well I imagine, and it's not 'as easy' to implement as the current one. Adding overhead to *all* function calls, even the ones without mutable defaults. That doesn't sound like an attractive tradeoff. What's good about the current behaviour is that it is easy to reason with (once you know what happens), even though you almost have to get bitten once. But using this to have static variable is extremely ugly IMHO. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 31, 2007 2:08 PM, Odalrick [EMAIL PROTECTED] wrote: On 31 Dec, 18:22, Arnaud Delobelle [EMAIL PROTECTED] wrote: On Dec 31, 10:58 am, Odalrick [EMAIL PROTECTED] wrote: On 30 Dec, 17:26, George Sakkis [EMAIL PROTECTED] wrote: On Dec 29, 9:14 pm, bukzor [EMAIL PROTECTED] wrote: Here's the answer to the question:http://www.python.org/doc/faq/general/#why-are-default-values-shared-... It looks like Guido disagrees with me, so the discussion is closed. Note that the FAQ mainly explains *what* happens, not *why* was this decision taken. Although it shows an example where this feature can be useful, it's neither the only way to do it nor is memoization as common as wanting fresh default arguments on every call. I'm surprised noone has said anything about the why of default mutables. I think it is becasue it isn't easy to do it an other way. [...] There is an easy enough way: evaluate default values when the function is called rather than when it is defined. This behaviour comes with its own caveats as well I imagine, and it's not 'as easy' to implement as the current one. Adding overhead to *all* function calls, even the ones without mutable defaults. That doesn't sound like an attractive tradeoff. And also removing the only way you can currently do early binding in Python. I agree that it's a gotcha, but unless someone comes up with an answer to the following questions, I'll stick with the status quo (Note that this is not blind Python group-think as a previous poster implied, but a pragmatic decision that this is the most practical solution): a) If we don't evaluate default arguments at function compilation, when do we do it? b) If you do it at call time, how do you implement early binding? c) If you want to introduce new syntax for the current behavior, what is it and can you justify it? d) What are the performance implications of your proposal versus the current behavior? Note that the desired behavior can be implemented under the current behavior, at the expense of verbosity - using factories and sentinel values as the default arguments, and then expanding them in the function. It's not possible to implement the current behavior of early-bound arguments if default arguments are evaluated with every call. This alone is a good reason to keep the current behavior until someone actually has a good alternative that covers the current use cases and isn't just upset by the behavior. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Jan 1, 3:22 am, Arnaud Delobelle [EMAIL PROTECTED] wrote:
On Dec 31, 10:58 am, Odalrick [EMAIL PROTECTED] wrote:
I'm surprised noone has said anything about the why of default
mutables. I think it is becasue it isn't easy to do it an other way.
[...]
There is an easy enough way: evaluate default values when the function
is called rather than when it is defined. This behaviour comes with
its own caveats as well I imagine, and it's not 'as easy' to implement
as the current one.
As Odalrick notes, there is no way to give different calls to a
function their own copies of mutable default arguments without re-
evaluating the defaults every time the function is called. The
horrendous performance implications mean that that simply isn't going
to happen. So the status quo, where the defaults are calculated once
when the function is defined and the result cached in the function
object is unlikely to change.
What's good about the current behaviour is that it is easy to reason
with (once you know what happens), even though you almost have to get
bitten once. But using this to have static variable is extremely ugly
IMHO.
The only thing it doesn't give you is a static variable that isn't
visible to the caller. Py3k's keyword-only arguments (PEP 3102) will
make those cases a little tidier, since it won't be possible to
accidentally replace the static variables by providing too many
positional arguments.
I believe the suggestion of permitting static variables after the **
entry in a function's parameter list was raised during the PEP 3102
discussions, but never gained much traction over a '_cache={}' keyword-
only argument approach (and the latter has the distinct advantage of
being *much* easier to test, since you can override the cache from the
test code to ensure it is being handled correctly).
Cheers,
Nick.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 29, 7:17 pm, Istvan Albert [EMAIL PROTECTED] wrote: On Dec 29, 12:50 pm, bukzor [EMAIL PROTECTED] wrote: Is this functionality intended? It seems very unintuitive. This has caused a bug in my programs twice so far, and both times I was completely mystified until I realized that the default value was changing. it is only unintuitive when you do not know about it once you realize how it works and what it does it can actually be very useful i. I agree it is a potentially useful feature, yet it can still bite you even after a decade of Python... Scenario: long running server process, Bug report: people aren't getting older, Code: def age(dob, today=datetime.date.today()): ... None of my unit tests caught that one :-) -- bjorn -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 30, 5:23 am, thebjorn [EMAIL PROTECTED] wrote: def age(dob, today=datetime.date.today()): ... None of my unit tests caught that one :-) interesting example I can see how it caused some trouble. A quick fix would be to write it: def age(dob, today=datetime.date.today ): and inside the definition invoke it as today() rather than just today. That way it still keeps the original spirit of the definition. i. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 29, 11:21 pm, bukzor [EMAIL PROTECTED] wrote: The standard library is not affected because the people who wrote code into it know how python works. Programming abounds with cases that some people think should work differently: a = b = [] a.append(1) is b empty or not at this point? Get informed, remember the rules, be happy and move on to write some cool code. There is little new in what you say. Every so often someone is having a confusing time with a feature and therefore proposes that the language be changed to match his/her expectations. i. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
This is a well-known python gotcha. See: http://www.ferg.org/projects/python_gotchas.html#contents_item_6 -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 29, 9:14 pm, bukzor [EMAIL PROTECTED] wrote:
Here's the answer to the
question:http://www.python.org/doc/faq/general/#why-are-default-values-shared-...
It looks like Guido disagrees with me, so the discussion is closed.
Note that the FAQ mainly explains *what* happens, not *why* was this
decision taken. Although it shows an example where this feature can
be useful, it's neither the only way to do it nor is memoization as
common as wanting fresh default arguments on every call.
For the record, I still think the following would be an improvement to
py3k:
In python25:
def f(a=None):
if a is None: a = []
...
In py3k becomes:
def f(a=[])
...
In python25 (this function from the FAQ linked above):
def f(a, _cache={}):
# Callers will never provide a third parameter for this function.
(then why is it an argument?)
...
In py3k becomes:
_cache = {}
def f(a):
global _cache
...
This follows the explicit is better and one best way principles of
Python, and greatly improves the intuitiveness. Also since the first
example is much more common, it reduces the overall verbosity of the
language.
I'm with you on this one; IMHO it's one of the relatively few language
design missteps of Python, favoring the rare case as the default
instead of the common one. Unfortunately, many Pythoneers become so
immersed with the language and whatever the current status quo is that
they rarely question the rationale of the few counter-intuitive design
choices.
George
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 30, 2:45 pm, Istvan Albert [EMAIL PROTECTED] wrote:
On Dec 30, 5:23 am, thebjorn [EMAIL PROTECTED]
wrote:
def age(dob, today=datetime.date.today()):
...
None of my unit tests caught that one :-)
interesting example I can see how it caused some trouble. A quick fix
would be to write it:
def age(dob, today=datetime.date.today ):
and inside the definition invoke it as today() rather than just today.
That way it still keeps the original spirit of the definition.
i.
The purpose of the argument was to be able to calculate the age at a
given point in time -- i.e. was the person 18 y/o at the time of the
incident?
Our coding standard now dictates:
def foo(arg=None):
if arg is None:
arg = default mutable value
(unless there's a very good reason to do it otherwise :-)
a close runner-up, that we specifically opted not to allow was
def foo(arg={}):
arg = arg or {}
even though it looks sexy and it's perhaps a bit more self-documenting
in some IDEs, it was rejected because it prevents false overrides of
the default argument.
For purely practical reasons we couldn't consider
def foo(arg=None):
arg = default mutable value if arg is None else arg
-- bjorn
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 30, 2:23 am, thebjorn [EMAIL PROTECTED] wrote: Scenario: long running server process, Bug report: people aren't getting older, Code: def age(dob, today=datetime.date.today()): ... A very interesting example, thanks. On Dec 30, 8:25 am, [EMAIL PROTECTED] wrote: This is a well-known python gotcha. See:http://www.ferg.org/projects/python_gotchas.html#contents_item_6 Just because it's well known doesn't mean we shouldn't think about it. For example, in the same list you linked, 3. Integer division is being fixed in py3k. On Dec 30, 8:26 am, George Sakkis [EMAIL PROTECTED] wrote: I'm with you on this one; IMHO it's one of the relatively few language design missteps of Python, favoring the rare case as the default instead of the common one. Well put. Although I've seen 'potentially useful' often used about this, I havn't found an instance of its use in production code. On Dec 30, 9:57 am, thebjorn [EMAIL PROTECTED] wrote: Our coding standard now dictates: def foo(arg=None): if arg is None: arg = default mutable value (unless there's a very good reason to do it otherwise :-) I believe this is very similar to most people's coding practices. It's the one I adopted, it's in the above FAQ, the above 'gotchas' list, and it's all over the standard library. This is a case of a language feature requiring coding practices to prevent bugs, with very little value added elsewhere. --Buck -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 30, 11:26 am, George Sakkis [EMAIL PROTECTED] wrote:
I'm with you on this one; IMHO it's one of the relatively few language
design missteps of Python, favoring the rare case as the default
instead of the common one.
George, you pointed this out this link in a different thread
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/521877
how would you rewrite the code below if you could not use mutable
default arguments (global variables not accepted)? Maybe there is a
way, but I can't think of it as of now.
---
def blocks(s, start, end):
def classify(c, ingroup=[0]):
klass = c==start and 2 or c==end and 3 or ingroup[0]
ingroup[0] = klass==1 or klass==2
return klass
return [tuple(g) for k, g in groupby(s, classify) if k == 1]
print blocks('the {quick} brown {fox} jumped', start='{', end='}')
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 30, 12:32 pm, Istvan Albert [EMAIL PROTECTED] wrote:
On Dec 30, 11:26 am, George Sakkis [EMAIL PROTECTED] wrote:
I'm with you on this one; IMHO it's one of the relatively few language
design missteps of Python, favoring the rare case as the default
instead of the common one.
George, you pointed this out this link in a different thread
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/521877
how would you rewrite the code below if you could not use mutable
default arguments (global variables not accepted)? Maybe there is a
way, but I can't think of it as of now.
---
def blocks(s, start, end):
def classify(c, ingroup=[0]):
klass = c==start and 2 or c==end and 3 or ingroup[0]
ingroup[0] = klass==1 or klass==2
return klass
return [tuple(g) for k, g in groupby(s, classify) if k == 1]
print blocks('the {quick} brown {fox} jumped', start='{', end='}')
Extremely simple
def blocks(s, start, end):
ingroup=[0]
def classify(c):
klass = c==start and 2 or c==end and 3 or ingroup[0]
ingroup[0] = klass==1 or klass==2
return klass
return [tuple(g) for k, g in groupby(s, classify) if k == 1]
print blocks('the {quick} brown {fox} jumped', start='{', end='}')
No globals, as you specified. BTW, it's silly not to 'allow' globals
when they're called for, otherwise we wouldn't need the 'global'
keyword.
--Buck
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 30, 3:41 pm, bukzor [EMAIL PROTECTED] wrote:
No globals, as you specified. BTW, it's silly not to 'allow' globals
when they're called for, otherwise we wouldn't need the 'global'
keyword.
okay, now note that you do not actually use the ingroup list for
anything else but getting and setting its first element. So why would
one really need it be a list? Let's replace it with a variable called
ingroup that is not a list anymore. See it below (run it to see what
happens):
--
def blocks(s, start, end):
ingroup = 0
def classify(c):
klass = c==start and 2 or c==end and 3 or ingroup
ingroup = klass==1 or klass==2
return klass
return [tuple(g) for k, g in groupby(s, classify) if k == 1]
print blocks('the {quick} brown {fox} jumped', start='{', end='}')
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Sun, 30 Dec 2007 12:41:57 -0800, bukzor wrote: BTW, it's silly not to 'allow' globals when they're called for, otherwise we wouldn't need the 'global' keyword. Nobody argues against allowing globals variables *when they're called for*, just pointing out that ninety-nine times out of a hundred, people use them when they're not called for and are positively harmful. And according to functional programmers, they're NEVER called for. -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Sun, 30 Dec 2007 13:34:07 -0800, Dennis Lee Bieber wrote: On Sun, 30 Dec 2007 12:11:50 -0800 (PST), bukzor [EMAIL PROTECTED] declaimed the following in comp.lang.python: Just because it's well known doesn't mean we shouldn't think about it. For example, in the same list you linked, 3. Integer division is being fixed in py3k. IMHO -- Py3K is /breaking/ integer division... as the division of two integers will differ from what happens in all the other languages I have used... All the others, to get a floating result from dividing two integers requires one to explicitly convert at least one term to a float first You need to use more languages :) Prolog uses / for division and // for integer division, just like Python. Apple's Hypertalk (and derivatives) don't distinguish between integer and floating point division. The / operator returns an integer result if the floating point result happens to be an integer. e.g. 10.0/5.0 = 2 while 11.0/5.0 = 2.2) I believe that Javascript behaves the same way. According to this table here: http://msdn2.microsoft.com/en-us/library/2hxce09y.aspx VisualBasic uses / for floating point division and \ for integer division, and both JScript and Visual FoxPro don't even offer integer division at all. No doubt there are others... -- as I would do with the current Python. The forthcoming change is going to require one to remember that if they want an integer result from two integers, they must use a different operator instead. How is that different from needing to remember to use a different algorithm if you want a floating point result? -- Steven. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 30, 3:34 pm, Steven D'Aprano [EMAIL PROTECTED] cybersource.com.au wrote: On Sun, 30 Dec 2007 12:41:57 -0800, bukzor wrote: BTW, it's silly not to 'allow' globals when they're called for, otherwise we wouldn't need the 'global' keyword. Nobody argues against allowing globals variables *when they're called for*, just pointing out that ninety-nine times out of a hundred, people use them when they're not called for and are positively harmful. And according to functional programmers, they're NEVER called for. -- Steven. I think you struck at the heart of the matter earlier when you noted that this is the simplest way to declare a static variable in python. Using the 'global' keyword is the other way, and is much more explicit, and much more widely used. I also see this as the main use of the 'notlocal' keyword to be introduced in py3k (it also fixes the example given by Istvan above). If the main value of this behavior is to declare a static variable, it seems like an argument to create a more explicit syntax for static variables. In the example above, the function only needed a static integer, but created a one-length list instead because this quirk doesn't work for immutable values. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Sun, 30 Dec 2007 20:00:14 -0800, bukzor wrote: I think you struck at the heart of the matter earlier when you noted that this is the simplest way to declare a static variable in python. Using the 'global' keyword is the other way, and is much more explicit, and much more widely used. I also see this as the main use of the 'notlocal' keyword to be introduced in py3k (it also fixes the example given by Istvan above). There doesn't appear to be any reference to a notlocal keyword in Python 3 that I can find. Have I missed something? It sounds like an April Fool's gag to me. Do you have a reference to a PEP or other official announcement? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
En Mon, 31 Dec 2007 05:01:51 -0200, Steven D'Aprano [EMAIL PROTECTED] escribió: On Sun, 30 Dec 2007 20:00:14 -0800, bukzor wrote: I also see this as the main use of the 'notlocal' keyword to be introduced in py3k (it also fixes the example given by Istvan above). There doesn't appear to be any reference to a notlocal keyword in Python 3 that I can find. Have I missed something? It sounds like an April Fool's gag to me. Do you have a reference to a PEP or other official announcement? No, it's a real keyword in python 3, but it's spelled nonlocal. See http://www.python.org/dev/peps/pep-3104/ -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
Is this functionality intended? Google for Python mutable default arguments (you can actually leave out Python). It's part of the language semantics, yes. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 29, 12:50 pm, bukzor [EMAIL PROTECTED] wrote: Is this functionality intended? It seems very unintuitive. This has caused a bug in my programs twice so far, and both times I was completely mystified until I!realized that the default value was changing. it is only unintuitive when you do not know about it once you realize how it works and what it does it can actually be very useful i. -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 29, 1:11 pm, Martin v. Löwis [EMAIL PROTECTED] wrote: Google for Python mutable default arguments and a mere 30 minutes later this thread is already one of the results that come up -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
Here's the answer to the question:
http://www.python.org/doc/faq/general/#why-are-default-values-shared-between-objects
It looks like Guido disagrees with me, so the discussion is closed.
For the record, I still think the following would be an improvement to
py3k:
In python25:
def f(a=None):
if a is None: a = []
...
In py3k becomes:
def f(a=[])
...
In python25 (this function from the FAQ linked above):
def f(a, _cache={}):
# Callers will never provide a third parameter for this function.
(then why is it an argument?)
...
In py3k becomes:
_cache = {}
def f(a):
global _cache
...
This follows the explicit is better and one best way principles of
Python, and greatly improves the intuitiveness. Also since the first
example is much more common, it reduces the overall verbosity of the
language.
Just my parting two cents,
--Buck
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Sat, 29 Dec 2007 11:14:30 -0800, bukzor wrote:
In python25 (this function from the FAQ linked above):
def f(a, _cache={}):
# Callers will never provide a third parameter for this function.
(then why is it an argument?)
The caller might want to provide it's own pre-prepared cache. Say, for
testing.
I think that this behaviour is a little unintuitive, and by a little I
mean a lot. Nevertheless, I am used to it, and I don't see any reason to
change it. There's very little about programming that is intuitive --
there's no intuitive reason to think that dictionary lookups are O(1)
while list lookups are O(n).
In the absence of a better solution, I'm very comfortable with keeping
the behaviour as is. Unfortunately, there's no good solution in Python to
providing functions with local storage that persists across calls to the
function:
(1) Use a global variable.
cache = {}
def foo():
global cache
print cache
(2) Use a function attribute.
def foo():
print foo.cache
foo.cache = {}
def foo():
try:
foo.cache
except AttributeError:
foo.cache = {}
print foo.cache
(3) Use an argument that isn't actually an argument.
def foo(cache={}):
print cache
#1, the global variable, is probably the worst solution of the lot.
Global variables are rightly Considered Harmful.
#2 has the disadvantages that you initialize the value *after* you write
the code that relies on it. Either that, or you waste time on every call
checking to see it if has been initialized. Also, like recursive
functions, it is difficult to rename the function.
#3 is, I think, the least-worse solution, but I would hardly call it
ideal.
_cache = {}
def f(a):
global _cache
...
This follows the explicit is better and one best way principles of
Python,
Declaring an argument is equally explicit.
And you are confused -- the Zen doesn't say one best way. People so
often get it wrong.
The Zen says:
There should be one-- and PREFERABLY only one --OBVIOUS way to do it.
(Emphasis added.)
At least you're not saying there should be only one way to do it. I
give you credit for that!
and greatly improves the intuitiveness. Also since the first
example is much more common, it reduces the overall verbosity of the
language.
I question that it is much more common. How do you know? Where's your
data?
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Sat, 29 Dec 2007 09:50:53 -0800, bukzor wrote: I've found some bizzare behavior when using mutable values (lists, dicts, etc) as the default argument of a function. This FAQ is so Frequently Asked that I sometimes wonder if Python should, by default, print a warning when it compiles a function with a list or dict as as default value. There's precedence for such a thing: the sum() built-in (un)helpfully raises an exception if you try to use it on strings. I say unhelpfully because the one time I wanted to use sum() on strings was specifically to demonstrate the difference between O(n**2) behaviour and O(n). I was quite put out that Python, which normally allows you to shoot yourself in the foot if you insist, was so unnecessarily protective in this case. Give me a warning, if you wish, but don't stop me. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
I think that this behaviour is a little unintuitive, and by a little I mean a lot. Thanks for acknowledging it. I question that it is much more common. How do you know? Where's your data? I did a dumb grep of my Python25/Lib folder and found 33 occurances of the first pattern above. (Use None as the default value, then check for None and assign empty list/dict) Although I spent at least double the amount of time looking for the second pattern, I found no occurances. (Use dict/list as default value and modify it in place.) Every single function that used a list or dict as a default value treated these variables as read-only. However, I did find two new ways to accomplish the above (further violating the Zen). /c/Python25/Lib/site-packages/wx-2.8-msw-ansi/wx/lib/ customtreectrl.py: def FillArray(self, item, array=[]): if not array: array = [] /c/Python25/Lib/site-packages/wx-2.8-msw-ansi/wx/lib/floatcanvas/ FloatCanvas.py: def __init__(self, ObjectList=[], InForeground = False, IsVisible = True): self.ObjectList = list(ObjectList) --Buck -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
Just for completeness, the mutable default value problem also affects classes: class c: def __init__(self, list = []): self.list = list self.list.append(LIST END) def __repr__(self): return Class a: %s % self.list import example2 print example2.c() Class a: ['LIST END'] print example2.c([]) Class a: ['LIST END'] print example2.c() Class a: ['LIST END', 'LIST END'] print example2.c([]) Class a: ['LIST END'] Again, we get different results if we supply an argument that is identical to the default value. There are many instances in the standard library where class values are assigned directly from the initializer, which has list or dict default values, so there is chance for errors cropping up here. The error scenario is this: 1. Use a mutable value as default value in a class constructor. 2. Assign class property from constructor arguments. 3. Instantiate class using default value. 4. Modify class property in place. 5. Instantiate (again) class using default value. The second instance will behave strangely because data from the first instance has leaked over. The standard library is not affected because it avoids one of these five steps. Most classes simply don't have mutable default values (1). Those that do generally treat them as read- only (4). Some classes are not useful using the default values (3). Some classes are not useful to be instantiated twice (5). The classes that don't avoid the problem at one of these four steps have to avoid it at (2) by using one of the three above patterns. --Buck -- http://mail.python.org/mailman/listinfo/python-list
Re: Bizarre behavior with mutable default arguments
On Dec 30, 3:21 pm, bukzor [EMAIL PROTECTED] wrote: Just for completeness, the mutable default value problem also affects classes: Simply, because methods are functions, and can have default arguments. You don't need to nail *another* zillion theses to the cathedral door :-) -- http://mail.python.org/mailman/listinfo/python-list
