On Jan 7, 2010, at 10:45 PM, Steven D'Aprano <st...@remove-this-cybersource.com.au > wrote an extensive answer to my questions about one function calling another in the same file being exec'd. His suggestion about printing out locals() and globals() in the various possible places provided the clues to explain what was going on. I would like to summarize what I have learned from this, because although I have known all the relevant pieces for many years I never put them together in a way that explains the odd behavior I observed.
Statements that bind new names -- assignment, def, and class -- do so in the local scope. While exec'ing a file the local scope is determined by the arguments passed to exec; in my case, I passed an explicit local scope. It was particularly obtuse of me not to notice the effects of this because I was intentionally using it so that an assignment to 'result' in the exec'd script would enable the exec'ing code to retrieve the value of result. However, although the purity of Python with respect to the binding actions of def and class statements is wonderful and powerful, it is very difficult cognitively to view a def on a page and think "aha! that's just like an assignment of a newly created function to a name", even though that is precisely the documented behavior of def. So mentally I was making an incorrect distinction between what was getting bound locally and what was getting bound globally in the exec'd script.
Moreover, the normal behavior of imported code, in which any function in the module can refer to any other function in the module, seduced me into this inappropriate distinction. To my eye I was just defining and using function definitions the way they are in modules. There is a key difference between module import and exec: as Steven pointed out, inside a module locals() is globals(). On further reflection, I will add that what appears to be happening is that during import both the global and local dictionaries are set to a copy of the globals() from the importing scope and that copy becomes the value of the module's __dict__ once import has completed successfully. Top-level statements bind names in locals(), as always, but because locals() and globals() are the same dictionary, they are also binding them in globals(), so that every function defined in the module uses the modified copy of globals -- the value of the module's __dict__ -- as its globals() when it executes. Because exec leaves locals() and globals() distinct, functions defined at the top level of a string being exec'd don't see other assignments and definitions that are also in the string.
Another misleading detail is that top-level expressions in the exec can use other top-level names (assigned, def'd, etc.), which they will find in the exec string's local scope, but function bodies do not see the string's local scope. The problem I encountered arises because the function definitions need to access each other through the global scope, not the local scope. In fact, the problem would arise if one of the functions tried to call itself recursively, since its own name would not be in the global scope. So we have a combination of two distinctions: the different ways module import and exec use globals and locals and the difference between top-level statements finding other top-level names in locals but functions looking for them in globals.
Sorry for the long post. These distinctions go deep into the semantics of Python namespaces, which though they are lean, pure, and beautiful, have some consequences that can be surprising -- more so the more familiar you are with other languages that do things differently.
Oh, and as far as using import instead of exec for my scripts, I don't think that's appropriate, if only because I don't want my application's namespace polluted by what could be many of these pseudo- modules users might load during a session. (Yes, I could remove the name once the import is finished, but importing solely for side- effects rather than to use the imported module is offensive. Well, I would be using one module name -- result -- but that doesn't seem to justify the complexities of setting up the import and accessing the module when exec does in principle just what I need.)
Finally, once all of this is really understood, there is a simple way to change an exec string's def's to bind globally instead of locally: simply begin the exec with a global declaration for any function called by one of the others. In my example, adding a "global fn1" at the beginning of the file fixes it so exec works.
################################ global fn1 # enable fn1 to be called from fn2! def fn1(val): return sum(range(val)) def fn2(arg): return fn1(arg) result = fn2(5) ################################ -- http://mail.python.org/mailman/listinfo/python-list
