Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-23 Thread Nick Coghlan
Brett Cannon wrote: I don't want to move it because this isn't some idea for a new feature that may or may not be useful; this isn't an idea, it's needed. It is needed, but it's only really needed in the test suite. The sys.modules hackery needed to get a Python-only version using the existing

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-23 Thread Brett Cannon
On Sun, Feb 22, 2009 at 22:41, Aahz a...@pythoncraft.com wrote: On Sun, Feb 22, 2009, Brett Cannon wrote: On Sat, Feb 21, 2009 at 20:12, Aahz a...@pythoncraft.com wrote: On Sat, Feb 21, 2009, Brett Cannon wrote: On Sat, Feb 21, 2009 at 15:46, Aahz a...@pythoncraft.com wrote: On Sat, Feb

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-23 Thread Brett Cannon
On Mon, Feb 23, 2009 at 04:02, Nick Coghlan ncogh...@gmail.com wrote: Brett Cannon wrote: I don't want to move it because this isn't some idea for a new feature that may or may not be useful; this isn't an idea, it's needed. It is needed, but it's only really needed in the test suite. The

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-23 Thread Steven Bethard
On Mon, Feb 23, 2009 at 04:02, Nick Coghlan ncogh...@gmail.com wrote: For example, a version that allows any number of extension modules to be suppressed when importing a module (defaulting to the Foo/_Foo naming): import sys def import_python_only(mod_name, *ext_names): if not

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-23 Thread Nick Coghlan
Brett Cannon wrote: Well, neither do I as your proposed approach below is what I do for warnings. It's possible I actually had test_warnings.py open in another window while writing that example function... ;) As Steven said, your concerns are precisely why I'm suggesting hiding this in a

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-23 Thread Brett Cannon
On Mon, Feb 23, 2009 at 13:23, Nick Coghlan ncogh...@gmail.com wrote: Brett Cannon wrote: Well, neither do I as your proposed approach below is what I do for warnings. It's possible I actually had test_warnings.py open in another window while writing that example function... ;) As

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread James Pye
On Feb 21, 2009, at 2:17 PM, Brett Cannon wrote: The other approach is having pickle contain code known not to be overridden by anyone, import _pypickle for stuff that may be overridden, and then import _pickle for whatever is available. This approach has the perk of using a standard

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread Steven Bethard
On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon br...@python.org wrote: But there is another issue with this: the pure Python code will never call the extension code because the globals will be bound to _pypickle and not _pickle. So if you have something like:: # _pypickle def A(): return

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread Michael Foord
Steven Bethard wrote: On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon br...@python.org wrote: But there is another issue with this: the pure Python code will never call the extension code because the globals will be bound to _pypickle and not _pickle. So if you have something like:: #

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread Brett Cannon
On Sat, Feb 21, 2009 at 20:12, Aahz a...@pythoncraft.com wrote: On Sat, Feb 21, 2009, Brett Cannon wrote: On Sat, Feb 21, 2009 at 15:46, Aahz a...@pythoncraft.com wrote: On Sat, Feb 21, 2009, Brett Cannon wrote: I am seeing two approaches emerging. One is where pickle contains all

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread Brett Cannon
On Sun, Feb 22, 2009 at 10:29, Michael Foord fuzzy...@voidspace.org.ukwrote: Steven Bethard wrote: On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon br...@python.org wrote: But there is another issue with this: the pure Python code will never call the extension code because the globals will

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-22 Thread Aahz
On Sun, Feb 22, 2009, Brett Cannon wrote: On Sat, Feb 21, 2009 at 20:12, Aahz a...@pythoncraft.com wrote: On Sat, Feb 21, 2009, Brett Cannon wrote: On Sat, Feb 21, 2009 at 15:46, Aahz a...@pythoncraft.com wrote: On Sat, Feb 21, 2009, Brett Cannon wrote: I am seeing two approaches emerging.

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread Jean-Paul Calderone
On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon br...@python.org wrote: On Fri, Feb 20, 2009 at 12:53, Aahz a...@pythoncraft.com wrote: On Fri, Feb 20, 2009, Brett Cannon wrote: On Fri, Feb 20, 2009 at 12:37, Brett Cannon br...@python.org wrote: On Fri, Feb 20, 2009 at 12:31, Daniel

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread Brett Cannon
On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone exar...@divmod.comwrote: On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon br...@python.org wrote: On Fri, Feb 20, 2009 at 12:53, Aahz a...@pythoncraft.com wrote: On Fri, Feb 20, 2009, Brett Cannon wrote: On Fri, Feb 20, 2009 at 12:37,

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread glyph
On 07:07 pm, br...@python.org wrote: On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone exar...@divmod.comwrote: But there is another issue with this: the pure Python code will never call the extension code because the globals will be bound to _pypickle and not _pickle. So if you have

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread Jean-Paul Calderone
On Sat, 21 Feb 2009 11:07:07 -0800, Brett Cannon br...@python.org wrote: On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone exar...@divmod.comwrote: On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon br...@python.org wrote: On Fri, Feb 20, 2009 at 12:53, Aahz a...@pythoncraft.com wrote: On

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread Brett Cannon
On Sat, Feb 21, 2009 at 11:43, gl...@divmod.com wrote: On 07:07 pm, br...@python.org wrote: On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone exar...@divmod.com wrote: But there is another issue with this: the pure Python code will never call the extension code because the globals will

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread Brett Cannon
On Sat, Feb 21, 2009 at 11:32, Jean-Paul Calderone exar...@divmod.comwrote: On Sat, 21 Feb 2009 11:07:07 -0800, Brett Cannon br...@python.org wrote: On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone exar...@divmod.com wrote: On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread Nick Coghlan
Brett Cannon wrote: On Sat, Feb 21, 2009 at 11:43, gl...@divmod.com mailto:gl...@divmod.com wrote: On 07:07 pm, br...@python.org mailto:br...@python.org wrote: On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone exar...@divmod.com mailto:exar...@divmod.comwrote:

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread Aahz
On Sat, Feb 21, 2009, Brett Cannon wrote: I am seeing two approaches emerging. One is where pickle contains all Python code and then uses something like use_extension to make sure the original Python objects are still reachable at some point. This has the drawback that you have to use some

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread Kevin Teague
So go ahead and tear this apart so that we can hopefully reach a consensus that makes sense so that at least testing can easily be done. If I was developing an application and wanted to deal with two different versions of the same library, I would simply make sure that the version I

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread Brett Cannon
On Sat, Feb 21, 2009 at 15:46, Aahz a...@pythoncraft.com wrote: On Sat, Feb 21, 2009, Brett Cannon wrote: I am seeing two approaches emerging. One is where pickle contains all Python code and then uses something like use_extension to make sure the original Python objects are still

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-21 Thread Aahz
On Sat, Feb 21, 2009, Brett Cannon wrote: On Sat, Feb 21, 2009 at 15:46, Aahz a...@pythoncraft.com wrote: On Sat, Feb 21, 2009, Brett Cannon wrote: I am seeing two approaches emerging. One is where pickle contains all Python code and then uses something like use_extension to make sure the

[Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Brett Cannon
With io getting rewritten as an extension module, I think it's time to try to come up with a good best practice scenario for how to be able to control when a module uses a pure Python implementation and when it uses extension module optimizations. This is really only important for testing as if

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Daniel Stutzbach
On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon br...@python.org wrote: Now, from what I can tell, Antoine is suggesting having _pyio and a _io and then io is simply: try: from _io import * except ImportError: from _pyio import * That works for testing as you can then have test

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Brett Cannon
On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach dan...@stutzbachenterprises.com wrote: On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon br...@python.org wrote: Now, from what I can tell, Antoine is suggesting having _pyio and a _io and then io is simply: try: from _io import * except

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Brett Cannon
On Fri, Feb 20, 2009 at 12:37, Brett Cannon br...@python.org wrote: On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach dan...@stutzbachenterprises.com wrote: On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon br...@python.org wrote: Now, from what I can tell, Antoine is suggesting having _pyio

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Michael Foord
Brett Cannon wrote: On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach dan...@stutzbachenterprises.com mailto:dan...@stutzbachenterprises.com wrote: On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon br...@python.org mailto:br...@python.org wrote: Now, from what I can tell,

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Aahz
On Fri, Feb 20, 2009, Brett Cannon wrote: On Fri, Feb 20, 2009 at 12:37, Brett Cannon br...@python.org wrote: On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach dan...@stutzbachenterprises.com wrote: A slight change would make it work for modules where only key functions have been rewritten.

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Brett Cannon
On Fri, Feb 20, 2009 at 12:53, Aahz a...@pythoncraft.com wrote: On Fri, Feb 20, 2009, Brett Cannon wrote: On Fri, Feb 20, 2009 at 12:37, Brett Cannon br...@python.org wrote: On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach dan...@stutzbachenterprises.com wrote: A slight change would

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Greg Ewing
Brett Cannon wrote: So while this alleviates the worry above, it does mean that anything that gets rewritten needs to have a name that does not lead with an underscore for this to work. You can use an __all__ list to explicitly say what is to be exported. -- Greg

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Nick Coghlan
Brett Cannon wrote: But there is another issue with this: the pure Python code will never call the extension code because the globals will be bound to _pypickle and not _pickle. So if you have something like:: # _pypickle def A(): return _B() def _B(): return -13 # _pickle

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Daniel Stutzbach
On Fri, Feb 20, 2009 at 5:27 PM, Nick Coghlan ncogh...@gmail.com wrote: Brett Cannon wrote: If you import pickle and call pickle.A() you will get -13 which is not what you are after. Ah, you may want to think about that a bit more. There's a reason globals are looked up when they're used

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Nick Coghlan
Daniel Stutzbach wrote: On Fri, Feb 20, 2009 at 5:27 PM, Nick Coghlan ncogh...@gmail.com mailto:ncogh...@gmail.com wrote: Brett Cannon wrote: If you import pickle and call pickle.A() you will get -13 which is not what you are after. Ah, you may want to think about that

Re: [Python-Dev] Choosing a best practice solution for Python/extension modules

2009-02-20 Thread Greg Ewing
Daniel Stutzbach wrote: No, I'm afraid Brett is quite right. Globals are looked up when the function is executed, true, but they are looked up within the module that defined the function. I was thinking you could fix that by going over the imported functions and stuffing the current globals