Re: [Python-Dev] PEP: per user site-packages directory
Jeroen Ruigrok van der Werven wrote: One thing I miss in this PEP and discussion is the point of view from a webhosting company and webhosting user. If the webhoster upgrades his hosting software and Python get updated by a revision (say 2.5 to 2.6) this would in the current case mean that the webhosting user using per-user site-packages is out of luck due to the hard-wired (and apparently needed) version identifier. Or is that a wrong assumption on my side? The problem exists for almost every Python extension. 3rd party packages with C modules have to be compiled for every Python version. Even pure Python packages don't always work on a new version. You can't expect software to run under every version of FOO, not even the next future version of FOO where FOO is Python, Java, C#, Perl, PHP ... you name it. It's not the intention of the PEP to solve the problem. A web hoster can support multiple versions of Python at once. You can install every minor version of Python from 2.0 to 3.0 (8 versions) without major conflicts. A *good* web hoster should give an user several months up to a year to migrate from - say - 2.5 to 2.6. Most Linux distributions (except Redhat) have at least 2.4 and 2.5 in their repositories. Right now Python faces a lot of problems in the webhosting world because it's tedious to set up and maintain for the webhosting user since they often have to compile and install their own Python in their home directory. The per-user site-packages will help in that aspect a lot, I think, but we do need to detail some more use-cases and scenarios (patterns if you like) on how such deployments would work. I think this is extremely important due to the proliferation of Python now more and more as a choice for webapp development. Can you write up some use cases for us? I'm especially interested in use cases that are not covered by PEP 370. So far only a few problems come into my mind: * Installation extensions for mod_python, requires root permissions or full control over the Apache process anyway * Multiple versions of an extension. The problem is mostly covered by pkg_resources and setup tools. * Installing a newer version of an extension, covered by PEP 370. * Registering more search paths. Drop a pth file into the user site directory or use a usercustumize.py. * easy_install packages to the user site directory. Setuptools are not part of the Python stdlib. I'm confident that Phil is going to add the feature as soon as the feature is implemented for Python 2.6 What's missing? Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On Mon, Jan 14, 2008 at 11:41:47PM +, Jon Ribbens wrote: It makes sense, but personally I have never heard before of ~/.local. Whereas ~/bin is something I am quite familiar with. Me too. python-dev is the only place I have heard of ~/.local. I have been using Linux (different distributions), Solaris and FreeBSD for quite a long time (though I have never used GNOME/KDE/etc.) Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On Jan 15, 2008 6:24 AM, Oleg Broytmann [EMAIL PROTECTED] wrote: On Mon, Jan 14, 2008 at 11:41:47PM +, Jon Ribbens wrote: It makes sense, but personally I have never heard before of ~/.local. Whereas ~/bin is something I am quite familiar with. Me too. python-dev is the only place I have heard of ~/.local. I have been using Linux (different distributions), Solaris and FreeBSD for quite a long time (though I have never used GNOME/KDE/etc.) Never heard of it either, would be completely baffled if caught unawares by it in the wild. Has anyone consulted with the LSB or a cross-platform filesystem layout guide to see what the recommended best-practice is? -Kevin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
On Tue, Jan 15, 2008 at 01:21:46PM +0100, Christian Heimes wrote: It took me a while to find a reference to .local. It's part of the FreeDesktop.Org standards: http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html The site only mentions $HOME/.local/share, there is no $HOME/.local/bin at the site. Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Kevin Jacobs [EMAIL PROTECTED] wrote: Never heard of it either, would be completely baffled if caught unawares by it in the wild. Has anyone consulted with the LSB or a cross-platform filesystem layout guide to see what the recommended best-practice is? It took me a while to find a reference to .local. It's part of the FreeDesktop.Org standards: http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Monkeypatching idioms -- elegant or ugly?
I ran into the need of monkeypatching a large number of classes (for what I think are very good reasons :-) and invented two new recipes. These don't depend on Py3k, and the second one actually works all the way back to Python 2.2. Neither of these allows monkeypatching built-in types like list. If you don't know what monkeypatching is, see see http://en.wikipedia.org/wiki/Monkey_patch. I think it's useful to share these recipes, if only to to establish whether they have been discovered before, or to decide whether they are worthy of a place in the standard library. I didn't find any relevant hits on the ASPN Python cookbook. First, a decorator to add a single method to an existing class: def monkeypatch_method(cls): def decorator(func): setattr(cls, func.__name__, func) return func return decorator To use: from somewhere import someclass @monkeypatch_method(someclass) def newmethod(self, args): return whatever This adds newmethod to someclass Second, a metaclass to add a number of methods (or other attributes) to an existing class, using a convenient class notation: def monkeypatch_class(name, bases, namespace): assert len(bases) == 1, Exactly one base class required base = bases[0] for name, value in namespace.iteritems(): if name != __metaclass__: setattr(base, name, value) return base To use: from somewhere import someclass class newclass(someclass): __metaclass__ = monkeypatch_class def method1(...): ... def method2(...): ... ... This adds method1, method2, etc. to someclass, and makes newclass a local alias for someclass. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Monkeypatching idioms -- elegant or ugly?
[EMAIL PROTECTED] wrote: On 03:37 pm, [EMAIL PROTECTED] wrote: I think it's useful to share these recipes, if only to to establish whether they have been discovered before, or to decide whether they are worthy of a place in the standard library. I didn't find any relevant hits on the ASPN Python cookbook. from somewhere import someclass class newclass(someclass): __metaclass__ = monkeypatch_class def method1(...): ... def method2(...): ... ... I've expressed this one before as class someclass(reopen(someclass)):, but have thankfully never needed to actually use that in a real program. It's been a helpful tool in explaining to overzealous Ruby-ists that reopenable classes are not as unique as they think. My feelings on monkeypatching is that it *should* feel a little gross when you have to do it, so the code I've written that does monkeypatching for real is generally a bit ugly. Yes, monkeypatching should never be formalized to the point where novices see it as other than a tool of last resort. Otherwise a user base will grow that uses monkeypatching instead of subclassing, for example (shudder). regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Monkeypatching idioms -- elegant or ugly?
On 03:37 pm, [EMAIL PROTECTED] wrote: I think it's useful to share these recipes, if only to to establish whether they have been discovered before, or to decide whether they are worthy of a place in the standard library. I didn't find any relevant hits on the ASPN Python cookbook. from somewhere import someclass class newclass(someclass): __metaclass__ = monkeypatch_class def method1(...): ... def method2(...): ... ... I've expressed this one before as class someclass(reopen(someclass)):, but have thankfully never needed to actually use that in a real program. It's been a helpful tool in explaining to overzealous Ruby-ists that reopenable classes are not as unique as they think. My feelings on monkeypatching is that it *should* feel a little gross when you have to do it, so the code I've written that does monkeypatching for real is generally a bit ugly. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Post import hooks
At 10:10 PM 1/11/2008 +0100, Christian Heimes wrote:
Phillip J. Eby wrote:
*sigh*. We seem to be getting further and further off course,
here. The more different you make the semantics of the system, the
harder it will be to verify that it's doing the right thing without
having real field experience with the new approach.
*sigh*, too. :/
This discussion has neither helping me nor you. Could you please write
an unit test or two to show me exactly what my implementation is doing
wrong and how you think the callbacks should be called. I know a simple
test won't proof the correctness of the implementation but a failing
test would at least show the incorrectness.
when_imported('a.b')(func_ab1)
when_imported('a.b')(func_ab2)
@when_imported('a')
def func_a1(module_a):
when_imported('a.b')(func_ab3)
notify_module('a.b') # - this is here to foil trivial implementations
when_imported('a')(func_a2)
notify_module('a.b')
This should produce the calling sequence:
func_a1, func_a2, func_ab1, func_ab2, func_ab3.
I'm still not sure which way is the correct way in your opinion and I
hate guessing what you are trying to explain to me.
The invariants to ensure are:
1. notification is only done once for a given module, ever, even if
the notification function is called more than once, even if it's
called during notifications for that module
2. notifications for a child module/package may not begin until the
notifications for the parent package have begun
3. two registrations for the same module must always be invoked in
the same order as they were registered, even if some of the
registrations are done during notification.
In order to implement these invariants, you will have to have a way
to know whether notifications have been begun for a given module. In
peak.util.imports, the module objects effectively keep track of this,
although they don't have a specific flag. For the Python
implementation, you could add a __notified__ field to module objects,
and implement the notify function thus:
def notify(name):
try:
module = sys.modules[name]
except KeyError:
raise ImportError(Module %s has not been imported % (name,))
if module.__notified__:
return
if '.' in name:
notify(name[:name.rfind('.')])
try:
module.__notified__ = True
for callback in post_import_hooks[name]:
callback(module)
finally:
post_import_hooks[name] = None
Of course, __notified__ would actually be a structure slot, rather
than an attribute, so as to avoid any attribute lookup issues with
module subtypes (e.g. lazy modules).
The register function would simply append a hook to the entry in
post_import_hooks if it's not None, or call the hook otherwise.
With this implementation, I could make a version of peak.util.imports
that did its own lazy modules, but used the base system for all the hooks.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Monkeypatching idioms -- elegant or ugly?
Guido van Rossum: I ran into the need of monkeypatching a large number of classes (for what I think are very good reasons :-) and invented two new recipes. These don't depend on Py3k, and the second one actually works all the way back to Python 2.2. Neither of these allows monkeypatching built-in types like list. If you don't know what monkeypatching is, see see http://en.wikipedia.org/wiki/Monkey_patch. I think it's useful to share these recipes, if only to to establish whether they have been discovered before, or to decide whether they are worthy of a place in the standard library. I didn't find any relevant hits on the ASPN Python cookbook. First, a decorator to add a single method to an existing class: def monkeypatch_method(cls): def decorator(func): setattr(cls, func.__name__, func) return func return decorator To use: from somewhere import someclass @monkeypatch_method(someclass) def newmethod(self, args): return whatever This adds newmethod to someclass I like it, but my first thought was, and if that method already exists? I'd extend monkeypatch_method to store a reference to the old method(s): def monkeypatch_method(cls): Add the decorated method to the given class; replace as needed. If the named method already exists on the given class, it will be replaced, and a reference to the old method appended to a list at cls._old_name. If the _old_name attribute already exists and is not a list, KeyError is raised. def decorator(func): fname = func.__name__ old_func = getattr(cls, fname, None) if old_func is not None: # Add the old func to a list of old funcs. old_ref = _old_%s % fname old_funcs = getattr(cls, old_ref, None) if old_funcs is None: setattr(cls, old_ref, []) elif not isinstance(old_funcs, list): raise KeyError(%s.%s already exists. % (cls.__name__, old_ref)) getattr(cls, old_ref).append(old_func) setattr(cls, fname, func) return func return decorator I chose a renaming scheme somewhat at random. The list allows you (or someone else ;) to call monkeypatch repeatedly on the same cls.method (but it's not thread-safe). And although it might seem to be making monkeypatches easier to perform, at least it's very explicit about what's going on as long as you keep monkeypatch in the name. Robert Brewer [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Monkeypatching idioms -- elegant or ugly?
Guido van Rossum guido at python.org writes: I ran into the need of monkeypatching a large number of classes (for what I think are very good reasons and invented two new recipes. These don't depend on Py3k, and the second one actually works all the way back to Python 2.2. Neither of these allows monkeypatching built-in types like list. If you don't know what monkeypatching is, see see http://en.wikipedia.org/wiki/Monkey_patch. I think it's useful to share these recipes, if only to to establish whether they have been discovered before, or to decide whether they are worthy of a place in the standard library. I didn't find any relevant hits on the ASPN Python cookbook. pypy has something that requires the base class to use a specific metaclass: https://codespeak.net/viewvc/pypy/dist/pypy/tool/pairtype.py?view=markup ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Monkeypatching idioms -- elegant or ugly?
On Jan 15, 2008 8:22 AM, Antoine Pitrou [EMAIL PROTECTED] wrote: pypy has something that requires the base class to use a specific metaclass: https://codespeak.net/viewvc/pypy/dist/pypy/tool/pairtype.py?view=markup That's no good for my particular use case. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
On Tue, 15 Jan 2008 06:57:02 -0500, Kevin Jacobs wrote: On Jan 15, 2008 6:24 AM, Oleg Broytmann [EMAIL PROTECTED] wrote: On Mon, Jan 14, 2008 at 11:41:47PM +, Jon Ribbens wrote: It makes sense, but personally I have never heard before of ~/.local. Whereas ~/bin is something I am quite familiar with. Me too. python-dev is the only place I have heard of ~/.local. I have been using Linux (different distributions), Solaris and FreeBSD for quite a long time (though I have never used GNOME/KDE/etc.) Never heard of it either, would be completely baffled if caught unawares by it in the wild. Has anyone consulted with the LSB or a cross-platform filesystem layout guide to see what the recommended best-practice is? I use ~/local, with a layout analogous to /usr, all the time. It's not a standard, but in my experience it is by far the best solution to installing things in the home directory. It doesn't matter much whether you call it local or .local or .pythonlocal (although that last would limit the utility somewhat, by implying that other things should be installed there). It does matter that it be a _subdirectory_ of ~, and that it be structured like /usr. To those folks who favor creating ~/bin, ~/lib, ~/share, ad nauseum, I point out that non-hidden, non-user-created files in ~ ought to be kept to a minimum. It is, after all, the user's default workspace. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
On Tue, Jan 15, 2008 at 12:11:37PM -0800, Daniel Arbuckle wrote: I use ~/local, with a layout analogous to /usr, all the time. It's not a standard, but in my experience it is by far the best solution to installing things in the home directory. It doesn't matter much whether you call it local or .local or .pythonlocal (although that last would limit the utility somewhat, by implying that other things should be installed there). It does matter that it be a _subdirectory_ of ~, and that it be structured like /usr. ~/.python ~/.python/bin ~/.python/lib ~/.python/lib/python2.5 Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
Daniel I use ~/local, with a layout analogous to /usr, ... Ditto. Makes things nice and clear. I install stuff without becoming root or polluting central directories. Daniel To those folks who favor creating ~/bin, ~/lib, ~/share, ad Daniel nauseum, I point out that non-hidden, non-user-created files in Daniel ~ ought to be kept to a minimum. It is, after all, the user's Daniel default workspace. Agreed. You're just asking for trouble if you set --prefix=$HOME. Eventually you will encounter some package which would have scribbled all over /usr/local and only disrupted other external packages (which you can always reinstall) but instead scribbles all over your home directory, wiping out your almost finished Ph.D. thesis in Library Science which you had saved to ~/lib. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
Oleg~/.python Oleg~/.python/bin Oleg~/.python/lib Oleg~/.python/lib/python2.5 The drawback of this approach is that it implies that Perl, Tcl, IPython, etc. belong in their own .whatever directory. The IT folks here at work do things that way (though not in home directories). If I want to build a package which relies on zlib, libpng, libtiff, libjpeg, etc., imagine what my CPPFLAGS, CFLAGS and LDFLAGS arguments look like. :barf: Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Monkeypatching idioms -- elegant or ugly?
Guido van Rossum writes: I think it's useful to share these recipes, But only to people who have demonstrated that they already know when, why and how to monkeypatch on their own. Lisp's `defadvice' plays a starring role in a number of my nightmares. Most recently, 15 minutes ago.wink Come to think of it, `defadvice' does have the advantage that it also automatically monkeypatches the docstring to say This function has class advice, and with a little cooperation from the advisor can also provide documentation of the changed semantics. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
On Tue, Jan 15, 2008 at 02:34:02PM -0600, [EMAIL PROTECTED] wrote: Oleg~/.python Oleg~/.python/bin Oleg~/.python/lib Oleg~/.python/lib/python2.5 The drawback of this approach is that it implies that Perl, Tcl, IPython, etc. belong in their own .whatever directory. How many users install (parts of) all of these into their homes? The IT folks here at work do things that way (though not in home directories). If I want to build a package which relies on zlib, libpng, libtiff, libjpeg, etc., imagine what my CPPFLAGS, CFLAGS and LDFLAGS arguments look like. :barf: Why not use GNU stow? Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Post import hooks
Phillip J. Eby wrote:
when_imported('a.b')(func_ab1)
when_imported('a.b')(func_ab2)
@when_imported('a')
def func_a1(module_a):
when_imported('a.b')(func_ab3)
notify_module('a.b') # - this is here to foil trivial
implementations
when_imported('a')(func_a2)
notify_module('a.b')
This should produce the calling sequence:
func_a1, func_a2, func_ab1, func_ab2, func_ab3.
My implementation calls the hooks in the right order but I had to insert
fake modules into sys.path.
# insert the modules into sys.modules to fake a 3rd party import
a = imp.new_module('a')
ab = imp.new_module('a.b')
a.b = ab
sys.modules[a] = a
sys.modules[a.b] = ab
# notify
imp.notify_module_loaded('a.b')
Otherwise the code fails to fetch the module objects from sys.path and
an exception is raised.
1. notification is only done once for a given module, ever, even if the
notification function is called more than once, even if it's called
during notifications for that module
The latter is not yet implemented. notify_module('a.b') currently fails
with a recursion limit exceeded. It's on my todo list.
2. notifications for a child module/package may not begin until the
notifications for the parent package have begun
This is already implemented for notification by name but not for
notification by module object. It's also on my todo list.
3. two registrations for the same module must always be invoked in the
same order as they were registered, even if some of the registrations
are done during notification.
I'm pretty but not yet absolute sure that my implementation guarantees
the last invariant. I've to go through my code several times and play
through use cases.
In order to implement these invariants, you will have to have a way to
know whether notifications have been begun for a given module. In
peak.util.imports, the module objects effectively keep track of this,
although they don't have a specific flag. For the Python
implementation, you could add a __notified__ field to module objects,
and implement the notify function thus:
That's a nice idea to fix the recursion issue with nested notifications.
Of course, __notified__ would actually be a structure slot, rather than
an attribute, so as to avoid any attribute lookup issues with module
subtypes (e.g. lazy modules).
Sure, I can use a slot for PyModule and its subclasses. Unfortunately
other implementations of lazy imports are adding a proxy object which is
not a subclass of PyModule. I've to use an attribute if not
isinstance(mod, moduletype).
The register function would simply append a hook to the entry in
post_import_hooks if it's not None, or call the hook otherwise.
My code queues up new hooks while a sequence of hooks is processed. It
makes sure that hooks for a parent aren't called in the middle of a
child's hook chain.
notification_in_progress = False
queue = []
def register_hook(hook, name):
if notification_in_progress:
queue.append((hook, name))
return
hooks = sys.post_import_hook_register(name, None)
if hooks is None:
module = sys.modules.get(name, None)
if modules:
sys.post_import_hook_register[name] = None
hook(module)
else:
hooks = []
sys.post_import_hook_register[name] = hooks
hooks.append(hook)
def notify_loaded(mod_or_name):
notification_in_progress = True
try:
...
finally:
notification_in_progress = False
while queue:
hook, name = queue.pop(0)
hook(sys.modules[name])
With this implementation, I could make a version of peak.util.imports
that did its own lazy modules, but used the base system for all the hooks.
Very good!
Christian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: per user site-packages directory
Right now Python faces a lot of problems in the webhosting world because it's tedious to set up and maintain for the webhosting user since they often have to compile and install their own Python in their home directory. I don't understand why they *have* to do that. I can believe they do that as they don't know better - but why can't they use the Python interpreter already available on the system, and just install additional packages in their home directory? I think this is extremely important due to the proliferation of Python now more and more as a choice for webapp development. For that, I think the requirements need to be much more explicit. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
Oleg Broytmann writes:
~/.python
To me, this strongly suggests user configuration files, not a place
where an app can store user-specific packages.
True, there are apps that store their stuff in such places, like most
GNOME apps. But they have no user-servicable parts (including config
files) anyway.
I think both for UI reasons (given above) and for API reasons (given
by others) there should be a separate ~/SOMETHING/{bin,etc,lib,share}
hierarchy for user-specific packaged contents. I like ~/.local a
little better than ~/local, but both work for me.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
On Wed, Jan 16, 2008 at 06:31:42AM +0900, Stephen J. Turnbull wrote:
I think both for UI reasons (given above) and for API reasons (given
by others) there should be a separate ~/SOMETHING/{bin,etc,lib,share}
hierarchy for user-specific packaged contents. I like ~/.local a
little better than ~/local, but both work for me.
Having ~/.python allows me remove it with just one command. It's harder
to clear ~/.local/{bin,lib} without affecting whatever I want to preserve
there.
Oleg.
--
Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED]
Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Monkeypatching idioms -- elegant or ugly?
On Jan 15, 2008 1:27 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: Second, a metaclass to add a number of methods (or other attributes) to an existing class, using a convenient class notation: I think this is similar to my partial classes: http://pypi.python.org/pypi/partial Indeed it is. I guess my only innovation is realizing that you don't have to create a real metaclass -- you can set __metaclass__ to a function that does the magic. I like your feature of refusing overrides unless flagged with @replace. I think that despite the objection that monkeypatching shoudn't be made too easy, it's worth at looking into a unification of the API, features, and implementation. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Post import hooks
At 10:14 PM 1/15/2008 +0100, Christian Heimes wrote: My code queues up new hooks while a sequence of hooks is processed. It makes sure that hooks for a parent aren't called in the middle of a child's hook chain. Notice that that's not necessary with the notification algorithm I gave, since the list in post_import_hooks suffices as a queue. So, just as in peak.util.imports, the registration code doesn't need to know whether callbacks are being run; it only needs to know whether they're *finished*. Of course, both the notification and registration functions must hold the import lock to prevent a race condition where one thread adds a hook to the list after another thread has just finished iterating over it and is about to replace the list with None. At least, they have to if they're executing any Python code that might cause the GIL to be released. The callbacks will release the GIL, of course, but the registration code probably doesn't... well, it will if it calls the hook, and ISTM that the hooks should always execute with the import lock held, even if they're fired at registration. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Monkeypatching idioms -- elegant or ugly?
At 01:51 PM 1/15/2008 -0800, Guido van Rossum wrote: On Jan 15, 2008 1:27 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: Second, a metaclass to add a number of methods (or other attributes) to an existing class, using a convenient class notation: I think this is similar to my partial classes: http://pypi.python.org/pypi/partial Indeed it is. I guess my only innovation is realizing that you don't have to create a real metaclass -- you can set __metaclass__ to a function that does the magic. I like your feature of refusing overrides unless flagged with @replace. I think that despite the objection that monkeypatching shoudn't be made too easy, it's worth at looking into a unification of the API, features, and implementation. I'm curious: has this affected your thoughts re: overloading existing functions? Note that overloading-in-place would provide the next_method idiom for calling the original function. (I'm assuming you still don't like the idea of changing a function's code to do it, just wondering about the non-implementation aspect. :) ) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Monkeypatching idioms -- elegant or ugly?
On Jan 15, 2008 2:08 PM, Phillip J. Eby [EMAIL PROTECTED] wrote: At 01:51 PM 1/15/2008 -0800, Guido van Rossum wrote: On Jan 15, 2008 1:27 PM, Martin v. Löwis [EMAIL PROTECTED] wrote: Second, a metaclass to add a number of methods (or other attributes) to an existing class, using a convenient class notation: I think this is similar to my partial classes: http://pypi.python.org/pypi/partial Indeed it is. I guess my only innovation is realizing that you don't have to create a real metaclass -- you can set __metaclass__ to a function that does the magic. I like your feature of refusing overrides unless flagged with @replace. I think that despite the objection that monkeypatching shoudn't be made too easy, it's worth at looking into a unification of the API, features, and implementation. I'm curious: has this affected your thoughts re: overloading existing functions? Note that overloading-in-place would provide the next_method idiom for calling the original function. (I'm assuming you still don't like the idea of changing a function's code to do it, just wondering about the non-implementation aspect. :) ) Not really -- I'm still thinking of a class as something with public internal structure while I think of a function as something with private internal structure (except for function attributes settable in its __dict__). While I have you, I've come across a need that I don't know how to do with GFs. Suppose I have a GF that implements some recursive function over container types, e.g. serialization or flattening. Now suppose I'd like to create *another* GF that implements the same algorithm except it does something different for one particular type; as a concrete example, suppose we want to treat tuples atomically when flattening. Is there a way to reuse the work of the first GF? It doesn't work to create a new GF that calls on the first GF for types it doesn't understand; e.g. a list could contain a tuple. Does your GF machinery let me do this in a relatively clean way? (I think of it as subclassing a GF, where I think of the GF as a class that has a method for each supported type. If GFs were implemented this way, I could simply subclass it and override the method for tuples.) You might want to change the subject if you care to respond. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Monkeypatching idioms -- elegant or ugly?
Second, a metaclass to add a number of methods (or other attributes) to an existing class, using a convenient class notation: I think this is similar to my partial classes: http://pypi.python.org/pypi/partial Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Extending generic functions
At 02:19 PM 1/15/2008 -0800, Guido van Rossum wrote: While I have you, I've come across a need that I don't know how to do with GFs. Suppose I have a GF that implements some recursive function over container types, e.g. serialization or flattening. Now suppose I'd like to create *another* GF that implements the same algorithm except it does something different for one particular type; as a concrete example, suppose we want to treat tuples atomically when flattening. Is there a way to reuse the work of the first GF? Yes. RuleDispatch actually has a 'clone()' feature for single-dispatch generics that does exactly what you're looking for: http://peak.telecommunity.com/DevCenter/VisitorRevisited (see the heading Extension and Reuse). It's probably not a bad idea to put a cloning feature on my extended to-do list for PEAK-Rules. In PEAK-Rules (the system after which PEP 3124 was modelled), a generic function has a RuleSet that contains its rules, and RuleSets can be subscribed to. So, you could create a listener that automatically takes the rules added to one function and adds them to others. It's not packaged as a convenient decorator or anything, but one certainly could make one. It'd also need to have some way to ensure that the rules from the original function were treated at a lower combination precedence than anything else, but that could be handled by with a custom method type pretty easily, I think. All in all, a cloning feature might be somewhere around 20-50 lines of code to add in -- and a third party could probably roll their own without changing PEAK-Rules' source. It doesn't work to create a new GF that calls on the first GF for types it doesn't understand; e.g. a list could contain a tuple. Does your GF machinery let me do this in a relatively clean way? It's relatively clean. One of my goals for the changed architecture in PEAK-Rules vs. RuleDispatch was to make it possible to do all sorts of things like this, by opening up the whole thing to extensibility. Btw, a lot of the credit for PEAK-Rules' design goes to you, in a roundabout way. Your tuple-of-types prototype made me see that it could be practical to implement generic functions using generic functions as a base -- getting rid of interfaces and adaptation altogether. I just needed to come up with a design that allowed separating the genericness of a function (e.g. the rules to be applied) from the implementation of genericness (the engine that turns rules into executability. In this way, a generic function can start out using just tuples of types, but then graduate to a full expression-based system, just by changing the engine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
OlegWhy not use GNU stow? Thanks for the reference. I'd never heard of it before. I suspect our IT folks may not have as well. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
Oleg Broytmann writes:
On Wed, Jan 16, 2008 at 06:31:42AM +0900, Stephen J. Turnbull wrote:
I think both for UI reasons (given above) and for API reasons (given
by others) there should be a separate ~/SOMETHING/{bin,etc,lib,share}
hierarchy for user-specific packaged contents. I like ~/.local a
little better than ~/local, but both work for me.
Having ~/.python allows me remove it with just one command. It's harder
to clear ~/.local/{bin,lib} without affecting whatever I want to preserve
there.
Good point, but I prefer ~/Library/Python to either of these.
Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Post import hooks
Phillip J. Eby wrote:
At 10:14 PM 1/15/2008 +0100, Christian Heimes wrote:
My code queues up new hooks while a sequence of hooks is processed. It
makes sure that hooks for a parent aren't called in the middle of a
child's hook chain.
Notice that that's not necessary with the notification algorithm I gave,
since the list in post_import_hooks suffices as a queue. So, just as in
peak.util.imports, the registration code doesn't need to know whether
callbacks are being run; it only needs to know whether they're *finished*.
Are you sure your proposed algorithm and output match for the test case?
I'm confident I got it right in C but I'm getting a different output.
Without the extra imp.notify_module_loaded('a.b') in func_a1(mod)::
['func_a1', 'func_a2', 'func_ab1', 'func_ab2', 'func_ab3']
With the extra imp.notify_module_loaded('a.b') in func_a1(mod)::
['func_a1', 'func_ab1', 'func_ab2', 'func_ab3', 'func_a2']
I can't see how your implementation results in the first output when
func_a1() calls the notification method.
Of course, both the notification and registration functions must hold
the import lock to prevent a race condition where one thread adds a hook
to the list after another thread has just finished iterating over it and
is about to replace the list with None. At least, they have to if
they're executing any Python code that might cause the GIL to be
released. The callbacks will release the GIL, of course, but the
registration code probably doesn't... well, it will if it calls the
hook, and ISTM that the hooks should always execute with the import lock
held, even if they're fired at registration.
I'm aware of the implications and my code already uses the lock. The
PyImport_NotifyLoaded() method excepts to be called with the importer
lock acquired. So I'm locking the importer lock in
imp_notify_module_loaded(). The PyImport_RegisterPostImportHook() method
does the locking before it accesses sys.modules and sys.post_import_hooks.
Christian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
Bill Janssen wrote: Good point, but I prefer ~/Library/Python to either of these. ~/Library/ is a Mac OS X thing. I haven't seen it on other Unix systems. I *could* add yet another environment variable PYTHONUSERHOME to set the base path but I prefer not. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Post import hooks
At 02:28 AM 1/16/2008 +0100, Christian Heimes wrote:
Phillip J. Eby wrote:
At 10:14 PM 1/15/2008 +0100, Christian Heimes wrote:
My code queues up new hooks while a sequence of hooks is processed. It
makes sure that hooks for a parent aren't called in the middle of a
child's hook chain.
Notice that that's not necessary with the notification algorithm I gave,
since the list in post_import_hooks suffices as a queue. So, just as in
peak.util.imports, the registration code doesn't need to know whether
callbacks are being run; it only needs to know whether they're *finished*.
Are you sure your proposed algorithm and output match for the test case?
I'm confident I got it right in C but I'm getting a different output.
I guess it's not right then. ;-) Though I shouldn't make fun, since
it turns out that my code sketch was not a correct translation of
peak.util.imports. (See below.)
Without the extra imp.notify_module_loaded('a.b') in func_a1(mod)::
['func_a1', 'func_a2', 'func_ab1', 'func_ab2', 'func_ab3']
With the extra imp.notify_module_loaded('a.b') in func_a1(mod)::
['func_a1', 'func_ab1', 'func_ab2', 'func_ab3', 'func_a2']
Right - that's why I put it in there, to foil trivial implementations
that don't really satisfy the invariant.
I can't see how your implementation results in the first output when
func_a1() calls the notification method.
Hm, you're right, my implementation sketch waits too long to set the
__notified__ flag. It should have read:
def notify(name):
try:
module = sys.modules[name]
except KeyError:
raise ImportError(Module %s has not been imported % (name,))
if module.__notified__:
return
try:
module.__notified__ = True
if '.' in name:
notify(name[:name.rfind('.')])
for callback in post_import_hooks[name]:
callback(module)
finally:
post_import_hooks[name] = None
That is, module.__notified__ has to be set *before* the recursive
notification call. This effectively happens in peak.util.imports
now, except that __notified__ isn't an explicit attribute, just a
side effect of other module state changes.
I'm aware of the implications and my code already uses the lock. The
PyImport_NotifyLoaded() method excepts to be called with the importer
lock acquired. So I'm locking the importer lock in
imp_notify_module_loaded(). The PyImport_RegisterPostImportHook() method
does the locking before it accesses sys.modules and sys.post_import_hooks.
Great!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
Bill Janssen wrote: Good point, but I prefer ~/Library/Python to either of these. ~/Library/ is a Mac OS X thing. I haven't seen it on other Unix systems. I *could* add yet another environment variable PYTHONUSERHOME to set the base path but I prefer not. Christian Sure, but it's clearly where this should be on an OS X system, by default. And I'm sure there's a different best place on Windows (for instance, all of our accounts are network roaming accounts, and you don't want to put anything in ~). And there are probably various right places for various flavors of Linux. Any PEP on this that doesn't take these OS-specific differences into account probably isn't worth reading, IMO. Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 54, Issue 57
Bill Janssen wrote: Sure, but it's clearly where this should be on an OS X system, by default. And I'm sure there's a different best place on Windows (for instance, all of our accounts are network roaming accounts, and you don't want to put anything in ~). And there are probably various right places for various flavors of Linux. Any PEP on this that doesn't take these OS-specific differences into account probably isn't worth reading, IMO. Your comment doesn't help at all. In fact my proposal uses different best places for Windows (%APPDATA%), Mac (~/Library/Python) and Unix (~/.local). I can't comment on OS2, Ricos and VAX because I've zero experience with the platforms. The right place for a platform is the place we decide is right. As long as the directory is somewhere inside the home directory of a user and the directory doesn't show up in the user data folder (hence APPDATA on Windows and a dot directory on Unix) I consider any sensible directory as the right one. Do you think APPDATA. ~/Library/Python or ~/.local are not sensible? Please stop bitching and bring in some useful, constructive criticism and use cases not covered by my PEP. Have you read the PEP at all? Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Post import hooks
At 04:40 AM 1/16/2008 +0100, Christian Heimes wrote: Phillip J. Eby wrote: I guess it's not right then. ;-) Though I shouldn't make fun, since it turns out that my code sketch was not a correct translation of peak.util.imports. (See below.) *gr* I spent more than hour to find my error ... Sorry about that - as I said, __notified__ is very much an implicit thing in peak.util.imports. And I believe I've also mentioned a lot of times how hard it is to get this stuff right... :) That is, module.__notified__ has to be set *before* the recursive notification call. This effectively happens in peak.util.imports now, except that __notified__ isn't an explicit attribute, just a side effect of other module state changes. It's done. Your proposed test cases passes together with my tests. The ref leak tests don't show a single missing reference. Congrats! Now all we need to do is get the authors of other lazy import/export/whatever systems to chime in with whatever additional invariants *they* might need... ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Post import hooks
Phillip J. Eby wrote: Sorry about that - as I said, __notified__ is very much an implicit thing in peak.util.imports. And I believe I've also mentioned a lot of times how hard it is to get this stuff right... :) *hehe* Indeed, you did! I should have been warned. :) Congrats! Now all we need to do is get the authors of other lazy import/export/whatever systems to chime in with whatever additional invariants *they* might need... ;-) Oh, no! You are kidding me ... But yes that's all we need to do. And I've to update the patch and explain why the implementation does all the crazy stuff. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
