Re: [Python-Dev] deja-vu .. python locking
Greg Ewing wrote: Martin Devera wrote: As I've written in Big reader lock paragraph of the original proposal, these objects could be handled by not blocking in read path But as was just pointed out, because of refcounting, there's really no such thing as read-only access to an object. What *looks* like read-only access at the Python level involves refcount updates just from the act of touching the object. Yes I was thinking about atomic inc/dec (locked inc/dec in x86) as used in G.Stein's patch. I have to admit that I haven't measured its performance, I was hoping for decent one. But from http://www.linuxjournal.com/article/6993 it seems that atomic inc is rather expensive too (75ns on 1.8GHz P4) :-( Greg, what change do you have in mind regarding that 3 instruction addition to refcounting ? thanks, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
Ah, I think I understand now. First the minor critique: I believe
the locking algorithm isn't thread-safe:
while (ob-owner_thread != self_thread()) {
unlock_mutex(thread_mutex[self_thread()])
// wait for owning thread to go to quiscent state
lock_mutex(thread_mutex[ob-owner_thread])
ob-owner_thread = self_thread()
unlock_mutex(thread_mutex[ob-owner_thread])
lock_mutex(thread_mutex[self_thread()])
}
If two threads are competing for the same object held by a third
thread, they may simultaneously enter the while loop, and then
simultaneously try to lock the owner_thread. Now, one will win,
and own the object. Later, the other will gain the lock, and
unconditionally overwrite ownership. This will cause two threads
to own the objects, which is an error.
oops .. well it seems as very stupid error on my side. Yes you are
absolutely right, I'll have to rethink it. I hope it is possible
to do it in correct way...
The more fundamental critique is: Why? It seems you do this
to improve efficiency, (implicitly) claiming that it is
more efficient to keep holding the lock, instead of releasing
and re-acquiring it each time.
I claim that this doesn't really matter: any reasonable
mutex implementation will be fast if there is no lock
contention. On locking, it will not invoke any system
call if the lock is currently not held (but just
atomically test-and-set some field of the lock); on
unlocking, it will not invoke any system call if
the wait list is empty. As you also need to test, there
shouldn't be much of a performance difference.
I measured it. Lock op in futex based linux locking is of the same
speed as windows critical section and it is about 30 cycles on my
P4 1.8GHz in uncontented case.
As explained in already mentioned http://www.linuxjournal.com/article/6993
it seems due to pipeline flush during cmpxchg insn.
And there will be cacheline transfer penalty which is much larger. So
that mutex locking will take time comparable with protected code itself
(assuming fast code like dict/list read).
Single compare will take ten times less.
Am I missing something ?
thanks, Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New relative import issue
On Tue, Sep 19, 2006 at 03:46:59PM +1200, Greg Ewing wrote: There really shouldn't be any such thing as sys.path -- the view that any given module has of the package namespace should depend only on where it is I do not understand this. Can you show an example? Imagine I have two servers, Linux and FreeBSD, and on Linux python is in /usr/bin, home is /home/phd, on BSD these are /usr/local/bin and /usr/home/phd. I have some modules in site-packages and some modules in $HOME/lib/python. How can I move programs from one server to the other without rewriting them (how can I not to put full paths to modules)? I use PYTHONPATH manipulation - its enough to write a shell script that starts daemons once and use it for many years. How can I do this without sys.path?! Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New relative import issue
Josiah Carlson wrote: As it stands, in order to work around this particular feature, one would need to write a 'loader' to handle importing and/or main() calling in subpackage1/module1.py . Yup. At the moment, you can rely on PEP 328, or an PEP 338, but not both at the same time. This was previously discussed back in June/July with Anthony convincing me that the solution to the current poor interaction shouldn't be rushed [1]. It is, however, pretty trivial to write a runpy.run_module based launcher that will execute your module and use something other than __name__ == '__main__' to indicate that the module is the main module. By letting run_module set __name__ normally, relative imports will just work. For example: #mypkg/launch.py # Runs a script, using the global _launched to indicate whether or not # the module is the main module if _launched not in globals(): _launched = False if (__name__ == __main__) or _launched: import runpy # Run the module specified as the next command line argument if len(sys.argv) 2: print sys.stderr, No module specified for execution else: del sys.argv[0] # Make the requested module sys.argv[0] run_module(sys.argv[0], init_globals=dict(_launched=True), alter_sys=True) Cheers, Nick. [1] http://mail.python.org/pipermail/python-dev/2006-July/067077.html -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New relative import issue
Greg Ewing wrote: Armin Rigo wrote: My (limited) understanding of the motivation for relative imports is that they are only here as a transitional feature. Fully-absolute imports are the official future. Guido does seem to have a dislike for relative imports, but I don't really understand why. The usefulness of being able to make a package self-contained and movable to another place in the package hierarchy without hacking it seems self-evident to me. What's happening in Py3k? Will relative imports still exist? there is no clean way from a test module 'foo.bar.test.test_hello' to import 'foo.bar.hello': the top-level directory must first be inserted into sys.path magically. I've felt for a long time that problems like this wouldn't arise so much if there were a closer connection between the package hierarchy and the file system structure. There really shouldn't be any such thing as sys.path -- the view that any given module has of the package namespace should depend only on where it is, not on the history of how it came to be invoked. This does, of course, assume that you're importing modules from the filestore, which assumption is no longer valid in the presence of PEP 302 importers. The current initialization code actually looks for os.py as a means of establishing path elements. This should really be better integrated with the PEP 302 mechanism: ideally Python should work on systems that don't rely on filestore for import (even though for the foreseeable future all systems will continue to do this). regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] RELEASED Python 2.5 (FINAL)
It's been nearly 20 months since the last major release of Python (2.4), and 5 months since the first alpha release of this cycle, so I'm absolutely thrilled to be able to say: On behalf of the Python development team and the Python community, I'm happy to announce the FINAL release of Python 2.5. This is a *production* release of Python 2.5. Yes, that's right, it's finally here. Python 2.5 is probably the most significant new release of Python since 2.2, way back in the dark ages of 2001. There's been a wide variety of changes and additions, both user-visible and underneath the hood. In addition, we've switched to SVN for development and now use Buildbot to do continuous testing of the Python codebase. Much more information (as well as source distributions and Windows and Universal Mac OSX installers) are available from the 2.5 website: http://www.python.org/2.5/ The new features in Python 2.5 are described in Andrew Kuchling's What's New In Python 2.5. It's available from the 2.5 web page. Amongst the new features of Python 2.5 are conditional expressions, the with statement, the merge of try/except and try/finally into try/except/finally, enhancements to generators to produce coroutine functionality, and a brand new AST-based compiler implementation underneath the hood. There's a variety of smaller new features as well. New to the standard library are hashlib, ElementTree, sqlite3, wsgiref, uuid and ctypes. As well, a new higher-performance profiling module (cProfile) was added. Extra-special thanks on behalf of the entire Python community should go out to Neal Norwitz, who's done absolutely sterling work in shepherding Python 2.5 through to it's final release. Enjoy this new release, (and Woo-HOO! It's done!) Anthony Anthony Baxter [EMAIL PROTECTED] Python Release Manager (on behalf of the entire python-dev team) pgpA8ImA53iae.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] release25-maint branch - please keep frozen for a day or two more.
Could people please treat the release25-maint branch as frozen for a day or two, just in case we have to cut an ohmygodnononokillme release? Thanks, Anthony -- Anthony Baxter [EMAIL PROTECTED] It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] release25-maint branch - please keep frozen for a day or two more.
Anthony Baxter wrote: Could people please treat the release25-maint branch as frozen for a day or two, just in case we have to cut an ohmygodnononokillme release? Thanks, Otherwise to be known as 2.5.005? regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Download URL typo
Hiho, in case noone didn't notice yet: the Windows MSI Installer link at http://www.python.org/download/releases/2.5/ points to Python 2.4! Regards, Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Download URL typo
Michael Walter schrieb: in case noone didn't notice yet: the Windows MSI Installer link at http://www.python.org/download/releases/2.5/ points to Python 2.4! Why is this a problem? The link is actually correct: The MSI documentation is the same. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Download URL typo
Martin v. Löwis schrieb: Michael Walter schrieb: in case noone didn't notice yet: the Windows MSI Installer link at http://www.python.org/download/releases/2.5/ points to Python 2.4! Why is this a problem? The link is actually correct: The MSI documentation is the same. I reconsidered. Even though the documentation was nearly correct (except that one limitation went away long ago), it's probably better to have the documentation state 2.5 throughout. So I copied it, changed the version numbers, and changed the links to refer to the copy. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deja-vu .. python locking
Martin Devera wrote: Greg, what change do you have in mind regarding that 3 instruction addition to refcounting ? I don't have any change in mind. If even an atomic inc is too expensive, it seems there's no hope for us. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New relative import issue
Steve Holden wrote: This does, of course, assume that you're importing modules from the filestore, which assumption is no longer valid in the presence of PEP 302 importers. Well, you need to allow for a sufficiently abstract notion of filesystem. I haven't really thought it through in detail. It just seems as though it would be a lot less confusing if you could figure out from static information which module will get imported by a given import statement, instead of having it depend on the history of run-time modifications to sys.path. One such kind of static information is the layout of the filesystem. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New relative import issue
Greg Ewing wrote: Steve Holden wrote: This does, of course, assume that you're importing modules from the filestore, which assumption is no longer valid in the presence of PEP 302 importers. Well, you need to allow for a sufficiently abstract notion of filesystem. For some value of sufficiently ... I haven't really thought it through in detail. It just seems as though it would be a lot less confusing if you could figure out from static information which module will get imported by a given import statement, instead of having it depend on the history of run-time modifications to sys.path. One such kind of static information is the layout of the filesystem. Less confusing, but sadly also less realistic. I suspect what's really needed is *more* importer behavior rather than less but, like you, I haven't yet thought it through in detail. All I *can* tell you is once you start importing modules for a database the whole import mechanism starts to look a bit under-specified an over-complicated. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Download URL typo
Martin v. Löwis wrote: Martin v. Löwis schrieb: Michael Walter schrieb: in case noone didn't notice yet: the Windows MSI Installer link at http://www.python.org/download/releases/2.5/ points to Python 2.4! Why is this a problem? The link is actually correct: The MSI documentation is the same. I reconsidered. Even though the documentation was nearly correct (except that one limitation went away long ago), it's probably better to have the documentation state 2.5 throughout. So I copied it, changed the version numbers, and changed the links to refer to the copy. As I write the situation is an ugly mess, since the most visible link is just plain wrong. The page http://www.python.org/download/releases/2.5/ has a block at the top right whose last link is Windows MSI installer. That links to http://www.python.org/download/releases/2.5/msi/ which *also* has a block at the top right whose last link is Windows MSI installer. Unfortunately that takes you to http://www.python.org/download/releases/2.5/msi/msi by which time you have completely lost contact with any style sheet, and despite the potential infinite regress have still not located the actual installer. The correct link is in-line: http://www.python.org/download/releases/2.5/python-2.5.msi I think the next time we redesign the web production system we should take the release managers' needs into consideration. They should have a simple form to fill in, with defaults already provided. As indeed should many other people ... regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
