Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-04-05 Thread Charles-François Natali
Hello, async.submit_work(func, args, kwds, callback=None, errback=None) How do you implement arguments passing and return value? e.g. let's say I pass a list as argument: how do you iterate on the list from the worker thread without modifying the backing objects for refcounts (IIUC you use

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-04-05 Thread Trent Nelson
On Thu, Apr 04, 2013 at 11:53:01PM -0700, Charles-Fran?ois Natali wrote: Hello, async.submit_work(func, args, kwds, callback=None, errback=None) How do you implement arguments passing and return value? e.g. let's say I pass a list as argument: how do you iterate on the list from the

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-04-04 Thread Charles-François Natali
Just a quick implementation question (didn't have time to read through all your emails :-) async.submit_work(func, args, kwds, callback=None, errback=None) How do you implement arguments passing and return value? e.g. let's say I pass a list as argument: how do you iterate on the list from the

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-04-04 Thread Trent Nelson
Hi Charles-François, On Thu, Apr 04, 2013 at 01:18:58AM -0700, Charles-François Natali wrote: Just a quick implementation question (didn't have time to read through all your emails :-) async.submit_work(func, args, kwds, callback=None, errback=None) How do you implement arguments passing

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-21 Thread Sturla Molden
Den 14. mars 2013 kl. 23:23 skrev Trent Nelson tr...@snakebite.org: For the record, here are all the Windows calls I'm using that have no *direct* POSIX equivalent: Interlocked singly-linked lists: - InitializeSListHead() - InterlockedFlushSList()

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-21 Thread Antoine Pitrou
Le Thu, 14 Mar 2013 15:23:37 -0700, Trent Nelson tr...@snakebite.org a écrit : Don't get me wrong, I grew up with UNIX and love it as much as the next guy, but you can't deny the usefulness of Windows' facilities for writing high-performance, multi-threaded IO code. It's decades

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-21 Thread Baptiste Lepilleur
2013/3/15 Trent Nelson tr...@snakebite.org On Thu, Mar 14, 2013 at 03:50:27PM -0700, Martin v. Löwis wrote: Am 14.03.13 12:59, schrieb Stefan Ring: I think you should be able to just take the address of a static __thread variable to achieve the same thing in a more portable way. That

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-21 Thread Trent Nelson
That's good to hear :-) (It's a fantastic facility, I couldn't imagine having to go back to manual TLS API stuff after using __thread/__declspec(thread).) This e-mail was sent from a wireless device. On 21 Mar 2013, at 09:30, Baptiste Lepilleur

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-21 Thread Trent Nelson
No, I haven't. I'd lose the excellent Windows pairing of thread pool IO and overlapped IO facilities if I did that. Not saying it isn't an option down the track for the generic submit work API though; that stuff will work against any thread pool without too much effort. But for now, the fact

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-21 Thread Trent Nelson
http://c2.com/cgi/wiki?BlubParadox ;-) Sent from my iPhone On 21 Mar 2013, at 06:18, Antoine Pitrou solip...@pitrou.net wrote: Le Thu, 14 Mar 2013 15:23:37 -0700, Trent Nelson tr...@snakebite.org a écrit : Don't get me wrong, I grew up with UNIX and love it as much as the next guy,

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-18 Thread Christian Tismer
Hi Trent, I just started to try to understand the idea and the implications. Removing almost all of your message since that is already too long to work with: The reference is http://mail.python.org/pipermail/python-dev/2013-March/124690.html On 3/14/13 11:45 AM, Trent Nelson wrote: On Wed,

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-18 Thread Trent Nelson
On Mon, Mar 18, 2013 at 05:27:33PM -0700, Christian Tismer wrote: Hi Trent, Hi Christian! Thanks for taking the time to read my walls of text ;-) So, the remaining challenge is preventing the use case alluded to earlier where someone tries to modify an object that hasn't been async

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-15 Thread Antoine Pitrou
On Thu, 14 Mar 2013 16:21:14 -0700 Trent Nelson tr...@snakebite.org wrote: Actually, what's really interesting is the new registered IO facilities in Windows 8/2012. The Microsoft recommendation for achieving the ultimate performance (least amount of jitter, lowest latency,

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-15 Thread Martin v. Löwis
Am 15.03.13 00:19, schrieb Antoine Pitrou: Does Microsoft change their recommendations every couple of years? :) Indeed they do. In fact, it's not really the recommendation that changes, but APIs that are added to new Windows releases. In the specific case, Windows 8 adds an API called

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Christian Heimes
Am 14.03.2013 03:05, schrieb Trent Nelson: Just posted the slides for those that didn't have the benefit of attending the language summit today: https://speakerdeck.com/trent/parallelizing-the-python-interpreter-an-alternate-approach-to-async Wow, neat! Your idea with

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Antoine Pitrou
Le Thu, 14 Mar 2013 13:21:09 +0100, Christian Heimes christ...@python.org a écrit : IMHO you should target x86, x86_64, ARMv6 and ARMv7. ARMv7 is going to be more important than x86 in the future. We are going to see more ARM based servers. Well we can't really see less of them, since there

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread a . cavallo
By the way on the arm (and any platform that can do cross-compiling) I've created a Makefile based build of the python 2.7.x: https://bitbucket.org/cavallo71/android Please don't be fooled by the Android name, it really can take any crosscompiler (provided it follows the gcc synatx). It

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Trent Nelson
On Thu, Mar 14, 2013 at 05:21:09AM -0700, Christian Heimes wrote: Am 14.03.2013 03:05, schrieb Trent Nelson: Just posted the slides for those that didn't have the benefit of attending the language summit today:

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Trent Nelson
On Wed, Mar 13, 2013 at 07:05:41PM -0700, Trent Nelson wrote: Just posted the slides for those that didn't have the benefit of attending the language summit today: https://speakerdeck.com/trent/parallelizing-the-python-interpreter-an-alternate-approach-to-async Someone

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Stefan Ring
Yup, in fact, if I hadn't come up with the __read[gf]sword() trick, my only other option would have been TLS (or the GetCurrentThreadId /pthread_self() approach in the presentation). TLS is fantastic, and it's definitely an intrinsic part of the solution (the Y part of if

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Trent Nelson
Cross-referenced to relevant bits of code where appropriate. (And just a quick reminder regarding the code quality disclaimer: I've been hacking away on this stuff relentlessly for a few months; the aim has been to make continual forward progress without getting bogged down in

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Trent Nelson
On Thu, Mar 14, 2013 at 02:30:14PM -0700, Trent Nelson wrote: Then it dawned on me to just add the snapshot/rollback stuff to normal Context objects. In retrospect, it's silly I didn't think of this in the first place -- the biggest advantage of the Context abstraction is that

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Trent Nelson
On Thu, Mar 14, 2013 at 12:59:57PM -0700, Stefan Ring wrote: Yup, in fact, if I hadn't come up with the __read[gf]sword() trick, my only other option would have been TLS (or the GetCurrentThreadId /pthread_self() approach in the presentation). TLS is fantastic, and it's

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Martin v. Löwis
Am 14.03.13 11:23, schrieb Trent Nelson: ARM CPUs don't have segment registers because they have a simpler addressing model. The register CP15 came up after a couple of Google searches. Noted, thanks! Yeah that's my general sentiment too. I'm definitely curious to see if other

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Trent Nelson
On Thu, Mar 14, 2013 at 03:50:27PM -0700, Martin v. Löwis wrote: Am 14.03.13 12:59, schrieb Stefan Ring: I think you should be able to just take the address of a static __thread variable to achieve the same thing in a more portable way. That assumes that the compiler supports __thread

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Martin v. Löwis
Am 14.03.13 12:59, schrieb Stefan Ring: I think you should be able to just take the address of a static __thread variable to achieve the same thing in a more portable way. That assumes that the compiler supports __thread variables, which isn't that portable in the first place. Regards, Martin

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-03-14 Thread Trent Nelson
On Thu, Mar 14, 2013 at 03:56:33PM -0700, Martin v. Löwis wrote: Am 14.03.13 11:23, schrieb Trent Nelson: Porting the Py_PXCTX part is trivial compared to the work that is going to be required to get this stuff working on POSIX where none of the sublime Windows concurrency,

[Python-Dev] Slides from today's parallel/async Python talk

2013-03-13 Thread Trent Nelson
Just posted the slides for those that didn't have the benefit of attending the language summit today: https://speakerdeck.com/trent/parallelizing-the-python-interpreter-an-alternate-approach-to-async Trent. ___ Python-Dev