Re: [9fans] current state of thread programming
Roman V. Shaposhnik writes: If we were to oversimplify things [then the] brain is, at its core, limited by a very fundamental biological constraint: speed at which cells can communicate. A sort of propagation delay if we were to use electronics as an analogy. It seems to be agreed upon(*) that we can safely assume this constraint to limit our brain to about couple of hundred of processing steps per second. This is known as a 100 steps rule. Something is really, really wrong with the computing model we base our technology on, if even the slowest of the computers we can consider useful required a clock rate of KHz. Either that or (like some brain scientists say) something is really, really wrong or suboptimal about the human brain.
Re: [9fans] current state of thread programming
On Tue, Jul 29, 2008 at 12:12:05PM -0700, Bakul Shah wrote: It is slightly depressing to think that the situation has not really changed since EWD wrote this in 1975. It will take some young whippersnapper of a Dijkstra or Hoare or Strachey or Iverson or Backus to find the critical insight that will make reasoning about parallel algorithm no more difficult than sequential ones. Is the human thought process parallel? For _my capacities_, I have the impression that I'm more multitask than parallel. And context switch is expensive because there is not only explicit data, but also implicit and I'm not able, if I'm really doing something involved, to restore the previous state without much ado. CSP is (for me) the best answer to problem involving blocking/waiting on input. But this is not parallelism. And for processing, finally threads are also called in some implementations: LWP, that is simply something that could have been solved with Processes, if it was not so costly. I have the impression that LWP is just a solution to poor process creation and to poor IPC tools (threads are the solution not for parallelism by itself, but because there is need to share resources between processes and that it is simplest to put them in the same address space). The most efficient is to have tools that match the way our brains work (or not...). I'm not convinced our brains are parallel (at least mines are not). -- Thierry Laronde (Alceste) tlaronde +AT+ polynum +dot+ com http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C
Re: [9fans] current state of thread programming
On Wed, 2008-07-30 at 13:35 +0200, [EMAIL PROTECTED] wrote: On Tue, Jul 29, 2008 at 12:12:05PM -0700, Bakul Shah wrote: It is slightly depressing to think that the situation has not really changed since EWD wrote this in 1975. It will take some young whippersnapper of a Dijkstra or Hoare or Strachey or Iverson or Backus to find the critical insight that will make reasoning about parallel algorithm no more difficult than sequential ones. Is the human thought process parallel? No. But to give you an example of why that shouldn't matter I would like to note that the human thought is, in my opinion, finite. Yet, we have developed very nice and efficient tools for comprehending and reasoning about infinity. The most efficient is to have tools that match the way our brains work (or not...). I'm not convinced our brains are parallel (at least mines are not). I disagree on philosophical grounds ;-) It's been one of the major engineering follies to always approach design from a just follow the nature standpoint. No wonder that before the Wright brothers everybody thought the best way to fly is to flap some kind of wings. Thanks, Roman. P.S. I guess, we are getting way off topic here.
Re: [9fans] current state of thread programming
I think useful parallel programming paradigms can very probably be abstracted from really big systems like a national health system or an army. How parallelism is employed in those systems, would be a good starting point for a deeper investigation. Especially a military system must have some very concrete, well tried and tested, ways of organising things in parallel. Government is another one, but I'm not sure if that's a good model ;-) Robby
Re: [9fans] current state of thread programming
On Wed, Jul 30, 2008 at 4:35 AM, [EMAIL PROTECTED] wrote: On Tue, Jul 29, 2008 at 12:12:05PM -0700, Bakul Shah wrote: It is slightly depressing to think that the situation has not really changed since EWD wrote this in 1975. It will take some young whippersnapper of a Dijkstra or Hoare or Strachey or Iverson or Backus to find the critical insight that will make reasoning about parallel algorithm no more difficult than sequential ones. Is the human thought process parallel? For _my capacities_, I have the impression that I'm more multitask than parallel. And context switch is expensive because there is not only explicit data, but also implicit and I'm not able, if I'm really doing something involved, to restore the previous state without much ado. CSP is (for me) the best answer to problem involving blocking/waiting on input. But this is not parallelism. It's not pure parallelism, no, but CSP arranged tasks may execute in parallel :-) Imagine if you had a CSP-style process that grabs data in chunks from a network, and then passes a message to another helper which does some filtering. The filtering process could be scheduled on another core/CPU, while the other task it getting I/O from the network and massaging it in other ways. It's not exactly CSP, but I wrote a prime sieve using communication channels in Haskell (typed STM based channels) and I got a pretty good boost by telling the scheduler there was more than one CPU. http://hpaste.org/7766 (was my first try with this kind of haskell, so it's probably not that pretty). And for processing, finally threads are also called in some implementations: LWP, that is simply something that could have been solved with Processes, if it was not so costly. I have the impression that LWP is just a solution to poor process creation and to poor IPC tools (threads are the solution not for parallelism by itself, but because there is need to share resources between processes and that it is simplest to put them in the same address space). The most efficient is to have tools that match the way our brains work (or not...). I'm not convinced our brains are parallel (at least mines are not). What Joe Armstrong (of Erlang) would tell you (I think) is that we operate concurrently, and we communicate with each other via message passing, and that we manage to get work done in parallel this way, so why not manage parallel applications in this same system model? If you have lots of cores, and you wrote your application in a concurrent manner, in theory, many of them *could* run in parallel, but obviously this makes it sound more like art than science :-) One must know when a given process depends on the results of others to show where things are really not in parallel but just compositions of serial operations. -- Thierry Laronde (Alceste) tlaronde +AT+ polynum +dot+ com http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C
Re: [9fans] current state of thread programming
On Wed, Jul 30, 2008 at 13:50, Roman V. Shaposhnik [EMAIL PROTECTED] wrote: On Wed, 2008-07-30 at 13:35 +0200, [EMAIL PROTECTED] wrote: The most efficient is to have tools that match the way our brains work (or not...). I'm not convinced our brains are parallel (at least mines are not). I disagree on philosophical grounds ;-) It's been one of the major engineering follies to always approach design from a just follow the nature standpoint. No wonder that before the Wright brothers everybody thought the best way to fly is to flap some kind of wings. I'd disagree with this example, as it is certainly not a viable one. First, the main thing that Wright brothers have created were control surfaces and a lighweight plane - people had been flying for few years before them, but nobody could get a light enough engine :) And flapping wings wasn't an error - the error was how it was implemented. And if you take a good look at bigger birds, you'll see that in their case, flapping is a reuse of equipment - the principles of flight are the same. Oh, and BTW, there's a lot of development for ornitopters. Da Vinci simply didn't have the tools (of which one of the most important is computer. With _lots_ of computing power...) P.S. I guess, we are getting way off topic here. Sorry for continuing with OT. -- Paweł Lasek
Re: [9fans] current state of thread programming
Is the human thought process parallel? For _my capacities_, I have the impression that I'm more multitask than parallel. And context switch is expensive because there is not only explicit data, but also implicit and I'm not able, if I'm really doing something involved, to restore the previous state without much ado. I, for one, think that this analogy is incorrect. It is your perception of what the brain is doing (higher order functions) that appears to be sequential, however, underneath it all, the brain is being highly parallel at doing all the functions that keep us alive: basic motor functions such as breathing, eyesight (which in itself is a highly parallel endeavour), hunger, circadian rhythms if you will. Even the neocortex is parallel: you can easily talk while, say, navigating the corridors of a building. The fact that most tasks that require extreme concentration (and using several of our neocortex functions in a coordinated fashion to solve a single problem) appear, to us, to be dealt with by 'multitasking', may simply be an evolutionary shortcoming to be fixed in later generations. Something like being able to SMS at 40 words per minute while headbanging at a concert :) So, in essence, I think the brain may be exactly what Dijkstra had in mind in the earlier quote: its workings are so implicitly parallel that the word 'parallel' carries no meaning when describing it.
Re: [9fans] current state of thread programming
On Wed, Jul 30, 2008 at 08:50:28AM -0400, erik quanstrom wrote: i don't see how csp is *not* parallel processing. as soon as you have more than 1 work process per client, i would call that parallel processing. It's a kind of parallelism, of course. But since it makes sense, it is not parallelism as the trend is today ;) [..] one could argue that isn't *really* parallel because the requests for the n blocks are made sequentially. but since drives, operating on the timescale of tens of milliseconds are competing with a processor working on a nanosecond timescale. that's effectively all at once. This is precisely the point: 1) It is not pure parallelism or parallelism du jour. But it is parallelism too. 2) Developers have not waited for the trend to make or use parallelism. 3) An algorithm is, for me, intrinsically sequential (the elementary unit is sequential). Parallelism is more at a higher level: methodology, engineering, the same level as, say, structural programming. -- Thierry Laronde (Alceste) tlaronde +AT+ polynum +dot+ com http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C
Re: [9fans] current state of thread programming
I disagree on philosophical grounds ;-) It's been one of the major engineering follies to always approach design from a just follow the nature standpoint. No wonder that before the Wright brothers everybody thought the best way to fly is to flap some kind of wings. off topic, but to note: biomimetics are making a comeback http://ngm.nationalgeographic.com/2008/04/biomimetics/tom-mueller-text/1
[9fans] current state of thread programming
found this snippet today and decided to share it with the list. every once in a while a look at how the rest of the world does things is beneficial :) I don't know about you, but every time I have to program with threads and shared resources, I want to remove my face incrementally with a salad fork. Locks, mutexes, the synchronized keyword; all of these things can strike fear into the heart of a green developer. Most seasoned developers just fall into a rut of depression when it's time for multi-threading. Developers like me simply talk our way out of it. It's easier than thinking. full article: http://www.theregister.co.uk/2008/07/28/sun_dziuba_tm/
Re: [9fans] current state of thread programming
On Mon, Jul 28, 2008 at 11:11:19AM -0600, andrey mirtchovski wrote: found this snippet today and decided to share it with the list. every once in a while a look at how the rest of the world does things is beneficial :) I don't know about you, but every time I have to program with threads and shared resources, I want to remove my face incrementally with a salad fork. Locks, mutexes, the synchronized keyword; all of these things can strike fear into the heart of a green developer. Most seasoned developers just fall into a rut of depression when it's time for multi-threading. Developers like me simply talk our way out of it. It's easier than thinking. On the same subject, this quote from Donald E. Knuth, Volume 4 fascicle 0 (new addition to The Art of Computer Programming, published in may 2008)---Preface: Furthermore, as in earlier volumes of this serie, I'm intentionnally concentrating almost entirely on _sequential_ algorithms, even though computers are increasingly able to carry out activities in parallel. I'm unable to judge what ideas about parallelism are likely to be useful five or ten years from now, let alone fifty, so I happily leave such questions to others who are wiser than I. Sequential methods, by themselves, already test the limits of my own ability to discern what the artful programmers of tomorrow will want to know. And as an illustration of the fun, the ongoing discussion on NetBSD kernel mailing list _between_ 1:1 and SA threading models (when the person working on SA revival proposes vel : 1:1 or/and SA ---pickup the one you want or need for backward compatibility), discussion with an amount of technical arguments of 5% or less. -- Thierry Laronde (Alceste) tlaronde +AT+ polynum +dot+ com http://www.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C
Re: [9fans] current state of thread programming
On Jul 28, 2008, at 1:11 PM, andrey mirtchovski wrote: salad fork. Locks, mutexes, the synchronized keyword; all of these things can strike fear into the heart of a green developer. Most That's what you get for using Java. On Jul 28, 2008, at 1:50 PM, [EMAIL PROTECTED] wrote: I'm unable to judge what ideas about parallelism are likely to be useful five or ten years from now, let alone fifty, By that time, $50 (not $500 or $5,000, but $50) computers will be around that will have processors with as many cores as Blue Gene, thus almost completely eliminating the problems of multithreaded programming. Data synchronization will be solved by simply turning off one of the processors until the critical code has completed. If only I could tell him that without having to wait for the snail!
Re: [9fans] current state of thread programming
Don Knuth: I'm unable to judge what ideas about parallelism are likely to be useful five or ten years from now, let alone fifty, so I happily leave such questions to others who are wiser than I. Pietro: By that time, ... If only I could tell him that without having to wait for the snail! I can't think of anything nice to say. Russ
Re: [9fans] current state of thread programming
Don Knuth: I'm unable to judge what ideas about parallelism are likely to be useful five or ten years from now, let alone fifty, so I happily leave such questions to others who are wiser than I. Pietro: By that time, ... If only I could tell him that without having to wait for the snail! I can't think of anything nice to say. Russ Pietro has a great future in talk radio.
