forks and threads (was: Re: META.yml how to declare the need for threaded perl?)
Christopher Brown schreef: I spent some time thinking about Dr. Ruud's reply over the past several days. Although I mostly share his views, I think that he has done a disservice to the discussion of the relative merits of threading and forking . Like most things in technology different approaches exploits different trade-offs. Usually not one approach is superior to all others in all situations. (cf. Python vs Perl vs Ruby, eg. ) Hello Christopher, thanks for your comments. Yes, I pulled harder on one side of the string than is practical. We haven't defined what we mean by threads versus forks, their differences and communalities. Some see threads as lightweight processes, other see the type of memory-space as the only signifcant difference. There is a hardware and a software oriented view on forks and threads. And then there is the Perl meaning of thread. (which might have been the one that was meant in the wrong question statement) Here is my opinion on the matter. Forking is a simpler and cleaner approach. By design the programmer does not have to worry about sharing of data and race conditions. When you algorithm is limited in memory requirements and completely compartamentalized, this is a good choice. And this is very often the case. Threading is not without its place, however. Notwithstanding, Dr. Ruud's argument: Most wishes for readily-shared memory result from (and to) bad design. This argument, stated without proof, makes a grand generalization. Yes, and I hope it was taken as that. Read also the arcticle The problem with threads, which is linked to on http://en.wikipedia.org/wiki/Thread_(computer_science) Deterministic ends should be accomplished with deterministic means. There are quite a few algorithms: sort, search, merge that greatily benefit from shared (or increased) memory. The simplest is using a very large shared hash to map values from a very, very large data stream. The ability of the shared memory will allow for more cooperative threads than forks where memory cannot be shared. Even for those cases there is often a good and less connected parallel solution available, where independent processes deal with considerable parts of the stream, and come together only at the end to sync and merge the results. Such a solution often means that double work is performed, and that significant more CPU cycles are used than would have been with readily-shared memory (but still the result can be achieved earlier and more reliable). It is possible to marshall, serialize, or use other IPC between forks, but then the balance of simpler, cleaner probably tilts toward a threading solution. Threads can run simultaneously in a multiprocessor system, but are susceptible to dead locks and to waste resources while polling. Forks share memory (code, read-only data, copy-on-write data) and files too. See also http://en.wikipedia.org/wiki/Non-blocking_synchronization about how to make multiprocessing more efficient. A lot of research is going on in that field. -- Affijn, Ruud Gewoon is een tijger.
Re: META.yml how to declare the need for threaded perl?
I spent some time thinking about Dr. Ruud's reply over the past several days. Although I mostly share his views, I think that he has done a disservice to the discussion of the relative merits of threading and forking . Like most things in technology different approaches exploits different trade-offs. Usually not one approach is superior to all others in all situations. (cf. Python vs Perl vs Ruby, eg. ) Here is my opinion on the matter. Forking is a simpler and cleaner approach. By design the programmer does not have to worry about sharing of data and race conditions. When you algorithm is limited in memory requirements and completely compartamentalized, this is a good choice. And this is very often the case. Threading is not without its place, however. Notwithstanding, Dr. Ruud's argument: Most wishes for readily-shared memory result from (and to) bad design. This argument, stated without proof, makes a grand generalization. There are quite a few algorithms: sort, search, merge that greatily benefit from shared (or increased) memory. The simplest is using a very large shared hash to map values from a very, very large data stream. The ability of the shared memory will allow for more cooperative threads than forks where memory cannot be shared. It is possible to marshall, serialize, or use other IPC between forks, but then the balance of simpler, cleaner probably tilts toward a threading solution. I hope this provides some clarity for the original poster. Best, Chris On Sun, Nov 2, 2008 at 8:03 AM, Dr.Ruud [EMAIL PROTECTED][EMAIL PROTECTED] wrote: Dana Hudes schreef: Dr.Ruud: Gabor Szabo: I guess we can implement everything with fork but I think - maybe because of my lack of experience in threads - that it will be better to use them than to fork. I think it was Randal Schwartz who said something like: If the answer involves threads, then the question was wrong. Then Randall is wrong No, he wasn't. And isn't either. You are, and not only because you misspelt his name. The Perl threading model is perhaps imperfect but the principle of threads is a part of many modern applications not to mention every modern OS including Solaris and MS windows xp And they are happily moving away from it, see for example the new IE design. With current hardware, forking nog longer has the issues that many people just can't appear to forget about. Many client/server apps benefit from threading. Parallel processing hugely benefits from threading b/c you have readily-shared memory Most wishes for readily-shared memory result from (and to) bad design. -- Affijn, Ruud Gewoon is een tijger.
Re: META.yml how to declare the need for threaded perl?
On Sat, Nov 01, 2008 at 01:25:17PM +, Ben Morrow wrote: Quoth [EMAIL PROTECTED] (Nicholas Clark): $ /home/nclark/Sandpit/588ish/bin/perl -Mthreads -e0 This Perl not built to support threads Compilation failed in require. BEGIN failed--compilation aborted. No, this still doesn't work. Every 5.8, threaded or not, has threads.pm installed, so using it as a dependancy won't ensure your perl is threaded. Trying to use it before writing the Makefile will Do The Right Thing: eval use threads || exit(0); WriteMakefile(...); because even if it's installed, you can't 'use' it unless you've got thread support. -- David Cantrell | London Perl Mongers Deputy Chief Heretic Vegetarian: n: a person who, due to malnutrition caused by poor lifestyle choices, is eight times more likely to catch TB than a normal person
Re: META.yml how to declare the need for threaded perl?
David Cantrell schreef: eval use threads || exit(0); Alternatively written as: exit(0) unless eval use threads; (just because I wondered how strong the || was in there) -- Affijn, Ruud Gewoon is een tijger.
Re: The value of threads (was Re: META.yml how to declare the need for threaded perl?)
* Chris Dolan [EMAIL PROTECTED] [2008-11-01 22:25]: On Nov 1, 2008, at 10:39 AM, Dr.Ruud wrote: I think it was Randal Schwartz who said something like: If the answer involves threads, then the question was wrong. Most likely that quote comes from a year when multi-processor systems were not ubiquitous. No, it’s as true as ever. Threads suck. They are a different beast in Perl than in other languages because Perl goes to great lengths not to make variables shared between threads unless explicitly asked to, but of course that makes Perl threads suck for an entirely different set of reasons. The fact of the matter is that hardware-level concurrency in Perl code on an OS without fork() is a non-starter. In other languages, you might have something like STM, or maybe the Erlang model, or some other approach – things which actually make sense and make it safe and sane to write concurrent code (like fork() does on Unix). Threads are always the wrong answer. To quote Brendan Eich[1], My default answer to questions such as the one I got at last May’s Ajax Experience, “When will you add threads to JavaScript?” is: “over your dead body!” [1]: http://weblogs.mozillazine.org/roadmap/archives/2007/02/threads_suck.html Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/
Re: META.yml how to declare the need for threaded perl?
Then Randall is wrong The Perl threading model is perhaps imperfect but the principle of threads is a part of many modern applications not to mention every modern OS including Solaris and MS windows xp Many client/server apps benefit from threading. Parallel processing hugely benefits from threading b/c you have readily-shared memory --Original Message-- From: Dr.Ruud To: module-authors@perl.org Sent: Nov 1, 2008 11:39 AM Subject: Re: META.yml how to declare the need for threaded perl? Gabor Szabo schreef: I guess we can implement everything with fork but I think - maybe because of my lack of experience in threads - that it will be better to use them than to fork. I think it was Randal Schwartz who said something like: If the answer involves threads, then the question was wrong. -- Affijn, Ruud Gewoon is een tijger. Sent from my BlackBerry® smartphone with Nextel Direct Connect
Re: META.yml how to declare the need for threaded perl?
On Fri, Oct 31, 2008 at 06:16:01PM -0500, Chris Dolan wrote: Just add a dependency on thread::shared or one of the other threading libraries. Push your problem up the chain! No, not threads::shared $ /home/nclark/Sandpit/588ish/bin/perl -Mthreads::shared -e0 $ threads looks a better idea: $ /home/nclark/Sandpit/588ish/bin/perl -Mthreads -e0 This Perl not built to support threads Compilation failed in require. BEGIN failed--compilation aborted. $ Nicholas Clark
Re: META.yml how to declare the need for threaded perl?
Quoth [EMAIL PROTECTED] (Nicholas Clark): On Fri, Oct 31, 2008 at 06:16:01PM -0500, Chris Dolan wrote: Just add a dependency on thread::shared or one of the other threading libraries. Push your problem up the chain! No, not threads::shared $ /home/nclark/Sandpit/588ish/bin/perl -Mthreads::shared -e0 $ threads looks a better idea: $ /home/nclark/Sandpit/588ish/bin/perl -Mthreads -e0 This Perl not built to support threads Compilation failed in require. BEGIN failed--compilation aborted. $ No, this still doesn't work. Every 5.8, threaded or not, has threads.pm installed, so using it as a dependancy won't ensure your perl is threaded. Ben
Re: META.yml how to declare the need for threaded perl?
Gabor Szabo schreef: I guess we can implement everything with fork but I think - maybe because of my lack of experience in threads - that it will be better to use them than to fork. I think it was Randal Schwartz who said something like: If the answer involves threads, then the question was wrong. -- Affijn, Ruud Gewoon is een tijger.
Re: META.yml how to declare the need for threaded perl?
* On Sat, Nov 01 2008, Dr.Ruud wrote: Gabor Szabo schreef: I guess we can implement everything with fork but I think - maybe because of my lack of experience in threads - that it will be better to use them than to fork. I think it was Randal Schwartz who said something like: If the answer involves threads, then the question was wrong. This assumes that you value snide comments over working code. Anyway, I wouldn't use threads for this, but that doesn't make threading an unreasonable approach. I would use POE. (Oh, and by would, I mean do: http://git.jrock.us/?p=Server-Stylish.git;a=summary ) Regards, Jonathan Rockway -- print just = another = perl = hacker = if $,=$
Re: META.yml how to declare the need for threaded perl?
On Sat, Nov 1, 2008 at 12:50 PM, Jonathan Rockway [EMAIL PROTECTED] wrote: Anyway, I wouldn't use threads for this, but that doesn't make threading an unreasonable approach. I would use POE. +1 I assume that POE::Loop::Wx should integrate into Padre pretty easily, yes? -- David
The value of threads (was Re: META.yml how to declare the need for threaded perl?)
On Nov 1, 2008, at 10:39 AM, Dr.Ruud wrote: Gabor Szabo schreef: I guess we can implement everything with fork but I think - maybe because of my lack of experience in threads - that it will be better to use them than to fork. I think it was Randal Schwartz who said something like: If the answer involves threads, then the question was wrong. Most likely that quote comes from a year when multi-processor systems were not ubiquitous. Every UI framework I've ever used was indeed written with a single- threaded event loop, but even in an editor it sure is handy to have background threads to do asynchronous work (syntax checking, rebuilding call graphs, Perl::Critic, etc) without having to explicitly chunk the work or coordinate multiple processes. Chris
Re: META.yml how to declare the need for threaded perl?
* On Fri, Oct 31 2008, Gabor Szabo wrote:
Hi,
currently I have this code in Build.PL to check if the perl where Padre
is being installed is threaded.
use Config;
if (not $Config{usethreads}) {
warn Padre requires a perl built using threads\n;
exit 0;
}
Probably off topic, but last time I tried Padre, I commented out all the
references to threads (non-threaded perl here) and it worked fine.
Maybe it doesn't really require threads?
Regards,
Jonathan Rockway
--
print just = another = perl = hacker = if $,=$
Re: META.yml how to declare the need for threaded perl?
On Fri, Oct 31, 2008 at 4:14 PM, Jonathan Rockway [EMAIL PROTECTED] wrote:
* On Fri, Oct 31 2008, Gabor Szabo wrote:
Hi,
currently I have this code in Build.PL to check if the perl where Padre
is being installed is threaded.
use Config;
if (not $Config{usethreads}) {
warn Padre requires a perl built using threads\n;
exit 0;
}
Probably off topic, but last time I tried Padre, I commented out all the
references to threads (non-threaded perl here) and it worked fine.
Maybe it doesn't really require threads?
Currently it only uses it in the Ack menu option.
I think as we introduce more and more long-running processes
(e.g. indexing all the pods, running your tests in the background etc)
we will need that.
I guess we can implement everything with fork but I think -
maybe because of my lack of experience in threads - that it will
be better to use them than to fork.
Id' be glad to hear your opinion too.
Gabor
Re: META.yml how to declare the need for threaded perl?
Just add a dependency on thread::shared or one of the other threading
libraries. Push your problem up the chain!
Chris
On Oct 31, 2008, at 7:31 AM, Gabor Szabo wrote:
Hi,
currently I have this code in Build.PL to check if the perl where
Padre
is being installed is threaded.
use Config;
if (not $Config{usethreads}) {
warn Padre requires a perl built using threads\n;
exit 0;
}
Is there any way to add this requirement to META.yml?
Gabor
