At a Kdb talk I attended recently, Fintan Quill mentioned that K does multi-processing but not multi-threading. He didn't go into details on this but, based on what they say in this talk and what I've heard elsewhere, the reason seems obvious: multi-threading is nearly impossible to get to work properly.
As they say in this talk about how "Shared Mutable State" in multi-threaded programming should be some of the most feared words in computing, Martin Thompson elaborates "It's a complete nightmare even with really, really good people." He then goes on to add "I haven't come across any multi-threaded application where people are mutating the same space, that isn't bug-ridden." So, in short, don't do this. Don't even try to do this: fine-grained, multi-threaded parallelism is probably a waste of time with the current level of tools we have for it. Multi-processing, on the other hand, is dead-easy to do in J for anything that can be parallelized at a coarse-grained level, which is many if not most things. I've written about this extensively over the past few years (all at jcode.jsoftware.com/wiki/): User:Devon_McCormick/ParallelizedJCodeExamples, Community/Conference2012/Talks/ParallelSimulationInJ, NYCJUG/2011-04-12/RedOrBlackGameSimulation . These latter two give detailed examples on how to achieve multi-processor parallelization. On another note, there's been some mention of processes like Agile for speeding up the development process. Keep in mind that the context for these kinds of "methodologies" is an idea that a small team might be five people. In the array-programming world, this would be a large team. On Sat, Sep 26, 2015 at 7:19 PM, Don Guinn <[email protected]> wrote: > Given a fork (f g h) f and h can be processed in parallel. But if f and h > have side effects (shared global names that are modified) then they should > not run in parallel. A programming practice that should be avoided > anyway. This is not a pipeline, but multiple processors could be used. But > even not running them in parallel I see nothing stating which, f or h, is > run first. So one should avoid code that depended on the order of f and h > execution anyway. > > On Sat, Sep 26, 2015 at 4:42 PM, Raul Miller <[email protected]> > wrote: > > > On Sat, Sep 26, 2015 at 3:04 PM, Vijay Lulla <[email protected]> > wrote: > > > quite easy. But what I'm very unclear about is how does one do > > > pipelining in J? Say we have functions f, g, and h (all used > > > monadically) and it is applied like f@g@h y and function g was > > > particularly costly how can we parallelize (maybe the user has to > > > program it himself or the interpreter can do some cost analysis [like > > > query planning in SQL databases]) it to make it faster? Isn't this > > > what the presenters were mentioning when they were using the example > > > of airbus pipeline system? > > > > "Pipelining" seems to describe a variety of topics. > > > > https://en.wikipedia.org/wiki/Pipeline_(computing) > > > > So, I would have to say that there is no general technique. If f, g > > and h are black boxes, you cannot pipeline them. If you want to > > reschedule g or make it more efficient, you'll need to know details > > about g. The more you know, the greater the odds are that you can do > > it (or the important parts of it) differently, in a more efficient > > manner. > > > > That said, I should also point out that a lot of the automated query > > planning systems are workarounds for bogus constraints underlying the > > sql standard. That effort could have gone in much more useful > > directions if people hadn't bought into those ideas. (But at this > > point, it has turned into a multi-billion dollar industry, so it's not > > going away. And there are some applications where the flaws are not > > all that important.) > > > > Thanks, > > > > -- > > Raul > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > -- Devon McCormick, CFA ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
