Andrew, On Fri, Aug 14, 2015 at 7:59 PM, J. Andrew Rogers <[email protected]> wrote:
> > > On Aug 14, 2015, at 6:53 PM, Logan Streondj <[email protected]> wrote: > > I agree that Hadoop is not useful for supercomputing applications. I'm > not sure if you are implying that this is the basis of your statement, > otherwise feel free to elaborate. > > > > Map/reduce/expand works well on GPU's in practice, since there are many > relatively slow processors which are independent of each other. > > > Scatter/Gather, Map/Reduce, etc style computing models, whether > implemented on a GPU or Hadoop, are very poor for many types of parallel > computing. It only works if your problem is in the tiny subset that is > embarrassingly parallel in a topological sense. Most interesting algorithms > are not in this class. > This gets down to what is "interesting". Certainly weather prediction falls into this class, as does neurological simulation. > > Put more simply, for most interesting applications of parallel > computation, the processors cannot be “independent of each other”. > So, you want absolutely maximum single-thread performance, so you can live with as few threads as possible. > The ability to design efficient parallel data flows between compute > elements is rather important for many classes of algorithm. Join > algorithms, for example, which are central to graph traversal. > With FPGAs you can design ANYTHING you can imagine. > > This, by the way, is why barrel processing architectures are far more > interesting than GPU architectures (or FPGAs for that matter) when it comes > to parallel computations. Barrel processors can efficiently express very > sophisticated topological parallelism; GPUs are optimized for pure data > parallelism. > I don't (yet) see this. The only successful application of barrels I know of was in peripheral processors attached to some early supercomputers designed by Cray. With really cheap but slow processors, they could dedicate a processor to every disk or tape drive - which is now no problem because they now already have several micros. Barrels make data chaining impossible, at which point you have thrown away an order of magnitude in memory bandwidth. > > > This is of course similar to the brain, where each neuron is relatively > independent, making it's decision to fire based on dendrites and possibly > micro-tubules. > > No, it is not similar to the brain. The brain has an incredibly > complicated computational topology between neurons. It is not convertible > into a simple data parallel problem — that tacitly assumes neurons do not > talk to each other. > Long ago ~1982 I presented a paper (the closing presentation at the 1st NN conference in San Diego) showing that if you chopped a human brain up into many processors that communicated over a single bus, that the communications rate would be only ~10^9 brief messages/second. Of course this was inconceivably fast for that time, but probably doable now. Steve > ------------------------------------------- > AGI > Archives: https://www.listbox.com/member/archive/303/=now > RSS Feed: https://www.listbox.com/member/archive/rss/303/10443978-6f4c28ac > Modify Your Subscription: > https://www.listbox.com/member/?&; > Powered by Listbox: http://www.listbox.com > -- Full employment can be had with the stoke of a pen. Simply institute a six hour workday. That will easily create enough new jobs to bring back full employment. ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
