> On Aug 13, 2015, at 12:01 AM, Steve Richfield <[email protected]> > wrote: > > THAT is why so many supercomputers start out with APL and FORTRAN compilers, > and why C compilers only vectorize simple loops that utilize a small subset > of the language.
Supercomputing has been almost exclusively C++ for well over a decade. FORTRAN was a legacy code thing. The reasons why C++ is nearly an ideal language for supercomputing is not immediately obvious to most people, but the primary reason is that most of the material parallelism lives outside the scope of a process. There isn’t even multithreading; each core lives in its own universe, which makes the code, within the boundaries of a process, very simple to write. Your primary optimization *within* a process is throughput optimization, and C++ is an excellent language for that. Most supercomputing codes are memory bandwidth bound; getting to that point is table stakes. Memory throughput optimizations are far outside the hands of the compiler but quite easily implemented in languages like C++. These kinds of optimizations are much more difficult to achieve in functional and/or garbage-collected languages which is why you rarely see those types of languages used in supercomputing. Parallelism in a supercomputing code comes almost entirely from the design of the structural topology between those processes. Within a process, it is all about throughput optimizations which have very little to do with classical parallelism per se nor are they the kinds of problems a clever compiler can solve if the code was not explicitly designed for it in the first place. > Unfortunately, this book never considered truly parallel implementations, > where everything runs at once - but rather they considered "parallel" > programming to be simple multi-threaded programing. Yes, many programmers have an unfortunate habit of conflating “concurrent”, “distributed”, and “parallel” even though they imply very different things at an architectural level. Andrew ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
