On Tuesday 15 April 2008 04:28:25 pm, Steve Richfield wrote: > Josh, > > On 4/15/08, J Storrs Hall, PhD <[EMAIL PROTECTED]> wrote: > > > > On Monday 14 April 2008 04:56:18 am, Steve Richfield wrote: > > > ... My present > > > efforts are now directed toward a new computer architecture that may be > > more > > > of interest to AGI types here than Dr. Eliza. This new architecture > > should > > > be able to build new PC internals for about the same cost, using the > > same > > > fabrication facilities, yet the processors will run ~10,000 times faster > > > running single-thread code. > > > > This (massively-parallel SIMD) is perhaps a little harder than you seem to > > think. I did my PhD thesis on it and led a multi-million-dollar 10-year > > ARPA-funded project to develop just such an architecture. > > > I didn't see any attachments. Perhaps you could send me some more > information about this? Whenever I present this stuff, I always emphasize > that there is NOTHING new here, just an assortment of things that are > decades old. Hopefully you have some good ideas in there, or maybe even some > old ideas that I can attribute "new" thinking to.
online: The CAM2000 Chip Architecture ftp://ftp.cs.rutgers.edu/pub/technical-reports/lcsr-tr-196.ps.Z look in http://www.amazon.com/exec/obidos/ASIN/0818676612 Associative Processing and Processors, Kirkelis & Weems, eds. order the dissertation from University Microfilms: Associative processing: Architectures, algorithms, and applications. by Hall, John Storrs. 9511479 Unfortunately I don't have it in machine-readable form. > The first mistake everybody makes is to forget that the bottleneck for > > existing processors isn't computing power at all, it's memory bandwidth. > > All > > the cruft on a modern processor chip besides the processor is there to > > ameliorate that problem, not because they aren't smart enough to put more > > processors on. > > Got this covered. Each of the ~10K ALUs has ~8 memory banks to work with, > for a total of ~80K banks, so there should be no latencies except for > inter-ALU communication. Have I missed something here? Either you're using static RAM (and getting a big hit in density and power) or DRAM, and getting a big hit in speed. YOU CAN"T AFFORD TO USE CACHE outside of a line buffer or two. You lose an order of magnitude in speed over what can be done on the CPU chip. Several "big" items that they put a few of on a cpu chip (besides cache) that you can't afford in each processing element: barrel shifters, floating point units, even multipliers. Instruction broadcast latency and skew. If your achitecture is synchronous you're looking at cross-chip times stuck into your instruction processing, which means TWO orders of magnitude loss from on-chip cpu cycle times. So instead of a 10K speedup you get a 100 speedup > The second mistake is to forget that processor and memory silicon fab use > > different processes, the former optimized for fast transistors, the latter > > for dense trench capacitors. You won't get both at once -- you'll give up > > at > > least a factor of ten trying to combine them over the radically > > specialized > > forms. > > Got that covered. Once multipliers and shift matrices are eliminated and > only a few adders, pipeline registers, and a little random logic remain, > then the entire thing can be fabricated with *MEMORY* fab technology! Note > that memories have been getting smarter (and even associative), e.g. cache > memories, and when you look at their addressing, row selection, etc., there > is nothing more complex than I am proposing for my ALUs. While the control > processor might at first appear to violate this, note that it needs no > computational speed, so its floating point and other complex instructions > can be emulated on slow memory-compatible logic. You need a collective function (max, sum, etc) tree or else you're doing those operations by Caxton Foster-style bit-serial algorithms with an inescapable bus turnaround between each bit. How are you going to store an ordinary matrix? There's no layout where you can both add and multiply matrices without a raft of data motion. Either you build a general parallel communications network, which is expensive (think Connection Machine) or your data-shuffling time kills you. Again, let me mention graphics boards. They have native floating point, wide memory bandwith, and hundreds of processing units, along with fairly decent data comm onboard. Speedups over the cpu can get up to 20 or so, once the whole program is taken into account -- but for plenty of programs, the cpu is faster. > The third mistake is to forget that nobody knows how to program SIMD. > > > This is a long and complicated subject. I spent a year at CDC digging some > of the last of the nasty bugs out of their Cyber-205 FORTRAN compiler's > optimizer and vectorizer, whose job it was to sweep these issues under the > rug. There are some interesting alternatives, like describing complex code > skeletons and how to vectorize them. When someone writes a loop whose > structure is new to the compiler, someone else would have to explain to the > computer how to vectorize it. Sounds kludgy, but co0nsidering the > man-lifetimes that it takes to write a good vectorizing compiler, this > actually works out to much less total effort. > > I absolutely agree that programmers will quickly fall into two groups - > those who "get it" and make the transition to writing vectorizable code > fairly easily, and those who go into some other line of work. Well, it's a high art to write code for GPU's now but they have APIs (e.g. OpenGL) that are a lor more adapted to the mainstream's capabilities. I have no doubt that associative processors would be the same way. But I think that give the current push to parallelism of the multicore style, there will be some new paradigms coming. Who knows. > Not even LISP. APL, which is the > > closest approach to a SIMD language, died a decade or so back. > > Yes. This is a political/psychological issue, as there were its > "practitioners" who learned its hieroglyphics, and the rest of the mere > mortals who simply ignored it. No one (that I know of)ever made the obvious > simple step of producing a humanized front-end to the language. > > BTW, APL is still alive in some financial modeling applications. If you don't like math-like symbols, check out NIAL (http://www.nial.com/). But actually Matlab (and the scientific packages for languages like Python) have data parallel primitives, as well as higher-level functions like FFT and principal components analysis... > Now frankly, a real associative processor (such as described in my thesis -- > > read it) would be very useful for AI. You can get close to faking it > > nowadays > > by getting a graphics card and programming it GPGPU-style. I quit > > architecture and got back into the meat of AI because I think that Moore's > > law has won, and the cycles will be there before we can write the > > software, > > so it's a waste of time to try end-runs. > > > Not according to Intel, who sees the ~4GHz limit as being a permanent thing. > I sat on my ideas for ~20 years, just waiting for this to happen and blow > Moore out of the water. Intel are going parallel too : http://www.news.com/2100-1006_3-6119618.html Intel pledges 80 cores in five years (and it's a real working silicon prototype today) (and note that each one of those has a floating-point unit) > Associative processing would have > > been REALLY useful for AI in the 80's, but we can get away without it, > > now. > > With enough ALUs, associative processing is just another programming style. > > Thanks. > > Steve Richfield > > ------------------------------------------- > agi > Archives: http://www.listbox.com/member/archive/303/=now > RSS Feed: http://www.listbox.com/member/archive/rss/303/ > Modify Your Subscription: http://www.listbox.com/member/?&; > Powered by Listbox: http://www.listbox.com > ------------------------------------------- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=101455710-f059c4 Powered by Listbox: http://www.listbox.com
