exponential information technology 1890-2014 10exp17 more MIPS per constant 2004 dollar in 124 years, Luke Muehlhauser, Machine Intelligence Research Institute 2014.05.12: Rich Murray 2014.12.27
since 1890, increase by 10 times every 7.3 years -- since 1950 -- 2014 = 54 years, with about 10exp13 times more = 10,000,000,000,000 times more per device, from vacuum tubes to multicore processors -- increase by 10 times every 4 years per constant 2004 dollar. !! Rich <http://intelligence.org/> <https://intelligence.org/careers/> - Research <https://intelligence.org/research/> - Team <http://intelligence.org/team/> - Get Involved <http://intelligence.org/get-involved/> - Donate <http://intelligence.org/donate/> - Blog <http://intelligence.org/blog/> Exponential and non-exponential trends in information technology May 12, 2014 | Luke Muehlhauser <https://intelligence.org/author/lukeprog/> | Analysis <http://intelligence.org/category/analysis/> *Co-authored with Lila Rieber.* In *The Singularity is Near* <http://smile.amazon.com/Singularity-Near-Humans-Transcend-Biology-ebook/dp/B000QCSA7C/>, Ray Kurzweil writes that “every aspect of information and information technology is growing at an exponential pace.”1 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_0_11027> In *Abundance <http://www.amazon.com/Abundance-Future-Better-Than-Think-ebook/dp/B005FLOGMM/>*, the authors list eight fields — including nanomaterials, robotics, and medicine — as “exponentially growing fields.”2 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_1_11027> *The Second Machine Age* <http://www.amazon.com/Second-Machine-Age-Prosperity-Technologies-ebook/dp/B00D97HPQI/> says that “technical progress” in general is “improving exponentially.”3 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_2_11027> These authors are correct to emphasize that exponential trends in technological development are surprisingly common (Nagy et al. 2013 <http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0052669>), and that these trends challenge the wisdom of our built-in heuristic <http://wiki.lesswrong.com/wiki/Absurdity_heuristic>to ignore futures that *sound* absurd. (To someone in the 1980s, the iPhone is absurd. To us, it is an affordable consumer good.) Unfortunately, these and other popular discussions of “exponential technologies” are often very succinct and therefore ambiguous, resulting in public and professional misunderstanding.4 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_3_11027> I (Luke) regularly encounter people who have read the books above and come away with the impression that all information technologies show roughly exponential trends all the time. But this isn’t true unless you have a *very* broad concept of what counts as “roughly exponential.” So, without speculating much about what Kurzweil & company intend to claim, we’ll try to clear up some common misunderstandings about exponential technologies by showing a few examples of exponential and not-so-exponential trends in information technology. A more thorough survey of trends in information technology must be left to other investigators.5 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_4_11027> Computations per dollar: still exponential It’s clear that Kurzweil himself does not *literally* mean that “every aspect of information and information technology is growing at an exponential pace,” for he has previously discussed examples of non-exponential growth in some aspects of information technology. For example, he’s well aware that the exponential trend in processor clock speed broke down in 2004, as shown in Fuller & Millett (2011a) <http://commonsenseatheism.com/wp-content/uploads/2014/03/Fuller-Millett-Computing-Performance-Game-Over-or-Next-Level-in-IEEE-Computer-Society.pdf> :6 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_5_11027> [image: Fuller & Millett figure 1] Because this is a logarithmic chart, a straight line represents an exponential trend. Notice that clock speed stopped improving exponentially in 2004, but transistors per chip has continued to increase exponentially via the jump from single-core to multicore processors. Elsewhere, Kurzweil tends to emphasize exponential trends in *price-performance ratios* specifically, for example *computations per dollar*. This is perfectly reasonable. Most of us don’t care about the fine details of processor architecture — we just care about how much *stuff we can do* per dollar.7 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_6_11027> And thus far, the exponential trend in computations per dollar has kept up.8 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_7_11027> It’s unclear, however, how much longer this trend can be maintained. In particular, the dark silicon problem <https://intelligence.org/2013/10/21/hadi-esmaeilzadeh-on-dark-silicon/>may slow the currently exponential trend in computations per dollar. Joel Hruska covers other recent challenges to the trend here <http://www.extremetech.com/computing/178529-this-is-what-the-death-of-moores-law-looks-like-euv-paused-indefinitely-450mm-wafers-halted-and-no-path-beyond-14nm>, including the halted production of the 450mm wafers that Intel, TSMC, and Samsung all bet their money on <http://www.extremetech.com/computing/132604-intel-invests-in-asml-to-boost-extreme-uv-lithography-massive-450mm-wafers> 18 months ago. DRAM capacity per dollar: a slowing trend Another important price-performance trend is *DRAM capacity per dollar*. This trend was almost precisely exponential for many years but recently the trend has slowed:9 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_8_11027> [image: DRAM chart] <https://intelligence.org/wp-content/uploads/2014/04/DRAM-chart-2.png> The same slowing trend for DRAM is also reported in Hennessy & Patterson (2011) <http://www.amazon.com/Computer-Architecture-Quantitative-Approach-Kaufmann-ebook/dp/B0067KU84U/>, on page 17. On page 100 they remark: DRAMs obeyed Moore’s law for 20 years, bringing out a new chip with four times the capacity every three years. Due to the manufacturing challenges of a single-bit DRAM, new chips only double capacity every two years since 1998. In 2006, the pace slowed further, with the four years from 2006 to 2010 seeing only a [single] doubling of capacity. SRAM capacity per dollar: a slowing trend What about another kind of computer memory, like SRAM <http://en.wikipedia.org/wiki/Static_random-access_memory>? The cost of SRAM dropped precipitously from 1980 to 1990 but has dropped more slowly since then.10 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_9_11027> [image: SRAM chart (2)] <https://intelligence.org/wp-content/uploads/2014/04/SRAM-chart-2.png> Hard drive capacity per dollar: interrupted by floods in Thailand Cost per gigabyte of hard drive storage had been dropping exponentially for about 30 years when suddenly hard drive prices actually *increased* <http://www.computerworld.com/s/article/9227829/Hard_drive_prices_to_remain_high_until_2014> for a while because October 2011 floods in Thailand <http://en.wikipedia.org/wiki/2011_Thailand_floods#Damages_to_industrial_estates_and_global_supply_shortages> destroyed some hard drive factories. Hard drive prices have been comparatively flat since then:11 <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#footnote_10_11027> [image: Hard drive storage cost] <https://intelligence.org/wp-content/uploads/2014/04/Hard-drive-storage-cost.png> The floods hurt many computing providers, who struggled (and sometimes failed) to offer services at the low prices they had anticipated based on exponential expectations: see e.g. the comments by BackBlaze <http://blog.backblaze.com/2013/11/26/farming-hard-drives-2-years-and-1m-later/> ,Intel <http://money.cnn.com/2011/12/12/technology/intel_earnings_revision/index.htm>, and Joyent <http://gigaom.com/2011/12/27/what-the-hdd-shortage-means-for-cloud-computing/> . Whereas the exponential trend for processor clock speed was brought to a halt by physics, the exponential trend in cost per gigabyte of storage was slowed (at least for now) by natural disaster. ------------------------------ 1. Page 85. In the same book, he also writes that “we see ongoing exponential growth of every aspect of information technology, including price-performance, capacity, and rate of adoption.” (p. 377). In *How to Create a Mind* <http://smile.amazon.com/How-Create-Mind-Thought-Revealed-ebook/dp/B007V65UUG/>, Kurzweil writes that “In the course of my investigation, I made a startling discovery: If a technology is an information technology, the basic measures of price/performance and capacity… follow amazingly precise exponential trajectories” (p. 254). ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_0_11027> 2. Page 57. In general, Diamandis & Kotler seem to agree with Kurzweil that all information technologies experience exponential growth curves. E.g. on page 99 they write that “Although [some agroecological] practices themselves look decidedly low tech, all the fields they’re informed by are information-based sciences and thus on exponential growth curves,” and on page 190 they write that “almost every component of medicine is now an information technology and therefore on an exponential trajectory.” ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_1_11027> 3. Page 10. The authors also seem to expect exponential trends for anything that becomes a digital process: “…batteries… haven’t improved their performance at an exponential rate because they’re essentially chemical devices, not digital ones…” (p. 52). ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_2_11027> 4. E.g. Kurzweil seems to use a fairly loose definition of “exponential.” For example in Kurzweil (2001) <http://www.kurzweilai.net/the-law-of-accelerating-returns> he gives this chart <https://intelligence.org/wp-content/uploads/2014/03/ISP-cost-performance.jpg>of ISP cost-performance as an example exponential trend. Sometimes this seems to cause confusion in dialogue. For example, in response to Ilkka Tuomi’s criticisms (2002 <http://firstmonday.org/ojs/index.php/fm/article/viewArticle/1000/921>, 2003 <http://meaningprocessing.com/personalPages/tuomi/articles/Kurzweil.pdf>) of claims of exponential trends in computing, Kurzweilwrote <http://www.kurzweilai.net/exponential-growth-an-illusion-response-to-ilkka-tuomi> that if Tuomi were correct, “I would have to conclude that the one-quarter MIPS computer costing several million dollars that I used at MIT in 1967 and the 1000 MIPS computer that I purchased recently for $2,000 never really existed… I admire his tenacity in attempting to prove that the world of information technology is flat (i.e., linear).” But Tuomi’s views don’t entail that, and Tuomi didn’t say that trends in information technology have been linear. The conflict appears to stem from the fact that Tuomi was using “exponential” in the strict sense, while Kurzweil was using the term in a very loose sense. This becomes clearer in Tuomi’s reply to Kurzweil <http://www.meaningprocessing.com/personalPages/tuomi/articles/ResponseToKurzweil.pdf> . ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_3_11027> 5. Our thanks to Jonah Sinick for his assistance in researching this post. ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_4_11027> 6. It should be noted, however, that the exponential trend line for clock speed on page 61 of *The Singularity is Near*(2005) is now known to be incorrect. Kurzweil’s graph used the 2002 ITRS report <http://www.itrs.net/Links/2006Update/2006UpdateFinal.htm> to project the trend line for 2001-2016, but actual growth in clock speed fell substantially short of the ITRS projection. ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_5_11027> 7. As Fuller & Millett (2011b) <http://www.amazon.com/The-Future-Computing-Performance-Level/dp/0309159512/> write, “When we talk about scaling computing performance, we implicitly mean to increase the computing performance that we can buy for each dollar we spend” (p. 81). ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_6_11027> 8. Kurzweil (2012) <http://www.amazon.com/How-Create-Mind-Thought-Revealed/dp/0670025291/>, ch. 10, footnote 10 shows “calculations per second per $1,000″ growing exponentially from 1900 through 2010, including several data points after 2004. However, we couldn’t find his data sources, and we don’t know whether he adjusted for inflation, so we’ve relied instead on a data set provided by Koh & Magee (2006) <http://web.mit.edu/cmagee/www/documents/15-koh_magee-tfsc_functional_approach_studying_technological_progress_vol73p1061-1083_2006.pdf>, extended by data we pulled from NotebookCheck <http://www.notebookcheck.net/>, PCStats <http://pcstats.com/>, Tom’s Hardware <http://www.tomshardware.com/>, and CPU-World <http://www.cpu-world.com/>. Our raw data are here <https://docs.google.com/spreadsheets/d/1qPBpgqxHsqQgcLLXJ5H-4yto9SPQinR4H0f9p5Dh4g4/edit?usp=sharing> and show a continuing exponential trend in MIPS per dollar, adjusted for inflation. ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_7_11027> 9. This chart uses data from Bryant & O’Hallaron (2011) <http://www.amazon.com/Computer-Systems-Programmers-Perspective-2nd-ebook/dp/B008VIXMWQ/>, p. 584 and from the Performance Curve Database <http://pcdb.santafe.edu/graph.php?curve=25>. Raw data here <https://docs.google.com/spreadsheets/d/1-gTuad7LZ7AstYAOsBB0BHyiGlfd7f_DfkAQqxMvJIU/edit?usp=sharing> . ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_8_11027> 10. Data from Bryant & O’Hallaron (2011) <http://www.amazon.com/Computer-Systems-Programmers-Perspective-2nd-ebook/dp/B008VIXMWQ/>, p. 584. Raw data here <https://docs.google.com/spreadsheets/d/1ELlkNbaeg4HSk-dEc76dIexCTiMuPa67v49f70JT7_M/edit?usp=sharing> . ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_9_11027> 11. Our data <https://docs.google.com/spreadsheets/d/16EvCAPcnijU1XXC2Iy9hvXYZsqKtipD5ERxknt4GFno/edit?usp=sharing> are drawn from Matthew Komorowski’s page on storage cost <http://www.mkomo.com/cost-per-gigabyte-update> (but adjusted for inflation), Koh & Magee (2006 <http://web.mit.edu/cmagee/www/documents/15-koh_magee-tfsc_functional_approach_studying_technological_progress_vol73p1061-1083_2006.pdf> ), Bryant & O’Hallaron (2011) <http://www.amazon.com/Computer-Systems-Programmers-Perspective-2nd-ebook/dp/B008VIXMWQ/>, and the Performance Curve Database <http://pcdb.santafe.edu/graph.php?curve=24>. ↩ <https://intelligence.org/2014/05/12/exponential-and-non-exponential/#identifier_10_11027> 8. Our raw data are here and show a continuing exponential trend in MIPS per dollar, adjusted for inflation. https://docs.google.com/spreadsheets/d/1qPBpgqxHsqQgcLLXJ5H-4yto9SPQinR4H0f9p5Dh4g4/edit#gid=952780094
