>From a [comment on the Smoothspan blog, by Damon Edwards][0] of dev2ops.org:
> A troubling trend I've noticed is how the benefits of "rock > star" software development teams (small, highly skilled, > highly motivated) are increasingly neutralized by poor > operations. > > In an ops heavy SaaS and on-demand world, the software > development phase becomes an increasingly smaller part of an > application's overall lifecycle. Time and time again we see > great code sitting behind the bottleneck of QA, staging, > performance testing, and then production deployment. > > On the project plan, the "rock star" teams repeatedly deliver > great code in record time... but at all but the smallest of > enterprises, their "delivery" of code is a long way from where > the business is actually realizing the benefit. I don't know whether this is true or false, but if it's true, it represents the reversal of a [trend touted by Philip Greenspun in 1998][1]: > When I graduated from MIT in 1982, my classmates and I had but one > choice if we wanted to get an idea to market: Join a big > organization. When products, even software, needed to be distributed > physically, you needed people to design packaging, write and mail > brochures, set up an assembly line, fill shelves in a warehouse, > fulfill customer orders, etc. We went to work for big companies like > IBM and Hewlett-Packard. Our first rude surprise was learning that > even the best engineers earned a pittance compared with senior > management. Moreover, because of the vast resources that were needed > to turn a working design into an on-sale product, most finished > designs never made it to market. "My project was killed" was the > familiar refrain among Route 128 and Silicon Valley engineers in > 1982. > > How does the Web/db world circa 1998 look to a programmer? If Joe > Programmer can grab an IP address on a computer already running a > well-maintained relational database, he can build an interesting > service in a matter of weeks. By himself. If built for fun, this > service can be delivered free to the entire Internet at minimal > cost. If built for a customer, this service can be launched without > further effort. Either way, there is only a brief period of several > weeks during which a project can be killed. That won't stop the site > from being killed months or years down the road, but very seldom > will a Web programmer build something that never sees the light of > day (during my entire career of Web/db application development, > 1994-1998, I have never wasted time on an application that failed to > reach the public). And [by Paul Graham in 2001][2]: > One of the most important changes in this new world is the way > you do releases. In the desktop software business, doing a > release is a huge trauma, in which the whole company sweats > and strains to push out a single, giant piece of code. Obvious > comparisons suggest themselves, both to the process and the > resulting product. > > With server-based software, you can make changes almost as you > would in a program you were writing for yourself. You release > software as a series of incremental changes instead of an > occasional big explosion. A typical desktop software company > might do one or two releases a year. At Viaweb we often did > three to five releases a day. I see four possible interpretations: 1. Damon Edwards is wrong, and in fact the software development phase is not "becoming an increasingly smaller part of an application's overall lifecycle" as a result of "SaaS", which is what Graham called "server-based software" and what Philip Greenspun called "a service", and in fact the per-user cost to deploy software is continuing to shrink. This is a plausible answer; Moore's Law continues to grind away giving us more MIPS per watt and MIPS per CPU, the cost of bandwidth was still falling last I checked, and perhaps more importantly, EC2 and S3 and Hadoop and Puppet and MogileFS and `aptitude` and Xen and `monit` and Cacti and `backuppc` and `nginx` and `perlbal` and `memcached` and Nagios and Erlang and Varnish and Capistrano are reducing the amount of human effort it takes to administer a given number of CPUs and increasing the number of users each MIPS can support. Maybe Damon sees operations becoming more difficult because the internet is still growing, and so the biggest services have to deal with more users now than in 2001 or 1998. 2. The effort required to deploy software on centralized servers really is growing out of proportion to the effort required to write it in the first place, and that's because of the dependence on centralized servers. If the software could run on the machines of its users, the way Firefox or BitTorrent or Skype or Emacs does, the users would be the ones deploying it. Of course, they might still find that difficult, but that wouldn't be visible to the software authors; they would just see that nobody was using their software, not the hours of frustration expended trying to install it. But a lot of that deployment effort can still be moved into software (that's the point of InstallShield, `aptitude`, `easy_install`, RubyGems, CPAN.pm, Fink, Darwin Ports, Xen, AppEngine, and so on, although the diversity of items in that list suggests that the job is far from over), and with the effort that can't be, people can avoid duplication of effort by sharing solutions online. Just because the software runs on its users' machines doesn't mean it can't be providing a networked service; consider BitTorrent or Skype or, for that matter, Sendmail, ircd, or INN. 3. The effort required is growing, but not because of centralized servers. But I do not know of any other plausible candidates. 4. A [post by Jesse Robbins on O'Reilly Radar][3] suggests that some startups get their operations highly automated early on, so they can easily deploy their changes, while others screw up and end up with a mess, and spend lots of time on operations. If this is correct, then Damon Edwards is wrong in thinking that operations *inevitably* consumes a greater and greater proportion of the resources available; he's just working with dumb groups who dig themselves into big holes. [0]: http://smoothspan.wordpress.com/2007/11/27/why-small-software-teams-grow-large-and-other-software-development-conundrums/#comment-1114 [1]: http://philip.greenspun.com/panda/future "The Future chapter of Philip and Alex's Guide to Web Programming" [2]: http://www.paulgraham.com/road.html "Paul Graham's Road Ahead" [3]: http://radar.oreilly.com/archives/2007/10/operations-advantage.html "Operations is a competitive advantage"
