http://online.wsj.com/article_print/0,,SB112743680328349448,00.html
Code Red
Battling Google, Microsoft Changes How It Builds Software
Delay in New Windows Version Drove Giant to Develop Simpler, Flexible
Product
Engineers Get Trip to 'Bug Jail'
By ROBERT A. GUTH
Staff Reporter of THE WALL STREET JOURNAL
September 23, 2005; Page A1
REDMOND, Wash. -- Jim Allchin, a senior Microsoft Corp. executive, walked
into Bill Gates's office here one day in July last year to deliver a
bombshell about the next generation of Microsoft Windows.
"It's not going to work," Mr. Allchin says he told the Microsoft chairman.
The new version, code-named Longhorn, was so complex its writers would never
be able to make it run properly.
The news got even worse: Longhorn was irredeemable because Microsoft
engineers were building it just as they had always built software.
Throughout its history, Microsoft had let thousands of programmers each
produce their own piece of computer code, then stitched it together into one
sprawling program. Now, Mr. Allchin argued, the jig was up. Microsoft needed
to start over.
Mr. Gates resisted at first, pushing for Mr. Allchin's group to take more
time until everything worked. Over the next few months, Mr. Allchin and his
deputies would also face protests from programmers who complained he was
trying to impose bureaucracy and rob Microsoft of its creativity.
"There was some angst by everybody," says Mr. Gates of the period. "It's
obviously my role to ask people, 'Hey, let's not throw things out we
shouldn't throw out. Let's keep things in that we can keep in.' "
Ultimately, Mr. Allchin's warning proved cathartic and led to what he and
others call a transformation in Microsoft's most important product. A key
reason: the growing threat from rivals such as Google Inc., Apple Computer
Inc. and makers of the free Linux operating system. In recent years these
companies have been dashing out some software innovations faster than
Microsoft. Google has grown particularly effective at introducing new
programs such as email and instant messaging over the Internet, watching how
they perform and regularly replacing them with improved versions.
Microsoft's Windows can't entirely replicate that approach, since the
software is by its nature a massive program overseeing all of a computer's
functions. But Microsoft is now racing to move in that direction: developing
a solid core for Windows onto which new features can be added one by one
over time.
As always, Microsoft's great fear is that it will lose its near-monopoly on
computer operating systems and basic office software. In the short term,
there is little danger of that. But the more Google and other software
makers encroach on Microsoft's turf, the greater the chance that someday
computer users will wake up and find Microsoft Windows superfluous.
"What happened when the American car companies failed to update their
manufacturing lines? There was a more efficient way to bring cars to market
for a lower price and they lost their market," says Microsoft Vice President
Chris Jones. "We're in a little bit of a different industry but it's the
same thing."
Microsoft's holy grail is a system that cranks out a new, generally bug-free
version of basic Windows every few years, with frequent updates in between
to add enhancements or match a competitor's offering.
The Longhorn crisis helps explain the sweeping restructuring that Microsoft
Chief Executive Steve Ballmer announced this week to organize the company
into three major business units. A key goal is to force Microsoft to be more
nimble in producing and delivering software.
Mr. Allchin's reforms address a problem dating to Microsoft's beginnings.
Old-school computer science called for methodical coding practices to ensure
that the large computers used by banks, governments and scientists wouldn't
break. But as personal computers took off in the 1980s, companies like
Microsoft didn't have time for that. PC users wanted cool and useful
features quickly. They tolerated -- or didn't notice -- the bugs riddling
the software. Problems could always be patched over. With each patch and
enhancement, it became harder to strap new features onto the software since
new code could affect everything else in unpredictable ways.
The 53-year-old Mr. Allchin, who joined Microsoft in 1990 and is now co-head
of the Platform Products and Services Division, says he always disdained the
fast-and-loose culture of PC software. The holder of a doctorate in computer
science, Mr. Allchin craved discipline in code writing. But in the booming
1990s, when it seemed Microsoft could do no wrong, there was little Mr.
Allchin could do. As soon as Microsoft was done with one version it pushed
on to the next. Mr. Allchin was haunted by what he calls his "little
demons."
In 2001 Microsoft made a documentary film celebrating the creation of
Windows XP, which remains the latest full update of Windows. When Mr.
Allchin previewed the film, it confirmed some of his misgivings about the
Windows culture. He saw the eleventh-hour heroics needed to finish the
product and get it to customers. Mr. Allchin ordered the film to be burned.
When the Longhorn project to build an XP successor got started, teams of
engineers set off to develop it as they always had. Mr. Gates was especially
eager for them to add a fundamental change to Windows called WinFS that
would let PC users search and organize information better. One goal was to
let users scour their entire computer for work they had done on a subject
without needing to go through every individual program or document.
Mr. Allchin says he soon saw his fears realized. In making large software
programs engineers regularly bring together all the new unfinished features
into a single "build," a sort of prototype used to test how the features
work together. Ideally, engineers make a fresh build every night, fix any
bugs and go back to refining their features the next day. But with 4,000
engineers writing code each day, testing the build became a Sisyphean task.
When a bug popped up, trouble-shooters would often have to manually search
through thousands of lines of code to find the problem.
Mr. Gates's WinFS project was so troublesome that engineers began talking
about whether they could make the "pig fly." Images of pigs with wings
started appearing in presentations and offices.
And Microsoft's culture was facing a new threat. The mass of patches and
agglomerations that made up Windows turned it into an easy target for
viruses and other Web-based attacks. Mr. Allchin had to divert top engineers
into the effort to fix security problems in existing versions of Windows.
"The ship was just crashing to the ground," Mr. Allchin says.
In late 2003, Mr. Allchin called on the help of two men. The first was one
of Microsoft's best-known "shippers," people known for their ability to turn
around troubled software projects. Windows veteran Brian Valentine had a
reputation for booming motivational speeches, beer bashes and stunts like
showing up to work functions as Elvis, the Easter Bunny or even once a hula
girl with a coconut bra.
The second man Mr. Allchin tapped was Amitabh Srivastava, now 49, a fellow
purist among computer scientists. A newcomer to the Windows group, Mr.
Srivastava had his team draw up a map of how Windows' pieces fit together.
It was 8 feet tall and 11 feet wide and looked like a haphazard train map
with hundreds of tracks crisscrossing each other.
[Amitabh Srivastava]
That was just the opposite of how Microsoft's new rivals worked. Google and
others developed test versions of software and shipped them over the
Internet. The best of the programs from rivals were like Lego blocks -- they
had a single function and were designed to be connected onto a larger whole.
Google and even Microsoft's own MSN online unit could quickly respond to
changes in the way people used their PCs and the Web by adding incremental
improvements.
In April 2004, Google, seemingly out of nowhere, introduced its Gmail
service, competing with Microsoft's Hotmail program. Tiny Internet browser
maker Mozilla Foundation beat Microsoft to market with browser features
planned for Longhorn.
Most alarming: By July 2004, it became clear that Google was working on a
"desktop search" tool for finding information on a PC -- offering some of
the features that Mr. Gates's WinFS program was supposed to bring to
Longhorn. Google, previously focused exclusively on the Internet, was now
stepping onto Microsoft's turf as the creator of software inside the PC.
While Windows itself couldn't be a single module -- it had too many
functions for that -- it could be designed so that Microsoft could easily
plug in or pull out new features without disrupting the whole system. That
was a cornerstone of a plan Messrs. Srivastava and Valentine proposed to
their boss, Mr. Allchin. Microsoft would have to throw out years of computer
code in Longhorn and start out with a fresh base. It would set up computers
to automatically reject bug-laden code. The new Longhorn would have to be
simple. It would leave bells and whistles for later -- including Mr. Gates's
WinFS, Messrs. Srivastava and Allchin say.
Mr. Allchin signed on to the plan and broke the news to Messrs. Gates and
Ballmer. Mr. Allchin remembers that Mr. Gates pushed him to keep going with
the original version of Longhorn, saying if the software writers needed more
time Microsoft could ship a scaled-down version in the interim. The
executives agreed to reserve a final decision until Mr. Ballmer returned
from a business trip, according to Mr. Allchin and Mr. Valentine, who was
also present.
Over the next few weeks, Mr. Gates expressed frustration. At one meeting on
Aug. 17, he berated Longhorn engineers for the mess, say people familiar
with the meeting. (Mr. Gates says he doesn't remember it.) Afterward, Mr.
Srivastava says he called his team together, acknowledging that he had
underestimated the scope of the challenge they faced in fixing Longhorn,
though he was heartened by the group's apparent willingness to change.
As Microsoft's chief software architect, Mr. Gates says that his role is
"almost paradoxical" because he has to push for innovation while being the
"ultimate realist" when problems arise on that quest. In this case, he says
he and Mr. Ballmer needed to make sure that the recommendations from Mr.
Allchin's group were sound.
On Aug. 27, 2004, Microsoft said it would ship Longhorn in the second half
of 2006 -- at least a year late -- and that Mr. Gates's WinFS advance
wouldn't be part of the system. The day before in Microsoft's auditorium,
Mr. Allchin had announced to hundreds of Windows engineers that they would
"reset" Longhorn using a clean base of code that had been developed for a
version of Windows on corporate server computers.
As he started to learn more about Mr. Srivastava's broader plan, Mr. Gates
was concerned that the unproven tools for keeping the Windows core clean
would levy a "tax" on engineers -- in other words, that they would spend so
much time trying to meet Mr. Srivastava's standards that they wouldn't be
able to devise innovations for Windows users. At a meeting on Sept. 8, Mr.
Srivastava's team was walking Mr. Gates through the plan when he challenged
them. Why, he wondered, weren't the reformers asking the mass of Windows
engineers for their view of the changes?
"It was all just, 'Hey, bless this process,' which I was unwilling to do,"
Mr. Gates says. "They're just talking about process and I'm frustrated we're
not talking about how the teams are responding to it."
By late October, Mr. Srivastava's team was beginning to automate the testing
that had historically been done by hand. If a feature had too many bugs,
software "gates" rejected it from being used in Longhorn. If engineers had
too many outstanding bugs they were tossed in "bug jail" and banned from
writing new code. The goal, he says, was to get engineers to "do it right
the first time."
Recognizing Mr. Gates's concerns over the impact on programmers, Mr.
Srivastava hit on a plan to win their hearts and minds. On Nov. 5, he
visited the computer-filled office of Dave Cutler, a revered elder statesman
among Windows engineers and a stickler for good code writing. Would he
publicly throw his weight behind the new approach? Mr. Srivastava asked.
On Dec. 1, Mr. Srivastava escorted Mr. Cutler to Microsoft's auditorium
where the software guru told 1,000 engineers that he had used the tools to
build Windows code that was nearly bug-free. That Mr. Cutler -- famous for
never attending meetings -- would emerge to back Mr. Allchin's revolution
helped persuade some engineers to drop their objections.
Others weren't so easily convinced. Responding to an attendee questioning
the merits of the new regime, Mr. Valentine, the enforcer, shot back, "Is
your code perfect? Are you perfect? If not, you should shut up and support
this effort," according to one of his team members, G.S. Rana. (Mr.
Valentine says he doesn't remember the remarks but doesn't dispute Mr.
Rana's recollection.)
As engineers began cooperating and Mr. Srivastava's team worked overtime to
refine the tools, the quality of the code flowing into Longhorn began to
improve. The time to create a new "build" fell to just a few days, allowing
a faster cycle of writing and testing new code. After the Windows group was
able to install a workable version of the system on their PCs four days
before Christmas, Mr. Srivastava says the group celebrated by not working
over the holidays.
Not everything went so quickly, as engineers grappled with the challenge of
making Longhorn more like Lego blocks. Microsoft missed its June deadline
for the first "beta" or test version of Longhorn. On the Fourth of July Mr.
Srivastava monitored the progress on his wireless laptop, set up next to his
grill as he cooked veggie burgers and teriyaki chicken for family guests.
Mr. Srivastava was so preoccupied with Longhorn that he inadvertently agreed
to his wife's plan to remodel their bedroom. He recalls that when he
protested, she joked, "You got the Windows job. I get this. It's a small
price to pay."
On July 27, Microsoft shipped the beta of Longhorn -- now named Windows
Vista -- to 500,000 customers for testing. Experience had told the Windows
team to expect tens of thousands of reported problems from customers.
Instead, there were a couple thousand problem reports, says Mr. Rana, the
team member.
And last month, Microsoft delivered a test version of Mr. Gates's WinFS idea
-- not as a part of Longhorn but as a planned add-on feature. Microsoft this
month said it would issue monthly test versions of Windows Vista, a first
for the company and a sign of the group's improved agility.
It could take years before Windows can be as flexible as Microsoft needs it
to be to pump out new features quickly. But the cultural shift is in swing.
Hours after showing off Windows Vista to software makers this month, Mr.
Gates in an interview noted how Microsoft's Office group is now using some
of Mr. Srivastava's tools to improve its code. "It's amazing the invention
those guys have brought forward," he said. "I wish we'd done it earlier."
This week Mr. Allchin announced that as part of the restructuring he will
retire next year after Windows Vista is in customers' hands. In a recent
interview he said his demons aren't fully exorcised. "There're weaknesses in
everything we're doing today," Mr. Allchin says. "But it's such a huge step
up from where we were."
Write to Robert A. Guth at [EMAIL PROTECTED]
URL for this article:
http://online.wsj.com/article/0,,SB112743680328349448,00.html
Hyperlinks in this Article:
(1) http://online.wsj.com/article/0,,SB112735486532848253,00.html
(2) http://online.wsj.com/article/0,,SB112724135950746370,00.html
(3) http://online.wsj.com/article/0,,SB112680183176941893,00.html
(4) mailto:[EMAIL PROTECTED]
You are a subscribed member of the infowarrior list. Visit
www.infowarrior.org for list information or to unsubscribe. This message
may be redistributed freely in its entirety. Any and all copyrights
appearing in list messages are maintained by their respective owners.
