Dogfooding and Microsoft

Every corporate culture has it’s own set of acronyms, TLAs (three letter abbreviations) and jargon, and Microsoft is no different. I try not to let it slip too much into my blog entries, but a comment from M.J. Easton reminded me that a while back I did use one without explanation. In an entry talking about the DirectCast operator, I said:

In addition to the fact that we like VB, it’s also a great way to dogfood the product.

I don’t believe the verb “to dogfood” is unique to Microsoft at all, but it’s certainly an integral part of our culture. It’s short for “to eat one’s own dogfood,” which means “to use the product yourself that you are trying to sell to your customers.” The purpose of dogfooding is severalfold, but the main reasons are:

1) It proves to customers that we believe in the product.

2) Because dogfooding usually means using beta (or pre-beta) software, it helps flush more bugs out of the product.

3) It makes us suffer the same bugs and design flaws that we inflict on users, thus giving us incentive to fix them.

4) It’s a valuable reality check that the product is actually as good as we say it is.

5) Because Microsoft is such a large organization, it can flush out problems that could not otherwise be found prior to full-scale rollout at launch. (This holds especially true for corporate server products such as Exchange, SQL, IIS, etc.)

6) We learn how our products actually work, which is more often than not not exactly how we think they work.

All in all, dogfooding is a extremely valuable, if not sometimes painful, thing that we do at Microsoft.

(You can find a deeper discussion of the etymology of the word in the Wikipedia entry on it.)

Was .NET a black hole project?

Eric Lippert astutely pointed out a hole that I consciously left in my discussion of black hole projects – namely, that they sometimes succeed. I left that part out because I figured it probably merited a little discussion of its own and I didn’t want to complicate the whole “black hole” narrative.

Eric’s completely correct that you could argue that .NET was a black hole project, albeit one that succeeded. It managed to hit most, if not all, of the bullet points I listed, including the last one (I started work on what would become VB.NET in mid-1998, after all, and some people had been working on it for several months by that point). I distinctly remember the first meeting I had with Brian Harry where he outlined a proposal for what would become the metadata engine in .NET. I thought to myself “this guy is completely crazy” because of his grandiose goals. A few months later, I remember sitting in a co-workers office talking about the emerging .NET strategy. “Do you think this thing will actually fly?” she asked me. “I have no idea.” I replied, “I give it about 50/50 chance. I guess we’ll see.”

However, I disagree with Eric a bit that it’s extremely hard to distinguish between a black hole project that’s going to impode and a black hole project that’s going to succeed. I think there are some traits that distinguish the projects that have succeeded. I’m less certain about them because 4 out of 5 black hole projects have failed, so I have less data to extrapolate from, but here’s my take on it…

The number one, no question about it, don’t leave home without it, trait of succesful black hole projects is:

  • Despite having grandiose goals, they are fundamentally a response to a serious and concrete competitive threat.

If .NET had started just as some abstract project to change the way that people program, then I’d be willing to wager that it would have gone down in flames at some point. What kept the teams focused and moving forward was the fact that, let’s be honest here, Java was breathing down Microsoft’s neck. Without that serious competitive pressure, odds are pretty decent that the .NET project would have eventually collapsed under its own weight because we wouldn’t have had the kinds of checks and balances that competition imposed on us. But the reality of needing to compete with Java helped keep our eyes on the prize, as it were, and counterbalanced the tendency of black hole projects to spin out of control.

Some other traits of successful projects are:

  • They tend to be led by people who, while they may be fantastically idealistic, are ultimately realists when push comes to shove.
  • It also seems to be vitally important that the project leadership be very, very technical. This allows them to stay in touch with what is actually happening in the project rather than having to rely on status reports (which are usually skewed in the optimistic direction) and allows them to ask the hard questions.
  • Hidden under the absurdly grandiose goals must be some set of real life problems that real life customers need solved.

I think what I’m starting to get at is that the line that divides success from failure is ultimately how in touch with reality the project team is. The Catch-22 is that to make large leaps, you often have to unmoor yourself a bit from reality so that you can see beyond what is into what could be. But once you’ve unmoored yourself from reality, it’s easy to go all the way and completely lose touch with reality. That’s the point at which things start to spiral out of control. The projects that succeed, in turn, manage to keep one foot in reality at all times. This keeps them from going absolutely insane. And, I think, it’s not as hard as it seems to get a sense of whether a particular project has one foot in reality or not. Usually, all you have to do is talk to a cross-section of the people working on it. They’ll know pretty well what the state of the project is.

I should close by saying my intention wasn’t to knock ambitious projects. I think they’re totally necessary to help our industry make those big leaps forward. It’s more of a question of how people go about doing ambitious projects that I sometimes have a problem with. Projects that think big but try as best they can to stay practical are really the best kind of projects to work on. I can’t say that every moment of working on .NET has been fun, but overall it’s been a wonderful experience and one that I’m very glad I got to be a part of…

Black hole projects

After hearing about a product named Netdoc from Scoble, Steve Maine takes the opportunity to reminsce about a similarly code-named project at Microsoft (that has nothing to do with the new product). He says:

The name “Netdocs” reminds me of my experience as an intern at MS in 2000. There was this mythical project codenamed “Netdocs”, and it was a black hole into which entire teams disappeared. I had several intern friends who got transferred to the Netdocs team and were never heard from again. Everyone knew that Netdocs was huge and that there were a ton of people working on it, but nobody had any idea what the project actually did.

I left Office just about the time that Netdocs really started going, but I do know a few people who invested quite a few years of their lives into it. I can’t say that I know much more than Steve about it, but it did get me thinking about other “black hole projects” at Microsoft. There was one I was very close to earlier in my career that I managed not to get myself sucked into and several others that I just watched from afar. None I can really talk about since they never saw the light of day, but it did get me thinking about the peculiar traits of a black hole project. They seem to be:

  • They must have absurdly grandiose goals. Something like “fundamentally reimagine the way that people work with computers.” Nobody, including the people who originate the goals, has a clear idea what the goals actually mean.
  • They must involve throwing out some large existing codebase and rewriting everything from scratch, “the right way, this time.”
  • They must have completely unrealistic deadlines. Usually this is because they believe that they can rewrite the original codebase in much, much less time than it took to write that codebase in the first place.
  • They must have completely unrealistic beliefs about compatibility. Usually this takes the form of believing you can rewrite a huge codebase and preserve all of the little quirks and such without a massive amount of extra effort.
  • They are always “six months” from from major deadline that never seems to arrive. Or, if it does arrive, another milestone is added on to the end of the project to compensate.
  • They must consume huge amounts of resources, sucking the lifeblood out of one or more established products that make significant amounts of money or have significant marketshare.
  • They must take over any group that does anything that relates to their absurdly broad goals, especially if that group is small, focused, has modest goals and actually has a hope of shipping in a reasonable timeframe.
  • They must be prominently featured as demos at several company meetings, to the point where people groan “Oh, god, not another demo of this thing. When is it ever going to ship?”
  • They usually are prominently talked up by BillG publicly years before shipping/dying a quiet death.
  • They usually involve “componetizing” some monolithic application or system. This means that not only are you rewriting a huge amount of code, you’re also splitting it up across one or more teams that have to all seamlessly work together.
  • As a result of the previous point, they also usually involve absolutely massive integration problems as different teams try madly to get their components working with each other.
  • They usually involve rewriting the application or system on top of brand-new technology that has not been proven at a large scale yet. As such, they get to flush out all the scalability problems with the new technology.
  • They are usually led by one or more Captain Ahabs, madly pursuing the white whale with absolute conviction, while the deckhands stand around saying “Gee, that whale looks awfully big. I’m not sure we can really take him down.”
  • Finally, 90% of the time, they must fail and die a flaming death, possibly taking down or damaging other products with it. If they do ship, they must have taken at least 4-5 years to ship and be at least 2 years overdue.

I’m kind of frightened at how easy it was to come up with this list – it all just kind of poured out. Looking back over 12.5 years at Microsoft, I’m also kind of frightened at how many projects this describes. Including some projects that are ongoing at the moment…