Monthly Archives: August 2011

Black Hole Projects

OK, so I may have reset my blog, but there were some interesting posts that probably shouldn’t disappear totally down the memory hole. This is one of them, which I am rescuing from back in 2004 because of it’s continuing relevance. It seems that six months can’t go by without something I hear making me think of this. Edited from the original for clarity and to bring it up-to-date.

Many, many years ago, Steve Maine took the opportunity to reminisce about a project at Microsoft that was being worked on while he was an intern. He says:

[ When I was an intern… ] there was this mythical project codenamed “Netdocs”, and it was a black hole into which entire teams disappeared. I had several intern friends who got transferred to the Netdocs team and were never heard from again. Everyone knew that Netdocs was huge and that there were a ton of people working on it, but nobody had any idea what the project actually did.

I also knew a few people who invested quite a few years of their lives into “Netdocs” and it got me thinking about the phenomenon of “black hole projects” at Microsoft (and elsewhere, I’ll wager). There was one I was very close to very early in my career that I managed to avoid, many others that I just watched from afar, and one or two that I got dragged into despite my best intentions. I can’t really talk about most of them since most never saw the light of day, but it did get me thinking about the peculiarly immutable traits of a black hole project. They seem to be:

  • They must have absurdly grandiose goals. Something like “fundamentally reimagine the way that people work with computers.” Nobody, including the people who originate the goals, has a clear idea what the goals actually mean.
  • They must involve throwing out some large existing codebase and rewriting everything from scratch, “the right way, this time.”
  • They must have completely unrealistic deadlines. Often this is because they believe that they can rewrite the original codebase in much, much less time than it took to write that codebase in the first place.
  • They must have completely unrealistic beliefs about compatibility. Usually this takes the form of believing you can rewrite a huge codebase and preserve all of its little quirks without a massive amount of extra effort.
  • They are always “six months” from from major deadlines that never seem to arrive. Or, if they do arrive, another milestone is added on to the end of the project to compensate.
  • They must consume huge amounts of resources, sucking the lifeblood out of one or more established products that make significant amounts of money or have significant market share.
  • They must take over any group that does anything that relates to their absurdly broad goals, especially if that group is small, focused, has modest goals, and actually has a hope of shipping in a reasonable timeframe.
  • They must be prominently featured as demos in public settings such as company meetings, all-hands, conferences, etc. to the point where people groan “Oh, god, not another demo of this thing. When is it ever going to ship?”
  • They usually are prominently talked up publicly by high level executives for years before dying a quiet death.
  • They usually involve “componentizing” some monolithic application or system. This means that not only are you rewriting a huge amount of code, you’re also splitting it up across one or more teams that have to all seamlessly work together.
  • As a result of the previous point, they also usually involve absolutely massive integration problems as different teams try madly to get their components working with each other.
  • They usually involve rewriting the application or system on top of brand-new technology that has not been proven at a large scale yet. As such, they get to flush out all the scalability problems with the new technology.
  • They are usually led by one or more Captain Ahabs, madly pursuing the white whale with absolute conviction, while the deckhands stand around saying “Gee, that whale looks awfully big. I’m not sure we can really take him down.”
  • Finally, 90% of the time, they must fail and die a flaming death, taking down other products with it (or at least severely damaging them). If they do ship, they must have taken at least 4-5 years to ship and be at least 2 years overdue.

It’s kind of frightening how easy it is to come up with this list – it all kind of just poured out. Looking back over 19 years at Microsoft, I’m also kind of frightened at how many projects this describes. Including some projects that are ongoing at the moment…

You should also follow me on Twitter here.