Black hole projects

After hearing about a product named Netdoc from Scoble, Steve Maine takes the opportunity to reminsce about a similarly code-named project at Microsoft (that has nothing to do with the new product). He says:

The name “Netdocs” reminds me of my experience as an intern at MS in 2000. There was this mythical project codenamed “Netdocs”, and it was a black hole into which entire teams disappeared. I had several intern friends who got transferred to the Netdocs team and were never heard from again. Everyone knew that Netdocs was huge and that there were a ton of people working on it, but nobody had any idea what the project actually did.

I left Office just about the time that Netdocs really started going, but I do know a few people who invested quite a few years of their lives into it. I can’t say that I know much more than Steve about it, but it did get me thinking about other “black hole projects” at Microsoft. There was one I was very close to earlier in my career that I managed not to get myself sucked into and several others that I just watched from afar. None I can really talk about since they never saw the light of day, but it did get me thinking about the peculiar traits of a black hole project. They seem to be:

  • They must have absurdly grandiose goals. Something like “fundamentally reimagine the way that people work with computers.” Nobody, including the people who originate the goals, has a clear idea what the goals actually mean.
  • They must involve throwing out some large existing codebase and rewriting everything from scratch, “the right way, this time.”
  • They must have completely unrealistic deadlines. Usually this is because they believe that they can rewrite the original codebase in much, much less time than it took to write that codebase in the first place.
  • They must have completely unrealistic beliefs about compatibility. Usually this takes the form of believing you can rewrite a huge codebase and preserve all of the little quirks and such without a massive amount of extra effort.
  • They are always “six months” from from major deadline that never seems to arrive. Or, if it does arrive, another milestone is added on to the end of the project to compensate.
  • They must consume huge amounts of resources, sucking the lifeblood out of one or more established products that make significant amounts of money or have significant marketshare.
  • They must take over any group that does anything that relates to their absurdly broad goals, especially if that group is small, focused, has modest goals and actually has a hope of shipping in a reasonable timeframe.
  • They must be prominently featured as demos at several company meetings, to the point where people groan “Oh, god, not another demo of this thing. When is it ever going to ship?”
  • They usually are prominently talked up by BillG publicly years before shipping/dying a quiet death.
  • They usually involve “componetizing” some monolithic application or system. This means that not only are you rewriting a huge amount of code, you’re also splitting it up across one or more teams that have to all seamlessly work together.
  • As a result of the previous point, they also usually involve absolutely massive integration problems as different teams try madly to get their components working with each other.
  • They usually involve rewriting the application or system on top of brand-new technology that has not been proven at a large scale yet. As such, they get to flush out all the scalability problems with the new technology.
  • They are usually led by one or more Captain Ahabs, madly pursuing the white whale with absolute conviction, while the deckhands stand around saying “Gee, that whale looks awfully big. I’m not sure we can really take him down.”
  • Finally, 90% of the time, they must fail and die a flaming death, possibly taking down or damaging other products with it. If they do ship, they must have taken at least 4-5 years to ship and be at least 2 years overdue.

I’m kind of frightened at how easy it was to come up with this list – it all just kind of poured out. Looking back over 12.5 years at Microsoft, I’m also kind of frightened at how many projects this describes. Including some projects that are ongoing at the moment…


46 thoughts on “Black hole projects

  1. Eric Lippert

    successful "fundamental change" products. For YEARS I was a huge .NET skeptic. But now, five years later, I’m a huge .NET proponent. It’s really changed the way I approach large-scale software development, for the better. .NET does to COM all those things that you mentioned — it had grandiose goals, wrote an entire threading system, memory allocator, security system, etc, from scratch, had absurd deadlines, has lots of weird compatibility issues with existing COM code, sucked the life out of scripting and OLE Automation, etc, etc, etc. That’s just life when you develop software at the level we do. We take huge risks — multi-billion-dollar risks — and when you take huge risks, lots of them don’t pay off. Ironically, those failures are a sign that we’re pursuing the RIGHT level of risk. If every project was so low risk that it succeeded, we wouldn’t be pushing the envelope hard enough; some competitor willing to take on risk would push harder and would eventually beat us.]]>

  2. Pingback: Rob Caron's Blog

  3. Pingback: Steve Makofsky's WebLog

  4. Pingback: Panopticon Central

  5. Pingback: Kevin Harder

  6. Pingback: Kevin Harder

  7. Pingback: Cook Computing

  8. Chris Kinsman

    You must mean Omega. A product in the Win 2.x days that was going to require 2MB of Expanded Memory just to start it!

  9. Pingback: TomorrowYesterday

  10. Pingback: TomorrowYesterday

  11. Pingback: decheung - The Dennis T Cheung B

  12. anon

    Avalon….. well at least there’s a CTP now πŸ™‚ so we’ll see. I couldn’t help but chime in on the accuracy of your bullet points.

  13. Pingback: David's blog

  14. Pingback: Signs on the Sand

  15. Pingback: Netdoc Blog

  16. Pingback: Arpan Desai's WebLog

  17. Pingback: XmlTeam's WebLog

  18. Pingback: The Cloak of Wisdom

  19. Pingback: The Cloak of Wisdom

  20. Pingback: Jiri's Notepad

  21. Chris

    I wonder which of those bullet points netdocs had going for it to send it down the great black hole. Anyone know? I’m curious now

  22. Pingback: Panopticon Central

  23. Pingback: Grammarware, Haskellware, XMLwar

Leave a Reply