Black hole projects

After hearing about a product named Netdoc from Scoble, Steve Maine takes the opportunity to reminsce about a similarly code-named project at Microsoft (that has nothing to do with the new product). He says:

The name “Netdocs” reminds me of my experience as an intern at MS in 2000. There was this mythical project codenamed “Netdocs”, and it was a black hole into which entire teams disappeared. I had several intern friends who got transferred to the Netdocs team and were never heard from again. Everyone knew that Netdocs was huge and that there were a ton of people working on it, but nobody had any idea what the project actually did.

I left Office just about the time that Netdocs really started going, but I do know a few people who invested quite a few years of their lives into it. I can’t say that I know much more than Steve about it, but it did get me thinking about other “black hole projects” at Microsoft. There was one I was very close to earlier in my career that I managed not to get myself sucked into and several others that I just watched from afar. None I can really talk about since they never saw the light of day, but it did get me thinking about the peculiar traits of a black hole project. They seem to be:

  • They must have absurdly grandiose goals. Something like “fundamentally reimagine the way that people work with computers.” Nobody, including the people who originate the goals, has a clear idea what the goals actually mean.
  • They must involve throwing out some large existing codebase and rewriting everything from scratch, “the right way, this time.”
  • They must have completely unrealistic deadlines. Usually this is because they believe that they can rewrite the original codebase in much, much less time than it took to write that codebase in the first place.
  • They must have completely unrealistic beliefs about compatibility. Usually this takes the form of believing you can rewrite a huge codebase and preserve all of the little quirks and such without a massive amount of extra effort.
  • They are always “six months” from from major deadline that never seems to arrive. Or, if it does arrive, another milestone is added on to the end of the project to compensate.
  • They must consume huge amounts of resources, sucking the lifeblood out of one or more established products that make significant amounts of money or have significant marketshare.
  • They must take over any group that does anything that relates to their absurdly broad goals, especially if that group is small, focused, has modest goals and actually has a hope of shipping in a reasonable timeframe.
  • They must be prominently featured as demos at several company meetings, to the point where people groan “Oh, god, not another demo of this thing. When is it ever going to ship?”
  • They usually are prominently talked up by BillG publicly years before shipping/dying a quiet death.
  • They usually involve “componetizing” some monolithic application or system. This means that not only are you rewriting a huge amount of code, you’re also splitting it up across one or more teams that have to all seamlessly work together.
  • As a result of the previous point, they also usually involve absolutely massive integration problems as different teams try madly to get their components working with each other.
  • They usually involve rewriting the application or system on top of brand-new technology that has not been proven at a large scale yet. As such, they get to flush out all the scalability problems with the new technology.
  • They are usually led by one or more Captain Ahabs, madly pursuing the white whale with absolute conviction, while the deckhands stand around saying “Gee, that whale looks awfully big. I’m not sure we can really take him down.”
  • Finally, 90% of the time, they must fail and die a flaming death, possibly taking down or damaging other products with it. If they do ship, they must have taken at least 4-5 years to ship and be at least 2 years overdue.

I’m kind of frightened at how easy it was to come up with this list – it all just kind of poured out. Looking back over 12.5 years at Microsoft, I’m also kind of frightened at how many projects this describes. Including some projects that are ongoing at the moment…

]]>

46 thoughts on “Black hole projects

  1. RichB

    < ![CDATA[The forerunner to MS Access is well known as a black hole project which prompted Microsoft to poach Adam Bosworth from Borland to come in and rescue Microsoft's desktop database development. I believe this is the product Steve Maguire talks about in "Writing Solid Code" There has also been at least one attempted MSWord rewrite. I know people who were on the the Word team 10 years ago and said the MSWord codebase was dreadful – but keeping a dreadful codebase limping along proved simpler than rewriting.]]>

    Reply
  2. Eric Lippert

    < ![CDATA[Well, sure, it's _easy_ to look at black hole projects after they died a flaming death and say that hey, that was an awful black hole project. What’s _hard_ is identifying which are going to be the flaming-death black hole products and which are actually going to be the
    successful "fundamental change" products. For YEARS I was a huge .NET skeptic. But now, five years later, I’m a huge .NET proponent. It’s really changed the way I approach large-scale software development, for the better.

    .NET does to COM all those things that you mentioned — it had grandiose goals, wrote an entire threading system, memory allocator, security system, etc, from scratch, had absurd deadlines, has lots of weird compatibility issues with existing COM code, sucked the life out of scripting and OLE Automation, etc, etc, etc.

    That’s just life when you develop software at the level we do. We take huge risks — multi-billion-dollar risks — and when you take huge risks, lots of them don’t pay off. Ironically, those failures are a sign that we’re pursuing the RIGHT level of risk. If every project was so low risk that it succeeded, we wouldn’t be pushing the envelope hard enough; some competitor willing to take on risk would push harder and would eventually beat us.]]>

    Reply
  3. Barry Dorrans

    < ![CDATA[What about the cool factor? Blackhole projects and their goals seem cool to geeks, so they clammer to join, even if they know deep down it's doomed. Heck the whole .com era was a bunch of companies based on blackhole projects.]]>

    Reply
  4. Steven Engelhardt

    < ![CDATA[I worked on NetDocs as a new college graduate SDE from July 2000 - August 2001 (at which point I moved away from Seattle for personal reasons). I got to see the death rattle of NetDocs and phoenix-like re-emergence as XDocs in February 2001 first-hand. I spent my time focusing on solving the techy problems and didn’t spend much time pondering the big picture or business implications of the product. I was young and a latecomer so I only heard second-hand of NetDocs ever-changing mission or corporate positioning. It would have been more enlightening, if horrificly more heartbreaking, to have been with NetDocs from the beginning; I only have an extremely vague guess who the "Captain Ahab"s of the project would be. However, one thing I learned exceptionally clearly was how technologies could be, and were, horribly misapplied. It saddens me that I’ve seen some of the same tech mistakes at my current company (and, believe it or not, some even worse). Regrettably, I arrived too late at my current company to prevent them from going through the same minefield; instead, I left the group. "Never again."]]>

    Reply
  5. Pingback: Rob Caron's Blog

  6. Pingback: Steve Makofsky's WebLog

  7. Joe Web Surfer

    < ![CDATA[Microsoft's not alone. I spent 5 years on Apple's failed "Pink" OS project in the 90's. And Netscape wasted years on Mozilla, but I guess that’s sort of succeededing now, so maybe it was worth doing. I think MS succeeds more often at big ambitious rewrite projects (NT, IE 3) than most other companies do. That’s probably one reason why they keep trying.]]>

    Reply
  8. Tonetheman

    < ![CDATA[I have worked on a couple of large black hole projects, mainly at MCI but it was always sad to do such work I think. It is interesting to hear that MS suffers the same fate on such large projects. I wish there was a place to have bad projects like those documented in detail. Edward Yourdon wrote a great book about such things called Death March.]]>

    Reply
  9. Pingback: Panopticon Central

  10. Pingback: Kevin Harder

  11. Pingback: Kevin Harder

  12. Pingback: Cook Computing

  13. Chris Kinsman

    You must mean Omega. A product in the Win 2.x days that was going to require 2MB of Expanded Memory just to start it!

    Reply
  14. Pingback: TomorrowYesterday

  15. Pingback: TomorrowYesterday

  16. Pingback: decheung - The Dennis T Cheung B

  17. anon

    Avalon….. well at least there’s a CTP now πŸ™‚ so we’ll see. I couldn’t help but chime in on the accuracy of your bullet points.

    Reply
  18. Christopher Coulter

    < ![CDATA[Might I add a few? πŸ™‚  They must be talked about constantly to the press, and worked-up into a soapy-lather by Public Relations firms, who are all reading from robotic talking-point scripts. Any good news or steady progress reports are to be leaked to the press.  They must foster a big money-wasting external Community; junkets and freebies alive. Such "enthusiasts" can be called-up on an emergency basis to give faint creedence to a "market". Any naysayers, are to be cut off and shunned. Be careful not to ask people for "feedback", just give them the script.  They must be the "hot meme" that every blogger is claiming will save the earth and cure cancer, simulateously. They must be the hot-topic among all the the A-List’ers. They must be filmed, packaged, and hyped up on company-sponsored blog projects. They must go out of their way to wine and dine these "new influencers".  They must be the featured, “hot item”, ‘can’t-miss-this-demo’, at all Developer Conferences. They must pull external projects away from the here-and-now present and into the murky future.  If the project fails, blame is collective. Failure is good, that much closer to success. But if it succeeds, credit is singular, one-man with one never-wavering vision made it all happen.  They must never be called a failure. Everything is recycled. Even if a failure, we will use it again in some other form. Also, nothing is ever a failure, just add more time. Soviet-like 5 year plans. All the long-term, "long-tail" talk.]]>

    Reply
  19. Pingback: David's blog

  20. Shaded

    < ![CDATA[Although there are other definitions associated with 'hole,' I prefer the most frequently used: A hollow space in a solid object. I believe choice of words "black" and “hole" to describe the strongest force in our universe was no coincidence. Nobody really knows what goes on inside celestial black holes, only because we have not yet been there and back again. Impossible? Maybe you’re thinking too small. They used to think the speed of sound was a brick wall in the sky waiting for people to run into it. Heck they used to think the earth was flat, and who can blame them? I’m sure we all did as children. My feeling is "Sign me up." Two words for anyone who can’t handle the pressure in Microsoft: Get Out. I for one am ready willing and able to take the job, I’m sure there are others. You want low risk predictable comfort? Go flip burgers. Not getting recognition? Welcome to the adult world. Neither is the guy that picks up your trash every week, but I don’t see him whining about the mayor getting all the credit.
    ]]>

    Reply
  21. Pingback: Signs on the Sand

  22. Pingback: Netdoc Blog

  23. Pingback: Arpan Desai's WebLog

  24. Pingback: XmlTeam's WebLog

  25. Pingback: The Cloak of Wisdom

  26. Pingback: The Cloak of Wisdom

  27. Jim

    < ![CDATA[I worked two years on a classic black whole product when I was the first developer at Dr[name withheld].com. We didn’t have a Captain Ahab, but we had an ex-Microsoftie who had worked on Outlook, so he convinced the boss that we needed to write what amounted to a new operating sytem with an outlook clone at its center. I learned a lot about windows programming from that experience, but quit when the Cloud of Doom became too much. I really liked the management team, and I just felt bad about helping to channel their money into this black whole.]]>

    Reply
  28. Pingback: Jiri's Notepad

  29. Alex

    < ![CDATA[Does anyone have any good models i could build for highschool project???? PLEASE Write back..
    Alex. earl of march, ont.]]>

    Reply
  30. Chris

    I wonder which of those bullet points netdocs had going for it to send it down the great black hole. Anyone know? I’m curious now

    Reply
  31. Pingback: Panopticon Central

  32. Pingback: Grammarware, Haskellware, XMLwar

Leave a Reply