Panopticon Central

a blog on Visual Basic, .NET and other stuff

  Home :: Contact :: Syndication  :: Login
  490 Posts :: 19 Stories :: 3231 Comments :: 622 Trackbacks

News

The information in this weblog is provided "AS IS" with no warranties, and confers no rights.

My Book

Picture

My Recent Posts

Article Categories

Archives

Post Categories

Microsoft Blogs

Samples

Technical Blogs

VB Links

After hearing about a product named Netdoc from Scoble, Steve Maine takes the opportunity to reminsce about a similarly code-named project at Microsoft (that has nothing to do with the new product). He says:

The name “Netdocs” reminds me of my experience as an intern at MS in 2000. There was this mythical project codenamed “Netdocs”, and it was a black hole into which entire teams disappeared. I had several intern friends who got transferred to the Netdocs team and were never heard from again. Everyone knew that Netdocs was huge and that there were a ton of people working on it, but nobody had any idea what the project actually did.

I left Office just about the time that Netdocs really started going, but I do know a few people who invested quite a few years of their lives into it. I can't say that I know much more than Steve about it, but it did get me thinking about other "black hole projects" at Microsoft. There was one I was very close to earlier in my career that I managed not to get myself sucked into and several others that I just watched from afar. None I can really talk about since they never saw the light of day, but it did get me thinking about the peculiar traits of a black hole project. They seem to be:

  • They must have absurdly grandiose goals. Something like "fundamentally reimagine the way that people work with computers." Nobody, including the people who originate the goals, has a clear idea what the goals actually mean.
  • They must involve throwing out some large existing codebase and rewriting everything from scratch, "the right way, this time."
  • They must have completely unrealistic deadlines. Usually this is because they believe that they can rewrite the original codebase in much, much less time than it took to write that codebase in the first place.
  • They must have completely unrealistic beliefs about compatibility. Usually this takes the form of believing you can rewrite a huge codebase and preserve all of the little quirks and such without a massive amount of extra effort.
  • They are always "six months" from from major deadline that never seems to arrive. Or, if it does arrive, another milestone is added on to the end of the project to compensate.
  • They must consume huge amounts of resources, sucking the lifeblood out of one or more established products that make significant amounts of money or have significant marketshare.
  • They must take over any group that does anything that relates to their absurdly broad goals, especially if that group is small, focused, has modest goals and actually has a hope of shipping in a reasonable timeframe.
  • They must be prominently featured as demos at several company meetings, to the point where people groan "Oh, god, not another demo of this thing. When is it ever going to ship?"
  • They usually are prominently talked up by BillG publicly years before shipping/dying a quiet death.
  • They usually involve "componetizing" some monolithic application or system. This means that not only are you rewriting a huge amount of code, you're also splitting it up across one or more teams that have to all seamlessly work together.
  • As a result of the previous point, they also usually involve absolutely massive integration problems as different teams try madly to get their components working with each other.
  • They usually involve rewriting the application or system on top of brand-new technology that has not been proven at a large scale yet. As such, they get to flush out all the scalability problems with the new technology.
  • They are usually led by one or more Captain Ahabs, madly pursuing the white whale with absolute conviction, while the deckhands stand around saying "Gee, that whale looks awfully big. I'm not sure we can really take him down."
  • Finally, 90% of the time, they must fail and die a flaming death, possibly taking down or damaging other products with it. If they do ship, they must have taken at least 4-5 years to ship and be at least 2 years overdue.

I'm kind of frightened at how easy it was to come up with this list - it all just kind of poured out. Looking back over 12.5 years at Microsoft, I'm also kind of frightened at how many projects this describes. Including some projects that are ongoing at the moment...

posted on Wednesday, December 01, 2004 9:00 AM

Feedback

# re: Black hole projects 12/1/2004 12:26 PM anonymous failure
That reminds me way too much of projects I worked on. And failed.

# re: Black hole projects 12/1/2004 12:52 PM RichB
The forerunner to MS Access is well known as a black hole project which prompted Microsoft to poach Adam Bosworth from Borland to come in and rescue Microsoft's desktop database development. I believe this is the product Steve Maguire talks about in "Writing Solid Code"

There has also been at least one attempted MSWord rewrite. I know people who were on the the Word team 10 years ago and said the MSWord codebase was dreadful - but keeping a dreadful codebase limping along proved simpler than rewriting.

# re: Black hole projects 12/1/2004 1:00 PM Eric Lippert
Well, sure, it's _easy_ to look at black hole projects after they died a flaming death and say that hey, that was an awful black hole project.

What's _hard_ is identifying which are going to be the flaming-death black hole products and which are actually going to be the
successful "fundamental change" products. For YEARS I was a huge .NET skeptic. But now, five years later, I'm a huge .NET proponent. It's really changed the way I approach large-scale software development, for the better.

.NET does to COM all those things that you mentioned -- it had grandiose goals, wrote an entire threading system, memory allocator, security system, etc, from scratch, had absurd deadlines, has lots of weird compatibility issues with existing COM code, sucked the life out of scripting and OLE Automation, etc, etc, etc.

That's just life when you develop software at the level we do. We take huge risks -- multi-billion-dollar risks -- and when you take huge risks, lots of them don't pay off. Ironically, those failures are a sign that we're pursuing the RIGHT level of risk. If every project was so low risk that it succeeded, we wouldn't be pushing the envelope hard enough; some competitor willing to take on risk would push harder and would eventually beat us.

# re: Black hole projects 12/1/2004 2:47 PM Barry Dorrans
What about the cool factor? Blackhole projects and their goals seem cool to geeks, so they clammer to join, even if they know deep down it's doomed.

Heck the whole .com era was a bunch of companies based on blackhole projects.

# re: Black hole projects 12/1/2004 9:55 PM Steven Engelhardt
I worked on NetDocs as a new college graduate SDE from July 2000 - August 2001 (at which point I moved away from Seattle for personal reasons). I got to see the death rattle of NetDocs and phoenix-like re-emergence as XDocs in February 2001 first-hand.

I spent my time focusing on solving the techy problems and didn't spend much time pondering the big picture or business implications of the product. I was young and a latecomer so I only heard second-hand of NetDocs ever-changing mission or corporate positioning. It would have been more enlightening, if horrificly more heartbreaking, to have been with NetDocs from the beginning; I only have an extremely vague guess who the "Captain Ahab"s of the project would be.

However, one thing I learned exceptionally clearly was how technologies could be, and were, horribly misapplied. It saddens me that I've seen some of the same tech mistakes at my current company (and, believe it or not, some even worse). Regrettably, I arrived too late at my current company to prevent them from going through the same minefield; instead, I left the group. "Never again."

# RE: Black hole projects 12/2/2004 12:16 AM Scott Mitchell
Heh, I was an intern @ Microsoft in 1999 and remember the "netdocs" buzz. Tee hee.

# Paul Vick on Black Hole Projects 12/2/2004 4:37 AM Rob Caron's Blog


# Black Hole Projects 12/2/2004 10:16 AM Steve Makofsky's WebLog


# re: Black hole projects 12/2/2004 11:16 AM Joe Web Surfer
Microsoft's not alone. I spent 5 years on Apple's failed "Pink" OS project in the 90's.

And Netscape wasted years on Mozilla, but I guess that's sort of succeededing now, so maybe it was worth doing.

I think MS succeeds more often at big ambitious rewrite projects (NT, IE 3) than most other companies do. That's probably one reason why they keep trying.

# re: Black hole projects 12/2/2004 11:34 AM Tonetheman
I have worked on a couple of large black hole projects, mainly at MCI but it was always sad to do such work I think.

It is interesting to hear that MS suffers the same fate on such large projects. I wish there was a place to have bad projects like those documented in detail.

Edward Yourdon wrote a great book about such things called Death March.

# Was .NET a black hole project? 12/2/2004 3:00 PM Panopticon Central


# Top 10 Signs you are working on a Black Hole Project 12/7/2004 12:12 AM Kevin Harder


# Top 10 Signs you are working on a Black Hole Project 12/7/2004 12:13 AM Kevin Harder


# WinFS Slips Again 12/12/2004 10:09 AM Cook Computing
Seen on Frans Bouma's blog, the news that WinFS has slipped yet again. Is WinFS just a huge blackhole project? Who is the Captain Ahad driving this project on?...

# re: Black hole projects 12/12/2004 9:45 PM Chris Kinsman
You must mean Omega. A product in the Win 2.x days that was going to require 2MB of Expanded Memory just to start it!

# Be concerned if your project sounds like this 12/13/2004 2:42 AM TomorrowYesterday
Black hole projects I'm kind of frightened at how easy it was to come up with this list - it all just kind of poured out. Looking back over 12.5 years at Microsoft, I'm also kind of frightened at how many projects this describes. Including some projects that are ongoing at the moment... A scary checklist to determine if your project is a blackhole....

# Signs you are on a blackhole project 12/13/2004 3:49 AM TomorrowYesterday
Black hole projects They must have absurdly grandiose goals. Something like "fundamentally reimagine the way that people work with computers." Nobody, including the people who originate the goals, has a clear idea what the goals actually mean. They must involve throwing out some large existing codebase and rewriting everything from scratch, "the right way, this time." They must have completely unrealistic deadlines. Usually this is because they believe that they can rewrite the original codebase in much, much less time than it took to write that codebase in the first place. They must have completely unrealistic beliefs about compatibility. Usually this takes the form of believing you can rewrite a huge codebase and preserve all of the little quirks and such without a massive amount of extra effort. They are always "six months" from from major deadline that never seems to arrive. Or, if it does arrive, another milestone is added on to the end of the project to compensate. They must consume huge amounts of resources, sucking the lifeblood out of one or more established products that make significant amounts of money or have significant marketshare. They must take over any group that does anything that relates to their absurdly broad goals, especially if that group is small, focused, has modest goals and actually has a hope of shipping in a reasonable timeframe. They must be prominently featured as demos at several company meetings, to the point where people groan "Oh, god, not another demo of this thing. When is it ever going to ship?" They usually are prominently talked up by BillG publicly years before shipping/dying a quiet death. They usually involve "componetizing" some monolithic application or system. This means that not only are you rewriting a huge amount of code, you're also splitting it up across one or more teams that have to all seamlessly work together. As a result of the previous point, they also usually involve absolutely massive integration problems as different teams try madly to get their components working with each other. They usually involve rewriting the application or system on top of brand-new technology that has not been proven at a large scale yet. As such, they get to flush out all the scalability problems with the new technology. They are usually led by one or more Captain Ahabs, madly pursuing the white whale with absolute conviction, while the deckhands stand around saying "Gee, that whale looks awfully big. I'm not sure we can really take him down." Finally, 90% of the time, they must fail and die a flaming death, possibly taking down or damaging other products with it. If they do ship, they must have taken at least 4-5 years to ship and be at least 2 years overdue. Wow... scary. Is your project a blackhole project? Thanks to Mr. Mini-Microsoft for finding it....

# Signs you are on a blackhole project 12/13/2004 3:50 AM decheung - The Dennis T Cheung Blog
Black hole projects They must have absurdly grandiose goals. Something like "fundamentally reimagine the way that people work with computers." Nobody, including the people who originate the goals, has a clear idea what the goals actually mean. They must involve throwing out some large existing codebase and rewriting everything from scratch, "the right way, this time." They must have completely unrealistic deadlines. Usually this is because they believe that they can rewrite the original codebase in much, much less time than it took to write that codebase in the first place. They must have completely unrealistic beliefs about compatibility. Usually this takes the form of believing you can rewrite a huge codebase and preserve all of the little quirks and such without a massive amount of extra effort. They are always "six months" from from major deadline that never seems to arrive. Or, if it does arrive, another milestone is added on to the end of the project to compensate. They must consume huge amounts of resources, sucking the lifeblood out of one or more established products that make significant amounts of money or have significant marketshare. They must take over any group that does anything that relates to their absurdly broad goals, especially if that group is small, focused, has modest goals and actually has a hope of shipping in a reasonable timeframe. They must be prominently featured as demos at several company meetings, to the point where people groan "Oh, god, not another demo of this thing. When is it ever going to ship?" They usually are prominently talked up by BillG publicly years before shipping/dying a quiet death. They usually involve "componetizing" some monolithic application or system. This means that not only are you rewriting a huge amount of code, you're also splitting it up across one or more teams that have to all seamlessly work together. As a result of the previous point, they also usually involve absolutely massive integration problems as different teams try madly to get their components working with each other. They usually involve rewriting the application or system on top of brand-new technology that has not been proven at a large scale yet. As such, they get to flush out all the scalability problems with the new technology. They are usually led by one or more Captain Ahabs, madly pursuing the white whale with absolute conviction, while the deckhands stand around saying "Gee, that whale looks awfully big. I'm not sure we can really take him down." Finally, 90% of the time, they must fail and die a flaming death, possibly taking down or damaging other products with it. If they do ship, they must have taken at least 4-5 years to ship and be at least 2 years overdue. Wow... scary. Is your project a blackhole project? Thanks to Mr. Mini-Microsoft for finding it....

# re: Black hole projects 12/13/2004 4:23 AM anon
Avalon..... well at least there's a CTP now :) so we'll see. I couldn't help but chime in on the accuracy of your bullet points.

# re: Black hole projects 12/13/2004 5:00 AM Christopher Coulter
Might I add a few? :)

• They must be talked about constantly to the press, and worked-up into a soapy-lather by Public Relations firms, who are all reading from robotic talking-point scripts. Any good news or steady progress reports are to be leaked to the press.

• They must foster a big money-wasting external Community; junkets and freebies alive. Such "enthusiasts" can be called-up on an emergency basis to give faint creedence to a "market". Any naysayers, are to be cut off and shunned. Be careful not to ask people for "feedback", just give them the script.

• They must be the "hot meme" that every blogger is claiming will save the earth and cure cancer, simulateously. They must be the hot-topic among all the the A-List'ers. They must be filmed, packaged, and hyped up on company-sponsored blog projects. They must go out of their way to wine and dine these "new influencers".

• They must be the featured, “hot item”, 'can't-miss-this-demo', at all Developer Conferences. They must pull external projects away from the here-and-now present and into the murky future.

• If the project fails, blame is collective. Failure is good, that much closer to success. But if it succeeds, credit is singular, one-man with one never-wavering vision made it all happen.

• They must never be called a failure. Everything is recycled. Even if a failure, we will use it again in some other form. Also, nothing is ever a failure, just add more time. Soviet-like 5 year plans. All the long-term, "long-tail" talk.

# Does this list sound familiar? 12/13/2004 5:36 PM David's blog


# re: Black hole projects 12/13/2004 7:54 PM Shaded
Although there are other definitions associated with 'hole,' I prefer the most frequently used: A hollow space in a solid object.

I believe choice of words "black" and “hole" to describe the strongest force in our universe was no coincidence. Nobody really knows what goes on inside celestial black holes, only because we have not yet been there and back again. Impossible? Maybe you're thinking too small. They used to think the speed of sound was a brick wall in the sky waiting for people to run into it. Heck they used to think the earth was flat, and who can blame them? I’m sure we all did as children.

My feeling is "Sign me up." Two words for anyone who can't handle the pressure in Microsoft: Get Out. I for one am ready willing and able to take the job, I'm sure there are others.

You want low risk predictable comfort? Go flip burgers.

Not getting recognition? Welcome to the adult world. Neither is the guy that picks up your trash every week, but I don’t see him whining about the mayor getting all the credit.


# A letter from a dead house 12/14/2004 2:32 PM Signs on the Sand
I was doing some catch up reading feeds I'm subscribed and I found this one item that made me feeling some sort of bitter nostalgia. It's right on MSDN TV site, a new episode where Mark Fussel explains new XML features in upcoming .NET 2.0. The episode is dated December...

# Netdocs and Netdoc 12/15/2004 3:09 PM Netdoc Blog
Netdocs and Netdoc blog thread seems to continue.What blew me out of my seat, was this news story on Microsoft's dropped Netdocs project."If you think of what a hosted version of Microsoft Office would look like, if it worked properly, you'd have Netdocs

# Timeshifting not always a good thing? 12/15/2004 6:44 PM Arpan Desai's WebLog


# XQuery support in SQL Server (soon) and .NET (maybe later) 12/17/2004 11:23 AM XmlTeam's WebLog


# re: Black hole projects 12/27/2004 1:28 AM mariah
i need to now were the black hole is!!!!!!!!!!!!!!!!!!!!!!!!!

# re: Black hole projects 12/28/2004 9:26 AM Ovidiu
Try hard. Fail. Try harder. Fail better.

# re: Black hole projects 1/15/2005 12:19 AM MBF
must be the Microsoft Business Framework (MBF) team

# re: Black hole projects 1/28/2005 5:20 PM tx
.net is cool, and real. but some projects in .net train sure are. like monad, winfs...

# re: Black hole projects 2/9/2005 7:24 AM Avalon
Avalon sounds exactly like a black hole.

#  2/11/2005 2:52 PM The Cloak of Wisdom


#  2/11/2005 2:53 PM The Cloak of Wisdom


# re: Black hole projects 2/14/2005 6:09 PM Jim
I worked two years on a classic black whole product when I was the first developer at Dr[name withheld].com.

We didn't have a Captain Ahab, but we had an ex-Microsoftie who had worked on Outlook, so he convinced the boss that we needed to write what amounted to a new operating sytem with an outlook clone at its center. I learned a lot about windows programming from that experience, but quit when the Cloud of Doom became too much. I really liked the management team, and I just felt bad about helping to channel their money into this black whole.

# re: Black hole projects 2/21/2005 6:55 PM Wendy Hernandez
Do black holes reallysuck up everything even metal ?

# re: Black hole projects 3/3/2005 9:44 PM Avoron
Wendy, they suck up poor comments with innane questions. Better start running

# Benefit of doubt 3/20/2005 8:24 AM Jiri's Notepad
Well, sure, it's _easy_ to look at black hole projects after they died a flaming death and say that hey, that was an awful black hole project. What's _hard_ is identifying which are going to be the flaming-death black hole...

# re: Black hole projects 6/1/2005 9:11 AM Alex
Does anyone have any good models i could build for highschool project???? PLEASE Write back..
Alex. earl of march, ont.

# re: Black hole projects 6/20/2005 10:34 AM Reshon
Way to long. And try to make it more interresting.

# re: Black hole projects 7/7/2005 1:10 PM Chris
I wonder which of those bullet points netdocs had going for it to send it down the great black hole. Anyone know? I'm curious now

# re: Black hole projects 3/8/2006 5:05 PM AARON
COOL I LIKE BLACK HOLES !!!!!!!!!!!!!!!!!!

# re: Black hole projects 6/5/2006 4:20 PM Brady
Uhhhhh!
Black holes are so boring. I think it is all a bunch od bulls**t

# Black hole projects, redux 6/26/2006 1:05 AM Panopticon Central


# re: Black hole projects 6/29/2006 2:51 AM NZ Web Guy
Good list, it's made me think about what we're up to with our rewrite :)

# Anyone want to write a Dragon book on mappings? 7/21/2006 4:16 AM Grammarware, Haskellware, XMLware
I am surrounded by X/R and O/R mapping geeks here at Microsoft. (It’s building 35. So there are fewer...

# re: Black hole projects 11/27/2006 10:42 PM Bill
XDocs? isn't it just InfoPath?

http://www.infoworld.com/article/03/02/20/08stratdev_1.html

is it really a failure?

Post Feedback

Title:
Name:
Url:
Comments: