Filling the pipeline…

I was on a thread today where a VB MVP asked if VB was ever going to integrate regular expression-type functionality directly into the language to make it easier (and more comprehensible) to work with strings. The immediate reaction in my head was, “Well, it’s in the pipeline, but no telling if it’ll ever come out…” Which bears a little explanation…

The traditional model for software development at Microsoft is something I’d call “punctuated insanity.” That is, you start a product version by running around like a chicken with your head cut off, trying to come up with what you think you should do in that version. Because you only just started thinking about what you think you need to do, the schedule goes through a phase of rapid expansion as people try and flesh out the initial ideas (which always cost more the more you think about them) and try to jam in all the other good ideas they had (because they know it’s going to be another couple of years before they’ll get to put new stuff into the product again). This is the “insanity” phase of the product, where reality takes an extended vacation and the product plan goes all over the place.

Usually, after some period of insanity, reality (a.k.a. management) comes back from its vacation, finds the house trashed, the fridge empty and the car at the bottom of the pool and freaks out. At this point, some combination of feature paring and schedule slippage happens as the product plan slowly starts to come back to something that is actually shippable. (The product likely goes through several rounds of this before making it to RTM.)

The problems with this model are pretty obvious on the back end—schedule slips and feature cuts are never a happy thing from either the company’s or customer’s perspective. But I think more damage is done on the front end of things. Because planning only ever happens at the beginning of a product cycle, our ability to react to ongoing changes in the market place is seriously compromised. Even worse, because all planning happens in a very short burst it’s very likely that any sufficiently complex feature isn’t going to get the kind of design time that it really needs. Put another way, features only get designed once and you have to hope the feature is right, because you have only a limited ability to react to external and internal feedback once you get the feature finished and you can actually use it.

There are enough obvious examples of this process in action that I don’t even need to bother to point them out.

Much better, when you can do it, is the “pipeline” model of software development that you see much more often in our web offerings. In the pipeline model, you’re never working on just one release. While the main bulk of the team may be working on the next release, you’ve always got some number of people working on features that are further back in the pipeline. What this means is that, first of all, you have people who are actively looking ahead beyond the current release and can start reacting to changes in the marketplace before you’ve finished the current version. It also means that more complex features can have longer “bake” times in the pipeline, allowing teams to try stuff out, see how it flies and discard or modify designs that don’t work. The result should be a more complete, more polished set of features when they finally reach the front of the pipeline (assuming they aren’t discarded entirely before that time).

LINQ has so far been a good example of pipelining in action, and it’s something that we’re actively trying to adopt within the VB team itself, at least on the language side. Regular expressions has been one of those things that have been hanging out there on the “consider one of these days” list for a long time. My hope is that as our pipeline gets going, these ideas will finally get their chance to be tried out and then either discarded or moved towards actual implementation. I can’t say whether regular expression integration will be a good idea or not, or whether it’ll ever show up in VB. But with a more forward-looking approach, I’m hoping we’ll find out…

One thought on “Filling the pipeline…

  1. bill burrows [MVP]

    Hi Paul,

    Great explanation of the "process". As I was reading it, my mind was thinking about the web folks and their release of Atlas, the "project model", the CSS generator, etc. within the product pipeline (as you talked about later in the blog). As you said, this seems to be a superior approach (certainly from the customer perspective). I encourage your efforts to bring this model to the VB language and features.



Leave a Reply

Your email address will not be published. Required fields are marked *