Monthly Archives: December 2006

IIF, a True Ternary Operator and Backwards Compatibility

One of the things we’ve been discussing for VB 9.0 is adding a true ternary operator to the language. It’s been a persistent source of annoyance for myself (and many others, to judge from suggestions and complaints we’ve gotten over the years) that there is no short-circuiting conditional expression operator in the language. True, there’s the IIF method which does most of what you want, but it doesn’t short-circuit. If you evaluate “IIF(x Is Nothing, 10, x.Foo())” and x is Nothing, then you’ll get an exception because we always evaluate all of the arguments of a method call (since IIF is just a regular method). In contrast, the C-style language’s ternary operator (i.e. “?:”, as in “x == null ? 10 : x.Foo()”) does short-circuit and it comes in mighty handy.

The annoyances get even worse with the introduction of queries into the language, because now there are a lot more places where you want to do in-line conditionals (since there aren’t statements in queries). In fact, I was a huge user of IIF back when I worked on the Access query designer in my previous life. So with LINQ coming, it’d be really nice to have a short-circuiting ternary operator available. So we’ve been pondering how best to do it. 

Rather than introduce some new operator, we’re considering doing to IIF what we did to AscW and ChrW and turning it into an intrinsic function. So even though IIF will still appear to be a function call, we’ll intercept the call and turn it into a true short-circuiting ternary operation. The nice thing is that you won’t have to learn any new syntax–things will just start working the way you expect them to. Probably 99% of programmers will never notice the change or like it. However, there probably will be that 1% of programmers (or maybe .1% of programmers or .01% of programmers) who will notice the change and be unpleasantly surprised. Perhaps you intentionally or unintentionally depended on the fact that all arguments to IIF would be evaluated, regardless of the value of the conditional expression. If so, when you recompile your application in VB 9, your program behavior will break or, even worse, silently change.

So what to do? Well, the language spec does say that we reserve the right to break compatibility for new features ”only if the impact would be extremely minimal and the benefit of the feature is high.” I think, based on feedback we’ve collected, the second part of the test is no problem. The first part is the question. For example, we introduced some small compatibility breaks in the language when we added support for unsigned types (that might cause some method calls to bind to different overloads than before), but no one to date has ever reported an issue. So that’s a case where a new feature had a minimal compatibility impact and a high benefit. But what about this case?

Well, we’d like to know what people think. Our take on it is that it is extremely unlikely that anyone is consciously depending on the evaluation of both branches of the IIF expression. Extremely unlikely. And that anyone who’s unconsciously depending on the evaluation of both branches of the IIF expression is actually having a bug in their program covered up by a limitation of the language. If this is the case, then changing the behavior of IIF should help most people write more correct programs and should negatively impact very few people, if any at all.

Or will it? What’s your take?

I should also note that IIF currently doesn’t do the cute typing rules that you’d like from a ternary operator — that is, the return type of IIF is Object, so you have to cast in a lot of cases where you really shouldn’t have to. We’d also fix this at the same time (although we could also fix this by introducing a generic IIF method and fixing a few limitations of our generic inference algorithm–currently if a type parameter infers to two or more types, the algorithm fails rather than taking into account the situation where the type are all related to one another).

And one last huge CAVEAT EMPTOR: since we’re talking about pre-release software, do NOT construe this as a promise this feature will make it in to this release. Even though I believe it will get in, surprises always happen. So don’t start counting this particular chicken until you get actual bits that contain the feature. And even then, nothing’s final until we release to manufacturing.

(Bonus question: Why is it the “ternary operator?” It’s a bit of an imprecise name. An operator that takes one operand is a unary operator. An operator that takes two operands is a binary operator. And an operator that takes three operands…? Since “?:” is usually the only operator in C-style languages that takes three operands, the general “ternary operator” is applied to this specific operator. I imagine there are other names for it, this is just the one I’m used to hearing.)

Language Specification: Useful? Not?

Random question for people: how many people out there actually ever look at the VB language specification? The original purpose of writing the language specification was purely for internal use, done so we’d have a reasonably authoritative explanation of the language aside from “whatever the compiler does” and “whatever MSDN happens to say.” And to have a place where historical thinking could be preserved for future generations of VB team members, so there’d be some chance in hell that we could avoid making the same mistakes twice. But how useful is it for the public? Not that much, I would imagine, but I’m curious.

There should be a second edition of the 8.0 spec coming out in the next few months. Besides a bunch of corrections, it also cleans up the formatting and also–surprise, surprise–actually ensures that all the examples work and do what the spec says they will do. Up until now, language specifications have been published in Word, but with the new PDF tools that are available for Word, that’s an option too. Do people like the Word format? Would you prefer PDF? Or both?

And finally, how tied are people to the structure of the specification? The Everett and Whidbey specifications only added on but didn’t change the ordering of existing sections. Orcas will likely do the same. But beyond that, I’ve been toying with the idea of restructuring the spec a bit to make it a little clearer and put topics that should go together, together. I’m wondering if that would really throw people, or whether people wouldn’t mind adapting. Not really a short-term issue, but…

Lambda expressions, Part II: Closures

Well, I’m glad to see that even with my writer’s block, people still seem to be reading the blog! Although there is definitely a diversity of opinion, the majority of people seem to prefer the “Function” syntax of the choices I laid out, which is not exactly what we expected. (We were wagering people would go for the more cryptic, compact syntax. Shows what we know…) That’s the syntax you should expect to see in the beta, and if public opinion shifts over time in the beta, we’ll deal with that feedback if and when we come to it. After all, that’s what a beta is for…

The other major topic to talk about with lambda expressions is closures. I don’t have a particular question this time, just an update on a topic we discussed about nine months ago. When I talked about closures back March, I raised the question of exactly how VB should treat local variables when lifting them into closures. I won’t rehash the entire discussion here–you can just go back and read the original entry itself–but the problem boiled down to something like this:

Module Module1
    Delegate Function AddDelegate(ByVal x As Integer) As Integer

    Sub Main()
        Dim lambdas(9) As AddDelegate

        For i As Integer = 0 To 9
            Dim y As Integer = i
            lambdas(i) = Function(x) x + y
        Next

        For Each l As AddDelegate In lambdas
            Console.WriteLine(l(0))
        Next

        Console.ReadLine()
    End Sub
End Module

This is the same example as the previous entry on closures, except that instead of using queries, I’m using lambda expressions directly. What I’m doing here is filling an array with lambda expressions that are supposed to add a particular value to the parameter. So lambdas(0) will add 0 to the parameter, lambdas(1) will add 1 to the parameter, etc. At least, that’s the intent. But now we run into the closure question that I asked originally–should each iteration get it’s own copy of the local variable y, or should they all share the same copy of y? If the former, I get the intended semantics. If the latter, then every lambda adds 9 to the parameter (because they share the same y and the final value of y is 9).

Just to make the problem clear, let’s look at an equally valid way (in VB) of writing the same code:

Module Module1
    Delegate Function AddDelegate(ByVal x As Integer) As Integer

    Sub Main()
        Dim lambdas(9) As AddDelegate

        For i As Integer = 0 To 9
            Dim y As Integer
            lambdas(i) = Function(x) x + y
            y += 1
        Next

        For Each l As AddDelegate In lambdas
            Console.WriteLine(l(0))
        Next

        Console.ReadLine()
    End Sub
End Module

Now instead of initializing y at the beginning of each iteration of the loop, I’m just incrementing it and letting the value carry over from one iteration of the loop to another. Now maybe we start to see the problem–if each iteration of the loop gets its own copy of y, then that seems to conflict with the idea that the value of the variable carries over from one iteration of the loop to another. We’re trying to eat our cake and have it too.

What we ended up deciding was to split the difference. It’s a little more complex that what you get in C#, for example, but it should give everyone the semantics they expect and not break any existing code (always a plus, in my book). Basically, we are going to treat the lifetime of local variables to be the scope that they are declared in. Thus, when you create a closure in a loop, each iteration of the loop does, indeed, get its own copy of the local variable. But to preserve the existing semantics, we’re going to add a caveat: when creating a new local variable, if a previous version of that local variable exists, we copy the value from that previous version into the newly created local variable. So the value of a local variable declared in a loop carries over from one iteration to the next. So lambdas work the way most people expect, and existing code continues to run as expected.

As with previous entries on closures, kudos to those who’ve bothered to read this far. It’s kind of arcane and, we believe, most people won’t ever have to think at all about the special semantics–things will just work.

Lambda expressions, Part I: Syntax

As I alluded to in my earlier entry, one of the features we’re working on for LINQ/Orcas is lambda expressions. Now, I might be tempted to say “You know, we should really call them ‘inline functions’ instead of ‘lambda expressions,’ because I think ‘inline function’ is a little clearer and less of a computer science-y term.” but you can rest assured that I learned my lesson. Lambda expressions they are. (Do you detect a trace of bitter sarcasm there? Perhaps just a little.)

For those not well versed in the arcana of the lambda calculus, a lambda expression is, uh, basically an inline function. Take this case, where you’ve got a function that takes another function and applies it to an array:

    Delegate Function TransformDelegate(input As Integer) As Integer

    Sub ApplyTransform(array() As Integer, transform As TransformDelegate)
        For i As Integer = 0 To array.Length - 1
            array(i) = transform(array(i))
        Next
    End Sub

Now, normally you’d have to create a whole function just to be able to use it:

    Function AddOne(x As Integer) As Integer
        Return x + 1
    End Function

    Sub Main()
        Dim a() As Integer = {1, 2, 3}
        ApplyTransform(a, AddressOf AddOne)
    End Sub

But this is kind of silly, right? What you’d really like to do is create an inline function that you could immediately pass to something that took a delegate. Then you could just express right there in the code what you want to do. Something like:

    Sub Main()
        Dim a() As Integer = {1, 2, 3}
        ApplyTransform(a, Function(x) x + 1)
    End Sub

This is all a lambda expression is, really, just an inline function that you can pass to something that takes a delegate type. However, lambda expressions can be quite powerful and form one of the major underpinnings of LINQ. For example, you’ll notice that the LINQ Where function takes a delegate type. So instead of using the higher level From … Where syntax, you could also write things like:

    Dim ys = xs.Where(Function(x) x < 5)

This calls the “Where” function on “xs,” passing in the inline function “x < 5″. The Where function then just applies that inline function to the elements of the collection to determine what to filter in or out of the collection. (You’ll notice I omitted all the types from the inline function; lambda expressions use type inference so that you don’t have to give all the types, but that’s a whole other blog entry.)

One of the major things we’re still finalizing, though, is the exact syntax of lambda expressions in VB (because, of course, the syntax is always the hardest thing to get right). C# is using a “fat arrow, no introducing token” syntax that looks something like this:

    var ys = xs.Where(x => x < 5)

This is certainly one possibility. It has both the advantage of being extremely concise and the disadvantage of being extremely concise. So it’s quick to use but kind of cryptic. Consistency between the languages is nice, but another possibility is the one I used in my initial examples:

    Dim ys = xs.Where(Function(x) x < 5)

This is more wordy and less concise than the “fat arrow, no introducing token” format, but it’s got the advantage that it’s consistent with the rest of the VB language and makes it really obvious what you’re doing (i.e. creating a function). There are some complications to the design having to do with how to distinguish these kinds of “expression-only” lambdas from multi-line lambdas, but we can gloss over those for the moment. (Also note that this is the syntax that some languages such as, I believe, Python use for their lambda expression syntax).

The final option we’ve been kicking around is somewhere in the middle of the two:

    Dim ys = xs.Where(x => x < 5)

The fat arrow has reappeared, but now there’s a backslash that introduces the lambda expression. The backslash is supposed to suggest the lambda character (?), but uses a character that is actually on people’s keyboards. The reason for even considering this hybrid syntax is that having an introducing token means we can give a better Intellisense/IDE experience, since the leading “” token allows us to recognize immediately that you’re writing a lambda expression instead of having to guess until we get to the “=>”.

I’m kind of curious what people think. Given the three options:

  1. C#-style, “x => x + 1″
  2. VB-style, “Function(x) x + 1″
  3. Hybrid-style, “x => x + 1″

I’m a little torn between the clarity of #2 and the conciseness of #3, but I may just be smoking crack on #3. There’s nothing intrinsically wrong with #1, but even after staring at it for a long time, it just doesn’t feel, you know, like VB…

Sometimes you forget you’re not the only one…

One of the teams I’ve been spending a lot of time talking to lately is the folks over in the CLR who bring you IronPython. Overall, it’s been a very enjoyable experience, as they’re a smart bunch of guys who have a lot of experience with building dynamic languages and libraries and such. One of the unexpected benefits, though, has been psychological. After spending the past nine years interacting mostly with languages that were derived from C, it’s amazingly refreshing to talk to people who work in a language that has more of a cultural affinity with our own language.

After all, if the C-style languages do things one way and VB does it another way, it’s a bit like a 4-on-1 situation since you have C, C++, C# and Java all lined up on one side and VB on the other. And as much as we relish being the maverick, it sometimes helps to be reminded that there are more things in heaven and Earth than are dreamt of in Kernighan and Richie’s philosophy. (And this is no knock on those guys, BTW, they did amazing work that’s stood the test of time.) This isn’t to say that VB and Python are amazingly similar–after all, Python is still case sensitive–but still… I think it also helps that the Python folks deal with the same kind of questions that we deal with vis-a-vis our communities and backwards compatibility. After all, unlike C#, we’ve both got a lot of history to deal with!

Anyway, just a thought motivated by some discussions we were having yesterday about lambda expressions. Oh, yes, which reminds me that that’s my next topic…

Writer’s block…?

Just for fun, as I was starting at the “Archives” in the left pane started working out what my “post per month” average was per year. It comes out to:

2003: 19.6 PPM (only 6 months of the year, though)
2004: 12.3 PPM
2005: 9.3 PPM
2006: 4.9 PPM (only 11 months so far)

As you can see, my average has dropped precipitously this year and I’m in danger of getting awfully close to zero next year if things continue as they have been. The real question is: why has my posting average gone down so much? I mean, I’m not surprised that my average dropped off after the first year or so–sooner or later the bloom has got to come off the rose–but why so anemic this year? Why was last year not particularly good either? Is there just nothing interesting left to talk about?

Well, no, for that last question. There’s lots of interesting stuff going on in VB. LINQ and Orcas is bubbling along. Lots of interesting thinking is happening.

I think I’ve been experiencing extended transition-induced writer’s block. By “transition-induced,” I mean that the 2005-2006 has been a period of a lot of various transitions for me. Obviously, I’m about to become a father, and that’s just a huge, major, massive transition in anyone’s life. And professionally, my role has shifted a lot over the past two years into something that I still feel like is a work-in-progress. (I think my next entry should be “What the Hell I Do,” since I’m not sure that’s clear anymore, least of all to myself at times.) And, even bigger than that at work, the product as a whole has been making a major transition into maturity. We’re not so far behind the design eight ball anymore that we can actually ponder a bit more where we’re going rather than just react to the next major thing that the .NET Framework is throwing at us. And .NET as a whole has been out there long enough to start to get a feel for what’s been working well and what hasn’t.

I’m hoping we’re far enough through the transition now for the writing mojo to return a bit. It feels like the writer’s block may be starting to loosen up, but I guess only time will really tell…