Category Archives: Visual Basic

Language Specification: Useful? Not?

Random question for people: how many people out there actually ever look at the VB language specification? The original purpose of writing the language specification was purely for internal use, done so we’d have a reasonably authoritative explanation of the language aside from “whatever the compiler does” and “whatever MSDN happens to say.” And to have a place where historical thinking could be preserved for future generations of VB team members, so there’d be some chance in hell that we could avoid making the same mistakes twice. But how useful is it for the public? Not that much, I would imagine, but I’m curious.

There should be a second edition of the 8.0 spec coming out in the next few months. Besides a bunch of corrections, it also cleans up the formatting and also–surprise, surprise–actually ensures that all the examples work and do what the spec says they will do. Up until now, language specifications have been published in Word, but with the new PDF tools that are available for Word, that’s an option too. Do people like the Word format? Would you prefer PDF? Or both?

And finally, how tied are people to the structure of the specification? The Everett and Whidbey specifications only added on but didn’t change the ordering of existing sections. Orcas will likely do the same. But beyond that, I’ve been toying with the idea of restructuring the spec a bit to make it a little clearer and put topics that should go together, together. I’m wondering if that would really throw people, or whether people wouldn’t mind adapting. Not really a short-term issue, but…

Lambda expressions, Part II: Closures

Well, I’m glad to see that even with my writer’s block, people still seem to be reading the blog! Although there is definitely a diversity of opinion, the majority of people seem to prefer the “Function” syntax of the choices I laid out, which is not exactly what we expected. (We were wagering people would go for the more cryptic, compact syntax. Shows what we know…) That’s the syntax you should expect to see in the beta, and if public opinion shifts over time in the beta, we’ll deal with that feedback if and when we come to it. After all, that’s what a beta is for…

The other major topic to talk about with lambda expressions is closures. I don’t have a particular question this time, just an update on a topic we discussed about nine months ago. When I talked about closures back March, I raised the question of exactly how VB should treat local variables when lifting them into closures. I won’t rehash the entire discussion here–you can just go back and read the original entry itself–but the problem boiled down to something like this:

Module Module1
    Delegate Function AddDelegate(ByVal x As Integer) As Integer

    Sub Main()
        Dim lambdas(9) As AddDelegate

        For i As Integer = 0 To 9
            Dim y As Integer = i
            lambdas(i) = Function(x) x + y
        Next

        For Each l As AddDelegate In lambdas
            Console.WriteLine(l(0))
        Next

        Console.ReadLine()
    End Sub
End Module

This is the same example as the previous entry on closures, except that instead of using queries, I’m using lambda expressions directly. What I’m doing here is filling an array with lambda expressions that are supposed to add a particular value to the parameter. So lambdas(0) will add 0 to the parameter, lambdas(1) will add 1 to the parameter, etc. At least, that’s the intent. But now we run into the closure question that I asked originally–should each iteration get it’s own copy of the local variable y, or should they all share the same copy of y? If the former, I get the intended semantics. If the latter, then every lambda adds 9 to the parameter (because they share the same y and the final value of y is 9).

Just to make the problem clear, let’s look at an equally valid way (in VB) of writing the same code:

Module Module1
    Delegate Function AddDelegate(ByVal x As Integer) As Integer

    Sub Main()
        Dim lambdas(9) As AddDelegate

        For i As Integer = 0 To 9
            Dim y As Integer
            lambdas(i) = Function(x) x + y
            y += 1
        Next

        For Each l As AddDelegate In lambdas
            Console.WriteLine(l(0))
        Next

        Console.ReadLine()
    End Sub
End Module

Now instead of initializing y at the beginning of each iteration of the loop, I’m just incrementing it and letting the value carry over from one iteration of the loop to another. Now maybe we start to see the problem–if each iteration of the loop gets its own copy of y, then that seems to conflict with the idea that the value of the variable carries over from one iteration of the loop to another. We’re trying to eat our cake and have it too.

What we ended up deciding was to split the difference. It’s a little more complex that what you get in C#, for example, but it should give everyone the semantics they expect and not break any existing code (always a plus, in my book). Basically, we are going to treat the lifetime of local variables to be the scope that they are declared in. Thus, when you create a closure in a loop, each iteration of the loop does, indeed, get its own copy of the local variable. But to preserve the existing semantics, we’re going to add a caveat: when creating a new local variable, if a previous version of that local variable exists, we copy the value from that previous version into the newly created local variable. So the value of a local variable declared in a loop carries over from one iteration to the next. So lambdas work the way most people expect, and existing code continues to run as expected.

As with previous entries on closures, kudos to those who’ve bothered to read this far. It’s kind of arcane and, we believe, most people won’t ever have to think at all about the special semantics–things will just work.

Lambda expressions, Part I: Syntax

As I alluded to in my earlier entry, one of the features we’re working on for LINQ/Orcas is lambda expressions. Now, I might be tempted to say “You know, we should really call them ‘inline functions’ instead of ‘lambda expressions,’ because I think ‘inline function’ is a little clearer and less of a computer science-y term.” but you can rest assured that I learned my lesson. Lambda expressions they are. (Do you detect a trace of bitter sarcasm there? Perhaps just a little.)

For those not well versed in the arcana of the lambda calculus, a lambda expression is, uh, basically an inline function. Take this case, where you’ve got a function that takes another function and applies it to an array:

    Delegate Function TransformDelegate(input As Integer) As Integer

    Sub ApplyTransform(array() As Integer, transform As TransformDelegate)
        For i As Integer = 0 To array.Length - 1
            array(i) = transform(array(i))
        Next
    End Sub

Now, normally you’d have to create a whole function just to be able to use it:

    Function AddOne(x As Integer) As Integer
        Return x + 1
    End Function

    Sub Main()
        Dim a() As Integer = {1, 2, 3}
        ApplyTransform(a, AddressOf AddOne)
    End Sub

But this is kind of silly, right? What you’d really like to do is create an inline function that you could immediately pass to something that took a delegate. Then you could just express right there in the code what you want to do. Something like:

    Sub Main()
        Dim a() As Integer = {1, 2, 3}
        ApplyTransform(a, Function(x) x + 1)
    End Sub

This is all a lambda expression is, really, just an inline function that you can pass to something that takes a delegate type. However, lambda expressions can be quite powerful and form one of the major underpinnings of LINQ. For example, you’ll notice that the LINQ Where function takes a delegate type. So instead of using the higher level From … Where syntax, you could also write things like:

    Dim ys = xs.Where(Function(x) x < 5)

This calls the “Where” function on “xs,” passing in the inline function “x < 5”. The Where function then just applies that inline function to the elements of the collection to determine what to filter in or out of the collection. (You’ll notice I omitted all the types from the inline function; lambda expressions use type inference so that you don’t have to give all the types, but that’s a whole other blog entry.)

One of the major things we’re still finalizing, though, is the exact syntax of lambda expressions in VB (because, of course, the syntax is always the hardest thing to get right). C# is using a “fat arrow, no introducing token” syntax that looks something like this:

    var ys = xs.Where(x => x < 5)

This is certainly one possibility. It has both the advantage of being extremely concise and the disadvantage of being extremely concise. So it’s quick to use but kind of cryptic. Consistency between the languages is nice, but another possibility is the one I used in my initial examples:

    Dim ys = xs.Where(Function(x) x < 5)

This is more wordy and less concise than the “fat arrow, no introducing token” format, but it’s got the advantage that it’s consistent with the rest of the VB language and makes it really obvious what you’re doing (i.e. creating a function). There are some complications to the design having to do with how to distinguish these kinds of “expression-only” lambdas from multi-line lambdas, but we can gloss over those for the moment. (Also note that this is the syntax that some languages such as, I believe, Python use for their lambda expression syntax).

The final option we’ve been kicking around is somewhere in the middle of the two:

    Dim ys = xs.Where(x => x < 5)

The fat arrow has reappeared, but now there’s a backslash that introduces the lambda expression. The backslash is supposed to suggest the lambda character (?), but uses a character that is actually on people’s keyboards. The reason for even considering this hybrid syntax is that having an introducing token means we can give a better Intellisense/IDE experience, since the leading “” token allows us to recognize immediately that you’re writing a lambda expression instead of having to guess until we get to the “=>”.

I’m kind of curious what people think. Given the three options:

  1. C#-style, “x => x + 1”
  2. VB-style, “Function(x) x + 1”
  3. Hybrid-style, “x => x + 1”

I’m a little torn between the clarity of #2 and the conciseness of #3, but I may just be smoking crack on #3. There’s nothing intrinsically wrong with #1, but even after staring at it for a long time, it just doesn’t feel, you know, like VB…

Hotfixes are, well, just that…

In response to my entry on VB hotfixes being made more available, Paul L brought up a point that’s worth addressing:

While this is great (and long overdue), the truth of the matter is that most companies do not have the time and resources to download these hotfixes, test them, and then propogate them to other teams. Indeed, after installing *all* of the hotfixes that you list we still had envrionment issues that remain unresolved. Sure, we can contact PSS every time we run into something, but aren’t we supposed to be spending our times working with tools that are just supposed to work out of the box…

I should have emphasized that the hotfixes are, well, hotfixes. We also call them “QFE”s, which stands for “Quick Fix Engineering,” which should give you a further appreciation of their nature. QFEs are produced in response to urgent customer product support requests and although we take as much care as possible with them, they are intended to solve the specific problem and are not put through the extremely rigorous tests that a full release or service pack gets. Thus, installing a hotfix is really only for people who need their particular problem solved. Everyone else should wait for VS 2005 SP1, which is already in beta and will contain lots of other fixes and will be much more rigorously tested. Seriously – if you can wait for SP1, you should. It’ll be much better.

Paul continues with:

Instead, we made a more strategic choice to simply convert all of our VB applications to C# and we haven’t looked back. In each case, the application conversion process took a day or two (thanks to the conversion features of SharpDevelop) and I can certainly say that our development experience has been on orders of magnitude more stable and reliable with the IDE experience than it was with VB.

I feel extremely regretful that the C# experience has been so much more stable for Paul than the VB experience was. I’ve been doing a lot of development work in a mixed C# and VB solution recently and haven’t noticed any difference between the two in terms of stability, but as is obvious from the hotfix and SP situation, there are some problems. There shouldn’t be any difference between the two IDEs, but I think that part of the problems, such as there is, stem from the greater ambitions of the VB IDE. I find it very annoying switching between C# and VB because I’m always missing my background compilation and much more accurate (in my experience) Intellisense when I flip over to C#. This comes directly from the greater integration between the compiler and the IDE, which also means that there’s more opportunity for problems and for us to screw things up.

The interesting thing to see is what happens in Orcas. With a much greater reliance on type information that type inference demands (both for local variable type inference and lambda expression inference), C# is going to need to forge much closer links between their compiler and their IDE, similar to what VB has done. What effect that will have is going to be interesting to see.

Bootstrapping VB

I could have sworn that I’d answered this before, but a Live search on “bootstrap” didn’t turn up anything, so I guess not. Anthony asked:

Do you think you’ll ever get VB to the point where it’s compiler is written in VB? Isn’t that like the ultimate programming language Right of Passage?

I’m not sure whether every major language has gone to the length of compiling itself in itself, but it’s something that would be nice to achieve one of these days with VB. There’s no technical reason I can think of why we couldn’t build the compiler in VB, but since most of the compiler codebase is currently in C++, that’s a pretty big barrier to migration. We wouldn’t even consider trying to move the codebase unless and until there was some compelling reason to do so, and so far that reason hasn’t appeared.

The initial choice of C++, of course, was due to the fact that we didn’t have a working VB .NET compiler to bootstrap us up…

Visual Basic Power Pack, CTPs and SPs, Oh my!

Lots of code has been flowing out of Redmond recently:

  • Two Visual Basic 2005 Power Packs have been released. The Microsoft Interop Forms Toolkit 1.0 and Microsoft PrintForm Component 1.0 are both targeted at people who use COM and VB6 and should make migration of code easier in many cases. You can get them here, and more are likely to come.
  • Beta 1 of VS 2005 Service Pack 1 has been released. This service pack includes a significant amount of work on the part of the Visual Basic compiler team and addresses a lot of the major issues (performance or otherwise) that we’ve found after release. I encourage people to try it out and report any problems you run into with it (I’ve got it installed!). You can get it on Microsoft Connect.
  • The September Community Technology Preview (CTP) of Orcas has been released. Unlike the previous LINQ CTPs, this CTP is actual Orcas production code instead of a prototype implementation. As a result, many of the features present in the last LINQ CTP aren’t in this CTP, so it’s going to be a step back in that regard. Also, because of the way the CTP schedule and the VB compiler schedule happened to (not) sync up, there are not a lot of new VB features in this CTP. Expect much more in the coming CTPs! You can get the CTP here (and hurrah! We’re finally moving to Virtual PC images!).

Enjoy all the tasty code!

Pseudo-Random Numbers, VB and Doing the (Mersenne) Twist…

Interesting how random things sometimes come together. I was checking out Chris William’s pre-beta rogue-like game Heroic Adventure! on CodePlex and noticed that although most of the game is written in VB, he had included some C# code in there. The motivation was the fact that the pseudo-random number generators that VB and the .NET platform provide (i.e. VB’s Rnd function and .NET’s System.Random) are not strong enough to satisfy the random number needs of his game. So he borrowed some C# code that implements the Mersenne Twister algorithm, which is a much better pseudo-random number generator.

I thought it was a shame for Chris to have to sully all his beautiful VB code with some C# (obligatory <g> for the humor impaired), so I took a look at the reference implementation of the algorithm and coded it up in VB (making sure, of course, the test output matched that of the reference implementation). For those of you with random number needs, I’ve attached it at the end. Please let me know if you find bugs.

Interestingly, at almost exactly the same time, the question of the strength (or lack thereof) of our pseudo-random number generator came up internally. Keep in mind that VB .NET’s Rnd function was written to remain completely compatible with VB 6.0’s Rnd function, which was compatible with VB 5.0’s Rnd function, etc., etc., all the way back to VB 1.0’s Rnd function. So that pseudo-random number generator is pretty freaking old — 15+ years and counting! We’re discussing what we might do about the situation, but since some people may depend on setting a particular seed and then getting a predictable sequence…

One caveat to keep in mind is that none of these pseudo-random number generators are strong enough to be used for cryptographic uses. For that, try System.Security.Cryptography.RNGCryptoServiceProvider.

'
' An implementation of the Mersenne Twister algorithm (MT19937), developed
' with reference to the C code written by Takuji Nishimura and Makoto Matsumoto
' (http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html).
'
' This code is free to use for any pupose.
'

Option Strict On

''' 
''' A random number generator with a uniform distribution using the Mersenne 
''' Twister algorithm.
''' 
Public Class MersenneTwister
    Private Const N As Integer = 624
    Private Const M As Integer = 397
    Private Const MATRIX_A As UInteger = &H9908B0DFUI
    Private Const UPPER_MASK As UInteger = &H80000000UI
    Private Const LOWER_MASK As UInteger = &H7FFFFFFFUI

    Private mt(N - 1) As UInteger
    Private mti As Integer = N + 1

    ''' 
    ''' Create a new Mersenne Twister random number generator.
    ''' 
    Public Sub New()
        Me.New(CUInt(Date.Now.Millisecond))
    End Sub

    ''' 
    ''' Create a new Mersenne Twister random number generator with a
    ''' particular seed.
    ''' 
    ''' The seed for the generator.
    Public Sub New(ByVal seed As UInteger)
        mt(0) = seed
        For mti = 1 To N - 1
            mt(mti) = CUInt((1812433253UL * (mt(mti - 1) Xor (mt(mti - 1) >> 30)) + CUInt(mti)) And &HFFFFFFFFUL)
        Next
    End Sub

    ''' 
    ''' Create a new Mersenne Twister random number generator with a
    ''' particular initial key.
    ''' 
    ''' The initial key.
    Public Sub New(ByVal initialKey() As UInteger)
        Me.New(19650218UI)

        Dim i, j, k As Integer
        i = 1 : j = 0
        k = CInt(IIf(N > initialKey.Length, N, initialKey.Length))

        For k = k To 1 Step -1
            mt(i) = CUInt(((mt(i) Xor ((mt(i - 1) Xor (mt(i - 1) >> 30)) * 1664525UL)) + initialKey(j) + CUInt(j)) And &HFFFFFFFFUI)
            i += 1 : j += 1
            If i >= N Then mt(0) = mt(N - 1) : i = 1
            If j >= initialKey.Length Then j = 0
        Next

        For k = N - 1 To 1 Step -1
            mt(i) = CUInt(((mt(i) Xor ((mt(i - 1) Xor (mt(i - 1) >> 30)) * 1566083941UL)) - CUInt(i)) And &HFFFFFFFFUI)
            i += 1
            If i >= N Then mt(0) = mt(N - 1) : i = 1
        Next

        mt(0) = &H80000000UI
    End Sub

    ''' 
    ''' Generates a random number between 0 and System.UInt32.MaxValue.
    ''' 
    Public Function GenerateUInt32() As UInteger
        Dim y As UInteger
        Static mag01() As UInteger = {&H0UI, MATRIX_A}

        If mti >= N Then
            Dim kk As Integer

            Debug.Assert(mti <> N + 1, "Failed initialization")

            For kk = 0 To N - M - 1
                y = (mt(kk) And UPPER_MASK) Or (mt(kk + 1) And LOWER_MASK)
                mt(kk) = mt(kk + M) Xor (y >> 1) Xor mag01(CInt(y And &H1))
            Next

            For kk = kk To N - 2
                y = (mt(kk) And UPPER_MASK) Or (mt(kk + 1) And LOWER_MASK)
                mt(kk) = mt(kk + (M - N)) Xor (y >> 1) Xor mag01(CInt(y And &H1))
            Next

            y = (mt(N - 1) And UPPER_MASK) Or (mt(0) And LOWER_MASK)
            mt(N - 1) = mt(M - 1) Xor (y >> 1) Xor mag01(CInt(y And &H1))

            mti = 0
        End If

        y = mt(mti)
        mti += 1

        ' Tempering
        y = y Xor (y >> 11)
        y = y Xor ((y << 7) And &H9D2C5680UI)
        y = y Xor ((y << 15) And &HEFC60000UI)
        y = y Xor (y >> 18)

        Return y
    End Function

    ''' 
    ''' Generates a random integer between 0 and System.Int32.MaxValue.
    ''' 
    Public Function GenerateInt32() As Integer
        Return CInt(GenerateUInt32() >> 1)
    End Function

    ''' 
    ''' Generates a random integer between 0 and maxValue.
    ''' 
    ''' The maximum value. Must be greater than zero.
    Public Function GenerateInt32(ByVal maxValue As Integer) As Integer
        Return GenerateInt32(0, maxValue)
    End Function

    ''' 
    ''' Generates a random integer between minValue and maxValue.
    ''' 
    ''' The lower bound.
    ''' The upper bound.
    Public Function GenerateInt32(ByVal minValue As Integer, ByVal maxValue As Integer) As Integer
        Return CInt(Math.Floor((maxValue - minValue + 1) * GenerateDouble() + minValue))
    End Function

    ''' 
    ''' Generates a random floating point number between 0 and 1.
    ''' 
    Public Function GenerateDouble() As Double
        Return GenerateUInt32() * (1.0 / 4294967295.0)
    End Function
End Class

Updated 09/22/2006: The dingo ate my XML comments! Fixed.

Updated 09/22/2006 (Later): The dingo ate my <g> as well! Fixed.

Lang .NET 2006 talk posted

If you are curious about my talk at the Lang .NET symposium that I talked about a while ago, you can now download the video of the talk here. As is usual, I can’t bear to watch the damn thing since I think my voice sounds just awful–it’s so much nicer sounding in my head. Oh well.

Overall, I think the talk was only OK. I kind of switched around what I was going to talk about late in the game and so I don’t think it was as interesting as I was hoping to be. It talks some about some of the issues we’ve run into with implementing LINQ and some thoughts about some future direction, but we had a snafu with the time (I thought I had much less time than I really did), so it kind of got truncated at the end. I’m hoping to make up for some of it on my blog this fall…

Localized programming languages

Omer van Kloten‘s entry on Internationalization of Programming reminded me of a (possibly apocryphal) story that I was told when I started working on OLE Automation. I asked why IDispatch::GetIDsOfNames takes an LCID and was told that once-upon-a-time, the VBA team conducted an experiment in localization with VBA in Excel (which was the first application to host VBA). Apparently, they attempted to localize the entire language–keywords, function names, etc.–into French, and possibly other languages. This mean you could write code along the lines of what Omer outlines in his entry, except in French instead of Dutch.

The problem was that because VBA wasn’t exactly compiled in those days, the Excel spreadsheet that you wrote your code in now depended on having localized Excel on the machine. If you sent your spreadsheet to your colleague in New York, they couldn’t run the macros because their English Excel didn’t understand the language…