Iterators in Visual Basic

WARNING: This is a speculative post. Caveat emptor.

Actually, in this case I don’t thing the above warning is strong enough. This is a super speculative post, because I believe the chance of it appearing in the next version of the language is not extremely high, not because it’s not a worthy feature but because it’s more than a little work and we’ve got a lot of other very worthy features we’re considering. However, since it’s something that’s valuable and something we keep getting requests for, we have decided to at least generally sketch out what iterators would look like in Visual Basic, with the idea that when the time comes that we have the resources to do them we can jump on it. Consider yourself double warned.

To start with, an iterator is just a structure that returns a sequence of values. Basically, when the IEnumerable(Of T) interface method GetEnumerator() returns an instance of the interface IEnumerator(Of T), that instance is an iterator. You can iterate through the collection by calling MoveNext on the enumerator and looking at the Current property until you’ve run out of values. Or you can use the For Each statement, which will do the iteration for you.

It’s possible to construct iterators in Visual Basic today–all you have to do is implement IEnumerable(Of T) and then implement IEnumerable(Of T). But the problem is that doing this involves a whole lot of boilerplate code that is the same in almost all iterators. On top of that, if an iterator involves a lot of conditions or branching, doing state management can be quite tricky. In an ideal world, what you’d like to be able to do is simply state the values that should be iterated over and have the compiler generate all the boilerplate code and state management for you. C# does this through the yield statement, which can be used in a method that returns an iterator to automatically generate the iterator:

using System;
using System.Collections.Generic;

class Program
{
    public static IEnumerable<int> FromTo(int low, int high)
    {
        if (low <= high)
        {
            yield return low;
            foreach (int i in FromTo(low + 1, high))
            {
                yield return i;
            }
        }
    }

    static void Main(string[] args)
    {
        foreach (int i in FromTo(1, 5))
        {
            Console.WriteLine(i);
        }
    }
}

In this example, the FromTo method returns an iterator (typed as IEnumerable(Of Integer)) which is automatically generated by the compiler. What we’re considering in Visual Basic is essentially the same feature, but with a slightly different design. Instead of tying the iterator to a method, as in C#, we’re considering extending the multi-line lambda syntax that I talked about in the previous post to make “anonymous iterators.” The above example in VB would look something like:

Module Module1
    Function FromTo(ByVal low As Integer, ByVal high As Integer) As IEnumerable(Of Integer)
        Return Iterator
                   If low <= high Then
                       Return low
                       Return Each FromTo(low + 1, high)
                    End If
               End Iterator
    End Function

    Sub Main()
        For Each i In FromTo(1, 5)
            Console.WriteLine(i)
        Next
    End Sub
End Module

There are a couple of differences to notice here:

  • An anonymous iterator is just an expression, so it can be used as the return value of a function but could also appear as the target of a For Each, used in a LINQ query, stored into a field, etc. (This is particularly useful when a function produces an iterator and you want to do some validation. In the C# model, the validation code is not run until MoveNext is first called on the iterator, not when the iterator is actually produced.)
  • By default the type of the iterator is IEnumerable(Of T), where T is inferred from all the types returned from the iterator using the same algorithm we use for multi-line lambdas. You could also explicitly state the iterator type using an As clause, and we would allow IEnumerator(Of T) as the iterator type just like C# does.
  • We don’t require the yield keyword since we’ve already marked the block as being special by using a new contextual keyword Iterator. (There would also be an Exit Iterator statement, not shown.)
  • Our plan would be to add an Each operator that “unwraps” a collection and returns each of its values one by one. This is generally useful but also addresses some performance issues with nested iterators which you can read more about in Wes Dyer’s post on iterators. (It might also later allow things like unwrapping collections in collection initializers, like {1, 2, 3, Each a, 5, 6, 7}, where a is an array of integers or something…)

We’re interested in people’s feedback on this proposed design and any other comments you might have.

18 thoughts on “Iterators in Visual Basic

  1. Jonathan Allen

    Wow, that’s amazing. I’m not sure if its a good idea or not, but it is definitely worth looking into.

    I really like the Each syntax and I hope you find ways to use it besides just collections initializers.

    Please don’t rush this one. This is something I would want to see in beta for a long, long time.

    Reply
  2. Konrad

    This is a design to my liking: much more powerful than its C# pendant (addresses all of my previous complaints) and yet very simple. Way to go!

    Reply
  3. Speednet

    Excellent, very happy to see that iterators may become part of the language. I’ve had to write some recently, and they’re currently no fun in VB.

    I like the use of lambdas (of the multi-line variety), and like most of VB, it seems a bit more logical than the C# version.

    I have 2 additional minor comments:

    1. The Return Iterator block does not seem consistent with other blocks, in that your End Iterator closes the "Iterator", not "Return Iterator". I would love to see a different form of "Return" in block form, maybe with a completely different keyword. For example, using the "Yield" term, you could replace "Return Iterator" with "Yield", and replace "End Iterator" with "End Yield".

    (I like the word yield, in that it is an action word, unlike iterator, which requires that a "verb" be placed before it. Or maybe it could be Iterate/End Iterate?)

    2. Is there a different syntax that would not require the two Returns within the If block? Many will not see how the second Return will ever get executed. How about something like:

    Return low Continue FromTo(low + 1, high)

    -Todd

    Reply
  4. Bill McCarthy

    Hi Paul,

    First a minor typo. You wrote "all you have to do is implement IEnumerable(Of T) and then implement IEnumerable(Of T). " you probably meant to write IEnumerator(Of T) for the last IEnumerable.

    Second, you said:

    "(This is particularly useful when a function produces an iterator and you want to do some validation. In the C# model, the validation code is not run until MoveNext is first called on the iterator, not when the iterator is actually produced.) "

    I’m not sure I follow what you mean there. At first it read like iterators in VB would not have delayed evaluation. But I am guessing what you meant was validation outside fo the iterator block, e.g:

    Function GetIterator(switch AS Boolean) As IEnumerable(Of Fruit)

    Dim var as IEnumerable(Of Fruit)

    If switch THen

    var = Iterator

    ……

    End Iterator

    Else

    var = me.innerlist

    End If

    Return var

    End Function

    Third, having Iterator a protected keyword could be a breaking change. Any thoughts on addressing that ?

    (other than the foxpro BEGIN Iterator … End Iterator of course <g>) Maybe:

    #Iterator

    #End Iterator

    Thanks. This looks good. Can we have it as an update to VS 2008 as well ? 🙂

    Reply
  5. Robert Taylor

    I like this design for VB Iterators.

    If we’re going to have "anonymous iterators" then why not also have "proper" iterators with similar syntax as Functions:

    Public Iterator Range(start as Integer, end as Integer) As IEnumerable(Of Integer)

    For i As Integer = start To end

    Return i

    Next

    End Iterator

    Also, i like the Each unwrapping statement.

    Reply
  6. Pingback: ??????

  7. Taiwo

    Robert Taylor’s suggestions a la: "Public Iterator Range…" looks interesting. However, shouldn’t that be:

    Public Iterator Function Range(…)



    End Function

    In any case this would prevent iterators from being embedded within methods.



    Taiwo

    Reply
  8. Tomas Herceg

    Public Iterator Function … is not compatible with anonymous iterator syntax. It ends with End Function but anonymous iterator ends with End Iterator. This leads to an inconsistence.

    Robert Taylor’s suggestion is different. If we could have anonymous functions and subs next to "classic" functions and subs, so we should have anonymous and proper iterators.

    On the other hand do we really need "proper" iterators? To create a read only property returning some generated collection we still have to use anonymous one.

    Or we will get Public Iterator Function … and Public Iterator ReadOnly Property … ? Then anonymous versions will be unnecessary.

    Reply
  9. Pingback: Anonymous

  10. Pingback: ???? Blog

  11. Pingback: Anonymous

  12. Pingback: Anonymous

  13. Pingback: Anonymous

  14. Pingback: Casino 1299836265

Leave a Reply