Category Archives: Visual Basic 2010

I haven’t gone AWOL…

…life just went and got very complicated for a while. In particular:

  • My wife Andrea has had some strange medical symptoms over the past few months that our neurologist thought indicated pretty strongly she has MS. So the past month has been spent getting MRIs and lumbar punctures to try and narrow down the diagnosis. The good news? It’s pretty unlikely she has MS or any of the related types of diseases (RA, Lupus, etc.). The bad news? Still not sure what’s going on. Still, since none of the symptoms are extremely serious, I’ll take “don’t know” for now, given the alternatives.
  • We’re pushing to get another LINQ preview out in the near future and I’ve been on the hook to write a number of new features for the preview. It’s been great to get the chance to write a lot of code again, but since that’s not my day job any more, it has contributed to a real time crunch.
  • The whole question of scripting languages, dynamic languages, dynamic environments, loose typing, etc, etc, etc. which has been bubbling around in the background for well over two years seems, all of a sudden, to be coming to quite a boil. Nothing in particular to talk about yet, but lots of interesting and exciting stuff bouncing around and perhaps some quite interesting things to talk about in the near future.

Anyway, I think I should be back now, barring any unexpected surprises. Time to get back to that question of local variables, eh?

See me on Channel9…

Two new videos, timed to coincide with the PDC are now up:

Paul Vick and Erik Meijer – Dynamic Programming in Visual Basic – This is me and Erik talking about dynamic stuff in VB. We were planning on talking about some of the new XML stuff as well, but didn’t even get to it! Another video, I’m sure…

Paul Vick and Amanda Silver – VB Language Futures – This is me and Amanda talking about a lot of the future stuff we talked about at the PDC. Check it out!

Hopefully more to come. If you have specific topics you’d like to see, let us know! Based on some of the comments, we may still have some kinks to iron out in the presentation department, but videos are easy enough to do and we’d love to talk about what you’re interested in hearing about!

Introducing LINQ!

Throughout this week and beyond, we’re going to be doing a lot of talking about Project LINQ, so I wanted to start with a relatively brief overview of what the project is, what its goals are, and how they apply to VB.

To start with, LINQ stands for “Language Integrated QUery.” LINQ fundamentally is about integrating query operations into the .NET platform in a comprehensive and open manner. It’s also about providing a unified way for you to query across any kind of data that you have in your program, whether it’s relational, objects or XML. This, we believe, will represent a tectonic shift in the way that VB programmers will work with data. The possibilities that having query capabilities always available right at your fingertips, regardless of the type of data you’re working with, are immense and will fundamentally alter the way people program.

The core of LINQ is a set of API patterns. I say “patterns” instead of “interfaces” or “classes” because we want to maintain maximum flexibility in our ability to provide querying for any kind of data, regardless of whether it has a particular object model or interface implemented. These API patterns specify how an object model can become “queryable,” and they cover all the basic query operations that we know and love: projection (select), filter (where), grouping (group by), ordering (order by), joining (join), etc. By implementing this API pattern, any data provider on .NET can become queryable. There is also a lot of flexibility as to how a data provider can become flexible — i.e., whether it wants to delegate most of the work on querying or whether it wants to do all the work itself.

(It’s also important to note that these patterns are fully compatible with .NET 2.0 and require no additional platform features. They’re totally implementable in VS 2005, so you won’t need a new CLR to use them.)

We will also be providing three implementations of the LINQ API patterns that cover 95% of the types of data that people need to query: relational, object and XML. First, we will provide a set of “standard query operators” that will make the interface IEnumerable(Of T), the fundamental collection interface on the .NET platforms, queryable. Second, we will provide a component called “DLinq” that implements a query-enabled object/relational mapping service which allows querying over remote relational data. And third, we will provide a component called “XLinq” that implements a very lightweight, query-enabled XML DOM. By hitting each element of the ROX equation, we believe we will provide a query solution for most situations right out of the box.

Now we get to the languages, which is where the fun really starts. Any .NET-enabled language is free to build extra language support on top of the LINQ API patterns, and VB and C# have already begun to do so. Although you are welcome to call the lower-level APIs directly if you like, VB will provide a set of “query comprehensions” that provide a very natural, SQL-like syntax on top of the LINQ APIs. For example, if you have an array of Customer objects, you can write a query that looks like this:

Dim custs() As Customer = ...
Dim waCusts = Select c.Name, c.City From c In custs Where c.State = “WA”

This high-level syntax then compiles down to calls to the standard LINQ APIs. This means that you can learn a very simple, natural query language that then applies across any component that is query-enabled, including the standard query operators, DLinq, and XLinq. (C# will also be providing a high level syntax that is similar, but not the same. We’ll be talking about some of the differences as we dig more into the syntax.)

So far we’ve been discussing things that are general to the platform or apply to both VB and C# (modulo the differences in approaches). However, there are a several things that are specific to VB that play a role in the LINQ project.

The first is that we’ve been working on several features that we believe will make working with dynamic data easier. By “dynamic data,” I mean data whose schema is flexible, malleable or likely to change. The best example of this is XML data — although there is plenty of highly schematized XML out there, there is a vast sea of XML data for which no schema information exists or for which there is no 100% solid schema (RSS is a perfect example). For languages that understand only static type information, this means that when you deal with dynamic data, you frequently have to do a lot of extra work to access information, for example you end up writing:

XElement ord = ...;
int amount = (int)ord.Element(“amount”);

instead of something much more natural like:

Dim ord As Order
Dim amount As Integer = ord.amount

So, what we’ve been doing is taking VB’s historical embrace of dynamic binding (aka “late binding”) and extending it further to make working with dynamic data such as XML and relational much easier.

The other major thing we’ve been working on is making it much more natural to read and produce XML data. XML has become the lingua franca of the Internet and XML should be as easy as possible to work with. We’ve been exploring ways to take the XLinq API and add syntax that looks and feels very natural, but still compiles down to regular API calls:

Dim orderAmount As Integer = 10
Dim order As XElement = _

<% orderAmount %>

Dim orderDate As Date = CDate(order.@date)

As you can see, the LINQ project is extremely ambitious and this blog entry has just scratched the surface. In conjunction with the PDC, we are releasing a bunch of preliminary information about LINQ and about VB LINQ support specifically. There are documents describing in more detail a lot of what I discussed here today, as well as pointers to preliminary bits. (I would emphasize that word “preliminary” as in “lots of things don’t work yet.” We expect to roll out new bits regularly as functionality comes online.) I encourage everyone who’s interested to check them out and then check back here regularly. I’ll be talking in more detail about a lot of this stuff as time goes on and we’re looking forward to your feedback!

Tag:

VB as a dynamic language

Joel Pobar has posted an excellent start of what should be an interesting set of blog entries on how dynamic/scripting languages work on the CLR. When listing out the dynamic languages available on the CLR he includes among them VB, which may come as a surprise to some people. In many places the cult of Option Strict may have convinced people that VB is an entirely statically typed language like C# is, but they’d be wrong. In fact, a decent chunk of our Whidbey time was spent rewriting our late binder to handle all the new language features in this release: unsigned types, generics, overloaded operators. I’m not sure how much people realize that they can now, say, add two objects together and we’ll correctly figure out the right user-defined operator (and/or conversion operators) to call. It’s one of those less-heralded features.

The interesting thing, of course, is that because the CLR class libraries are much more statically typed than most COM libraries ever were, there are some ways in which dynamic typing has become much less important than it once was. If you were going to, say, work with the Office object model, late binding is pretty much a necessity in many instances. However, the .NET Framework has really gone out of its way to introduce strong typing wherever possible, partially because of general typing goodness and partially so that languages like C# don’t completely fall over in the face of too many things typed as Object. This has had the effect of really swinging the typing pendulum pretty far towards the strong typing side, so I’m really glad to see a lot of creative ferment in the scripting/dynamic typing space. Because, folks, let’s face it — strong typing can be a pain in the ass sometimes.

One of the places where this is most evident, I think, is in the data access space. As anyone who’s used a data-intensive application from Excel to Access to FoxPro to SQL Server can tell you, data is extremely flexible when it comes to typing. A lot of data out there is only semi-typed and as you work with data it frequently needs to change type (and shape, as you add and remove columns). Strongly typed systems often times have a hell of a time dealing with this flexibility, and so it comes as no surprise that pretty much every single application I listed above takes a weakly-typed approach (some more than others) when working with data.

As I’ve hinted at before, one of the major investments we’re going to be making in Visual Basic post-VB 2005 is in integrating data (both XML and relational) more deeply into the language than ever before. We’re planning on talking a lot more about specifics at the PDC, but this general topic definitely has a large impact on our thinking about data. It’s also been very interesting to watch as C# has also thought about data integration (as Brad also hints in his post) because they’re a much more statically typed language, which means there are some places where we may be going farther than they’re willing or able to…

Anyway, the benefits of dynamic typing are hardly limited to data access, and as we think beyond our current release we’re investigating ways in which we can take even more advantage of it in the future. In doing so, we’re really trying to straddle both sides of the religious war between the static typers and the dynamic typers. To paraphrase Erik Meijer (who we’ve been working a lot with lately) the ideal principle seems to be:

Static typing where possible, dynamic typing where needed.

This seems to be the sweet spot, the best of both worlds. Will we be able to achieve it? Well, we’re going to take our best whack at it…

Updated 07/08/05: Peter Drayton pointed out to me that he and Erik wrote a paper on their principle, so I added a link. Also fixed the link to Joel’s entry thanks to mikhar!

The Tyranny of the Suite

Another common comment around refactoring has been: why do we have to wait for a whole other release to get refactoring? Why can’t you just do the features and then ship them when they’re ready?

The answer, for the moment, is that we are subject to the “tyranny of the suite.” Which is to say, we are so tightly bound to the rest of the suite of products that we ship with (C#, C++, J#, debugger, IDE, SQL Server, CLR, etc.) that it is extremely difficult for us to ship separately from the rest of them. In effect, it’s like we’re running some freaky twenty person version of a three-legged race. Because all our legs are tied together, nobody can move forward unless we all do, and coordinating everyone to step at the same time is a major undertaking.

This binding was absolutely necessary to ship the initial version of .NET because it was a v1.0 product and we needed every piece of the puzzle in place to make the system work. Now that we’ve moved beyond that, though, it does seem like it should be possible to move more towards a “ship early, ship often” strategy that allows individual pieces to innovate separate from the rest of the suite. Some things, like generics, would still require simultaneous work across the entire stack, but other things, like refactoring, could be done in smaller, incremental releases.

Now, whether this actually happens remains to be seen. A lot of the machinery that goes into release management is shared across the suite, and it’s not clear how possible it is to disentagle it all. Plus, the butterfly effect holds: small, out-of-band releases increase the test matrix and make it more likely that, say, a minor VB release could conflict with a minor ASP.NET release in ways that were very unfortunate.

Anyway, I think what can be said best is that we are always looking to improve our release strategies and get new features into the hands of our users as soon as possible. Time will just tell.

Dynamic languages/dynamic environments

The .NET Languages blog recently pointed me to an SD Times article by Larry O’Brien entitled “Dynamic Do-Over.” Most of the later part of the article talked about IronPython and Jim Hugunin, but the earlier part touched on something that I’ve discussed earlier: the question of language strictness when it comes to typing. The more I think about it, the more I believe that static typing is a good thing and something that should be encouraged wherever possible. But when I say that, I don’t mean to say that there isn’t something of value in all those scripty-like dynamic languages out there. I think Larry hits the nail on the head in his article: what makes dynamic languages so great is not their loose type systems, but their dynamic environments.

In the end, I think anything that helps the average programmer be more productive is a good thing. By and large, static typing satisfies this dictum: static typing enables all kinds of programmer productivity features like Intellisense, better error messages at compile time, etc. (One could argue, I suppose, that you could lose the static typing and use type inferencing instead, but I wonder whether it would be possible to build a complete enough type inferencing ruleset that: a) was implementable, b) made some kind of sense, and c) could compete with just stating the damn type of your variables.) Dynamic environments also do this: edit and continue (pace Franz et al.), continuable exceptions, being able to call functions at design time, etc. So I think marrying the two worlds has some facinating possibilities.

I should add, though, that I don’t believe loose typing has no use. One application for loose typing that I’m particularly interested in is modeling unstructured or semi-structured data such as XML. I think the work that the E4X group has been doing is particularly interesting…

Time ever marches on…

Now that VB 2005 Beta 1 has shipped, the time has come to start doing some “long lead” work on versions beyond the 2005 release. What this means is that while 98% of the team remains focused full-time on finishing the current product, a few people get to start spending some of (though hardly all of) their time thinking about the kinds of big issues that we might want to solve in the next version(s). A good way of thinking about it is that before a version ships we try to lay some of the groundwork for the next release so that when the rest of the team is ready to engage on the next version down the road, we can hit the ground running. (Other teams are also starting to do some of this work, too.)

At the moment, I’m spending my long lead time thinking about two major things. One is: what can we learn from scripting languages? We sort of discussed this back in March, and now I’m returning to the question, both in terms of the question of development environment and in terms of the question of dynamism. (Which is also interesting given that the CLR hired a scripting language implementer specificially to work on making the CLR better for dynamic languages.) The other is: what can we do to improve data access? This is something that other languages are thinking about as well. My interest stems not just from the fact that people find it difficult at times, but from the fact that my first job at Microsoft involved working a lot with SQL and data access (I worked on the query designer in Access) and I’d like to see us improve in that area.

Those are just the two things that I’ve started with, but there’s certainly room for other things as well. What other things would you like to see? I’m not talking about features (you can go submit those requests here and the product teams will see them), I’m talking about big picture areas like the ones I mentioned above. What areas do you think we could do better in? What areas do you think there is untapped potential? Can’t promise I’ll look at everything, but I’m curious to see what people might be thinking.

(Since we MSFT bloggers often get dinged for talking only about the future and never about the present, I should add the disclaimer that I don’t plan on talking too much about this long lead work for the forseeable future. This blog is still going to properly remain focused on VB 2003 and VB 2005, not on random ideas about the future that may or may not ever come to fruition. There’s been way too much of that at Microsoft in the past.)

Boo to you, too!

Don pointed to the release of a new .NET programming language called boo. I downloaded the boo manifesto and took a look and it seems very interesting (and it’s nice to see some language experimentation going on on the CLR!). Some of the ideas aren’t unfamiliar – C# already has iterators and we’ve kicked around the idea of being able to enable late binding on a per-variable basis (called “duck typing” in the manifesto) – but others are less familiar. One idea I’m curious about seeing how it plays out is extensibility in the language. In the manifesto Rodrigo says:

I wanted a language I could extend with my own constructs.

This is one of those ideas that’s always seemed to me to be great on the face of it but questionable because of the law of unintended consequences. On the one hand, how can you argue with giving people the ability to extend the language as they see fit, especially given the fact that compiler releases can be a year or more? On the other hand, it’s so easy to screw up language design even when you’ve been doing it a long time that I wonder if you won’t quickly have a language that’s no longer comprehensible by man or machine.

One thing that Rodrigo loves about .NET is the nice way that the entire set of class libraries are architected together in a pretty clean and consistent manner. This is a great advance over the hodgepodge approaches of the past where fundamental libraries were incompatible or worked in entirely different ways. I wonder, though, if allowing extensible languages isn’t taking a step backwards into that world. Central control of a language ensures some reasonable amount of consistency in thought and design – if anyone is allowed to add extensions, will the language gradually devolve into the language of Babel? Or will only extensions that preserve the core style survive while the others die off? I guess we’ll just have to see.

I will add that one major problem with our compilers today is that entirely too much information is locked up in them. We’ve started exposing compilation information through mechanisms such as the Visual Studio code model, but we need to go a lot further to enable more advanced and extensible tools. It’s certainly possible to expose a lot more information about what exists in the code that’s being compiled without having to make that information modifiable outside of the compiler, and I think that’s something that’s unquestionably worth pursuing in the long run.

Simpler IDE + more permissive VB language = script magic?

I just wanted to take a moment to acknowledge an interesting series of entries by Mike Schinkel (here and here) that were followed up by a reply by Eric (here) and a further reply by Mike (here). To be honest, my reading attention span can be very short and I find my eyes glazing over when I start to read longer-winded blog entries, but these did catch my attention. (My short attention span is, of course, very ironic given that I can be kind of long-winded myself…)

To grossly oversimplify Mike’s points, he feels that

  1. The Visual Studio IDE is too complex for many users.
  2. The Visual Basic language as it stands now is too difficult for many users.

I have absolutely no argument with point #1. The kitchen-sink approach to VS IDE development has had its upsides, to be sure, but the resulting behemoth is a bit too complex even for my taste. And I’ve gotten this same feedback from others as well. This is definitely something we’re working to address in the future.

Point #2 is a little more interesting. There’s the usual problem with the overwhelming size of the Framework and how to make that digestible, but Mike also raises the question of strictness. He makes the argument (echoed by Don Box and others) that many programmers would do better with a world that’s more typeless and less strict than the one they get on .NET. As someone who lives day to day in a strongly-typed world (so to speak), this seems somewhat counter-intuitive: less typing usually equals less performance and less compile-time checking, leaving problems to be discovered at runtime. In fact, one of the major features of this release, generics, is specifically about making it easier to have *more* type safety and better performance at runtime. So the persistent voices in favor of scripting appear to be swimming against the tide. The question is: are they right?

To be honest, I think the whole question deserves more thought. We purposefully spent a lot of time making sure that things like operator overloading and generics work with late binding in Whidbey, so it’s not like we’re closing down that avenue of possibility. But, hmmmm…. Dunno. I’m just going to have to think about it some more.

(“Top minds are working on it“? Does that mean I also get to take a look at the Ark of the Covenant too?)

Why do hobbyists matter?

After Kathleen worried about losing the hobbyist programmer on .NET, Rory came back with the question “Should the hobbyist programmer matter to Microsoft?” His thesis, in a nutshell, was:

I say that we don’t worry about the hobbyists – don’t dissuade them from coding in .NET, but don’t cater to them either.

I understand where he’s coming from, but I think that the terminology is confusing the issue. When we talk about “hobbyist programmer,” it evokes images of guys tinkering in their garages or in their basements on the weekend. And, yeah, maybe if it was just the equivalent of a bunch of guys (or gals) building model trains or making furniture or rebuilding old cars, it wouldn’t matter so much. But the reality is that the hobbyist programmer doesn’t just program on the weekends – they’re also programming during the week at their “real” jobs.

Before I started working on VB, I worked on Access. And I cannot count the number of times that customer testimonials started along the lines of “I was fooling around with Access one day and managed to write this small app to help manage my group. Once my department found out about it, they started using it to manage the department. Now my whole company uses it!” One of the key aspects of Access’s success was this kind of “viral adoption” where some tinkerer used it to solve some local problem that ended up solving a company-wide problem. The same holds for VB – lots of VB applications in corporations started life as someone’s side project. As I put it in a recent presentation, “Throwaway applications have a way of becoming mission critical applications.” And where do those throwaway applications come from? Hobbyist programmers.

With the spread of computing into more and more industries, the people who don’t consider themselves programmers become more and more important because they’re the beachhead for “real programming” to make its way in. For example, the throwaway applications that hobbyists write ultimately helps drive demand for professional programmers to come in and “professionalize” the applications so that they scale correctly for the corporation. Also, as hobbyist applications make companies more open to the benefits of technology, they open the door to commercial software that can augment or replace the homegrown applications and maybe do a better job. And, of course, hobbyist programmers usually need lots of help, which drives demand for websites, magazines, books, consultants, etc.

So, in much the same way that small businesses serve a vital function in keeping the economy going so that large corporations can thrive, hobbyists play a vital role in sustaining the ecosystem that supports the professional programmers. Even if the professional programmers don’t always appreciate that…