Category Archives: Visual Basic 2008

Integrating XML: Playing with Fire?

12 Replies

I’ve been thinking about something Wes said last week about C# 3.0 and VB 9.0. After viewing my PDC presentation slides, he noted:

The difference in focus between the C# and VB team is becoming much more evident: The VB team is opting for greater relevance and convenience in the short term, while the C# team is trying to avoid creating long-term baggage.

I think this succinctly describes what’s going on in regards to VB’s embrace of XML integration into the language and C#’s cautiousness around it. If you talk to the C# team about this, you get some very explicit concerns about the long-term impact that integrating XML into the language might have. I would characterize these concerns in my own words as:

  1. It is extremely risky to put some portion of your language design in the hands of another language design team. In other words, if VB integrates XML as it stands today into our language, what happens in the future as the XML standard continues to evolve? What we do today may be sufficient and there may not be any demand for adding any further features the XML standard may add. But maybe not. And those features will not necessarily have been designed with an eye towards whether they’ll work well in VB. Heaven knows we’re going to have enough issues shoehorning XML into the language, what happens if we hit some really sticky issue in the future? What if XML heads in a direction we’re not willing or able to go in?
  2. It is extremely risky to tie yourself explicitly to a technology that may or may not be here 15 years from now. Right now, XML is king. But what happens if some other technology comes along and knocks it off its perch? What if things radically shift in some other direction and XML suddenly becomes a side track instead of the main line?

In truth, both of these concerns are valid. In fact, VB’s own history has borne them out — all one has to do is replace “XML” with “COM” and you have the whole VB6 to VB.NET question in a nutshell. I mean, it doesn’t seem likely today that XML will ever lose its primacy, but if you could take a time machine and go back to the pre-.NET days and ask Don Box if COM was going to be supplanted in a few years time, what do you think he would have said?

So, if integrating XML is so gosh-darn risky, then why are we doing it? Are we nuts? My answer is: not exactly.

Like so many things in life, it’s all about tradeoffs. The C# team looks at the problem and says, “It’s more important to us that we design a language that is as minimal as possible and whose pieces are guaranteed (as much as anything ever is) to retain relevance for the forseeable future. Therefore, we’re willing to sacrifice some usability and some programmer speed to ensure this goal is reached.” Perfectly valid. The VB team, on the other hand, looks at the problem and says, “It’s more important to use that we design a language that allows people to solve the problems that are confronting them today in as simple and straightforward manner as possible. Therefore, we’re willing to take the chance that some feature that is fantastically useful right now may lose relevance in the future and perhaps even become almost entirely useless (c.f. Gosub, On … Goto, Open/Close, etc.).” Also perfectly valid.

What it boils down to is that we’re willing to make the bet that the overall benefit of integrating XML now is going to, in the long run, outweigh any possible downside. Are there people who are going to say we’re crazy? Of course. How can we tell who’s right? Check back in 15 years. In the meantime, I leave you with the conclusion to Wes’s quote above:

The end result is that Visual Basic is definitely going to be much more fun to program.

 See you in 2020…

Is it Select/From or From/Select?

31 Replies

Over in the LINQ MSDN forums, the question came up: which is better, Select/From (as VB does it) or From/Select (as C# does it)? To quote from the thread, here’s the C# side of it, straight from Anders:

There are a multitude of reasons why select comes at the end and not the beginning of a query in C# 3.0. The more important ones are:

(1) Statement completion (Cyrus’ blog has a good explanation).

(2) Order of execution. The C# query syntax lists operations in the order they are executed.

(3) Scope of “from” variables. SQL is strange in that scope flows both upward and downward. In C#, scope extends from the point of introduction to the end of the query, which seems much more intuitive.

(4) Large select expressions. In SQL, because results are rectangular rowsets, select clauses tend to be fairly simple. However, when querying objects and XML it is quite common to have large select expressions that construct entire object graphs, possibly with multiple nested queries. Trying to understand a large select expression written in terms of variables that haven’t been introduced yet (and may not even be visible on the screen) is quite confusing.

(5) Even if we picked SQL’s ordering, the similarity would be skin deep. There are lots of other differences. For example, C#’s built-in operators and quite different from those of SQL. I actually think the different ordering is a benefit because it makes it quite clear that this is not SQL.

Note that XQuery’s FLWR (“flower”) expressions have the same ordering as C#’s query expressions–I suspect for some of the same reasons.

Anders

 and here’s the VB side of it, straight from me:

Speaking from the VB team perspective…

As is already obvious from this thread, Select/From ordering is going to be another one of those religious issues like case insensitivity that people are going to be arguing about for the next twenty years and beyond. As with case sensitivity, both sides can marshal perfectly reasonable arguments as to why their choice is the One True Way(tm) and why the other side is Consorting With the Devil(tm). I say this knowing that nothing I say now can influence that outcome and that this discussion (in the larger sense) must always end in tears, recriminations and Godwin’s law. But it’s worth saying nonetheless.

With that out of the way, I think this thread has already covered many of the relevant points relating to why VB chose to say Select/From instead of From/Select:

* For many, many programmers and for many, many VB programmers, specifically, SQL is a very familiar language. Leveraging a huge existing base of knowledge (and programmed muscle memory) makes the LINQ support more understandable and usable straight out of the gate.

* The SQL ordering of clauses is a time-tested convention that has been in continuous use for decades.

* VB emphasizes English readability. As noted in the beginning of the thread, the SQL ordering is more English readable than the obverse.

Anders raises some objections to the SQL ordering, but on the balance, we believe that the benefits outweigh the limitations. Specifically, I’d say:

* Statement completion is a significant question. We have a bunch of ideas as to how we could finesse this in the IDE, but we haven’t reached a point of being really able to try them out. This may be a real sticking point, time is just going to tell.

* I don’t agree with Anders’s points about the difficulty of understanding SQL’s clause ordering. Although some variants of SQL have invented phenomenally complex and bizarre sets of rules about how binding works, the basic rules about binding seem to have been graspable by millions of SQL programmers over the years without too much pain and suffering. As long as the rules remain relatively simple and straightforward, there don’t appear (to me, at least) to be major hurdles in implementing a very understandable Select statement where From is in the middle of the statement. Reasonable people can disagree on this point, however.

* Ultimately, we don’t believe that people will be confused as to whether this syntax is SQL because it will become almost immediately obvious that it isn’t once a programmer works with it for more than a few minutes. It’s a problem that will sort itself out on its own relatively quickly.

I would close by saying that the VB team, overall, values practicality over dogmatism. From a pure syntatic perspective, clause ordering is not fixed in stone – one could support both a Select/From order and a From/Select order without much difficulty. Indeed, we’d like to look at how clauses such as Where and Order By can be used on their own without requiring a Select or, perhaps, even a From. So while we will continue to support Select/From, other possibilities may be investigated if it becomes obvious that they make the language more usable in some strong way.

That’s about it. We now return you to your regularly scheduled religious debate…

Paul
VB Team

And that’s where it stands at the moment… I’m sure I’ll get a few comments on that, though.

See me on Channel9…

2 Replies

Two new videos, timed to coincide with the PDC are now up:

Paul Vick and Erik Meijer – Dynamic Programming in Visual Basic – This is me and Erik talking about dynamic stuff in VB. We were planning on talking about some of the new XML stuff as well, but didn’t even get to it! Another video, I’m sure…

Paul Vick and Amanda Silver – VB Language Futures – This is me and Amanda talking about a lot of the future stuff we talked about at the PDC. Check it out!

Hopefully more to come. If you have specific topics you’d like to see, let us know! Based on some of the comments, we may still have some kinks to iron out in the presentation department, but videos are easy enough to do and we’d love to talk about what you’re interested in hearing about!

TLN308: Visual Basic: Future Directions in Language Innovation

Leave a reply

Now that I’m back, safe and sound, in Seattle, I can start to catch up on some of my blog backlog! So let’s start with the highlight of the trip, my talk. I was a little nervous about it, but the worried turned out to be for naught — the session went really well! Lots of people showed up and we had a good time talking about all the ideas we’ve been working on for VB 9.0.

I’m not sure what the plans are in terms of making slides available outside of the PDC. You can find a bunch of the VB 9.0 PDC information here, and I’ll be talking more in upcoming months about the stuff we discussed in the talk. As I said in the talk, too, there’s a lot more going on than we had time to talk about in the session, so we’ll be talking more about that stuff, too!

The audience reaction was very positive. People were extremely excited about LINQ, the VB query syntax and, especially, the integration of XML into the language. Got a lot of comments on that.

This is just the beginning, more to come…

Tag:

It gets inside your head, man…

4 Replies

Still not at a point to post more about how the session went (long story, short: very good), but I had a funny moment just a second ago. I was looking at the internal website that has speaker evaluation information to see how people liked the session. Pulled up the data and sorted it in Excel. Then I was thinking, “You know, I really want to filter out sessions that didn’t get a lot of feedback reports because they might skew the data.” Since I’m not an Excel expert, the thought occured to me:

You know, I could just fire up my LINQ-enabled VB that I demo’ed today, save the Excel file to XML, read the XML into XLinq and then write a query on that data and then I’d be able to use VB instead of having to figure out all that Excel stuff.

What happens when you drink your own Kool-Aid? I guess I’ve found out…

Tag:

Introducing LINQ!

25 Replies

Throughout this week and beyond, we’re going to be doing a lot of talking about Project LINQ, so I wanted to start with a relatively brief overview of what the project is, what its goals are, and how they apply to VB.

To start with, LINQ stands for “Language Integrated QUery.” LINQ fundamentally is about integrating query operations into the .NET platform in a comprehensive and open manner. It’s also about providing a unified way for you to query across any kind of data that you have in your program, whether it’s relational, objects or XML. This, we believe, will represent a tectonic shift in the way that VB programmers will work with data. The possibilities that having query capabilities always available right at your fingertips, regardless of the type of data you’re working with, are immense and will fundamentally alter the way people program.

The core of LINQ is a set of API patterns. I say “patterns” instead of “interfaces” or “classes” because we want to maintain maximum flexibility in our ability to provide querying for any kind of data, regardless of whether it has a particular object model or interface implemented. These API patterns specify how an object model can become “queryable,” and they cover all the basic query operations that we know and love: projection (select), filter (where), grouping (group by), ordering (order by), joining (join), etc. By implementing this API pattern, any data provider on .NET can become queryable. There is also a lot of flexibility as to how a data provider can become flexible — i.e., whether it wants to delegate most of the work on querying or whether it wants to do all the work itself.

(It’s also important to note that these patterns are fully compatible with .NET 2.0 and require no additional platform features. They’re totally implementable in VS 2005, so you won’t need a new CLR to use them.)

We will also be providing three implementations of the LINQ API patterns that cover 95% of the types of data that people need to query: relational, object and XML. First, we will provide a set of “standard query operators” that will make the interface IEnumerable(Of T), the fundamental collection interface on the .NET platforms, queryable. Second, we will provide a component called “DLinq” that implements a query-enabled object/relational mapping service which allows querying over remote relational data. And third, we will provide a component called “XLinq” that implements a very lightweight, query-enabled XML DOM. By hitting each element of the ROX equation, we believe we will provide a query solution for most situations right out of the box.

Now we get to the languages, which is where the fun really starts. Any .NET-enabled language is free to build extra language support on top of the LINQ API patterns, and VB and C# have already begun to do so. Although you are welcome to call the lower-level APIs directly if you like, VB will provide a set of “query comprehensions” that provide a very natural, SQL-like syntax on top of the LINQ APIs. For example, if you have an array of Customer objects, you can write a query that looks like this:

Dim custs() As Customer = ...
Dim waCusts = Select c.Name, c.City From c In custs Where c.State = “WA”

This high-level syntax then compiles down to calls to the standard LINQ APIs. This means that you can learn a very simple, natural query language that then applies across any component that is query-enabled, including the standard query operators, DLinq, and XLinq. (C# will also be providing a high level syntax that is similar, but not the same. We’ll be talking about some of the differences as we dig more into the syntax.)

So far we’ve been discussing things that are general to the platform or apply to both VB and C# (modulo the differences in approaches). However, there are a several things that are specific to VB that play a role in the LINQ project.

The first is that we’ve been working on several features that we believe will make working with dynamic data easier. By “dynamic data,” I mean data whose schema is flexible, malleable or likely to change. The best example of this is XML data — although there is plenty of highly schematized XML out there, there is a vast sea of XML data for which no schema information exists or for which there is no 100% solid schema (RSS is a perfect example). For languages that understand only static type information, this means that when you deal with dynamic data, you frequently have to do a lot of extra work to access information, for example you end up writing:

XElement ord = ...;
int amount = (int)ord.Element(“amount”);

instead of something much more natural like:

Dim ord As Order
Dim amount As Integer = ord.amount

So, what we’ve been doing is taking VB’s historical embrace of dynamic binding (aka “late binding”) and extending it further to make working with dynamic data such as XML and relational much easier.

The other major thing we’ve been working on is making it much more natural to read and produce XML data. XML has become the lingua franca of the Internet and XML should be as easy as possible to work with. We’ve been exploring ways to take the XLinq API and add syntax that looks and feels very natural, but still compiles down to regular API calls:

Dim orderAmount As Integer = 10
Dim order As XElement = _
    
        <% orderAmount %>
    
Dim orderDate As Date = CDate(order.@date)

As you can see, the LINQ project is extremely ambitious and this blog entry has just scratched the surface. In conjunction with the PDC, we are releasing a bunch of preliminary information about LINQ and about VB LINQ support specifically. There are documents describing in more detail a lot of what I discussed here today, as well as pointers to preliminary bits. (I would emphasize that word “preliminary” as in “lots of things don’t work yet.” We expect to roll out new bits regularly as functionality comes online.) I encourage everyone who’s interested to check them out and then check back here regularly. I’ll be talking in more detail about a lot of this stuff as time goes on and we’re looking forward to your feedback!

Tag:

Coming soon to a VB near you… LINQ!

Leave a reply

Today at the PDC, we’re launching “Project LINQ,” and it’s big. Really big. I mean, really, I wish I had the money to hire those guys who do the movie trailer voiceovers (and you know who I’m talking about):

In a world torn apart by incompatible conflicting and incompatible data domains, one company dared to stand up to the status quo. (Shots of scowling programmers looking at computer screens. One desperate programmer exclaims “We just can’t make it work!”) Doing what they said couldn’t be done, that company cut through the artificial barriers separating the world’s data, making querying relational data, object data and XML data as easy as a stroll in the park. (Voice over of programmer gasps, “Oh my god!”, fade out to black.) Now, a project that was millions of years in the making (well, two actually, but this is a trailer, after all) arrives to change all the rules: Project LINQ!

(This project not yet rated. No one under 17 admitted with out parent or guardian. See website for more details.)

But enough foolishness. Let’s talk real turkey…

Tag: