You Can Go Home Again (or, Returning to Programming Languages)

About a year and a half after doing something I swore I’d never, ever do, I have to admit… it didn’t stick. While working in Windows during the Windows 10 initial cycle was fascinating and fun, it turns out that I just miss working in programming languages too much to become a long-term member of the Windows team. I had a great time working with some great people (and a few of the things I worked on are just about going to go public…), but in the end, I’ve been lured back to the PL world.

Interestingly, I’ve decided to return to the team I left seven-odd years ago just as they were beginning to start this crazy re-write of the VB and C# compilers into managed code. Now that Roslyn has shipped, I felt it was finally safe to return… So I’ve accepted a position back on the Visual Languages team. I don’t plan on doing anything specific with Visual Basic — that language is in very good hands at the moment — but will be working on some ideas about how we might leverage Roslyn to do some interesting stuff. More information to come as it becomes available!

You should also follow me on Twitter here.

Edge JsRT: Miscellany

To close out the series on Edge JsRT, there are two minor APIs that were added that didn’t fit any of the other categories.

Filling in a hole I accidentally left when I originally did the APIs, JsNumberToInt allow extracting an integer value from a JS Number value.

And to aid in debugging and other diagnostics, JsCreateNamedFunction allows creating a function that has a name associated with it that will show up in callstacks, toString, etc.

You should also follow me on Twitter here.

Edge JsRT: Universal Windows Platform

The last major chunk of new APIs in Edge JsRT relate to the Universal Windows Platform, or UWP. Hopefully, you’re familiar with the UWP already, but if not, the UWP is the “modern” way of building Windows applications that go into the Windows Store. (Thus, UWP apps are also called “Store apps”.) At it’s most basic, the UWP is a set of curated APIs that UWP applications can use to talk to the system. UWP APIs are built in a special way (similar to the way .NET APIs are built) that allow them to be written once and then be usable in C++/CX, JavaScript, and C#/VB.

The way that UWP APIs are exposed in each language is called projection. To make projection work, each language has a special piece of code that knows how to take a UWP API and make it appear correctly to programs written in that language. To turn on that special code in Chakra requires some calls that were not available to JsRT hosts in the past, but which have now been exposed in Edge JsRT. The main one is called JsProjectWinRTNamespace. (WinRT is yet another legacy name for the UWP.)

JsProjectWinRTNamespace will take a root UWP namespace (such as “Windows”) and project it into the current context, making that namespace available in the global JS namespace. Great, right! Well, there’s one catch: unless your host is a UWP application (i.e. a Store app), you can only project the Windows namespace. That’s because projecting anything outside of the Windows namespace requires loading some UWP component that supports that namespace. And UWP components can only be loaded by UWP applications. This is a somewhat entirely arbitrary restriction, but since the UWP platform is built that way, oh well. It does mean that non-UWP hosts can still get access to some useful UWP system APIs.

The other API is a little more esoteric, but still important. JsSetProjectionEnqueueCallback is needed to support asynchronous UWP APIs. If you’ve been following along at home, you’ll remember that my previous post talked about setting up a callback that allows asynchronous JavaScript APIs to let you know to run some code when an ES6 async API is finished. Well, this is basically the same thing, but for the UWP platform as opposed to ES6. So go back to read that post if you want to understand a bit more what this is for. The shape is slightly different (confusingly) but the purpose is basically the same.

You should also follow me on Twitter here.

Edge JsRT: Promises

One of the major additions to the ES6 API surface was Promises. I’m not going to go into the deep nitty-gritty about Promises (you can find more here), but in a nutshell a Promise represents an asynchronous operation. So you can call an API that does something long running, and it can immediately return to you a Promise that represents the result of that action in the future. You then can say “When this promise is fulfilled (i.e. the operation is completed), please call me back and I’ll do something with the result.”

Mostly, hosts don’t have anything special to do with Promises except for that last bit–the “please call me back” part. When the long-running operation completes and it notifies the Promise object that it has been fulfilled, there has to be a way for the Promise to tell the host, “OK, now you need to run this bit of code (i.e. the part that wants to be called back when the Promise is fulfilled.)”. Hosts can now set this up by calling JsSetPromiseContinuationCallback. It allows the host to provide a callback to Chakra that Chakra can call when a Promise completion callback needs to be queued up. The parameter will be a JS function that will need to be called whenever the host has some available time (i.e. when something else isn’t running in the associated Chakra context).

You should also follow me on Twitter here.

Edge JsRT: Typed Arrays, Array Buffer and Data View

Probably the biggest change surface area-wise for JsRT in Edge relates to APIs to support typed arrays. Typed arrays are data structures that allow data exchange/protocols to be implemented more efficiently by allowing buffers to be accessed in strongly typed (i.e. efficient) ways. In particular, it allows buffers to be accessed specifically as integral values rather than the general Number type that JS supports. Obviously, these can be extremely useful for native apps hosting JS, so support for working with typed arrays is a major addition to Edge JsRT.

There are three levels of objects, which I won’t go into extreme detail on because they’re well-documented elsewhere:

  • The lowest level are ArrayBuffers, which represent the raw buffers of bytes. These can be manipulated using JsCreateArrayBuffer and JsGetArrayBufferStorage. You can’t replace the storage of an ArrayBuffer–it’s basically a buffer that Chakra allocates on your behalf.
  • The next level up is a DataView, which wraps an ArrayBuffer and lets you read and write from it in a strongly typed way. These can be manipulated using JsCreateDataView and JsGetDataViewStorage. You can call the JS methods defined on the object to do the strongly typed read/writes.
  • The top level is TypedArrays. A TypedArray wraps some portion of an ArrayBuffer and can be used as a fixed-size array of values of one particular type (i.e. an array of 5 Int32 values). These can be manipulated using JsCreateTypedArray and JsGetTypedArrayStorage. You can use the JS index operation to access the members of the TypedArray.

You should also follow me on Twitter here.

Edge JsRT: External indexed properties

Several of the features that Chakra added to the hosting APIs for Edge were inspired/required by the port of node.js on top of Chakra. One of the biggies, I believe, is the new ability to override the numerically indexed properties on an object using a native blob of data. The APIs in question are JsHasIndexedPropertiesExternalData, JsSetIndexedPropertiesExternalData, and JsGetIndexedProeprtiesExternalData.

The way they work is that you have a blob of some particular native type (say, 32-bit integer) and you can say “hey, when someone asks for this object’s numerically indexed properties (i.e. obj[1], obj[2], etc.), give them back the answer from this blob.” It’s a quick way to make a JS object look like an array and back the array with a native blob (instead of having to set every individual index).

The V8 hosting API has a very similar API, and I believe the node.js Buffer object uses this to quickly set the contents of a Buffer before passing it into JS. When I worked on the original port of node.js on top of Chakra, there were all sorts of hacky hacks we had to make to get this to work–and it was always fragile–so I’m not surprised this API makes an appearance. (Ironically, I think that everyone might be better off using ArrayBuffer/TypedArray, but you work with the code you have, not the code you want to have…)

You should also follow me on Twitter here.

Edge JsRT: Symbols

In ES6, the new Symbol language feature allows people (especially hosts) to extend JavaScript objects without polluting their property namespaces. So you can add, say, a Boolean property to an object and instead of giving that property a name like “isCounted,” you can instead create a Symbol (“var isCountedSymbol = Symbol();”) and then use that symbol as the property name (“obj[isCountedSymbol] = true;”). Then people iterating over the property names on that object won’t see your symbol and won’t get tripped up. And you can be fairly sure no one else is going to accidentally stomp on your property or collide with it.

(You can read more about the language feature here.)

Symbols are values, so JsGetValueType now has a new type it will return, JsSymbol. You can create a new Symbol through JsRT using JsCreateSymbol. You can get a property ID from a symbol using JsGetPropertyIdFromSymbol, and you can get the symbol from a property ID (assuming it is a property ID created from a symbol) using JsGetSymbolFromPropertyId. JsGetPropertyIdType will tell you whether a property ID was created using a symbol or a property name. You can use GetOwnPropertySymbols to get any properties on an object that are indexed using symbols (as they won’t show up in GetOwnPropertyNames).

You should also follow me on Twitter here.

Edge JsRT: Object Before Collection Callbacks

There are a number of new features in the Edge version of JsRT, and since I haven’t seen anything discussing them on the web, I thought I’d walk through them.

The first one I thought I’d discuss is object before collection callbacks. In the first version of JsRT, you could create an “external” object that could have a callback that would be called when the object was finalized. But this only allowed you to know about objects that you create. What if you need to know when an object that you’ve attached some special significance to is collected? Now you can call JsSetObjectBeforeCollectCallback, provide it the reference to track, some state, and a callback function. When the GC is going to collect the object, you’ll get a callback to let you know.

Useful for managing lifetime of native objects that track some JS object!

You should also follow me on Twitter here.

The Two Faces of JsRT (in Windows 10)

In terms of Windows 10, probably the biggest thing to happen to the JsRT APIs in Windows was the IE/Edge browser split.

For anyone who’s been living under a rock, Microsoft introduced a new browser in Windows 10 called Edge (neé Project Spartan). Edge basically represents a fork in the road for the web browser codebase at Microsoft: down one path the IE code will continue on, largely unchanged and preserving all that crufty backwards compatibility we all love to hate, and down another path will go Edge, discarding all the legacy shackles of IE and committing to a thoroughly modern browser experience.

And, just as the browser now splits into IE and Edge, so does the underlying JavaScript engine, Chakra. There are now two (yes, two!) JsRT APIs:

  • The “legacy” JsRT APIs that are exposed by jscript9.dll (the one that IE uses), have not changed since Windows 8.1, and will not be added to.
  • The “Edge” JsRT APIs that are exposed by chakra.dll (the one that Edge uses), have some breaking changes since Windows 8.1, and will be updated as the Edge browser moves forward.

You can find more details on the breaking changes that were made to the Edge JsRT on MSDN, but in a nutshell they are:

  • In C++ you #define USE_EDGEMODE_JSRT before including jsrt.h, and you link against chakrart.lib instead of jsrt.lib.
  • In C# and VB, you target your PInvoke declarations at chakra.dll instead of jscript9.dll.
  • The “version” parameter of JsCreateRuntime disappears (since there will only ever be one evergreen version of the Edge Chakra).
  • The debugging no longer requires an IDebugApplication. Thus, JsStartDebugging and JsCreateContext lose a parameter.

The last bullet point may seem a little odd and arbitrary. The reason is that the Edge version of the JsRT API is now supported as a part of the UWP (Universal Windows Platform). That means that a “modern” UWP application can host the Edge Chakra runtime and still qualify for the Windows Store (i.e. it won’t be rejected for using non-UWP APIs). To get there, though, the Chakra team needed to remove things that don’t work on the UWP, namely talking to the PDM (Process Debug Manager), which is what you need to get an IDebugApplication. Since it turns out in almost every possible case you don’t need to be able to supply your own IDebugApplication, Chakra now just acquires one on your behalf.

I’ve updated my chakra-host sample on GitHub to include samples that show hosting both “legacy” and “edge” JsRT on Windows 10. I’ve also added a new sample to the Windows Samples showing this. The Chakra team also now has a GitHub repo where they host their own samples.

You should also follow me on Twitter here.