Author Archives: paulvick

Running node.js on Chakra!

It’s been a very busy year since I last posted, and a lot has been happening. As a result, I’ve kind of fallen behind a bit on all the stuff that came out around the Windows 10 launch. And one of the biggest things that I missed blogging about was the revelation of a (temporary) fork of node.js that runs Node on Chakra instead of V8!

This was actually the culmination of some work that I started several years ago as an internal project to enable some teams at Microsoft to author node.js applications that ran on Chakra instead of V8. The problem, then as now, was that there were some platforms where V8 didn’t or couldn’t run. So I spent a fun month or so working with another team concocting a shim over the nascent JSRT APIs that made them look a lot like the V8 APIs. The end result was a node.js binary that used Chakra and ran all the stuff that the team I was working with needed. Then the code went sort of into hibernation, and when I left the Chakra team, my expectation was that nothing more would come of it.

But, lo and behold, the IoT folks wanted node.js and they wanted it on ARM, which V8 doesn’t have an official code generator for, so that code miraculously got a second chance at life! After a lot of extra work on the part of the Chakra team (since the original shim only covered the scenarios the internal team needed), it was polished up and is now running out in the wild. It’s great to see that work continuing, especially since node.js can really be a joy to program in. Combining node.js with the ability of Chakra to access UWP (Universal Windows Platform) APIs also opens intriguing possibilities.

I’ve been thinking that I wanted to clone the fork to see if there was anything I could contribute, but things have been just too busy. Soon, maybe!

You should also follow me on Twitter here.

Windows, Diagnostics, and Me

Although, as I said before, I can’t talk specifically about what I’m working on in Windows for the time being, there are some general things I can say. The main thing I’m doing right now is some investigations around the diagnostic API surface of Windows. By “diagnostic,” I mean any information that Windows may possess regarding the internal state of the system at any particular moment that might be useful for understanding what’s going on. This is a pretty wide-ranging definition, covering everything from the current state of a thread up to the current state of the operating system itself.

The thing is, Windows contains an enormous amount of information about the state of things at any particular moment, but getting to that information can often be difficult or impossible. For example, a lot of components in Windows expose Event Tracing for Windows (ETW) data which can be used to look at what’s going on inside that component. The problem is that you have to know: a) which of the 100s of components that expose ETW events you should be looking at, b) which events that component surfaces you should be looking at, and c) what they mean. And usually there’s little or no documentation for any of it. In other cases, there isn’t even ETW events available–the data is just locked up in the internals of the system (unless Mark Russinovich has found it, of course).

The question I’m looking at is whether there are things that we could do to make Windows diagnostic information more available to developers (who could then use it to better understand their applications, or to build better tools for developers). I have no answers as of yet and, of course, nothing may come of this, but there you are. If you happen to be a Windows developer and are interested seeing some aspect of Windows diagnostics become more available to you, drop me a line in the contact form, I’d love to hear from you!

You should also follow me on Twitter here.

Restored some old VB parser samples…

It occurred to me the other day that my old VBParser samples for VB 7.1 and VB 8.0 had gotten lost in the ether somewhere along the line and were no longer on my website anywhere. So I dug up the sources and added them to my GitHub profile. I kind of seriously doubt that they’re a lot of use anymore, given that they are at least, what, three versions out of date now? But who knows?

You should also follow me on Twitter here.

NodeRT: Accessing WinRT from Node.js

If you’re using node.js on Windows and are interested in getting easy access to some of the cool WinRT-based APIs in Windows 8.0 and later, you should check out the new NodeRT project on Github. It’s a tool that allows you to automagically generate native node.js modules from WinRT winmd metadata files, making them easily available in node:

Since node.js runs as a regular Win32 application and not a Windows Store application (is that what we’re calling Metro these days?), you won’t be able to get access to Store-only WinRT APIs, but there are still quite a few useful WinRT APIs available to desktop apps.

Pretty awesome, check it out!

You should also follow me on Twitter here.

JsRT: Wrappers updated to v1.1

Even though I may not be working on the Chakra team any more, I’m still doing some noodling on making the hosting APIs easier to use. I’ve got some other interesting things coming soon, but on the way there I made some improvements/fixes to the wrappers that I released a while back.

Changes to v1.1:

  • PDB files are generated along with the LIB, and debug builds are included. All flavors are now packaged together.
  • Header file name is changed to jsrt-wrappers.h to be consistent with the other files.
  • Added equality operators for handles.
  • Added function_base::create which can deduce the strongly typed function wrapper from the function signature.
  • Fixes the handling of optional parameters in functions.
  • Changes the handling of rest parameters to use std::vector rather than a JavaScript array, which is more efficient.
  • Simplified prototype handling.
  • Bug fixes.

Let me know if there are any problems!

You should also follow me on Twitter here.

I Just Did Something I Swore I’d Never, Ever, Ever Do

OK, so I’ve worked at Microsoft… a long time. This summer, it’s going to be twenty two years since I first started full time, and twenty three years since I was an intern. During that time, I’ve worked in a lot of different places in the company. The Developer Division. Office. Developer Division again. A little division that doesn’t exist anymore, the Connected Systems Division. SQL. Developer Division again. And in all of that time, I had one cardinal rule: I was never, ever, ever going to work in Windows. No matter what happened, no matter how desperate I was, I was just never going to take a job anywhere in the Windows organization. No way, no how.

It’s not that I had anything against Windows or the Windows group, it’s just that for most of my tenure here so far, the Windows group has seemed… a little nuts. Not nuts in terms of what they’ve produced (although the whole Longhorn thing, ug), but in terms of how they seemed to work. From the outside, it looked like Windows was always on one kind of death march or another, from Windows 95 all the way through Windows Vista. And as much as I might like the OS, I just never thought that I was hardcore or badass enough to be able to withstand that kind of pressure.

There was also the fact that most of my career has centered around developers in one way or another, and for most of my time at Microsoft, Windows has been content to simply outsource the majority of the developer relationship to the Developer Division. This isn’t shocking or anything–hey, “Developer” is right there in the name!–but it meant that if I wanted to be working on developer-related stuff, there was only one place to be.

By now, I’m sure you’ve guessed where this is heading. After spending two very enjoyable years noodling on JavaScript hosting (and other related stuff) on the Chakra team, I’ve decided to break my longstanding vow and join Windows. Shocking, I know.

I decided to do this for two reasons. The first one is that since Windows Vista the Windows team seems to have adopted a much more sane attitude towards the whole work/life balance, something I got to see up close while working on a component that ships in Windows. They still work hard, don’t get me wrong, just that they no longer employ burly fellows to walk along the hall cracking whips every half hour.

The other reason is that since the Windows 8 cycle, the Windows team has a much more serious attitude towards supporting developers. They’ve even got a whole organization dedicated to it that works closely with the Developer Division and everything! And so when some acquaintances started chatting with me about some of the ideas the developer team in Windows had for future versions of Windows, I got interested. And then did an interview. And then accepted. So now here I am! Most of what I’m going to do isn’t public for the time being, so not a lot to say about that at the moment, but suffice it to say, I think it’s going to be interesting!

You should also follow me on Twitter here.

JsRT: Wrappers for C++

One unfortunate side-effect of the fact that the JsRT API is a flat Win32-style API is that it can be a little… painful to use in C++. Some of the conventions, particularly the use of error code returns everywhere and the use of return parameters, mean that you have to write a lot of boilerplate code. And marshalling values between C++ and JavaScript can be very cumbersome. So with that in mind, I’ve written some sample C++ wrappers that make the process of working with the JsRT APIs a lot nicer in C++.

You can get them from my GitHub account here. For more information, here’s part of the readme, which covers some of the highlights of the wrappers:

A Tour Through the Helpers

Most of the wrappers simply provide a nice object-oriented interface over the JsRT API. For example, creating a runtime and a context works like this:

jsrt::runtime runtime = jsrt::runtime::create();
jsrt::context context = runtime.create_context();

There are, however, a number of special features in the wrappers.

Context scopes

A context scope (jsrt::context::scope) automatically sets the current context when it comes into scope and clears the current context when it goes out of scope. This greatly simplifies working with scopes. For example:

jsrt::runtime runtime = jsrt::runtime::create();
jsrt::context context = runtime.create_context();
{
    jsrt::context::scope scope(context);
    ... work with context ...
}
// Context has automatically been set back to original value.
runtime.dispose();

Pinned references

To keep a reference to a Chakra object alive, it must be visible to the Chakra garbage collector (i.e. on the stack or stored in the garbage collected heap) or it has to be pinned using JsAddRef. The pinned<T> class functions as a smart-pointer template that keeps a reference alive using JsAddRef and JsRelease automatically. For example:

{
    jsrt::pinned<jsrt::object> pinned_obj = jsrt::object::create();
    // Object reference does not have to stay on the stack or in the GC heap
}
// Reference can now be garbage collected

Value translation

Value getter and setter functions can be strongly-typed, allowing automatic marshalling of JavaScript values to C++ values and vice versa. For example:

jsrt::object obj = jsrt::object::create();
obj.set_property(jsrt::property_id::create(L"boolProperty"), true);
bool bool_value = obj.get_property<bool>(jsrt::property_id::create(L"boolProperty"));
obj.set_property(jsrt::property_id::create(L"stringProperty"), L"foo");

The wrappers marshal types in the following way:

  • number values are marshalled to/from double

  • string values are marshalled to/from std::wstring

  • Boolean values are marshalled to/from bool

Strongly-typed arrays

Similarly, arrays can be strongly typed and accessed using numeric indexes:

jsrt::array<double> darray = jsrt::array<double>::create(1);
darray[0] = 10;
darray[1] = 20;

Functions

Perhaps the most useful aspect of the wrappers is the way that JavaScript functions can be wrapped and their arguments and return values marshalled to C++ types, and vice versa. For example:

auto f = (jsrt::function<double, double, double>)jsrt::context::evaluate(
    L"function f(a, b) { return a + b; }; f;");
double a = f(jsrt::context::undefined(), 1, 2);

Native functions can also be wrapped:

double add(const jsrt::call_info &info, double a, double b) { return a + b; }
auto nf = jsrt::function<double, double, double>::create(add);
jsrt::context::global().set_property(jsrt::property_id::create(L"add"), nf);
jsrt::context::run(L"add(1, 2)");

When calling a function wrapper, the first argument to the function call is the this value for the call. Function wrappers can also be bound to a particular value of this. For example:

auto bf = jsrt::bound_function<jsrt::value, double, double, double>::create(
    jsrt::context::undefined(),
    (jsrt::function<double, double, double>)jsrt::context::evaluate(
        L"function f(a, b) { return a + b; }; f;"));
double ba = bf(1, 2);

You should also follow me on Twitter here.

Using Jake with TypeScript

I’ve been playing around a lot lately with TypeScript and node.js, and have been using Jake as my make system. Oddly, I haven’t been able to find any npms that simplify Jake/TypeScript integration (maybe I’m not looking hard enough?), so I wrote a little one of my own, jake-typescript. It exposes two methods:

  • batchFiles takes a list of files and produces a Jake task that compiles each of the files separately (i.e. the default tsc behavior).
  • singleFile takes a list of files and produces a Jake file task that compiles all of the files into a single .js file (i.e. the –out behavior).

There’s also a structure to specify command-line arguments for the compile. It’s my first npm and I’m still getting up to speed on all the various conventions, definitely looking for feedback…

You should also follow me on Twitter here.

The Avalanche Theory of Programming Language Evolution

One of the other things that listening to Bjarne Stroustrup reminded me of is an idea that I’ve had kicking around in my head for quite some time about the way programming languages evolve. I can’t exactly quote Bjarne here, but I think he said something to the effect of, “One of the reasons I give this talk is because people still think of C++ as it existed back in 1986, not as it exists today.” Which reminded me of something I read a long time ago about avalanches and the way that they work.

The interesting thing about avalanches is that, should you ever be unlucky enough to be caught in one, they go through two distinct phases. In the first phase, while the avalanche is making its way down the slope, it behaves almost as if it were a liquid. Everything is moving, and it’s theoretically possible (if you don’t get bashed in the head by a rock or tree or something, and you can tell which way is “up”) to “swim” to the top of the avalanche and sort of float/surf on top of it. And it turns out that it’s pretty important to do this if you happen to be caught in an avalanche. Because once the avalanche hits the bottom of the slope and stops moving, it suddenly transitions to another state: a solid. At this point, if you aren’t at least partially free of the avalanche (i.e. on top of it), you’re totally screwed because you are now basically encased in a block of concrete. Not only will you be totally unable to move, but you’re quickly going to suffocate because there’s no way to get fresh air. Unless someone is close by with a locator and a shovel, you’re basically dead.

Programming languages, as far as I can tell, seem to evolve in a way similar to avalanches. Once a programming language “breaks free”–and, to be honest, most never do–it starts accelerating down the slope. At this point, the design of the language is very fluid as new developers pour into the space. And by “design of the language,” I don’t just mean the language spec but everything that makes up the practical design of a language, from the minute (coding standards, delimiter formatting, identifier casing) to the large-scale (best practices for APIs, componentization, etc.). Everything is shifting and changing and growing and it’s very exciting!

And then… boom! It all stops and freezes into place, just like that. Usually, this happens around maybe the second or third major release of the language, long enough for the initial kinks to get worked out and for things to cool and take shape. And the interesting thing about it is that it’s not so much that the language designers are done, as it is that from that point on the language design effectively become fixed in the minds of the public. There are number of reasons, but I think that it has to do with reaching a critical mass of things like books, blog posts, articles, samples, course materials, etc. and a critical mass of developers who were trained in a particular version of a language. Once enough of that stuff is out there and enough people have adopted a particular usage of the language, they effectively become the “standard” for the language.

And from that point on, I think, the language designer is essentially like those poor souls trapped at the bottom of the avalanche–they’re still alive and kicking (well, for a while, at least) but they increasingly find that they can’t really move anything. They can pump out new features and new ideas, but smaller and smaller slices of the developers in those languages are even aware of those features, much less willing to retrain (and rethink) to take advantage of them.

It makes me seriously wonder about all the work we did in VS 2008 to add things like LINQ to VB and C#. I suspect the .NET development avalanche largely came to rest around the time of VS 2005 (or maybe even VS 2003), and while I think a lot of high-end developers like and use LINQ, I don’t know that it ever penetrated the huge unwashed masses of .NET developers. I’m not saying we shouldn’t have done it–I think at very least, it helps push the language design conversation forward and influences the next generation of programming languages as they start their descent down the slopes–but just that I wonder.

And I also have to say that I’m very much a participant in this process. I originally learned C++ all by myself back in the ancient year of 1989 by buying myself Bjarne’s first C++ book and a copy of Zortech C++ for my IBM 386 PC. For a long, long time, C++ was effectively the C++ that I learned way back when C++ compilers were just front-ends for C compilers. Even with lots of exposure to more modern programming concepts while working on VB and so on, it’s taken me a long time to break the old habits and stretch within the C++ language. And, I have to admit, it’s really not a bad language once I did it. But I suspect I’m part of a somewhat small portion of the C++ market.

Anyway, all it really means is that I expectantly scan the metaphorical slopes, waiting for the large “BANG!” that will herald the descent of a new avalanche and the chance to try and surf it once again…

You should also follow me on Twitter here.

The Secret to Understanding C++ (and why we teach C++ the wrong way)

A little over a year ago, I asked “How on earth do normal people learn C++?” which reflected some of my frustration as I re-engaged with the language and tried to make sense of what “modern” C++ had become. Over time, I’ve found that as I’ve become more and more familiar with the language again, things have begun to make more sense. Then a couple of days ago I went to a talk by Bjarne Stroustrup (whose name, apparently, I have no hope of ever pronouncing correctly) and the secret of understanding C++ suddenly crystallized in my mind.

I have to say, I found the talk quite interesting which was a huge accomplishment because: a) I usually don’t like sitting listening to talks under the best of circumstances, and b) he was basically covering the “whys and wherefores” of C++, which is something I’m already fairly familiar with. However, in listening to the designer of C++ talk about his language, I was struck by a realization: the secret to understanding C++ is to think like the machine, not like the programmer.

You see, the fundamental mistake that most teachers make when teaching C++ to human beings is to teach C++ like they teach other programming languages. But with the exception of C, most modern programming languages are designed around hiding as many details of how things actually happen on the machine as possible. They’re designed to allow humans to explain to the computer what they want to do in a way that’s as close to the way humans think (or, at least, how engineers think) as possible. And since C++ superficially looks like some of those languages, teachers just apply the same methodology. Hence, when you learn about classes, teachers tend to spend most of their time on the conceptual level, talking about inheritance and encapsulation and such.

But the way Bjarne talks about C++, it’s clear that everything that C++ does is designed while thinking hard about the question how will this translate to the machine level? This may be a completely obvious point for a language whose main purpose in life is a systems programming language, but I don’t think I’d ever really groked how deeply that idea is baked in to C++. And once I really looked at things that way, things make a lot more sense. Instead of teaching classes at just the conceptual level, you really need to teach classes at the implementation level for C++ to make sense.

For example, I’ve never been in a programming class that discussed how C++ classes are actually implemented at runtime using Vtables and such. Instead, I had to learn all that on my own by implementing a programming language on the Common Language Runtime. The CLR hides a lot of the nitty-gritty of implementing inheritance from the C# and VB programmer, but the language implementer has to understand it at a fairly deep level to make sure they handle cross-language interop correctly. As such, I find myself continually falling back on my CLR experience when looking at C++ features and thinking, “How is this supposed to work?” I can’t imagine how people who haven’t had to confront these kinds of implementation-level details figure it out.

It makes me wonder if a proper C++ programming course would actually work in the opposite direction of how most classes (that I’ve seen) do it. Instead of starting at the conceptual level, start at the machine level. Here is a machine: CPU, registers, memory. Here’s how the basic C++ expression map to them. Here’s how basic C++ structures map to them. Here’s how you use those to build C++ classes and inheritance. And so on. By the time you got to move semantics vs. copy semantics, people might actually understand what you’re talking about.

You should also follow me on Twitter here.