External Requests Versus Internal Requirements

2 Replies

A commenter observed that a side-effect of my blog reset is that now any answers on Stack Overflow that pointed to my blog are broken. This is an unfortunate situation that I didn’t consider when I decided to reset my blog, and it did give me some pangs of regret when I looked at the list of answers that referred to me. It also got me thinking about the inevitable conflict in life between external requests (i.e. things other people want from me) and internal requirements (i.e. things that I want for myself).

From the Internet’s perspective, it would be happy if every piece of information that ever appears on the web would: a) stay on the web forever, and b) stay in exactly the same place forever. This makes total sense from the collective perspective—I’ve been amazed by the number of times that I’ve gone to research some obscure thing (such as how to unbind a DOS executable from a DOS 4/GW extender… don’t ask me why) and found some answer from back in the dark ages (like, 1996). Information disappearing, from a public utility standpoint, is a Very Bad Thing because you never know how useful that information might be some day. In college I was always amazed at the variety of interesting historical information that could be deduced from everyday things like private letters, diaries, commercial correspondence, etc. The true enemy of the scholar indeed is people who throw away… well, anything, really.

But from my perspective, it’s a lot of dead weight. I mean, a lot of the stuff I dumped off my blog was written five or six years ago. Some of it was wrong and a lot of it was irrelevant when looked at through the lens of later events. Some of it reminded me of things that I’d rather forget all about. And some of it was, frankly, embarrassing. One could say, “Well, then, just don’t look at it!” And, most of the time, I didn’t. But, you know, it was still there, taunting me from the archive list on the right hand side of the blog.

Some changes in my personal life recently motivated me to go through a bunch of boxes that had been sitting in storage for years. They were mostly full of stuff from my high school and college years—old papers, letters, other random stuff. I’d been holding on to a lot of it because, well, it was my stuff. But in looking through it, I realized how much of a burden most it had become. Like the blog, I didn’t look at the boxes very often, but there they always were, taking up space, having to be moved around, making me keep track of them. So I decided to go through and throw away anything that didn’t have a strong, tangible, positive, personal meaning to me. I’d say I threw away about 90% of what I had stored in those boxes, tons of stuff I no longer even remembered anything about. And the wonderful thing? Once I was over the initial trauma, I felt a lot better, freer, and lighter. It’s amazing what wonders getting rid of old stuff can do for you, even if it does make my theoretical future biographer’s job harder.

So my apologies to the Internet: I realize that losing my miniscule contributions to global knowledge might make life a little more difficult, and I’m sorry about that. But I have to say: I feel a whole lot better letting go of that stuff. I’m sure it’s going to cost me some (or even a lot) of visitors, but it seems like a small price to pay. At least, for me.

You should also follow me on Twitter here.

Landed…

3 Replies

As I warned last week, I’ve moved my blog entirely over to WordPress now. Please let me know if you encounter any difficulties.

Hitting the Big Red Switch

7 Replies

As you’ve probably noticed (and I’ve commented on from time to time) my blogging output over the past several years has fallen off precipitously, to the point where there has basically been nothing written on the blog at all in the past six months. I spent a while debating whether it was just time to go ahead and shut Panopticon Central down but in the end I decided that it’s likely that I’m going to have a few more things to say as I start to really settle in to the SQL Server team.

One thing I did decide I wanted to do was get out of the blog hosting business entirely. Those of you who have been exceptionally loyal readers may remember that this blog started off with a hand-built engine running on a server in my basement and has steadily moved in the direction of requiring less and less of my time to administer it and keep it running: first to using standard software (.TEXT and then SubText), then to a hosting provider (Server Intellect). To be honest, though, even administering a standard blog package on a hosting provider is more than I want to be doing. So after a lot of thinking I’ve decided to move wholesale to WordPress in the hopes that I can forever be done with such fun and exciting tasks as upgrading my blogging engine database.

That leaves the question, though, as to what to do with the large amount of previous stuff on the blog. Reading through the archives, most posts seemed to fall into the following categories, in order of frequency:

Inside baseball discussions of the design/shipping process for previous versions of Visual Basic.
Highly technical discussions of design decisions for features shipped in previous versions of Visual Basic.
Random moment-in-time personal commentary.
Highly confused (and often very inaccurate) discussions of now-cancelled or re-directed projects.

After some serious consideration, I’ve decided that rather than trying to bring all the past posts and comments over to WordPress, I’m going to wipe the slate clean and start over with a fresh, clean blog. It was surprisingly difficult to decide to toss out eight years of accumulated blogging detritus, but as I started to look through some of my old posts, I was convinced it was time. There just wasn’t much there that seemed particularly important to save for posterity. OK, maybe posts that fall into category #2 above might provide some value for people, but even a lot of that information is quietly starting to rot or become irrelevant. And, to be honest, I’d rather just expunge some of the things I talked about regarding future plans for Visual Basic (most of which haven’t turned out to be true) or Oslo/”M” (ditto), to the degree that’s possible in today’s Internet.

If you’re seeing this in a RSS reader, then you’ve probably got a narrow window of opportunity to grab what you want off the old blog, as the first step of the move is going to be redirecting my RSS feed to my WordPress blog but not moving anything else. After that’s been given a little while to settle, I’ll move the entire domain over and the old blog will be decommissioned. I will dump all of the old blog into a database, so if you find yourself missing some pearl of my wisdom, let me know and I can probably dig it up for you. There might also be some select gems that I might republish, but we’ll just have to see.

So thanks for still being a reader, and look for newer and fresher content soon! And you should also follow me on Twitter here.

[Ed note—in case you’re too young to remember it, the “big red switch” was the power switch on the back of the original IBM PC. With only floppy drives, turning off the power meant resetting the machine entirely and starting fresh.]

Murphy’s Computer Law

8 Replies

A long time ago, my family took a trip to Expo `86 in Vancouver, with stop offs in San Francisco and Los Angeles. In LA, we went on the Universal studio tour, something which I basically have no memory of. I did get a memento, though-a poster entitled “Murphy’s Computer Law” with a bunch of humorous computing “laws” on it. This poster went up in my room, accompanied me to college and has been in most of my offices at Microsoft. However, a few years ago, a corner ripped off in a move. Then while it was sitting around waiting to be repaired, it got a bit stained. And then I realized just how dated and ratty the thing looked. So, I figured it’s time to retire it. However, I would like to hang on to the “laws” since some of them are are still quite pertinent, even if some are quite outdated. So here they are, on my “permanent record:”

Murphy’s Computer Law:

Murphy never would have used one.
Murphy would have loved them.

Bove’s Theorem: The remaining work to finish in order to reach your goal increases as the deadline approaches.

Brooks’ Law: Adding manpower to a late software project makes it later.

Canada Bill Jones’ Motto: It’s morally wrong to allow na‹ve end users to keep their money.

Cann’s Axiom: When all else fails, read the instructions.

Clarke’s Third Law: Any sufficiently advanced technology is indistinguishable from magic.

Deadline-Dan’s Demo Demonstration: The higher the “higher-ups” are who’ve come to see your demo, the lower your chances are of giving a successful one.

Deadline-Dan’s Demon: Every task takes twice as long as you think it will take. If you double the time you think it will take, it will actually take four times as long.

Demian’s Observation: There is always one item on the screen menu that is mislabeled and should read “ABANDON HOPE ALL YE WHO ENTER HERE.”

Dr. Caligari’s Come-back: A bad sector disk error occurs only after you’ve done several hours of work without performing a backup.

Estridge’s Law: No matter how large and standardized the marketplace is, IBM can redefine it. [ed, later “Microsoft”, now “Apple,” I guess]

Finagle’s Rules:

To study an application best, understand it thoroughly before you start.
Always keep a record of data. It indicates you’ve been working.
Always draw your curves, then plot the reading.
In case of doubt, make it sound convincing.
Program results should always be reproducible. They should all fail in the same way.
Do not believe in miracles. Rely on them.

Franklin’s Rule: Blessed is the end user who expects nothing, for he/she will not be disappointed.

Gilb’s Laws of Unreliability:

At the source of every error which is blamed on the computer you will find at least two human errors, including the error of blaming it on the computer.
Any system which depends on human reliability is unreliable.
Undetectable errors are infinite in variety, in contrast to detectable errors, which by definition are limited.
Investment in reliability will increase until it exceeds the probable cost of errors, or until someone insists on getting some useful work done.

Gummidge’s Law: The amount of expertise varies in inverse proportion to the number of statements understood by the general public.

Harp’s Corollary to Estridge’s Law: Your “IBM PC-compatible” computer grows more incompatible with every passing moment.

Heller’s Law: The first myth of management is that it exists.

Hinds’ Law of Computer Programming:

Any given program, when running, is obsolete.
If a program is useful, it will have to changed.
If a program is useless, it will have to be documented.
Any given program will expand to fill all available memory.
The value of a program is proportional to the weight of its output.
Program complexity grows until it exceeds the capability of the programmer who must maintain it.
Make it possible for programmers to write programs in English, and you will find that programmers cannot write English.

Hoare’s Law of Large Programs: Inside every large program is a small program struggling to get out.

The Last One’s Law of Program Generators: A program generator creates programs that are more “buggy” than the program generator.

Meskimen’s Law: There’s never time to do it right, but always time to do it over.

Murphy’s Fourth Law: If there is a possibility of several things going wrong, the one that will cause the most damage with be the one to go wrong.

Murphy’s Law of Thermodynamics: Things get worse under pressure.

Ninety-Ninety Rule of Project Schedules: The first ninety percent of the task takes ninety percent of the time, and the last ten percent takes the other ninety percent. [ed: words to live by]

Nixon’s Theorem: The man who can smile when things go wrong has thought of someone he can blame it on.

Nolan’s Placebo: An ounce of image is worth a pound of performance.

Osborn’s Law: Variables won’t, constants aren’t.

O’Toole’s Commentary on Murphy’s Law: Murphy was an optimist.

Peer’s Law: The solution to a problem changes the problem.

Rhode’s’ Corollary to Hoare’s Law: Inside every complex and unworkable program is a useful routine struggling to be free.

Robert E. Lee’s Truce: Judgment comes from experience; experience comes from poor judgment.

Sattinger’s Law: It works better if you plug it in.

Shaw’s Principle: Build a system that even a fool can use, and only a fool will want to use it. [ed: also known as “Bob’s Law”]

SNAFU Equations:

Given an problem containing N equations, there will be N+1 unknowns.
An object or bit or information most needed will be least available.
Any device requiring service or adjustment will be least accessible.
Interchangeable devices won’t.
In any human endeavor, once you have exhausted all possibilities and fail, there will be one solution, simple and obvious, highly visible to everyone else.
Badness comes in waves.

Thoreau’s Theories of Adaptation:

After months of training and you finally understand all of a program’s commands, a revised version of the program arrives with an all-new command structure. [ed: also known the “Office Principle”]
After designing a useful routine that gets around a familiar “bug” in the system, the system is revised, the “bug” is taken away, and you’re left with a useless routine.
Efforts in improving a program’s “user friendliness” invariably lead to work in improving user’s “computer literacy.”
That’s not a “bug”, that’s a feature!

Weinberg’s Corollary: An expert is a person who avoids the small errors while sweeping on to the grand fallacy.

Weinberg’s Law: If builders built buildings the way programmers write programs, then the first woodpecker that came along would destroy civilization.

Zymurgy’s First Law of Evolving System Dynamics: Once you open a can of worms, the only way to recan them is to use a larger can.

Wood’s Axiom: As soon as a still-to-be-finished computer task becomes a life-or-death situation, the power fails.

The Five Levels of Incompetence

2 Replies

In my “Learning and Teaching” post last week, I talked about the different stages of learning, from “unconscious incompetence” up to “unconscious competence.” It occurred to me today, though, that there really are different levels within those levels and, in particular, there are some very distinct levels of incompetence that I’ve encountered in my nearly (yikes!) two decades of working in the industry. The reason why the levels of incompetence are somewhat more important than the various levels of competence, it seems to me, is that incompetent people are often a very real threat to the stability of teams that they work in, while competent people usually aren’t.

The five levels of incompetence are, in increasing order of danger:

Level 1: The N00b.

There’s not much to say about the n00b since, let’s face it, we’ve all been there. Hiring n00bs is unavoidable in most situations. Being a n00b is a basic fact of life.

Danger: Low, assuming that they are properly sandboxed or are experienced enough to sandbox themselves. (Otherwise, they’re likely to hit “launch” instead of “lunch” and then you’re really in trouble.)

Level 2: Out of their depth.

Typically, this is someone who’s not really incompetent in general but who’s just been pushed up to a level of responsibility beyond their capabilities (a.k.a. a victim of the Peter Principle.). I’ve seen this happen most often in situations where a senior, experienced person leaves the team and the leadership decides that they have to put someone equally senior in their place regardless of whether that person can, you know, actually do the job. So they pluck someone who’s senior but not as experienced and plops them into the departing person’s chair.

Danger: Moderate. Because the person isn’t completely incompetent, they tend to be able to give the appearance of competence and avoid leading the team totally off the rails but usually end up leading the team in circles. So the team doesn’t make any forward progress and people eventually wise up and leave.

Level 3: Dumb and Dumber.

Now we start getting into the fun levels. This person is just plain incompetent, someone placed into a totally inappropriate position for them (which, for the most part, is going to be any position). I’ve actually encountered very few instances of this in my career, and it usually happens when someone transfers between two wildly different kinds of jobs. That tends to mask, for a little while anyway, their completely lack of ability to actually do anything under the guise of just being a n00b.

Danger: High to moderate. It really depends on how fast everyone figures out just how incompetent the person is-usually, truly incompetent people get shunted aside as soon as everyone figures out what’s going on. If that takes too long, competent people tend to get pissed off and leave.

Level 4: Bozo.

The difference between a Bozo and a Dumb and Dumber is that a Bozo is Dumb and Dumber who thinks he is competent. Bozo’s tend to believe that they are as good or better than everyone else, deserve special treatment, and that their genius is being under-rewarded. And they ignore the fact that they have absolutely no idea what they are doing.

My best Bozo story is a Program Manager that I worked with a long time ago. I implemented a feature he specified. He entered a bug saying that the feature didn’t work correctly. I resolved the bug “by design” after I verified that the feature worked exactly the way he specified it. He then came to my office and started to argue with me that I shouldn’t have resolved the bug “by design,” because the feature didn’t work correctly. Finally, I pulled out a copy of his specification and pointed at the paragraph that said exactly how the feature should work. He then got totally exasperated with me and started ranting that I was supposed to implement the feature “the way he wanted the feature to work, not the way he specified it!”

Danger: High. Bozos are always on the lookout to get ahead (in line with their great abilities), so they often manage to worm their way in to management positions. They then tend to lead the team the way Mr. Toad drives motorcars: careening all over the road until they finally end up in the ditch. Bozo’s are adept at bringing down even the most experienced team in a surprisingly short amount of time.

Level 5: Evil Genius.

I was debating whether this is even a level of incompetence at all, because in many ways Evil Geniuses are not incompetent people. Quite the contrary, they are often quite adept at many things, including manipulation, spin, intimidation, self-aggrandizement, and sucking up. But I think in a deep sense, Evil Geniuses are just a more highly evolved form of Bozos because the end result tends to be the same: the team blows up in a very spectacular way. However, while a Bozo usually does this in a totally oblivious way (“What happened?”), it’s often all part of an Evil Genius’s plan to use the force of the explosion to propel them ever higher. These are the kind of guys who end up running major corporations and then running them totally into the ground. And then jumping ship to run an even bigger corporation. But, at the core, I think that Evil Geniuses act this way because they couldn’t actually figure out h
ow to do things in an above-board manner. Thankfully, I’ve met very few Evil Geniuses in my day. And those I have met, I’ve been able to largely avoid.

T-SQL Tuesday #8: Learning and Teaching

3 Replies

Since I’m joining the T-SQL community, I thought I’d try my hand at a “T-SQL Tuesday” that I could actually have an opinion about. This week’s question (hosted by Robert Davis, a.k.a. @SQLSoldier on Twitter) is “How do you learn? How do you teach? What are you learning or teaching?” and is very relevant for me because, of course, I just joined the T-SQL team a short while ago and am doing a whole lot of learning at the moment.

How I learn

I was going to say “by doing,” but I don’t think that’s accurate enough because there are lots of kinds of “doing.” I’m reminded of something they said when I was learning to ballroom dance for my wedding reception. They said that when learning anything new, people tend to go through four distinct stages: “unconscious incompetence” (i.e. you don’t know how bad you are), “conscious incompetence” (i.e. you know exactly how bad you are), “conscious competence” (i.e. you’re good but you have to pay attention), and “unconscious competence” (i.e. you’re good and it seems effortless). So when I’m starting something new, I’m doing a lot things but most of what I’m doing is learning just how little I actually know. That’s helpful and necessary, but it’s not exactly what I call “real” learning. The real learning seems to come between the second and third stages-when I’ve discovered just how bad I am and am now working on figuring out how to be less bad. When I get to the fourth stage, the learning starts to taper down and that’s when I really get to enjoy the state of knowing (which I think is also called the state of “flow”) and I get to have a lot of fun.

The interesting implication of this is that when I’m entering a new area, my first attempts are necessarily going to not be that great because I don’t know what I don’t know yet. So the initial doing isn’t really very helpful in learning the area, nor is it likely to look much like what I’m going to end up with if I keep on learning. But it’s only when I’ve got something and I know, at least at some level, how bad it is that I can start learning the area. Ironically, when the true learning starts it mostly looks like anal-retentiveness and neat-freakishness-going over and over and over something I’ve done, trying to make it better and suck less. In other words, to start really learning something I have to take something I’ve already done and go back and start pulling at the loose threads, seeing how it unravels and then figuring out how to reweave it properly. That’s when I really get to figure out how the things are supposed to work.

(I’ll note here that this is the number one mistake that I’ve seen most new programmers make. They’re like the verse from The Rubaiyat of Omar Khayyam:

The Moving Finger writes; and, having writ,
Moves on: nor all thy Piety nor Wit
Shall lure it back to cancel half a Line,
Nor all thy Tears wash out a Word of it.

They write their code once and then abandon it, never returning, always moving on to the next thing. Thus, they never actually get the chance to learn how to do things properly and always stay in that state of unconscious incompetence.)

Ironically, the situation I’m stepping into in SQL Server is perfect for “real” learning because I get to largely shortcut through the first stage of unconscious incompetence. That is to say, there’s already this large, mature artifact (i.e. the SQL Server codebase), so I don’t need to go through the trouble of creating something imperfect-someone’s already done that for me. I can spend just a few short weeks realizing just how little I actually know about anything and then jump straight to pulling threads and seeing what starts coming apart. Metaphorically, of course. I’m not gunning to have SQL Server fall apart on me or anything.

I actually think this can be more fun than starting something brand new and blazing the path, which isn’t the way the world sees it, oftentimes.

How I teach

I think I’m actually going to touch on this in more detail soon, but the short answer is: “by writing.” I’m pretty consciously incompetent when it comes to standing up in front of people and teaching them things, but I’ve been practicing writing for a whole lot longer and am better at it than speaking. And writing is just another form of what I was talking about in the previous section-first, I sit down and try to write down an explanation of whatever it is I’m trying to say. Then I realize how pathetically inadequate it is (most of the time) at saying what I want it to say. Or how little I understand what I’m trying to talk about. So I start pulling on the threads again and seeing what I can unravel and rework. And I find myself learning more not only about the process and practice of writing, but also more about whatever it is I’m trying to explain.

I think writing can be a wonderful way to teach people things, but I think it only really works-even technical writing!-if you follow the dictum of “writing what you know (and love).” In the end, I guess any teaching medium works if the teacher is interested enough in the subject, knowledgeable enough about it, and has a real passion for teaching (as opposed to a passion for having people listen to them, which is something entirely different).

Well, that’s about it. Hope this was interesting!

There are only three types of programmers in the world…

10 Replies

..and they are:

Programmers who want to write an operating system
Programmers who want to write a compiler
Programmers who want to write a database

It’s not that every programmer ever actually works on one of these, just that every programmer seems to dream of doing one of these things. It’s the primary reason why things like Linux exist. Yes, open source, blah, blah, blah, OS choices, blah, blah, blah, evil Microsoft, blah, blah, blah. But I would bet my bottom dollar that 9 out of 10 of the people donating their valuable time to the Linux project do so not because they want an alternative to Windows but because they always dreamed of being OS hackers. It’s also why there are so many damn programming languages out there, all the people who sit around dreaming of being, I don’t know, James Gosling or something.

(I think with the advent of the Internet, it’s likely that there’s now a fourth kind of programmer who wants to write websites, but I’m not totally sure about that yet.)

The interesting thing about these categories is that the Venn diagram tends, in my experience, to be pretty distinct-most “data” guys aren’t also “language” guys, and most “language” guys aren’t also “OS” guys, and so on. My theory is that it’s like the parable of the blind men and the elephant: although we all grapple with basically the same set of problems, each kind of programmer grapples with a different aspect of it.

I say all this because although I started out working in databases, it’s clear to me that I’ve always been a “language” guy. In college, I did so-so in the OS course and never touched a database course (I’m not even sure they were offered), but my compiler course netted me a special letter of commendation from the professor (the only one I ever got). Anyway, now I’m back in the “data” world as an even more confirmed “language” guy and the most interesting thing is how many of the problems are the same, but the way they’re conceptualized, handled, or even talked about, are different from what I’ve been used to working on programming languages. It’s kind of. refreshing to see things in a different light. More on that soon.

Relaunching on Twitter

“Fixing” T-SQL

9 Replies

In a comment to my previous post, Rich asked

Does this mean you’re the person to fix T-SQL programmability?

I honestly don’t know the answer to that question because, coming from the outside, I’m not sure about everything that’s wrong with T-SQL. I’d love to hear more from anyone who’s got an opinion (and any pointers to complaints around the web would be welcome as well). You can also feel free to use my contact form to talk to me directly. I can’t promise I can do anything at this point, but I’m looking to learn.

What I can do is offer two thoughts about why I chose to move over to the T-SQL team:

Even though I’ve been interfacing with SQL Server for nearly 20 years in one capacity or another, I only know a surface amount about it-basically, some standard SQL and that’s about it. A lot of the reason for that is because T-SQL always seemed, well, a little arcane from the outside. I sort of got the basics of querying, but beyond that I never felt I really had the time to figure things because they looked. complicated. Some of that, I’m sure, is just the usual “people look strange when you’re a stranger,” but I have always wondered what might be done to make SQL Server and T-SQL a little more approachable.
Along those lines, one recurring situation that I’ve found with SQL Server (both personally and observing others) is that the perceived impenetrability of T-SQL causes people to waste the wonderful opportunity to leverage the power of the server and instead use up precious time sucking data down to the client just to do a bit of processing that could easily have been done as a part of the query. The question comes to mind: what could we do to enable people to more easily capitalize on the server resources that are available to them, and how can T-SQL play a part in that?

So that’s what my hopes are, we’ll just have to see how it plays out.

Another transition…

6 Replies

After spending a year and a half working on “M”, I’ve decided to make another change in what I’m doing and and move over to the SQL Server Programmability team. That’s the team responsible for things like the T-SQL language and runtime in SQL Server. Working on “M” was a lot of fun and the team was great, but after spending a good, long while down in the bowels of a GLR parser, I decided that that was enough and that it was time to do something else. Working on SQL Server programmability is, in some ways, a combination of all my previous jobs-a bit of data from Access, a bit of runtime from OLE Automation, and a bit of programming language from Visual Basic and “M”. It’s also an interesting challenge-a product that’s both well established and confronting a lot of new challenges. I think it’s going to be quite a bit of fun!

It does mean saying goodbye to “M”, and that was sad (although, really, they’re still in the same division and not that far away), but that’s the way it goes. I’ll be looking forward to their next CTP, which is where people will see a lot of the hard work that’s been going on and the overall direction that the language is headed. There’s a lot of cool stuff coming, and I think people will find it very interesting!

Changing jobs also means that I’m back to drinking from the firehose, learning the ins and outs of the guts of the SQL Server engine, as well as T-SQL. Interesting stuff. Any good T-SQL/SQL Server blogs anyone can recommend?