Please Come Again

I’ve been playing Dwarf Fortress lately. It’s a very fun game. However, as I was doing something in game, it occurred to me that maybe I was doing something wrong…

Then again, maybe I’m doing it right…

Last time, we determined that destruction and finalization are two completely different things. One is a consensual agreement between an object and its owner to clean up after itself, while the other is an ultimatum in a world of instability and destruction.

Since we don’t have proper destructors in C#, how can we get destructor semantics?

Easy, just implement IDisposable!

interface IDisposable {
    public void Dispose();
}

This Dispose method is as close to a proper destructor (in the C++ sense) that we are going to get. Sort of. Let’s implement it, and see how it works.

class MyClass : IDisposable {
 
    ManagedResource managedRes;
    IntPtr unmanagedRes;
 
    public MyClass() {
        AcquireUnmanagedResource();
        managedRes = new ManagedResource();
    }
 
    private void AcquireUnmanagedResource() {
        unmanagedRes = //...
    }
 
    private void ReleaseUnmanagedResource() {
        //...
        unmanagedRes = IntPtr.Zero;
    }
 
    bool alreadyDisposed = false;
 
    public void Dispose() {
        Dispose(true);
    }
 
    private void Dispose(bool disposing) {
        if(alreadyDisposed) return;
 
        alreadyDisposed = true;
 
        if(disposing) {
            managedRes.Dispose();
            managedRes = null;
 
            GC.SuppressFinalization(this);
        }
 
        ReleaseUnmanagedResource();
    }
 
    ~MyClass() {
        Dispose(false);
    }
}

In our sample class here, we want to manage two resources. One is a managed resource, the other unmanaged.

Before we continue, I need to point something. Once you wrap an unmanaged resource in a class, that class is then a managed resource (because the class itself is managing the resource). This is an important distinction, as we will quickly find out.

In our constructor, we acquire both resources (following the RAII — resource acquisition is initialization — pattern), and we retain them until our object is destroyed. I added two helper methods to acquire and destroy the managed resource, just to make everything clear.

First off, you should notice that we have both IDisposable AND a Finalizer. But, beyond that, we have two Dispose methods! The heck?

It’s actually easier than it looks. The second dispose method (the one taking the boolean) is not part of IDisposable, and so it could have been named

private void Bagel(bool whatever)

for all we care. However, since it is actually the method doing the heavy work of the disposal, we name it Dispose for clarity.

You see, when an object is being Finalized, it’s in a very peculiar situation. Again, unlike deterministic Destruction, Finalization is non-deterministic. Another consequence of this is that its children may well have been finalized before it, and their references nulled. In the case of a circular reference (a child referring to their parent), this is always the case. So, the number one big rule of finalization is:

During finalization, you may not touch any managed resources.

In other words, by the time ~MyClass is invoked, it’s too late for managedRes. It could very well have already died, so there’s no point trying to destroy it any more. We still want to take care of our unmanaged resources, since the GC can’t handle those, but the managed resource is already gone. To indicate that we don’t want to touch any managed resources, we have ~MyClass call Dispose(false).

Conversely, when we’re being disposed normally (by someone calling MyClass.Dispose()), we DO want to dispose our managed resources, along with our unmanaged resources. That’s why Dispose() calls Dispose(true);

Sure enough, Dispose(bool) follows along with this pattern by only freeing managedRes if the parameter is true. The unmanaged resource is freed regardless.

Once we’ve been disposed, it suddenly becomes obvious that we don’t need to be finalized any more. After all, our resources are gone, what’s left to free? To that end, we invoke GC.SuppressFinalization(this), which does exactly what it says on the tin. Normally, if an object has a finalizer, the GC needs to do special book keeping to ensure that the finalizer runs (when/if it gets around to it). By suppressing the finalizer, we tell the GC to relax, and not worry about it.

Finally (see what I did there?), there’s one last rule regarding IDisposable. Dispose() should always be a valid call (i.e., not throw any exceptions), even if the object has already been disposed. That’s why we added the boolean and check, so that if we’ve already been disposed, we don’t try to dispose ourselves again.

Bonus Reading: As I was writing this series, I thought that I would also write an article about WHY we have finalization in the first place. However, it seems that Raymond Chen has beaten me to the punch, so I encourage you to read his take on the matter!

Raymond Chen’s blog yesterday reminded me of a topic I wanted to blog about. This is strange, since I work primarily in C#/.NET, while Raymond is an old school Win32 guy. Fortunately, it seems that this week is CLR week on the Old New Thing!

Anyway, now that he’s segueing into the world of managed code, I will begin by hearkening back to the glory days of C++. Remember back when you wrote a class, you had the constructor, and some fields, a few methods, maybe add some inheritance. Oh, and let’s not forget the destructor! (Oh, who am I kidding? Who could forget the cute ~ before the class name? :3)

The semantics of a destructor in C++ are fairly clear: It gets called exactly once in the object’s lifetime, just before it dies. The object is responsible to free any resources it holds, and ensure that any child objects are destructed as well (generally, just deleting any new’d objects).

Ah, yes, it was easy back then… Memory leaks aside. And writing exception safe code. Okay, it wasn’t easy at all. But, you could understand it.

Now a days, the world is a bit different. In C#, we still have those funny ~ characters, but it doesn’t mean what it used to. They aren’t destructors any more, no siree. Those are finalizers, and they have a whole new set of semantics.

Finalizers are still linked to object destruction, that much is true. However, there is no delete keyword in C#. We simply let the object go. Once it gets garbage collected, that’s when the finalizer is called. We don’t have any control over the garbage collector, which means that we don’t know when it will run, and when the object will be destroyed. In other words, object destruction is non-deterministic.

But, it will be collected eventually, right?

Raymond says in the linked article:

If the amount of RAM available to the runtime is greater than the amount of memory required by a program, then a memory manager which employs the null garbage collector (which never collects anything) is a valid memory manager.

Obviously, if memory is plentiful, why bother going through the trouble of recovering it all?

A correctly-written program cannot assume that finalizers will ever run at any point prior to program termination.

The only time the garbage collector will run is at the end of the program, as the runtime is shutting down.

But, he continues:

A correctly-written program cannot assume that finalizers will ever run.

Whaaaaa? Really?

As it turns out, yes.

There are two situations where this can happen:

1. If the process crashes (eg, a P/Invoke gone horribly wrong) or is killed in some fashion (an overeager user armed with Task Manager, for example), the CLR won’t have a chance to invoke the garbage collector.
2. If an object throws an exception in its finalizer, the garbage collector has no choice but to call it quits, and it takes the process down with it.

Strictly speaking, #2 is a special case of #1, since you’re effectively crashing the garbage collector. Either way, the net effect is that the process dies, and Windows has to take care of your mess.

So, what good is a finalizer, if it isn’t guaranteed to run at any point? To be honest, not much at all.

If you consider deterministic (C++) destruction as a polite request for the object to clean up, then non-deterministic finalization is putting the object in front of the firing squad, and asking “Any last words?”

Fortunately, we have yet a third option, which I will discuss next time.

I recently answered a question on StackOverflow about a strange error message in a class file. The class looked something like this:

class Foo {
    private int bar;
    private int baz;
 
    public Foo(int b1) {
        this.bar = b1;
    }
 
    public Foo(int b1, int b2) {
        this(b1);
 
        this.baz = b2;
    }
}

The error message this guy was getting was (paraphrasing from the question):

variable baz might already have been assigned

And, it was pointing at the constructor with two arguments.

After seeing that, I invoked my psychic debugging skills, and determined the problem.

The constructors in this class attempt to implement a common pattern, whereby each member only has one place where it’s initialized. The idea is to chain constructors together. However, he did it backwards.

See, the Java runtime has to ensure that after a constructor is complete, the object state is defined. Not necessarily consistent, but defined. Consider what happens when you run the single argument constructor in the first class. Foo.bar gets a value, but Foo.baz doesn’t!

To make up for this, the Java compiler inserts a hidden chunk of code to initialize any members that don’t get a value. Incidentally, this is the same code that runs when you don’t have a constructor at all.

So, the first constructor does, in fact, initialize both members, after a fashion. Now, let’s consider what happens when the second constructor runs.

First, it calls the first constructor. This initializes both members for us. Now, we then initialize Foo.baz.

Oops, it’s already been initialized! So, the compiler emits the warning.

Now, here’s how the class should have looked:

class Foo {
    private int bar;
    private int baz;
 
    public Foo(int b1) {
        this(b1, 0);
    }
 
    public Foo(int b1, int b2) {
        this.bar = b1;
        this.baz = b2;
    }
}

See the difference?

Right, all the initialization is done in the constructor with two arguments. The constructor with one argument simply forwards the argument and a default to the second one. The first constructor doesn’t get any code to initialize anything implicitly, since everything is explicitly done in the second constructor.

This is the proper way to do constructors in Java, C# and probably most other statically typed languages, so you will probably be seeing it a lot.

I recently came across a curious problem while writing a C# Winform application. I wanted to do something when the user pressed an arrow key. “Simple!” I thought. I knew that I couldn’t use the KeyPress event, since that only applies to keys that produce a visible character. So, I set the form’s “KeyPreview” property to true, and added a handler to the KeyDown event:

private void Form1_KeyDown(object sender, KeyEventArgs e) {
    switch (e.KeyCode) {
        case Keys.Left:
            //do something
            break;
        case Keys.Right:
            //do something
            break;
        case Keys.Up:
            //do something
            break;
        case Keys.Down:
            //do something
            break;
    }
}

However, I soon discovered that my handler wasn’t being called. I added a few other keys, such as “A”, and they worked. So, what’s so special about the arrow keys?

Clue #1: I should point out that my form had a number of controls on it, including custom controls that contained other controls.

Clue #2: The tab key also wasn’t being handled in the event.

If you haven’t guessed already, the answer is simple: By default, a form uses those keys to handle navigation amongst any child controls it has. So, it was processing the arrow keys long before I even got near them!

So, what was I to do? I wasn’t particularly worried about losing control navigation, since that wasn’t important to my application. But, I wasn’t sure how to recover the key presses. So, I started at the top, by overriding the WndProc method in the form:

protected override void WndProc(ref Message msg) {
    switch(msg.Msg) {
        case 0x100; //WM_KEYDOWN
            //do something
            break;
        default:
            base.WndProc(msg);
            break;
    }
}

Unfortunately, that didn’t work either. I saw the WM_KEYDOWN messages for the other keys, but not that one.

I continued overriding methods until I finally found one that got the appropriate WM_KEYDOWN message. That ended up being the ProcessDialogKey method. As expected, this is the method that handles the arrow keys and the tab key. My implementation of this method was really easy:

protected override bool ProcessDialogKey(Keys keyData) {
    return false;
}

Many of the message processing methods have the following contract: If you handle the message, return true. If you can’t handle it, but you have a base, return whatever the base returns. Otherwise, just return false to indicate you can’t handle it.

So, by us returning false here, we pretend that we asked our base class, and they also said no. Thus, the message gets handled as a regular KeyDown event instead.

Also as expected, this killed the default handling of the arrow keys and Tab. However, if you have a good reason to override these keys (say, in a game), then you also probably don’t want the form doing anything with them either.

If, say, you still wanted the default behaviour, but also wanted the events, you could do this instead:

protected override bool ProcessDialogKey(Keys keyData) {
    OnKeyDown(new KeyEventArgs(keyData));
    return base.ProcessDialogKey(keyData);
}

This will manually simulate the KeyDown handler, and also do the normal dialogue stuff. Just a warning: Don’t do this if the arrow keys also manipulate the control with focus, since that will move when the key is pressed!

Last time, we figure out how to create a pointer, but we left on the question of how to set a pointer’s value. As I mentioned, there are four ways to obtain a pointer:

1. Since it’s an integer, code the address in manually.
2. Get the address of a variable.
3. Allocate some memory.
4. Modify another pointer.

Let’s look at each of these in turn.

1. Since it’s an integer, code the address in manually.

Back in the old days, when C was brand new, this was a common technique. You knew that your program was the only program running, and you knew that your data was located at 0×400, so you can just stick 0×400 into the pointer, right?

Also, hardware in the pre-”plug ‘n’ play” days were located at fixed addresses in memory, which meant that you could communicate with (say) the mouse by reading from a fixed address. (Note: Memory-mapped IO is beyond the scope of this article, so look elsewhere for details)

These days, however, it’s not such a good idea. Heck, even under DOS, the potential for hardware address conflicts existed, necessitating the existence of those stupid DIP switches and config.sys. And, when multitasking systems became common, programs that existed on them needed to be aware that they weren’t necessarily located in the same spot they were used to. (Note: This is a gross and inaccurate simplification of how things work. Just wait until the virtual-memory post, where I’ll clear everything up — I promise!)

These days, fixed-address pointers are a thing of the past (on non-specialized devices); a footnote in the annals of IT history.

2. Get the address of a variable.

This is one of the more common uses. Every variable in your program has an address in memory, and so you can create a pointer that points to it:

int myVariable;
int * myPointer;
 
myPointer = &myVariable;

Now, myPointer contains the address of myVariable (it “points” to it). If you then modify the pointer, it will show up in the variable, and vice versa:

*myPointer = 123; //we'll talk about this next time
printf("%d", myVariable); //prints 123

(I know we haven’t discussed how to actually USE a pointer, just assume I’m sticking the 123 into the place where myPointer points)

You can also pass the address of a variable to a function (that expects it), and the function can modify it too:

void myFunction(int * intptr) {
   *intptr = 123;
}
 
int myVariable;
myFunction(&myVariable);
printf("%d", myVariable); //again, prints 123

As you can see, you don’t necessarily need a separate variable to store the pointer. In this case, since the thing you’re pointing at has a name (myVariable), you can ask for the address any time.

3. Allocate some memory.

This is trickier than working with local variables. Some times, at compile time, you don’t know how much memory you need. Maybe you’re working with a string that can be any length (a common source of bugs!). Perhaps you have a dynamically sized array. Either way, you can’t tell the compiler, and so it can’t manage the memory for you.

Fortunately, we have pointers.

#include 
 
int * myArray = (int *) malloc(sizeof(int) * 100);

Okay, this is a considerable step up from the previous example, so let’s look at it in detail.

int * myArray =

You’ve seen this before, this is a pointer variable.

(int *)

We will deal with this in a moment.

malloc

malloc (Memory ALLOCator) is the function that pulls a block of memory out of thin air for you. It’s not important (right now) to know where it gets it from, but only that it does.

malloc doesn’t know what you’re going to do with the memory, and so it can’t even begin to guess what type it should be. So, the return type is void* (that is, a pointer with no type). That’s why we need this bit:

(int *)

This is a cast. It tells the compiler “The return value of this function is an integer pointer, even if it says otherwise”. Since void* and int* are not the same thing, you need to cast before the compiler will allow you to make the assignment.

NOTE: Technically, in regular C, you don’t need the cast; the compiler allows implicit casting from void* to any other pointer type. However, in C++, the cast IS required. Frankly, it’s better to just always cast — it doesn’t cost anything, so go nuts.

sizeof(int) * 100

When we allocate memory, we need to tell malloc how much memory we want. In this case, we want enough room for 100 integers. Since we all know that an integer is four bytes, we can just write 400, right?

Well, no. Remember in part one, when I said that I was assuming 32-bit platforms, unless I say otherwise? I’m saying otherwise now. on a 64-bit platform, an integer can be up to eight bytes! So, if we allocate 400 bytes, we only have enough room for 50 ints. That’s bad, and is the root cause for a lot of bugs.

Fortunately, we can ask the compiler how big an integer is. That’s where the sizeof operator comes in (techincally, sizeof is an operator, not a function. It’s complicated, don’t ask.). Pass any type or variable, and it will be replaced with that type or variable’s size. The net result is that on 32-bit platforms, we’ll allocate 400 bytes, while on a 64-bit platform, we’ll allocate 800 bytes. Neato!

After all that, myArray now points to a block of memory 100 ints long, and we can treat it as anything else.

HOWEVER! There’s a lot more to using malloc and friends, which we’ll talk about in part five. Don’t even think about using malloc until you read it, because you’ll shoot yourself in the foot!

4. Modify another pointer.

In the last section, we learned how to allocate an array of arbitrary length. For completion’s sake, it’s also possible to allocate a static array of fixed length:

int myArray[100];

That’s a lot less complex, and does exactly what it seems to on both 32-bit and 64-bit platforms. The downside is that you need to know how big the array is at compile time. You can easily access the array using the [] notation:

printf("%d", myArray[0]);

What does this have to do with pointers? Well, I’ll let you in on a secret. In both of these example, myArray has the same type: int*. Even though you never have to go anywhere near the asterisk key with static arrays, it’s actually a pointer.

Which brings us to the topic of modifying pointers. Recall that accessing a pointer looks like this:

*myArray

The leading asterisk means “I am not talking about myArray, I’m talking about the thing myArray points at”. Now, consider this:

*(myArray + 0)

Wait, what did I just do? Technically, I didn’t do anything. Heck, the compiler will probably take the + 0 out by itself, and shake its head in bemusement. But, now, let’s change it once more:

*(myArray + 1)

What this say is “I am not talking about myArray, I’m talking about the thing after the thing myArray points at”.

Notice the similarity to arrays? Well, that’s because they’re the exact same thing. This:

myArray[1]

Translates exactly to:

*(myArray + 1)

That’s what I mean by “modifying” the pointer. We’re modifying where the pointer points at, to access an array.

Just for clarification, the + 1 doesn’t mean “one byte later”. It means “one int later”. Think about how the array access translates into the pointer addition, and it should make sense. They are 100% equivalent.

The upshot is that it’s 100% legal to go the other way, and index an explicit pointer:

int * myPointer = ...; //points to whatever
myPointer[10] = 123;

Craaazy, huh?

Last time, we talked about what a pointer is, and touched on what you can do with one. However, you may have been left wondering where exactly we get pointers. Do we go to the pointer store and buy them in packs of 50?

No, it’s much cheaper to buy them in bulk!

Just kidding, pointers are a programming construct, so there’s no purchase necessary. To create a pointer, you just use your favourite language’s syntax. For example, in C, a pointer is declared by adding an asterisk (*) to a type’s name:

int notAPointer;
int * aPointer;

In FreeBASIC, you add the keyword Ptr to the type:

dim notAPointer as integer
dim aPointer as integer ptr

One thing you may notice is that a pointer always has an associated type. Always. In these cases, the type was that of a standard integer.

That said, you can make a pointer for any type at all:

struct myStruct {
   int a, b, c, d, e;
};
 
int * intPointer;
float * floatPointer;
myStruct * myStructPointer;

Another property of pointers is that since they only point at data, and are themselves just integers, they are all the same size. Although the struct I just made a pointer to (myStruct) is 20 bytes (on 32-bit platforms), the pointer to it is just 4 bytes.

Earlier, when I said that a pointer always had a type, I very slightly fibbed. Sometimes, you have a pointer whose type is void. As you know, void means that there is no type. When used as a function’s return type, it means that the function doesn’t return anything. And, as you know, you cannot declare a void variable:

void voidVar; //illegal!

So, what’s a void pointer, then?

A void pointer is a pointer with an unknown type. This property means that you can store any pointer into it, and you can cast the pointer into any type (if you have experience with other object oriented languages such as Java or C♯, this is the same idea as an object variable).

However, just because the thing a void pointer points at doesn’t have a type doesn’t mean that the pointer itself doesn’t have a type. It is still a pointer, and is exactly the same size as every other pointer. So, don’t get them mixed up!

So, now you know a couple different ways you can create pointers. However, that still leaves the question about what the pointers point to. There’s actually 4 different ways you can get a pointer:

1. Since it’s an integer, code the address in manually.
2. Get the address of a variable.
3. Allocate some memory.
4. Modify another pointer.

However, we’re going to save this topic until next time.