Classes

One has no right to love or hate anything if one has not acquired a thorough knowledge of its nature. Great love springs from great knowledge of the beloved object, and if you know it but little you will be able to love it only a little or not at all.

Leonardo da Vinci

We’re eleven chapters in, and the interpreter sitting on your machine is nearly a complete scripting language. It could use a couple of built-in data structures like lists and maps, and it certainly needs a core library for file I/O, user input, etc. But the language itself is sufficient. We’ve got a little procedural language in the same vein as BASIC, Tcl, Scheme (minus macros), and early versions of Python and Lua.

If this were the ’80s, we’d stop here. But today, many popular languages support “object-oriented programming”. Adding that to Lox will give users a familiar set of tools for writing larger programs. Even if you personally don’t like OOP, this chapter and the next will help you understand how others design and build object systems.

12 . 1OOP and Classes

There are three broad paths to object-oriented programming: classes, prototypes, and multimethods. Classes came first and are the most popular style. With the rise of JavaScript (and to a lesser extent Lua), prototypes are more widely known than they used to be. I’ll talk more about those later. For Lox, we’re taking the, ahem, classic approach.

Since you’ve written about a thousand lines of Python code with me already, I’m assuming you don’t need a detailed introduction to object orientation. The main goal is to bundle data with the code that acts on it. Users do that by declaring a class that:

1. Exposes a constructor to create and initialize new instances of the class

2. Provides a way to store and access fields on instances

3. Defines a set of methods shared by all instances of the class that operate on each instances’ state.

That’s about as minimal as it gets. Most object-oriented languages, all the way back to Simula, also do inheritance to reuse behavior across classes. We’ll add that in the next chapter. Even kicking that out, we still have a lot to get through. This is a big chapter and everything doesn’t quite come together until we have all of the above pieces, so gather your stamina.

12 . 2Class Declarations

Like we do, we’re gonna start with syntax. A class statement introduces a new name, so it lives in the declaration grammar rule.

declaration    → classDecl
               | funDecl
               | varDecl
               | statement ;

classDecl      → "class" IDENTIFIER "{" function* "}" ;

The new classDecl rule relies on the function rule we defined earlier. To refresh your memory:

function       → IDENTIFIER "(" parameters? ")" block ;
parameters     → IDENTIFIER ( "," IDENTIFIER )* ;

In plain English, a class declaration is the class keyword, followed by the class’s name, then a curly-braced body. Inside that body is a list of method declarations. Unlike function declarations, methods don’t have a leading fun keyword. Each method is a name, parameter list, and body. Here’s an example:

class Breakfast {
  cook() {
    print "Eggs a-fryin'!";
  }

  serve(who) {
    print "Enjoy your breakfast, " + who + ".";
  }
}

Like most dynamically typed languages, fields are not explicitly listed in the class declaration. Instances are loose bags of data and you can freely add fields to them as you see fit using normal imperative code.

Over in our AST generator, the classDecl grammar rule gets its own statement node.

# lox/ast.py
@dataclass
class Class(Stmt):
    name: Token
    methods: list[Function]

It stores the class’s name and the methods inside its body. Methods are represented by the existing Stmt.Function class that we use for function declaration AST nodes. That gives us all the bits of state that we need for a method: name, parameter list, and body.

A class can appear anywhere a named declaration is allowed, triggered by the leading class keyword.

# lox/parser.py Parser.declaration() match/case
    ...
    case "CLASS":
        return self.class_declaration()
    ...

That calls out to:

# lox/parser.py method of Parser
def class_declaration() -> Class:
    self.consume("CLASS", "Expect 'class' keyword.")
    class_name = self.consume("IDENTIFIER", "Expect class name.")
    methods = []
    self.consume("LEFT_BRACE", "Expect '{' before class body.")
    while not self.check("RIGHT_BRACE"):
        methods.append(self.function("method"))
    self.consume("RIGHT_BRACE", "Expect '}' after class body.")
    return Class(class_name, methods=methods)

There’s more meat to this than most of the other parsing methods, but it roughly follows the grammar. We’ve already consumed the class keyword, so we look for the expected class name next, followed by the opening curly brace. Once inside the body, we keep parsing method declarations until we hit the closing brace. Each method declaration is parsed by a call to function(), which we defined back in the chapter where functions were introduced.

Like we do in any open-ended loop in the parser, we also check for hitting the end of the file. That won’t happen in correct code since a class should have a closing brace at the end, but it ensures the parser doesn’t get stuck in an infinite loop if the user has a syntax error and forgets to correctly end the class body.

We wrap the name and list of methods into a Class node and we’re done. Previously, we would jump straight into the interpreter, but now we need to plumb the node through the resolver first.

# lox/resolver.py after resolve_node()
@resolve_node.register
def _(stmt: Class, env: Env):
    env.declare(stmt.name)
    env.define(stmt.name)
    ...

We aren’t going to worry about resolving the methods themselves yet, so for now all we need to do is declare the class using its name. It’s not common to declare a class as a local variable, but Lox permits it, so we need to handle it correctly.

Now we interpret the class declaration.

# lox/interpreter.py add after exec()
@exec.register
def _(stmt: Class, env: Env) -> None:
    klass = LoxClass(stmt.name.lexeme)
    env[stmt.name.lexeme] = klass

And adjust the imports:

# lox/interpreter.py at top
from lox.runtime import LoxClass

type Value = ... | LoxClass  # Add the LoxClass type

This looks similar to how we execute function declarations. We declare the class’s name in the current environment. Then we turn the class syntax node into a LoxClass, the runtime representation of a class. We circle back and store the class object in the variable we previously declared. That two-stage variable binding process allows references to the class inside its own methods.

We will refine it throughout the chapter, but the first draft of LoxClass looks like this:

# lox/runtime.py
@dataclass(eq=False)
class LoxClass:
    name: str

    def __str__(self) -> str:
        return self.name

Literally a wrapper around a name. We don’t even store the methods yet. Not super useful, but it does have a __str__() method so we can write a trivial script and test that class objects are actually being parsed and executed.

class DevonshireCream {
  serveOn() {
    return "Scones";
  }
}

print DevonshireCream; // Prints "DevonshireCream".

12 . 3Creating Instances

We have classes, but they don’t do anything yet. Lox doesn’t have “static” methods that you can call right on the class itself, so without actual instances, classes are useless. Thus instances are the next step.

While some syntax and semantics are fairly standard across OOP languages, the way you create new instances isn’t. Ruby, following Smalltalk, creates instances by calling a method on the class object itself, a recursively graceful approach. Some, like C++ and Java, have a new keyword dedicated to birthing a new object. Python has you “call” the class itself like a function. (JavaScript, ever weird, sort of does both.)

I took a minimal approach with Lox. We already have class objects, and we already have function calls, so we’ll use call expressions on class objects to create new instances. It’s as if a class is a factory function that generates instances of itself. This feels elegant to me, and also spares us the need to introduce syntax like new. Therefore, we can skip past the front end straight into the runtime.

Right now, if you try this:

class Bagel {}
Bagel();

You get a runtime error. eval(Call) checks to see if the called object implements LoxCallable and reports an error since LoxClass doesn’t. Not yet, that is.

# lox/runtime.py
# Replace class LoxClass:
...
class LoxClass(LoxCallable):
    ...

Implementing that interface requires two fields.

# lox/runtime.py in LoxClass
    ...
    arity : int = 0

    def call(self,
             env: Env,
             arguments: list[Value]) -> LoxInstance:
        return LoxInstance(self)
    ...

The interesting one is call(). When you “call” a class, it instantiates a new LoxInstance for the called class and returns it. The arity attribute is how the interpreter validates that you passed the right number of arguments to a callable. For now, we’ll say you can’t pass any. When we get to user-defined constructors, we’ll revisit this.

That leads us to LoxInstance, the runtime representation of an instance of a Lox class. Again, our first implementation starts small.

# lox/runtime.py
@dataclass(eq=False)
class LoxInstance:
    klass: LoxClass

    def __str__(self) -> str:
        return f"{self.klass.name} instance"

Like LoxClass, it’s pretty bare bones, but we’re only getting started. If you want to give it a try, here’s a script to run:

class Bagel {}
var bagel = Bagel();
print bagel; // Prints "Bagel instance".

This program doesn’t do much, but it’s starting to do something.

12 . 4Properties on Instances

We have instances, so we should make them useful. We’re at a fork in the road. We could add behavior first—methods—or we could start with state—properties. We’re going to take the latter because, as we’ll see, the two get entangled in an interesting way and it will be easier to make sense of them if we get properties working first.

Lox follows Python and JavaScript in how it handles state. Every instance is an open collection of named values. Methods on the instance’s class can access and modify properties, but so can outside code. Properties are accessed using a . syntax.

someObject.someProperty

An expression followed by . and an identifier reads the property with that name from the object the expression evaluates to. That dot has the same precedence as the parentheses in a function call expression, so we slot it into the grammar by replacing the existing call rule with:

call           → primary ( "(" arguments? ")" | "." IDENTIFIER )* ;

After a primary expression, we allow a series of any mixture of parenthesized calls and dotted property accesses. “Property access” is a mouthful, so from here on out, we’ll call these “get expressions”.

12 . 4 . 1Get expressions

The syntax tree node is:

# lox/ast.py
@dataclass
class Get(Expr):
    object: Expr
    name: Token

Following the grammar, the new parsing code goes in our existing call() method.

# lox/parser.py inside Parser.call()
# Inside the while, before else
    ...
    elif self.match("DOT"):
        msg = "Expect property name after '.'."
        name = self.consume("IDENTIFIER", msg)
        expr = Get(expr, name)
    ...

The outer while loop there corresponds to the * in the grammar rule. We zip along the tokens building up a chain of calls and gets as we find parentheses and dots, like so:

Instances of the new Get node do not need any special treatment from the resolver.

OK, not much to that. Since properties are looked up dynamically, they don’t get resolved. During resolution, we recurse only into the expression to the left of the dot. The actual property access happens in the interpreter.

# lox/interpreter.py after eval()
@eval.register
def _(expr: Get, env: Env) -> Value:
    obj = eval(expr.object, env)
    if isinstance(obj, LoxInstance):
        return obj.get(expr.name)
    msg = "Only instances have properties."
    raise LoxRuntimeError(msg, expr.name)

First, we evaluate the expression whose property is being accessed. In Lox, only instances of classes have properties. If the object is some other type like a number, invoking a getter on it is a runtime error.

If the object is a LoxInstance, then we ask it to look up the property. It must be time to give LoxInstance some actual state. A dict will do fine.

We need to update the imports and definitions in the interpreter module:

# lox/interpreter.py at top
from lox.runtime import  LoxInstance

type Value = ... | LoxInstance  # Add LoxInstance to Value type

And tweak the LoxInstance class

# lox/runtime.py LoxInstance add attribute
    ...
    fields: dict[str, Value] = field(default_factory=dict)

Each key in the map is a property name and the corresponding value is the property’s value. To look up a property on an instance:

# lox/runtime.py LoxInstance method
def get(self, name: Token) -> Value:
    if name.lexeme in self.fields:
        return self.fields[name.lexeme]
    msg = f"Undefined property '{name.lexeme}'."
    raise LoxRuntimeError(msg, name)

This method uses a few imports:

# lox/runtime.py at top
from lox.errors import LoxRuntimeError
from lox.tokens import Token
from dataclasses import field

An interesting edge case we need to handle is what happens if the instance doesn’t have a property with the given name. We could silently return some dummy value like nil, but my experience with languages like JavaScript is that this behavior masks bugs more often than it does anything useful. Instead, we’ll make it a runtime error.

So the first thing we do is see if the instance actually has a field with the given name. Only then do we return it. Otherwise, we raise an error.

Note how I switched from talking about “properties” to “fields”. There is a subtle difference between the two. Fields are named bits of state stored directly in an instance. Properties are the named, uh, things, that a get expression may return. Every field is a property, but as we’ll see later, not every property is a field.

In theory, we can now read properties on objects. But since there’s no way to actually stuff any state into an instance, there are no fields to access. Before we can test out reading, we must support writing.

12 . 4 . 2Set expressions

Setters use the same syntax as getters, except they appear on the left side of an assignment.

someObject.someProperty = value;

In grammar land, we extend the rule for assignment to allow dotted identifiers on the left-hand side.

assignment     → ( call "." )? IDENTIFIER "=" assignment
               | logic_or ;

Unlike getters, setters don’t chain. However, the reference to call allows any high-precedence expression before the last dot, including any number of getters, as in:

Note here that only the last part, the .meat is the setter. The .omelette and .filling parts are both get expressions.

Just as we have two separate AST nodes for variable access and variable assignment, we need a second setter node to complement our getter node.

# lox/ast.py
@dataclass
class Set(Expr):
    object: Expr
    name: Token
    value: Expr

In case you don’t remember, the way we handle assignment in the parser is a little funny. We can’t easily tell that a series of tokens is the left-hand side of an assignment until we reach the =. Now that our assignment grammar rule has call on the left side, which can expand to arbitrarily large expressions, that final = may be many tokens away from the point where we need to know we’re parsing an assignment.

Instead, the trick we do is parse the left-hand side as a normal expression. Then, when we stumble onto the equal sign after it, we take the expression we already parsed and transform it into the correct syntax tree node for the assignment.

We add another clause to that transformation to handle turning an expr.Get expression on the left into the corresponding expr.Set.

# lox/parser.py Parser.assignment()
# Add case to inner if block
    ...
    elif isinstance(expr, Get):
        return Set(expr.object, expr.name, value)
    ...

That’s parsing our syntax.

Again, like Get, the property itself is dynamically evaluated, so there’s nothing to resolve there. All we need to do is recurse into the two subexpressions of Set, the object whose property is being set, and the value it’s being set to.

That leads us to the interpreter.

# lox/interpreter.py
@eval.register
def _(expr: Set, env: Env) -> Value:
    obj = eval(expr.object, env)
    if not isinstance(obj, LoxInstance):
        msg = "Only instances have fields."
        raise LoxRuntimeError(msg, expr.name)
    value = eval(expr.value, env)
    obj.set(expr.name, value)
    return value

We evaluate the object whose property is being set and check to see if it’s a LoxInstance. If not, that’s a runtime error. Otherwise, we evaluate the value being set and store it on the instance. That relies on a new method in LoxInstance.

# lox/runtime.py LoxInstance method
def set(self, name: Token, value: Value) -> None:
    self.fields[name.lexeme] = value

No real magic here. We stuff the values straight into the Python dict where fields live. Since Lox allows freely creating new fields on instances, there’s no need to see if the key is already present.

12 . 5Methods on Classes

You can create instances of classes and stuff data into them, but the class itself doesn’t really do anything. Instances are just maps and all instances are more or less the same. To make them feel like instances of classes, we need behavior—methods.

Our helpful parser already parses method declarations, so we’re good there. We also don’t need to add any new parser support for method calls. We already have . (getters) and () (function calls). A “method call” simply chains those together.

That raises an interesting question. What happens when those two expressions are pulled apart? Assuming that method in this example is a method on the class of object and not a field on the instance, what should the following piece of code do?

var m = object.method;
m(argument);

This program “looks up” the method and stores the result—whatever that is—in a variable and then calls that object later. Is this allowed? Can you treat a method like it’s a function on the instance?

What about the other direction?

class Box {}

fun notMethod(argument) {
  print "called function with " + argument;
}

var box = Box();
box.function = notMethod;
box.function("argument");

This program creates an instance and then stores a function in a field on it. Then it calls that function using the same syntax as a method call. Does that work?

Different languages have different answers to these questions. One could write a treatise on it. For Lox, we’ll say the answer to both of these is yes, it does work. We have a couple of reasons to justify that. For the second example—calling a function stored in a field—we want to support that because first-class functions are useful and storing them in fields is a perfectly normal thing to do.

The first example is more obscure. One motivation is that users generally expect to be able to hoist a subexpression out into a local variable without changing the meaning of the program. You can take this:

breakfast(omelette.filledWith(cheese), sausage);

And turn it into this:

var eggs = omelette.filledWith(cheese);
breakfast(eggs, sausage);

And it does the same thing. Likewise, since the . and the () in a method call are two separate expressions, it seems you should be able to hoist the lookup part into a variable and then call it later. We need to think carefully about what the thing you get when you look up a method is, and how it behaves, even in weird cases like:

class Person {
  sayName() {
    print this.name;
  }
}

var jane = Person();
jane.name = "Jane";

var method = jane.sayName;
method(); // ?

If you grab a handle to a method on some instance and call it later, does it “remember” the instance it was pulled off from? Does this inside the method still refer to that original object?

Here’s a more pathological example to bend your brain:

class Person {
  sayName() {
    print this.name;
  }
}

var jane = Person();
jane.name = "Jane";

var bill = Person();
bill.name = "Bill";

bill.sayName = jane.sayName;
bill.sayName(); // ?

Does that last line print “Bill” because that’s the instance that we called the method through, or “Jane” because it’s the instance where we first grabbed the method?

Equivalent code in Lua and JavaScript would print “Bill”. Those languages don’t really have a notion of “methods”. Everything is sort of functions-in-fields, so it’s not clear that jane “owns” sayName any more than bill does.

Lox, though, has real class syntax so we do know which callable things are methods and which are functions. Thus, like Python, C#, and others, we will have methods “bind” this to the original instance when the method is first grabbed. Python calls these bound methods.

In practice, that’s usually what you want. If you take a reference to a method on some object so you can use it as a callback later, you want to remember the instance it belonged to, even if that callback happens to be stored in a field on some other object.

OK, that’s a lot of semantics to load into your head. Forget about the edge cases for a bit. We’ll get back to those. For now, let’s get basic method calls working. We’re already parsing the method declarations inside the class body, so the next step is to resolve them.

# lox/resolver.py at resolver(Class)
    ...
    for method in stmt.methods:
        resolve_function(method, "METHOD")

We iterate through the methods in the class body and call the resolve_function() method we wrote for handling function declarations already. The only difference is that we pass in a new FunctionContext enum value.

# lox/resolver.py replace FunctioContext
type FunctionContext = Enum["FUNCTION", "METHOD", None]

That’s going to be important when we resolve this expressions. For now, don’t worry about it. The interesting stuff is in the interpreter.

# lox/interpreter.py exec(Class)
# Before return
    ...
    for method in stmt.methods:
        function = LoxFunction(method, env)
        klass.methods[method.name.lexeme] = function
    ...

When we interpret a class declaration statement, we turn the syntactic representation of the class—its AST node—into its runtime representation. Now, we need to do that for the methods contained in the class as well. Each method declaration blossoms into a LoxFunction object.

We take all of those and wrap them up into a map, keyed by the method names. That gets stored in LoxClass.

# lox/runtime.py LoxClass add attribute
    ...
    methods: dict[str, LoxFunction] = field(default_factory=dict)
    ...

Where an instance stores state, the class stores behavior. LoxInstance has its map of fields, and LoxClass gets a map of methods. Even though methods are owned by the class, they are still accessed through instances of that class.

# lox/runtime.py LoxInstance.get() before raising
    ...
    method = self.klass.find_method(name.lexeme)
    if method is not None:
        return method
    ...

When looking up a property on an instance, if we don’t find a matching field, we look for a method with that name on the instance’s class. If found, we return that. This is where the distinction between “field” and “property” becomes meaningful. When accessing a property, you might get a field—a bit of state stored on the instance—or you could hit a method defined on the instance’s class.

The method is looked up using this:

# lox/runtime.py LoxClass method
def find_method(self, name: str) -> LoxFunction | None:
    return self.methods.get(name)

You can probably guess this method is going to get more interesting later. For now, a simple map lookup on the class’s method table is enough to get us started. Give it a try:

class Bacon {
  eat() {
    print "Crunch crunch crunch!";
  }
}

Bacon().eat(); // Prints "Crunch crunch crunch!".

12 . 6This

We can define both behavior and state on objects, but they aren’t tied together yet. Inside a method, we have no way to access the fields of the “current” object—the instance that the method was called on—nor can we call other methods on that same object.

To get at that instance, it needs a name. Smalltalk, Ruby, and Swift use “self”. Simula, C++, Java, and others use “this”. Python uses “self” by convention, but you can technically call it whatever you like.

For Lox, since we generally hew to Java-ish style, we’ll go with “this”. Inside a method body, a this expression evaluates to the instance that the method was called on. Or, more specifically, since methods are accessed and then invoked as two steps, it will refer to the object that the method was accessed from.

That makes our job harder. Peep at:

class Egotist {
  speak() {
    print this;
  }
}

var method = Egotist().speak;
method();

On the second-to-last line, we grab a reference to the speak() method off an instance of the class. That returns a function, and that function needs to remember the instance it was pulled off of so that later, on the last line, it can still find it when the function is called.

We need to take this at the point that the method is accessed and attach it to the function somehow so that it stays around as long as we need it to. Hmm . . . a way to store some extra data that hangs around a function, eh? That sounds an awful lot like a closure, doesn’t it?

If we defined this as a sort of hidden variable in an environment that surrounds the function returned when looking up a method, then uses of this in the body would be able to find it later. LoxFunction already has the ability to hold on to a surrounding environment, so we have the machinery we need.

Let’s walk through an example to see how it works:

class Cake {
  taste() {
    var adjective = "delicious";
    print "The " + this.flavor + " cake is " + adjective + "!";
  }
}

var cake = Cake();
cake.flavor = "German chocolate";
cake.taste(); // Prints "The German chocolate cake is delicious!".

When we first evaluate the class definition, we create a LoxFunction for taste(). Its closure is the environment surrounding the class, in this case the global one. So the LoxFunction we store in the class’s method map looks like so:

When we evaluate the cake.taste get expression, we create a new environment that binds this to the object the method is accessed from (here, cake). Then we make a new LoxFunction with the same code as the original one but using that new environment as its closure.

This is the LoxFunction that gets returned when evaluating the get expression for the method name. When that function is later called by a () expression, we create an environment for the method body as usual.

The parent of the body environment is the environment we created earlier to bind this to the current object. Thus any use of this inside the body successfully resolves to that instance.

Reusing our environment code for implementing this also takes care of interesting cases where methods and functions interact, like:

class Thing {
  getCallback() {
    fun localFunction() {
      print this;
    }

    return localFunction;
  }
}

var callback = Thing().getCallback();
callback();

In, say, JavaScript, it’s common to return a callback from inside a method. That callback may want to hang on to and retain access to the original object—the this value—that the method was associated with. Our existing support for closures and environment chains should do all this correctly.

Let’s code it up. The first step is adding new syntax for this.

# lox/ast.py
@dataclass
class This(Expr):
    keyword: Token
    depth: int = -1

Parsing is simple since it’s a single token which our lexer already recognizes as a reserved word.

# lox/parser.py Parser.primary() if
    ...
    if self.match("THIS"):
        return This(self.previous())
    ...

You can start to see how this works like a variable when we get to the resolver.

# lox/resolver.py
@resolve.register
def _(expr: This, resolver: Resolver):
    resolve_local(expr, expr.keyword, env)

We resolve it exactly like any other local variable using “this” as the name for the “variable”. Of course, that’s not going to work right now, because “this” isn’t declared in any scope. Let’s fix that over in resolve(Class).

# lox/resolver.py resolve_node(Class)
# Add lines
    ...
    for method in stmt.methods:
        role = "METHOD"
        method_env = env.push(function_context=role)
        method_env["this"] = "DEFINED"
        resolve_function(method, role, method_env)

We are pushing a new scope to the environment stack, modifying its enclosing function and class context to indicate to the resolver that everything will happen inside a method of a class.

We override the push() method to accept optional parameters for the new fields.

# lox/resolver.py Env method
def push(self, **kwargs) -> Env:
    kwargs.setdefault("function_context", self.function_context)
    return Env(enclosing=self, errors=self.errors, **kwargs)

Before we step in and start resolving the method bodies, we push a new modified scope and define “this” in it as if it were a variable. Now, whenever a this expression is encountered (at least inside a method) it will resolve to a “local variable” defined in an implicit scope just outside of the block for the method body.

The resolver has a new scope for this, so the interpreter needs to create a corresponding environment for it. Remember, we always have to keep the resolver’s scope chains and the interpreter’s linked environments in sync with each other. At runtime, we create the environment after we find the method on the instance. We replace the previous line of code that simply returned the method’s LoxFunction with this:

# lox/runtime.py LoxInstance.get()
# Before returning the method
    ...
    if method is not None:
        return method.bind(self)
    ...

Note the new call to bind(). That looks like so:

# lox/runtime.py LoxFunction method
def bind(self, instance: LoxInstance) -> LoxFunction:
    env = self.closure.push()
    env["this"] = instance
    return LoxFunction(self.declaration, env)

There isn’t much to it. We create a new environment nestled inside the method’s original closure. Sort of a closure-within-a-closure. When the method is called, that will become the parent of the method body’s environment.

We declare “this” as a variable in that environment and bind it to the given instance, the instance that the method is being accessed from. Et voilà, the returned LoxFunction now carries around its own little persistent world where “this” is bound to the object.

The remaining task is interpreting those this expressions. Similar to the resolver, it is the same as interpreting a variable expression.

# lox/interpreter.py
@eval.register
def _(expr: This, env: Env) -> Value:
    try:
        return env.get_at(expr.depth, "this")
    except NameError as error:
        msg = f"Undefined variable '{error}'."
        raise LoxRuntimeError(msg, expr.keyword)

Go ahead and give it a try using that cake example from earlier. With less than twenty lines of code, our interpreter handles this inside methods even in all of the weird ways it can interact with nested classes, functions inside methods, handles to methods, etc.

12 . 6 . 1Invalid uses of this

Wait a minute. What happens if you try to use this outside of a method? What about:

print this;

Or:

fun notAMethod() {
  print this;
}

There is no instance for this to point to if you’re not in a method. We could give it some default value like nil or make it a runtime error, but the user has clearly made a mistake. The sooner they find and fix that mistake, the happier they’ll be.

Our resolution pass is a fine place to detect this error statically. It already detects return statements outside of functions. We’ll do something similar for this. In the vein of our existing FunctionContext enum, we define a new ClassContext one.

# lox/resolver.py at top
type ClassContext = Enum["CLASS", None]

Yes, it could be a Boolean. When we get to inheritance, it will get a third value, hence the enum right now. We also add a corresponding field, class_context. Its value tells us if we are currently inside a class declaration while traversing the syntax tree. It starts out None which means we aren’t in one.

# lox/resolver.py Env
    ...
    class_context: ClassContext = None
    ...

When we begin to resolve a class declaration, we change that.

# lox/resolver.py resolve_node(Class)
# Add lines to the start
    current_context = env.class_context
    env.class_context = "CLASS"
    ...

We store the previous value of the field in a local variable. This lets us piggyback onto the Python VM to keep a stack of current_context values in its stack frames. That way we don’t lose track of the previous value if one class nests inside another.

Once the methods had been resolved, we “pop” that stack by restoring the old value.

# lox/resolver.py resolve_node(Class)
# Add line to the end
    ...
    env.class_context = current_context

When we resolve a this expression, the class_context field gives us the bit of data we need to report an error if the expression doesn’t occur nestled inside a method body.

# lox/resolver.py resolve_node(This)
# Add lines to the start
    ...
    if env.class_context is None:
        msg = "Can't use 'this' outside of a class."
        env.error(expr.keyword, msg)
    ...

That should help users use this correctly, and it saves us from having to handle misuse at runtime in the interpreter.

12 . 7Constructors and Initializers

We can do almost everything with classes now, and as we near the end of the chapter we find ourselves strangely focused on a beginning. Methods and fields let us encapsulate state and behavior together so that an object always stays in a valid configuration. But how do we ensure a brand new object starts in a good state?

For that, we need constructors. I find them one of the trickiest parts of a language to design, and if you peer closely at most other languages, you’ll see cracks around object construction where the seams of the design don’t quite fit together perfectly. Maybe there’s something intrinsically messy about the moment of birth.

“Constructing” an object is actually a pair of operations:

1. The runtime allocates the memory required for a fresh instance. In most languages, this operation is at a fundamental level beneath what user code is able to access.

2. Then, a user-provided chunk of code is called which initializes the unformed object.

The latter is what we tend to think of when we hear “constructor”, but the language itself has usually done some groundwork for us before we get to that point. In fact, our Lox interpreter already has that covered when it creates a new LoxInstance object.

We’ll do the remaining part—user-defined initialization—now. Languages have a variety of notations for the chunk of code that sets up a new object for a class. C++, Java, and C# use a method whose name matches the class name. Ruby and Python call it init(). The latter is nice and short, so we’ll do that.

In LoxClass’s implementation of LoxCallable, we add a few more lines.

# lox/runtime.py in LoxClass.call()
# Replace the return statement
    ...
    init = self.find_method("init")
    if init is not None:
        init.bind(instance).call(env, arguments)
    return instance

When a class is called, after the LoxInstance is created, we look for an “init” method. If we find one, we immediately bind and invoke it just like a normal method call. The argument list is forwarded along.

That argument list means we also need to tweak how a class declares its arity.

# lox/runtime.py LoxClass
# Replace line `arity: int = 0`
@property
def arity(self) -> int:
    init = self.find_method("init")
    if init is None:
        return 0
    return init.arity

If there is an initializer, that method’s arity determines how many arguments you must pass when you call the class itself. We don’t require a class to define an initializer, though, as a convenience. If you don’t have an initializer, the arity is still zero.

That’s basically it. Since we bind the init() method before we call it, it has access to this inside its body. That, along with the arguments passed to the class, are all you need to be able to set up the new instance however you desire.

12 . 7 . 1Invoking init() directly

As usual, exploring this new semantic territory rustles up a few weird creatures. Consider:

class Foo {
  init() {
    print this;
  }
}

var foo = Foo();
print foo.init();

Can you “re-initialize” an object by directly calling its init() method? If you do, what does it return? A reasonable answer would be nil since that’s what it appears the body returns.

However—and I generally dislike compromising to satisfy the implementation—it will make clox’s implementation of constructors much easier if we say that init() methods always return this, even when directly called. In order to keep pylox compatible with that, we add a little special case code in LoxFunction.

# lox/runtime.py LoxFunction.call()
# Add at the end
    ...
    if self.is_initializer:
        return self.closure.get_at(0, "this")

If the function is an initializer, we override the actual return value and forcibly return this. That relies on a new is_initializer field.

# lox/runtime.py LoxFunction add field
    ...
    is_initializer: bool = False
    ...

We can’t simply see if the name of the LoxFunction is “init” because the user could have defined a function with that name. In that case, there is no this to return. To avoid that weird edge case, we’ll directly store whether the LoxFunction represents an initializer method. That means we need to go back and fix the few places where we create LoxFunctions.

Since we gave a default value of False to the is_initializer field, we don’t need to change the code that creates LoxFunctions for normal function declarations. We only need to change the code that creates them for methods.

# lox/interpreter.py in exec(Class)
# Replace the line that creates the LoxFunction
    ...
    is_initializer=method.name.lexeme == "init"
    function = LoxFunction(method, env, is_initializer)
    ...

And then in bind() where we create the closure that binds this to a method, we pass along the original method’s value.

# lox/runtime.py in LoxFunction.bind()
# replace the return statement
    ...
    return LoxFunction(self.declaration, env, self.is_initializer)

12 . 7 . 2Returning from init()

We aren’t out of the woods yet. We’ve been assuming that a user-written initializer doesn’t explicitly return a value because most constructors don’t. What should happen if a user tries:

class Foo {
  init() {
    return "something else";
  }
}

It’s definitely not going to do what they want, so we may as well make it a static error. Back in the resolver, we add another case to FunctionContext.

# lox/resolver.py
# Add case to FunctionContext
type FunctionContext = Enum[..., "INITIALIZER"]

We use the visited method’s name to determine if we’re resolving an initializer or not.

# lox/resolver.py at resolve_node(Class)
# After line `role = "METHOD"`
    ...
    if method.name.lexeme == "init":
        role = "INITIALIZER"
    ...

When we later traverse into a return statement, we check that field and make it an error to return a value from inside an init() method.

# lox/resolver.py resolve(Return)
# Inside the `if stmt.value is not None:` block
    ...
    if env.function_context == "INITIALIZER":
        msg = "Can't return a value from an initializer."
        env.error(stmt.keyword, msg)
    ...

We’re still not done. We statically disallow returning a value from an initializer, but you can still use an empty early return.

class Foo {
  init() {
    return;
  }
}

That is actually kind of useful sometimes, so we don’t want to disallow it entirely. Instead, it should return this instead of nil. That’s an easy fix over in LoxFunction.

# lox/runtime.py in LoxFunction.call()
# Before `return error.value`
    ...
    if self.is_initializer:
        return self.closure.get_at(0, "this")
    ...

If we’re in an initializer and execute a return statement, instead of returning the value (which will always be nil), we again return this.

Phew! That was a whole list of tasks but our reward is that our little interpreter has grown an entire programming paradigm. Classes, methods, fields, this, and constructors. Our baby language is looking awfully grown-up.

power = breadth × ease ÷ complexity