Scripting, Part 1: Clearing Things Up a Bit

Now that I have my nostalgia-fueled introduction out of the way, we can get to business. Before we get to coding, or even designing code structure, however, let's clear up what implementing a scripting language actually means. There are, after all, some valid concerns with such an endeavor.

Why implement a scripting system at all?

For me, answering that question is easy: My life does not depend on my ability to deliver a product and I wanted to do it for a while now. It's a fun challenge for me, but there is some value in it:

• Behaviour as data: Being able to save parts of the game logic with your other assets becomes valuable once you implement things like skill systems, that drive behavior and are subject to design changes and balancing.
• Separation of concerns: It should be the designers' job to make sure that a certain enemy type explodes upon death and deals damage to everyone in range. It should be the programmers' job to make sure they have the tools for that. A scripting language is such a tool.
• Sandboxing behaviour: Assuming you allow user created content, you probably would want them to have access to similar tools. If you allow them to go too deep into the engine, those tools become a potential threat, unless you yourself can manage what can and can't be done with them. This is true for most integrated scripting systems.

Why not Lua? Or Python?

Or any other language that can be integrated into most engines with very little trouble, for that matter. There are, after all, a number of compelling reasons for it:

• Documentation: Established languages like Lua have a detailed documentation and a wide range of tutorials. When a problem occurs anyway, it is likely that a quick Google search will give you a solution. If you already document your code, the former should not be as much of a problem since large parts of the interpreter's documentation will overlap with that of the language; the latter is harder, if not impossible to replace.
• Tools: Unless you want to write byte code in a hex editor, you will need, at the very least, some sort of compiler. Established languages will usually offer a debugger and a decompiler, possibly even a dedicated IDE, which you should consider valuable tools, even for simple scripts. If you plan for this, implementing the interpreter will give you a barebones debugger and decompiler for nearly free, and creating a compiler isn't as hard as people make it out to be, assuming you keep the feature count low. Parts 7 through 10 are actually dedicated to compilers, but for now just keep in mind that tool support for established languages is a lot better.
• Support: The tools for an established language, including the interpreter, have been a lot longer in development than anything you will implement in the future. Therefore, they already have gone through countless hours of debugging and the QA department that is the internet. You can expect bugs to be known and fixed quickly. On the flipside, you will have to do that yourself for any language you could implement. Using automated testing can and will keep bugs to a minimum, but someone will still have to write those tests.

There are some arguments against using established systems like Lua, mainly sandboxing behaviour. These languages are pretty powerful, after all, and if you allow using things like libraries, that may become a safety concern. Also, you might not actually want control flow in some cases; the game should not crash because a script that was supposed to populate a behaviour tree got stuck in an infinite loop, for instance.

Other than that, I'll admit that I can't really think of other reasons to implement your own language.
...Well, it is a pretty cool feat to brag about to your programmer friends, if nothing else.

How to approach the problem

If you're still with me now, let's talk about how to tackle the task. What I will explain and implement in the following posts is essentially a variation of the bytecode pattern.

It revolves around the idea that commands can be stored as bytes (or shorts or integers or whatever) that can be executed one by one. That means that

• you will store scripts in byte arrays,
• keep track of the index of the next command within that array, and
• execute commands by determining the next command and calling the apropriate function.

From there you can add functionality as needed.

• The first article I linked suggests adding a stack to evaluate expressions and pass parameters. It's a pretty good fit for those tasks, but not the only solution. You could, for instance use an array for variables and interpret the bytes following the command as the index to that array. Either way, you're going to need some way to read extended opcodes (i.e. commands that take up more than one byte).
• For control flow, you will also need a callstack, on which previous values of the program counter will be pushed so that you can return to previous code paths.
• Finally, you'll want some way to hook up the interpreter with the rest of your game engine, so that the interpreter, when it comes across a spawnParticle command, can actually spawn particles.

In the next part, we'll discuss some patterns to use for our interpreter. I'll introduce you to the fairly simple class architecture to support all of the above, plus easy tool implementation. After that, we finally program the interpreter, followed by the debugger and decompiler. Then I'll walk you through the design and implementation of an examplary language. Eventually, we'll discuss some alternatives to compiler construction before getting our hands dirty with that as well.