Matrix Transformations in Color Space

When I started programming in C# five years ago, I quickly latched onto XNA for game programming. I didn't have constant internet access back then, so I bought books to learn from instead. The particular book I learned XNA from had a chapter on shader programming. Back then it seemed like magic to me - my school didn't teach me linear algebra until a year or two ago, I forget - and even today, I grasp the general concept, but have little experience, especially with vertex shaders.
However, one sample shader caught my attention recently: The black-and-white shader. It used a dot product operation to convert a color vector to its monochrome luminance. Other than that, and maybe sepia filters, I never really heard of people using transformation matrices on their color vectors. A quick google search reveals that it's not an entirely new concept - that page predates my birth by over three years. There is quite a bit of cool stuff you can do with this, so let's get crackin!

Prerequisites

Before we can work on the color transformations though, we need to linearize our colors. There's a nice chapter of GPU Gems explaining the concept and general importance of that, and since we use additive and subtractive blending for lighting effects, we should be doing that anyway.
I haven't produced any textures for this project yet, so we can, as advised, assume that all loaded textures are already linearized. However, we still need to perform a gamma correction in the post processing effect, and that has to happen in a pixel shader. Additionally, we should apply the calculated transformation matrix as we render so we'll actually need two different shaders.

I've dabbled in shader programming in HLSL for XNA, but I know that MonoGame uses its own shader system to compile for different platforms. I was pleasantly surprised to find documentation and example code on that at least.
Note that, unless you are using SpriteSortMode.Immediate, applying the Effect manually won't actually work. You need to pass it to SpriteBatch.Begin, but then you'll face a compatibility issue between the sprite batch and you effect: the camera transformation matrix is no longer applied, and neither is the OrthographicOffCenter matrix used internally by every sprite batch. You'll have to apply these manually with a vertex shader. Here is a SpriteBatch compatible pass-through effect that does that:

Color Transformations

Okay, this section is basically about porting Paul Haeberli's color transformations to C#, it's not really my code per se. The first transformation, Changing Brightness/Color Balance is basically provided by XNA, as it only scales the color by the specified parameter; we can use Matrix.CreateScale for that. We can also Apply Offsets to Color Components by using Matrix.CreateTranslation.

Modifying Saturation

This one is pretty simple: the first variant, Converting to Luminance, is a specific case of the saturation problem, and is solved by using a weighted average of all channels for all channels. To arbitrarily Scale Saturation, you need to interpolate between Matrix.Identity and the luminance matrix. I implemented both in one function, since you can just pass 0 to get the monochrome picture:
This is pretty cool, because you can actually pass values outside of (0; 1) to get inverted colors at -1 and boosted saturation for values greater than 1.

Shifting Hue

I'll admit that I don't really understand the math behind this either. I mean, I know how the matrices do their job, but I know too little about color spaces and how they relate to each other to be sure about why you need to do these exact operations. Sorry.
To be honest, it's not even that useful an operation. If we're tinting the map and objects to simulate different lighting situations, hue doesn't usually shift by a fixed offset. Rather, it shifts towards a certain value, generally yellow for increased light and blue for decreased light.

Just combining the other three operations gets us quite far, though; these are the exact same scenes, only rendered with a different color transformation matrix:

The ugliest island map in the world, day and night.

ABDUCT: About functional programming, patent laws and a dead project

About a week ago, while I was working on the entity system for my game, I grew tired of my code base; I'm not really happy about all decisions I made, both early on and with my tile map rendering, and it all accumulated into some anxiety over sprite rendering.
So I decided to take a break and come back later instead of burning out and leaving the project unfinished. I didn't want to stop contributing to the game, however, so I considered my other options. I was also kind of obsessed with audio stuff at the time, since I spent so much time researching it just prior, and really wanted to work on authentic BRR compression, with better tool support than the shady tools I found on romhacking.net.
Finally, I wanted to try functional programming, see what all the hype was about. Since audio compression seemed like a great fit, and F# like a good bridge into the paradigm, everything seemed to line up perfectly.

Since tool naming is important, I thought long and hard (heh) about a naming scheme for my own tools, and came up with the Authentic Bit rate reDUCtion Tool, the Barely Usable Rigid Game Level Editor and other silly backronyms for criminal verbs.
Then I got to work, setting up an F# project Abduct.Core, which should do all the heavy lifting, and the C# project Abduct.UI, which should handle all the boring stuff, such as communicating with NAudio, the library I decided to use for handling audio files (and resampling, which, as it turns out, you shouldn't try to do yourself, at least not if you want quality output).

And, as if in the second act of a tragedy, things started to go down hill from here.
The plan was relatively simple: use NAudio to load WAV files and convert them into 16 bit mono. Resample to a user defined sample rate, perform length correction - BRR compressed sounds come in chunks of 16 samples, so you either have to pad in silence for sound effects, or truncate looped samples. Then you apply the actual compression, and decompress again - we're not actually running on a SNES, so the compressed data is of no use to us.
The user should be able to play back a sound at any sample rate (without resampling!) to hear what the sample would sound like after pitch modulation, with or without looping, to examine its behaviour, and finally export the file as an uncompressed wave file.

As mentioned above, I use NAudio to resample, so that's not really an accomplishment. I did get length correction to work, I think. It's currently intertwined with the non-functioning compression code, and even before, it was a bit hard to test.
Convering different wave formats with NAudio involves mucking around with .NET streams. A lot. So much so, that I couldn't find a way to easily do it for any wave file I might load, even with a clearly defined target format. So, for prototyping, I just allowed wave files in the proper format.
Playing back audio with NAudio is also an atrocious process. Well, it is if you have to change the output wave format dynamically because you're trying to modulate pitch.
The real killer of this project, though, was the actual compression, for two reasons:

1. I didn't bother to really learn the concepts of functional programming. I just jumped in with a new project in Visual Studio and this reference. Inevitably, I encountered a compiler error that I still haven't wrapped my mind around yet.
2. As the wikipedia article on BRR compression mentions - which I didn't notice during my initial research - there's loads and loads of patents involved. I don't have the patience and legalese to read all of these, and I can't afford (and frankly, don't want to pay) a patent attorney to do it for me. I'm not clear on American patent laws, and the fact that I'm located in Germany doesn't make the whole situation any simpler.

To cut to the chase, even if I did get around all these other problems - bite the bullet and write all that code to properly use NAudio, learn all about the functional paradigm - I still couldn't complete and use the tool with confidence, much less release the code in good conscience.
I will, however, strip away the bits of compression code I wrote so far, get it to a somewhat presentable state, and release that code on github later this week. If anyone from the home brew and rom hacking communities is more confident in their legalese (or Nintendo's unwillingness to sue over this stuff), they are of course free to do with it whatever they want.

As for myself, the inability to get accurate SNES sound is a bit of a disappointment. I'm left with just a few options:

• Skip the compression and only use length correction and resampling. This would make our samples significantly higher in quality - maybe using 8 bit samples might combat that somewhat.
• I could, of course, try other compression algorithms, or even create a shitty one myself.
• Use a chip tunes soundtrack. Most samples qualified for that would barely suffer from the compression anyway. It would also get us closer to the GBA sound, which wouldn't be too bad.
• Giving up on authenticity and using a CD quality sound track is tempting, but not really my style.

I'll have to think about the specifics of this later. It's 4 am over here, and I need some sleep.
Anyway, that's what I've been up to this week. I'll muck around with functional programming some more, but I'll return to doing engine work soon, promise!

NOTE: I usually wrtite blog posts a week or three in advance, so the posts will keep coming as scheduled, this post is the one in the wrong time line.

UPDATE: The cleaned up code for this project can now be found on github: https://github.com/teryror/ABDUCT-Audio-Processor.

Handling Multiple Resolutions with a Virtual Screen

My current retro 16 bit style guidelines dictate that my game use a virtual screen of 320x180 pixels. At first, that seems easy enough: when the game is running in window mode, we can just dictate the size to be a multiple of that and zoom in accordingly. In full screen 720 or 1080p, we can just scale by a factor of 4 or 6, respectively. Common laptop resolutions, like 1366x768, are not so easy, but we can give the option to either scale unevenly, by a factor of roughly 4.26 in this case, or scale evenly by a factor of 4, and have a black border around the virtual screen. On displays with different aspect ratios, we generally face the same problem, except even the unevenly scaled version would require letterboxing.

To handle all of these different cases, as well as the very concept of a virtual display, I created the class VirtualScreen. It uses a RenderTarget2D that is a multiple of 320x180 in size, and provides a CaptureFrame method that sets the render target on the GPU, and a DrawFrame method that renders the target on the screen.
The hairy bit is the Refresh method, which sets the size of the render target and calculates a destination rectangle for DrawFrame. There's also the Reset and Save methods, which load the properties of the class from the configuration and calls Refresh, and write the current state of the screen back into the configuratin and save it, respectively.

This is the bad boy in question:
It looks pretty hairy at first glance, and it is a bit heavy on the arithmetic side, but the real problem is, that it just explodes in size as we add rendering options, and there just is no design pattern or algorithm to save us here. We could factor out a RefreshFullScreen and a RefreshWindowed function, but that would only take away a couple lines of code and a level of indentation; it wouldn't help with the why does-this-math-work-question.
The real problem is, that this thing just explodes in complexity as you add video options - I added a SuperSampling property that doubles the size of the render target while keeping the destination rectangle the same, slightly blurring the pixel art but also smoothing out sprite movement (once we get to that, anyway).

But setting up the back buffer and render target isn't the only thing I wrote this class for.
The Scale property the Refresh method sets before calculating a destination is not configurable. Instead, it's supposed to be used to create transformation matrices for rendering on the virtual display. I now use it in my Camera class in place of its own scale member, so the camera automatically adjusts to changes to the rendering options.
Since we have a texture with the rendered world and UI by the time DrawFrame gets called, we can also do post-processing there, which I'll at least touch on in the next post.
I will also render a Super Game Boy inspired background image behind the render target, to make potential letterboxing less awful to look at. The original The Binding of Isaac did that to great effect, I find.

Matrix Transformations in 2D Space - The Camera

While I was factoring out the useful bits of the map rendering proto-code, I tested them a lot more thoroughly and a fixed a couple of bugs, mostly just missing calls into classes I wanted to plug into the engine. One bug in particular, however, was related to the way we currently rotate tiles;
When we set the origin in our draw call to (8 8), we not only rotate tiles around their center, as I'd assumed, but also shift them to the top left by the same amount. To fix that, I removed the origin parameter and used some evil bit magic again:
This is not today's topic though, so explaining how that works is left as an exercise for the reader.

Doing Math on Camera

With the refactoring done, we could go straight to implementing map objects, but I decided to do some polishing first. We can kill quite a few birds with one stone here, and that stone is a 2D camera.
We usually talk about cameras in 3D games, because their control is usually fundamental to the gameplay. But it's also common to have a camera in a 2D game, because it's very convenient for the programmer: If you can just move the camera and the environment magically scrolls on the screen, there are so many fewer headaches involved in just figuring out where to render a sprite.
The easiest way to implement such magic is with a transformation matrix that you pass to the GPU, which will use that to automatically figure out where sprites should be rendered, and at what size.
XNA/MonoGame also provide us with a Matrix structure, so we don't even have to worry about the math behind the magic too much. We just call a couple static functions, multiply the results together, and pass the product to SpriteBatch.Begin.

However, in case you actually are unfamiliar with the math, I highly recommend you read up on it. There's tons of more focused resources on linear algebra than this blog, and you need it constantly when you're doing any sort of graphics or physics work. In any case, here's a brief rundown of the essentials:
A matrix is a NxM field of real numbers. The (x y) vectors we've been working with are essentialy 1x2 matrices. Other than vectors, the most common kind of matrix in graphics programming is the 4x4 matrix, which is also what a Matrix in XNA is.
You can use arithmetic operators on matricess; the two most important operations for us are multiplying 1x4 matrices - Vector4 - with 4x4 matrices to get another Vector4, and multiplying one 4x4 Matrix with another to get a third 4x4 Matrix.
The latter lets us combine multiple transformation matrices into one, so we can perform multiple transformations of a vector with just one matrix multiplication.
The former works by calculating four linear combinations of the elements in the original vector, with the columns of the matrix serving as the coefficients for each elemnt in the new vector. Since we mostly work with 2- or 3-component vectors, the remaining elements are usually padded with 1.0, giving us the power to do translations, i.e. shifting the vectors around by a constant offset. We can also scale a vector by setting the factors on the main diagonal.

Our camera doesn't even need to do any more than that:
First, we want to zoom in by a factor of 16. This lets us render tiles into destination rectangles of size 1, which should remove a lot of potential *16 and /16 from our code, such as in the bug fix above.
Then, as the camera's position goes to the bottom right, we want to move the scene to the top left. This lets us render the map with the origin at (0 0), no matter which part of the level should currently be focused. We could also easily add a screenshake offset, without anything having to change in our rendering code.
Finally, we want to zoom in some more so our virtual 320x180 screen doesn't look so small on a 1080p display. Having this configurable easily will help us once we actually code the virtual screen to deal with multiple resolutions.

I use a full class TopDownCamera for this, with properties and dirty flags, but at its heart is just this one line of code:
Passing that to SpriteBatch.Begin does all the rest. However, there's one other thing we can do, now that we have the camera working:

Polishing Tiles

At this point, I added a Vector4 Viewport property to my camera class that would give me the left, top, right and bottom limits of the visible area of the world. Using that, we can calculate the upper and lower limits of the x and y loops in our DrawLayer method. This has two major advantages:
We only consider tiles that are actually on screen - we can remove map size from the performance equation.
We also get to consider values that are not on the map. This may sound stupid at first, but whenever the camera gets close to the edge of the screen, you see Cornflowe Blue, or whatever other color you clear your screen with. If the map data contained a set of border blocks we could render there instead, we can easily detect when to render border blocks, depending on whether the x and y indices are within the bounds of the blocks array.

Depending on how much attention you want to give this, determining which tile to render can actually be more difficult. If you just define a single border block, you can just go
I, however, wanted the border blocks to be almost full featured maps of themselves. They share the same layer settings and obviously won't get any collision data, but I defined a ushort[,,] blocks, which should be repeatedly rendered around the map.
Finding the proper index into this array is a bit more complicated. You could just modulate the coordinates by the length of the array in the respective dimension, but then you'd still get negative values. Negating these won't work, though, because that would inverse the block pattern upwards and to the left of the map. However, you could add the length of the array and modulate again, which gives the correct pattern, like this:

Finally, here is the code in action:

The ugliest island map in the world!
The water is all border blocks, by the way.

Tile Maps: I'll Show You The World

After fixing the hot mess from installing the full release of Visual Studio 2015 without first uninstalling the Release Candidate, which left me unable to start up or uninstall either (if you're having the same problem, try running the installer from the command line with the /u /force flags), I can finally do the rendering code for tile maps.

However, to see if it works, we'll need a tileset and a map. We can hardcode the tile map, but for the tileset, I decided to use KenneyNL's Roguelike tile sets, just for proto typing.

And now my watch begins

To get an idea of where to even start, let's think of when a tile map would be visible.

• First and foremost, during exploration, when you're controlling your character,
• during scripted events, when you're not in control, and
• when you're in an overlaid menu, where you control the cursor, rather than your character.

These are three inherently different game states, that share some logic - they are substates to the encompassing TileMapView game state.
Just to get something on screen quickly, I took a quick stab at implementing it. It's chaotic, it's not feature complete, and it's not even documented. It's a refactoring waiting to happen. However, it also renders this:

Not all that pretty. Yet.

I think the code is mostly self-explanatory, but before we go on to add the missing features, let's discuss the evil bit level hacks and other non-obvious choices.

Starting at the top, the MakeTileID method would probably have been a macro, if those existed in C#. It's purely a convenient function for now, but it does document the bit level layout of a tile ID: There's integers X and Y, each ranging from 0 to 63, taking up bits 0 through 5 and 6 through 11, respectively. There's a SpriteEffects in there, which is a field of two flags, and finally a two bit integer called r. That's an index into the ROTATIONS array above, allowing blocks to be rotated individually.

The constructor is pretty simple initialization. All these loops programmatically fill the map. The last line is there to test the rotation and mirroring features. I tested them individually as well, but here the combination of both flips and a rotation by 180° gives us the original orientation of the tile.

Finally, the DrawLayer method does the heavy lifting: we Begin a SpriteBatch for each layer because, as the additive/subtractive lighting layers go in between, we'll have to begin different batches anyway. The layers in between are also the reason this is its own method in the first place, and not just the body of a loop in Draw.
Then, we declare and initialize our local variables; the source and destination Rectangles for the call to SpriteBatch.Draw, as well as rot, an index into ROTATIONS, and a SpriteEffects variable. I do this outside the loop because it's probably going to be the hottest loop in our code base for quite a while and it's a quite intuitive optimization.
Within the loop itself, we split all the info in the tile ID and update the local variables as necessary, before finally drawing the tile. The Vector2 origin we pass here refers to the center of rotation, relative to the top-left corner of the sprite. Since all tiles are 16x16 pixels, passing (8 8) rotates tiles around their mid-point.

The light that brings the dawn

I already mentioned the light maps that go between layers, an those will be our next priority. To make this make sense, let's place a light source first. The fire pit will do, so that's MakeTileID(14,26) for me.
As for the light itself, I used gradients and the Posterize adjustment in Paint.NET to make a pixely, fading blob of orange on black and added that to the tile set. At this point, my proto-tileset looked like this. I have my doubts stuffing the lighting data into the tile set like this is viable in the long run, but for now, this will certainly do.

To blend layers, we need to pass a BlendState when calling SpriteBatch.Begin, so we'll just add that as a parameter to our DrawLayer function. After adjusting our code in Draw and adding a couple tiles to the additive layer, we can now render this:

...still not impressed? Me neither.

While BlendState.Additive is provided in XNA/MonoGame, we'll have to make our own subtractive blend mode. We can't really set the opacity either, but for that, we can use the color parameter to SpriteBatch.Draw.
It is used to tint sprites by multiplying each pixel's color by the provided value. Since we only use the opacity for additive and subtractive blending, we can provide a shade of grey that gets us the same effect as setting opacity.
The rules say we get exactly different 16 opacities, so I decided to make another array of constants to reference with an opacityIndex parameter to the DrawLayer method. Choosing an opacity of fifty percent (that's #808080) makes the lighting in this proto type a lot prettier:

Meh.

The fire that burns against the cold

Our little camp fire just doesn't look right with just the light map, wouldn't you say? The thing it lacks is movement. I've never seen a still fire, so we need to animate it.
I already mentioned using a Dictionary for that in my last post, and here's how that works:
You need the X and Y parts of a tile ID, both as values and as keys. When rendering, get the lower 12 bits by ANDing the tile ID with 0xFFF, put them in a local variable, and if (dictionary.ContainsKey(id)), set id = dictionary[id].
This lets you alias tile IDs, which is a great way to troll level designers, but not all that useful by itself. To animate a tile, you need to dynamically change the contents of the dictionary. There are seemingly endless ways to time your animations, but no matter which you choose, in the end, the result will look somewhat like this:

Converting this into a gif seems to have fucked the colors, but at least it's moving.

With that, we have the basic feature set down. The new main focus, obviously, is to refactor this mess into something useful. You'll hear from me again once that's done and something interesting is in the making again. Until then!

Tile Maps: How to Represent a World

After the last post, which covered an awful lot of code, this will mostly be theory again, mostly because I'd like to go in with a plan, and I do not yet have one.

With a good tile set, you hardly notice the grid.

I want you to think of classical console RPGs. Final Fantasy VI (pictured on left), Chrono Trigger, Secret of Mana and the like. While their art styles differ somewhat, they have one thing in common: They share an angled top-down view of their game worlds, projected orthogonally, i.e. without perspective. Levels are made up of 16 by 16 pixel square tiles.
Pokémon, Golden Sun and most RPG Maker games use the very same world view. Yet, with all of these games, there are some technical differences in how they do their rendering; they all use different data structures to represent their levels, so they all have slightly different 'max-specs' in terms of level design.

Layers

Seiken Densetsu 3, showing the ocean
and the sky beyond the cliffs.

Some SNES games opted to use one of their precious two background layers for a parallax background.

Especially popular with the RPG Maker crowd, but also used in Chrono's room in Chrono Trigger, you could use additive blending for lighting effects.

You could use subtractive blending for shadow effects in murkier environments, alpha blending for translucent overlays, e.g. for weather effects or transparent objects in the game world.

By sacrificing layers for visual effects like these, however, you limit how many layers you can effectively use for maps. For instance, Pokémon Ruby and Sapphire had bridges you could go both under and over, solely because the GBA had more background layers, making a multi-layered environment possible.
Considering you also need to render UI elements somehow, you run out of layers fast. In my post on technical limitations, I said I'd use around eight layers. Let's see where that gets us, assuming we want all of the above:

1. Parallax Background
2. Map (Underneath objects)
3. Map (Above objects)
4. Light Map (Additive Blending)
5. Shadow Map (Subtractive Blending)
6. Parallax Overlays (Arbitrary Blending)
7. UI Background (Blending)
8. UI Text

That's not very far. We could do objects crossing bridges over and under, but there could be no elements in the map that cover these bridges, so that's already pretty limiting. With both the light and shadow map above the map layers, you couldn't have light sources or shadows that only cover the lower level of the map.

We'll have to bend our minds somewhat to justify more layers.
The obvious target would be UI; Assuming the background doesn't blend, you could render it into a single layer. You could also use sprites for text and other non-transparent objects. Either way, that's one layer we get back.
You might argue that few maps would use both light and shadow maps, and that having both go in front of all map layers is almost certainly a waste of either. Reordering the layers so that you get one transparent layer with arbitrary blending in front of both map layers would be more useful.
Finally, you might say that having both an overlay and a parallax background would look weird, since some layers in the middle would be the only scrolling ones. If you allow either, but not both at the same time, you get the same fundamental feature set, and an additional layer back. However, I've never seen both effects used at the same time and thus would like to withhold judgement in this case.

Still, our layer reservations now look like this:

1. Parallax Background
2. Map (Underneath objects)
3. Light Map (Arbitrary Blending)
4. Map (Above objects)
5. Light Map (Arbitrary Blending)
6. Map (Above everything)
7. Parallax Overlays (Arbitrary Blending)
8. UI

I might want to reshuffle layers 2 through 6 later on, or even provide facilities to determine their order on a per-map basis, but that's basically what I'll be going for. And I'm quite happy with it. We have three Map Layers, which should be enough. We might even get a fourth, depending on my judgement on the background+overlay issue.
All that said, it should be clear that there can really only be two layers of map objects, which will be a major factor in how you can layout a dungeon, for example.

Tile Sets

These are the deciding factor in what can actually go inside a map. I barely touched on these in my post on technical constraints, so it's time to talk about them now.
A tile set is essentially a texture you use to construct maps. A map is a multidimensional array of indices into an array of tiles. Tiles are regularly sized pieces of the tile set.
As mentioned above, most RPGs from the era in question had a 16x16 pixel grid, but both the SNES and GBA had hardware support for maps with a 8x8 pixel grid. Generally, you would find tile set meta data, that provides the 8x8 tile set to the hardware, as well as block data, which would combine four small tiles into one 16x16 block, possibly with some gameplay data attached. This gives you a block set, from which the actual maps are constructed.

Since there's no hardware or even library support for tile maps, I don't have to add this complication, using a tile set with a 16x16 grid from the outset.
However, that begs the question how large our tile sets should be. Depending on the graphics mode, the SNES supported up to 1024 8x8 tiles in a set. Each can be displayed in one of eight palettes, giving us 8192 possible tiles, four of which are combined into one block, giving us upwards of 4.5 quadrillion possible blocks, more than we could index with a 32 bit integer.
Usually, only a handful of tiles would be used with different palettes, and even then, rarely more than two palettes. Assuming all tiles could be reused that way, this gets us down to about 17.6 trillion. If none are to be reused, that puts us to just below 1.1 trillion, still more than fits into an integer.

Maybe looking at actual technical limitations is in order. XNA runs on DirectX 9. That's so 2010. Anyway, DX9 supported textures up to 4096x4096 pixels, though not all DX9 GPUs actually implemented that. The most commonly recommended maximum texture size I remember from these days is 1024x1024 pixels.
Now, I'm not sure how MonoGame fares compared to that, but a texture of that size gives us 64x64 (=4096) tiles. In comparison, that's nothing, but it's a much more reasonable size for a block set. You can index that with a 16 bit integer and have 4 bits left over, e.g. for flipping blocks along either axis (one bit per axis) or rotating by a multiple of 90 degrees (four rotations fit into two bits).

As for colors, because of the way blocks are constructed from four tiles, each of which might use one of eight palettes of 15 colors, one block might contain up to 60 different colors, but only 15 colors per quarter block. That's not only confusing, but also somewhat limiting.
You can't really enforce it from a software perspective either, so I just decided to go with up to 128 colors per tileset, and try to avoid scenarios with many palettes on a block. If breaking this rule makes for a prettier tileset, though, I won't think twice about it.

Animation

Whether it's a waterfall crashing into a riverbed, the mill wheel rotating downstream or just the grass beside it swaying in the wind, if it's in your game world, you'll want to see it moving. Since the tiles themselves can't really move, and that would make for a horrible 'animation' to boot, we have to figure out another method.
Generally, you animate pixel art with old-school, hand drawn frames that get swapped according to some carefully chosen timing.

Ideally, the tile ID we read from the map should be translated directly into coordinates in the tile set texture, so this leaves us in a bit of a predicament.
On the SNES, you'd have a set of animation definitions, process them, and hot-swap the animated tiles in the tile set for other graphics. We could use SetData on the tile set texture, though I'm not sure how feasible it is to do that mid-frame.
If we want to stick to the ideal tile ID to texture coordinate translation, the only real option is to modify the map data instead of the tile set data. That's naive and counter intuitive at the same time; the map refers to an animated block, why does the animation change the map? It would also take varying amounts of time, depending on map size.
The most viable solution I can see, would be an additional step in translating the tile ID to a texture coordinate. Have a dictionary of tile IDs, which is updated by the animation code, and switch tiles as they're rendered.

Meta Data

With that, we could go in and write a renderer for this specific tile map system, but there are some questions left to make it into a game. These questions primarily concern meta data:
Which tiles can you actually walk on? Do these tiles have any special behaviour attached? How should NPCs and other event data be stored?

Starting with collision data, I want you to think of two games in particular: The Legend of Zelda - A Link to the Past, and Tales of Phantasia. In both games, you move your character with no respect for the grid, unlike Pokémon, where your character is always aligned with it.
In Tales of, a block's collision data was essentially four boolean values, creating a smaller 8x8 grid of colliders. In A Link to the Past, in addition to full-block colliders, there were diagonal walls, that, when walked against, had you slide along them.
The point is, there's many ways to relate collision data to the tile map, and you have to decide on one concept.

Logical layers also tie into this. Consider bridges again. While you're on top, you collide with their edges, but can walk over just fine. When you're below, the inverse is true.
The Pokémon games handled this strangely. They have one byte of movement data attached to each block. Two bits of that are dedicated to the basic collision behaviour (walkable, impassable, walkable*, surfable), with some of the rest representing various layers.
You can cross from walkable blocks to walkable blocks of the same layer, but not of a different layer. The walkable* behaviour allows you to cross layers.
Certain values of the movement byte are reserved for horizontal and vertical bridges, as well as some others with unknown behaviour. If you are on a walkable block above a certain layer, you can enter horizontal bridge tiles from the sides, though you can walk freely between them. At the same time, you could only enter vertical bridge tiles from the top and bottom. If you are below that layer, the inverse is true.

I really like this approach, because you only need one collision map. The Pokémon developers only ever used, like, two of the dozens of available layers, confirming again that that should be enough, while also freeing up some space in the collision data structure.
Let's say we use three bits to determine the basic collider shape. There's passable (0), impassable (1), four variations of diagonal walls (4-7), and two undefined values. When used on the lower layer, these may just be impassible, but on the upper layer, they might represent bridges. Add one bit to indicate the layer, and we have four bits per collider.
We could easily use four of these per block, which would make for a total of 16 bits of collision data.

Block behaviour is a whole other issue. These should be hard-coded event handlers for entering, exiting or clicking on a block with the given behaviour. Seeing that they're hard coded, you could use a simple list of behaviours and index that with a byte, but I think you could parametrize behaviours opcode style.

NPCs and Objects should probably get their own post, especially with the recent buzz around Entity-Component-System designs. For the time being, I think the most important bits are already covered.

Anyway, we won't need most of that meta data for quite a while. We will first focus on getting tile maps to render in the first place, then we can worry about map objects, collision detection, the character controller, and all the other good stuff.
Personally, I'm excited this is finally packing some steam and look forward to continue work on this! Until then!

Self-Imposed Restrictions: The Sound of a Generation

As I hinted in my last post, I've been researching sound technology so I could finally complete the rule set simulating technical restrictions of the SNES and GBA. The fundamental problem I should have noticed the first time around is that, while the two systems had really similar graphical capabilities, their sound systems are radically different.

Which Generation, though?

The GBA had to be backwards compatible with the Game Boy and Game Boy Color, so it came with all the hardware of the GBC inside it, including its sound system: four analog channels, one for white noise, two for square waves with varying duty cycles, and one "programmable channel", which was essentially a four bit digital channel. It would have been a waste not to use these for GBA games as well, so all that was added for those were two digital 8 bit channels.
Compare that to the S-SMP, the audio processing unit in the SNES, which sported eight digital 16 bit channels, at 32 kHz, which is almost CD quality (which was never utilized, because of limited memory and lossy compression). A cheesy echo effect was famously added as well, which saw quite a bit of overuse.

The two consoles encouraged completely different approaches to sound design. One had all the utensils for high quality chip tunes, the other was designed for sampled music. Lo-fi samples, but samples nonetheless.
This is the reason people barely complained about the music in Donkey Kong Country 3, which had its sound track remade entirely for the GBA port, while almost everyone complained about the "shitty audio" in, say, Final Fantasy VI.

When designing the rules for graphical limitations, I cherry picked the better properties from both systems. My virtual controller has an X and Y button; my virtual screen is modeled after the SNES resolution; and I won't emulate the GBA's display, at least not by default.
The choice seems obvious: I'll design my rules around the S-SMP, leaving out GBA tech. Even if I decided to use a chip tunes sound track instead - you can make some kick-ass tunes with eight sampled channels, even more so than with four analog and two sampled ones.

The Rules

The rules themselves are quite simple, actually:
Since there is only eight channels, you can't play more than eight different samples at a time. Since you need a channel or two for sound effects, that leaves you with two options: reserve a channel or two, or drop one out dynamically whenever an effect plays. I won't bother programming the latter, so we'll stick to the former: No more than six notes may be played at any one time in a song, though I there will be no limit to the number of sound effects playing at a time. It does, however, include the tails of notes: samples were generally chosen such that even when played in rapid succession, they wouldn't overlap with themselves.
The S-SMP's memory was also limited to a mere 64 Kilobytes, and after compression, 9 bytes of data in there correspond to 32 bytes of sample data. Leaving some space for instructions, this means that no more than 110 Kilobytes of sample data are permitted per song.
With so little memory, multisampling, i.e. using multiple samples of the same instrument playing different notes, wasn't really done. So only one sample per instrument. On the other hand, sounds based purely on white noise, such as a synthesized snare drum, do not count towards the data total, as the S-SMP came with a noise synthesizer.
As to the quality of the samples, the SNES could play back any sample rate up to 32 kHz, but because of the strict memory limitations, none of the games I looked at used sample rates above 8 kHz. Also noteworthy is the compression algorithm used: Bit Rate Reduction produces very unique, audible artifacts. Luckily, I found a tool that BRR compresses sample data.
We'll also make good use of the famous echo effect. It could be enabled independently for all channels, but they had to share the same settings. The delay could be set to any multiple of 16ms from 16 to 240ms. Even echo feedback is supported.
Finally, the eight channels could be "mastered": There's registers for pitch modulation and ADSR envelopes, as well as panning and volume.

With all that in mind, I'll use a tracker to compose the music and render it into uncompressed *.wav files, and possibly master them with an equalizer. This would be wholly unauthentic, but it's what Shovel Knight did, and that game had a great sound.
Either way, this is entirely in asset production, so my sound engine code can be quite simple, and I can focus on other things, which is always good.

Problem is that I don't know jack about music theory, so I'll use inauthentic place holder sounds until I (a) find someone willing to work under these conditions; or (b) learn music theory.