LuaGhost for Spyro the Dragon #

Written 2023-04-24 — Updated 2023-05-18

LuaGhost is a Lua script I created that runs on the BizHawk emulator that adds ghost recording and playback functionality to the PlayStation 1 game Spyro the Dragon. Think Mario Kart, but you’re racing against pre-recorded ghosts of previous speedruns.

I built it for two reasons.

First, I was a speedrunner at the time, looking to get back into running Spyro games, and I’d been wanting a practice tool like this for years. So I made one.

Second, I was streaming on Twitch and had a small audience that didn’t know much about Spyro speedruns. I hoped putting ghosts on screen would create some excitement for the audience. You don’t have to know all the particulars of the route or the game’s physics engine to watch overtakes happening on screen.

Obviously, you can’t submit a run to any normal leaderboard if you run while using LuaGhost. I found I enjoyed running against ghosts and didn’t care much about leaderboards, so I never went back to doing official speedruns. But I know others in the community who used the tool for practice to help improve their official times.

Here’s the promo video I made to show people what to expect from the script:

And here’s a completed run I did. LuaGhost has the ability to show ghosts of full runs or individual levels. For this run, I had the individual level ghosts turned off, so just the full game ghosts are showing:

World Space and Frame Buffers #

I’d been hacking the game and learning Lua scripting by this point long enough that I knew extracting Spyro’s position each frame would be easy. Without looking it up, and having not worked on it in over nine months, I can tell you that Spyro’s x coordinate is stored at memory address 0x078A58. I could easily extract Spyro’s coordinates 30 times per second and figure out how to save them and recall them later. Some compression would probably be a good idea, but I knew that would be solvable.

But how do I show the locations of the ghosts? At that time, I didn’t have enough understanding of the game’s rendering engine to have a hope of injecting custom geometry. So I needed some other way of making it work.

This is where the emulator comes in. BizHawk lets scripts print text and draw primitive shapes to an overlay that sits on top of the game. It’s good for printing debug information and I’ve seen people use it to draw hitboxes around sprites in 2D games. It’s not very efficient. It lags badly if you draw more than a few lines. But that’s enough to get some simple wire frames drawn onto the screen.

However, BizHawk doesn’t know anything about the world or the camera. The overlay drawing functions all work in screen space.

I won’t go into the details because it’s messy and I’m sure I didn’t do it in the best way, but I eventually figured out how to extract the camera position from the game and convert coordinates from world space into screen space! To test it, I made a script that drew a wireframe outline around some static world geometry I already knew the coordinates of. It took some tweaking to get the aspect ratio right, but I got it working. Almost.

Whenever I moved Spyro or the camera, it went all out of sync and wouldn’t line up again until I stopped moving. When I realized that, I was worried the project wouldn’t be able to progress. It really was very far out of sync and there’s no way that amount of lag would be acceptable. It looked weird, uncanny even, but it seemed BizHawk had some serious delay built into its overlay renderer.

After messing with it for a while, trying to figure out a way around it, I realized why it looked uncanny. The wireframes weren’t trailing behind the camera. They were jumping ahead of it.

BizHawk actually renders its overlay faster than the game. Spyro the Dragon uses a frame buffer where it draws onto one image while a different image is being sent to the display. When a frame is done, the game switches switches which image it’s drawing to. Each image also gets drawn twice to the screen, but that’s just some interlacing nonsense we can safely ignore. Effectively, the game runs at 30 frames per second. But I’m getting sidetracked. The point is that I’m drawing wireframes based on extracted game data, but it will take two frames before that data gets drawn to the screen by the game.

So I created a buffer and started tracking camera positions over several frames. I’d render each frame using old camera positions. At this point, I still wasn’t certain how much delay I needed, so I played with the buffer size until it worked.

When I hit “run” for the first time and my test wireframes stayed perfect synced with the level geometry, even through camera moves, that was a moment of such profound relief. I was giddy with excitement. I was still almost 0% of the way through what I knew would be a massive project, but I had solved the only part I wasn’t certain was possible. I paused the game and danced around my room to get my excitement out. This project was going to work!

Then I went back to my computer and unpaused the game. The wireframes were immediately out of sync again. Um…

Interlacing #

The amount of buffer I needed seemed to vary. I variably needed to delay for two or three frames and I couldn’t figure out why. Several days of debugging later, I finally worked out what was going on. Spyro doesn’t actually run at 30 frames per second.

The game expects to be drawing the picture to a tv running at 60 fps, but it’s interlaced. The game renders only the even scanlines on one frame and then only the odd scanlines on the next. Theoretically, the game could render half an image 60 times per second and have the game run at a true 60 fps with half the detail missing. Spyro doesn’t bother with that. It renders a full image every 30th of a second and then sends it to the tv over two frames at 60 fps, at least normally.

What happens if two 60ths of a second pass and the game doesn’t have the next image ready yet? That can happen if there’s too much going on in the game. I assumed it would just render the old image for another two frames. That made sense. It didn’t even occur to me that the game wouldn’t stay in sync with the parity of the interlaced display.

It turns out Spyro doesn’t care. If it takes three frames to draw the next image, then the last image gets drawn for three frames. Each image is drawn for a minimum of two frames (at 60 fps), but there’s nothing forcing that number to be even. One moment, each image is having its even frames drawn first. Then there’s a lag frame and the next moment each image is drawing its odd frames first.

Testing became easier when I realized that pausing and unpausing the game always generates an odd number of lag frames, which gave me a way of swapping the parity on command.

Once I figured all that out, I was able to quickly fix my renderer to take account of the current parity. Then the wireframes stayed in sync.

The result of this was that I had to do some serious thinking about how I was going to record and play back ghosts. What if the player and the ghost got different amounts of lag? In the end, it only required a small change. The plan was always to keep the file sizes small by discarding as many frames as possible and interpolating between the remaining keyframes during playback. I just had to ensure each keyframe included the correct number of 60 fps frames since the previous keyframe.

Sidenote: Buffers and Latency #

Throughout the rest of the project, I was careful to ensure I delayed everything to stay in sync with the frame that was currently being drawn to the screen. That meant delaying the camera movements, but also delaying the positions of the ghosts.

By this time, I was digging into the internal working of the game’s renderer enough that I could create effects such as disabling rendering of level geometry or enemies or messing with levels of detail. On a whim, I disabled Spyro and placed a ghost at his current coordinates in each frame. The camera position was still delayed to sync up with the world as it was being drawn to the screen, but I didn’t delay Spyro’s position data.

It’s weird how we get used to things. I hadn’t realized how much of a delay there was between me doing anything on the controller and Spyro responding to it. I had spent enough time moving back and forth between running on console and emulator that I knew emulator had a little more lag on it. I was used to adjusting my input timings to compensate for the different amounts of delay. Bypassing the frame buffer completely changed that.

When I pressed X, Spyro jumped. He just jumped. No delay I could perceive. Seeing Spyro move without waiting for the frame buffer was shocking to me. It was a bit jarring to look at since the camera couldn’t keep Spyro centered on the screen as well as it normally did. Never before, on emulator or on console, had Spyro seemed to react to my thoughts instead of my hands.

It made me sad. I knew I’d have to turn the game’s rendering of Spyro back on and go back to waiting for Spyro to respond to my inputs. Having had a taste of what it was like to play with no frame buffer, I’d always regret not being able to play like that all the time.

There are emulators that can do something called “run ahead” which can remove the frame buffer. I’ve done it for playing Super Mario World romhacks, but I don’t currently know of any emulators that can do it for PlayStation 1 games. I’d love to be able to do Spyro speedruns without the frame buffer, even if the runs wouldn’t be valid for leaderboards. Maybe one day.

Spyro’s a PAL #

I keep throwing the numbers 30 and 60 out when talking about the frame rate. But Spyro was also released for PAL systems, which run at 50 fps interlaced. I decided early in development that I wanted to ensure LuaGhost would run on either system. There are many PAL runners in the Spyro community, but they get overshadowed a lot since PAL is slower to run than NTSC (the 60 fps version). Accessibility is something I consider very important, so I wanted LuaGhost to work for everyone. Also, most of the world runs on 50Hz power, so it sucks that the USA dominates many speedrunning communities.

Interestingly, Spyro actually runs faster on flat ground on PAL. A bug was introduced into the PAL version of the game when porting it that allows Spyro to run about 20% faster on flat ground (I’m trying to remember the number from studying this a long time ago, so my memory may be off). PAL loads data from disc far more slowly. The loads are so much slower that it completely wipes out the advantage PAL runners would otherwise have.

If the community ever created a loadless timer for timing speedruns, PAL players would destroy the Americans. The speed bug also introduces some complexity that makes PAL movement more interesting to optimize, but I digress.

It can be demoralizing to run LuaGhost on PAL against NTSC ghosts that gain time for free at every loading point. But being able to cruise past them on the straitaways is very satisfying.

A Conclusion, I Guess #

I think that’s all the interesting points. The rest was just integrating everything together and building the UX. It’s the first time I’ve released a project this large to the public. I was worried it wouldn’t work somehow. I was terrified people were going to spend hours recording ghosts and then the ghosts would get corrupted somehow. I spent a lot of time testing to make it as reliable as possible. I didn’t have any major problems on release, so that was a relief.

Thank you for reading and have a nice day!