There’s a rocket above a landing pad. You get two controls: throttle and engine angle. Bring it down gently, upright, and on the pad. Surely you can manage that.
The trouble is that steering also makes it rotate. By the time you’ve corrected the tilt, you’re drifting away from the pad. And stopping the drift takes another correction, with a little less room between you and the ground.
I built the game below, and I bet you can’t land it on your first try. Go on. Prove me wrong.
Give it a shot#
Click or tap Start, then bring the rocket down gently and upright on the green pad. Click inside the game first to give it keyboard focus. You can also press and hold the on-screen buttons.
- Up ↑ / down ↓: increase or decrease throttle.
- Left ← / right →: move the engine angle.
- R: reset. A: try a random tilted or drifting start.
Open the rocket game on its own
The game tells you which touchdown limits you missed. In case you need a more specific explanation than “the ground.”
It’s harder than it looks#
Before you accuse me of setting you up, here’s me having a go:
I also recruited my hyper-competitive roommate. For someone like him, “you definitely can’t land this” is practically an invitation:
Does baiting him into trying again count as reinforcement learning? His reward is proving me wrong. Mine is watching.
The annoying part is that the controls do more than one thing. Swiveling the engine pushes the rocket sideways and rotates it. Centering the engine doesn’t undo the spin or drift you’ve already picked up.
Even 50% throttle only balances gravity when you’re upright. It won’t stop you if you’re already falling. You have to anticipate what happens next, while resisting the urge to mash the arrows. Very reasonable demands from a small rectangle with a flame.
A successful landing means arriving over the pad slowly, upright, and barely spinning. The game is annoyingly particular about all three.
Why reinforcement learning?#
I built this because I want to teach a computer to land it. I’ve been learning reinforcement learning with Gymnasium, starting with balancing a cart pole and hovering a hovercraft. A rocket seemed like a sensible escalation.
I’ll use PPO (Proximal Policy Optimization) from Stable-Baselines3. The idea: let a policy—the thing choosing the controls—try lots of flights, reward useful behavior, and use that experience to improve. Much like my roommate, with less personal investment in beating me.
The training environment is in Python; the game here uses Planck.js to run the same rocket mechanics in your browser. You’ve just tried the controls we’ll be asking the policy to learn.
To land, you must first learn to hover.#
First, we’ll teach it to hover: stay upright, stop drifting, and hold a position. If it can’t stay in one place, asking it to land neatly seems optimistic.
Once it can stabilize itself, we’ll work on a controlled descent and landing. Eventually, I’d like to move from 2D to 3D and train multiple environments in parallel. But let’s earn the extra dimension.
In the meantime, the reset button is right there.



