how i added online co-op to my port of crimsonland
crimsonland is a 2003 game i reverse engineered from its stripped bi | Hanami
how i added online co-op to my port of crimsonland
crimsonland is a 2003 game i reverse engineered from its stripped binary and ported to compile to wasm. it was never written for netplay: its two-player mode assumes one machine. late in a survival run the world holds hundreds of creatures, projectiles, effects and decals. a full state is ~450 kb, so streaming it at 60 hz is not feasible.
i anticipated this early, so my port has bended the game to use a deterministic simulation. this gave us interesting properties like lightweight replay format and verified scores in the online leaderboards.
so for netplay we only send inputs. both browsers run the same game.wasm at fixed 60 hz ticks and exchange each player’s input per tick. that only works if the simulation is bit-exact everywhere. we pinned the transcendental math (musl), emulate the x87’s 24-bit precision the original ran with, keep msvc’s argument evaluation order for every rand() call, and route rendering-time rng to its own stream. the original also had real bugs here: an uninitialized float in two-player targeting, and a music latch that changed the rng stream depending on whether audio was on. all thoroughly fixed.
inputs arrive with a small delay: 2 ticks minimum, more if the round trip needs it. each machine runs up to 8 ticks ahead on a prediction of its partner’s input, then 4 after 15 minutes of survival. the original game has a blowup there with the creature spawn interval going negative. when a contradictory input arrives, we restore a snapshot and re-simulate. snapshots are raw memcpys into a 16-slot ring buffer, about 7 µs. simulating one tick usually takes 30 µs to 150 µs in the late game, so a full 12-tick rollback costs 1.8 ms, well inside a 16.7 ms frame budget.
side effects are the hard part. a rolled-back tick must leave no trace. sounds from re-simulated ticks stay quiet. blood and corpses only bake into the terrain once a tick has settled. the open perk menu stops the game from producing new ticks. in my interpretation whoever opens it picks, and both games stop at that exact tick. every confirmed tick’s state is hashed and compared, so any desync is caught at the tick where it happens.
a cloudflare durable object is the room: signaling, the invite link, and turn credentials. webrtc carries two datachannels. an unreliable one sends each tick’s inputs plus up to 8 unacked ones. a reliable one carries control messages and repairs gaps. if webrtc fails, the room’s websocket relays everything.
both machines write a byte-identical replay, and the server re-simulates it with the same core to verify the score. a replay header can be rewritten, so the room attests the pair: each player proves their key to the room, the room issues the run’s seed to exactly those two keys, and each player signs the replay for their seat. one partner can’t claim a run alone.
the tests boot two copies of the original game against each other over a seeded lossy, jittery pipe: 60 hz vs 144 hz, stalls, packet bursts, perk races, pause and leave. every confirmed tick must hash the same on both machines and when re-simulated from the recorded inputs, and v8, native and wasmtime builds agree snapshot for snapshot.
design doc with all the measurements: http://crimson.banteg.xyz/rewrite/network-coop
code: http://github.com/banteg/crimson