ā† kevin.md

šŸ¤‘ I Put Smash Bros Melee in My Living Room

Ā· 13 min read Ā· meleemeta-questmixed-realityreverse-engineeringslippiai

On September 7, I saw that Super Smash Bros. Meleeā€™s decompilation had reached 100%. By that night, I was wearing a Meta Quest and watching Fox and Marth fight in my apartment. I posted a screen recording, and within a couple of days it had passed a hundred thousand views.

The part I wanted to build was pretty simple to describe: take a real Melee match, put the stage in my room, and let me watch it from wherever I wanted. A little tournament on the table. Or make it huge and let me stand beside the fighters. Iā€™ve watched games through a screen my whole life; I wanted to see what happened when I could move around one.

Fox and Marth fighting above my living-room table, with the couch and room visible behind them

A later version of the experiment: Fox and Marth fighting over my table. Watch the demo on YouTube.

What the 100% means

When I checked the Melee decompilation tracker in September, it read 100%, fully linked. The community has reconstructed C code that compiles to match the gameā€™s original PowerPC assembly. Thatā€™s a much stricter target than writing a game that looks or behaves similarly: the compiled instructions themselves have to match.[1]

That gives people a way to read, study, and modify the machinery inside a game from 2001. It doesnā€™t recover all the original variable names or comments, and the resulting project still targets the GameCube. A native port requires more workā€”the decompā€™s own README says so.[2]

For my project, the milestone was the reason to start looking. Much of the work I ended up using already existed before the counter reached 100%. Slippi had the replays, modding tools could read the assets, and the decomp had readable tables and functions to consult. The interesting opportunity was connecting those pieces to a different kind of display.

Iā€™ve also been looking at what people are doing with other games from that era. Modern Warfare 2 has projects such as OpenIW4, which reconstructs engine functions, and IW4L, an experimental replacement runtime that loads the original gameā€™s assets. Halo 3 has recompilation work targeting a 2007 pre-release build; Halo: Combat Evolved has a separate decompilation research project. These efforts have different goals and levels of completion, but they make me wonder what else I could do with games I grew up playing once their internals become accessible.[8]

Getting the match into the headset

I started by brainstorming with Claude. It suggested mods, physics visualizations, analysis tools, and a VR replay spectator. That last one was the one I wanted to see. I took the conversation into Codex, using Astra, and asked it to build the Quest app. The agents did the bulk of the implementation; I picked the direction, tried the builds in the headset, and kept pointing out what looked or felt wrong.

The scope was one match: Mangoā€™s Fox against Zainā€™s Marth on Final Destination, from the Smash Summit 11 replay archive. I wanted something recognizable, and people in the replies recognized it. Slippi publishes tournament replay bundles, and its parser exposes the recorded frame data I needed to drive the scene.[3]

There were two inputs to bring together. The replay supplied the fightersā€™ positions, facing directions, and action states over time. A local game disc image supplied the models, textures, skeletons, and animations. An extractor built around Ploajā€™s HSDLib read the gameā€™s asset files on my Mac and converted them into data Unity could load.[4] The app then placed the characters at their recorded positions and played the corresponding animations.

This is where the decomp helped directly. An action-state number in a replay has to become the right animation on the right character. I derived those mappings from the decompiled common, Fox, and Marth motion-state tables. Later, when something looked wrong, the source gave me places to investigate: animation timing, model visibility, sword trails, and Foxā€™s special moves.[5]

The first version rendered that reconstructed scene in Unity. The match followed the replay, so I could change my view but not what the fighters did. It wasnā€™t running the full Melee engine. That distinction became more interesting once I started adding live battles.

Fox and Marth fighting on a miniature Final Destination in front of my kitchen, captured through the Quest headset

A frame from the first recording. The early stage floated in the room; fitting it to actual furniture came next.

The first build had the characters moving around the stage, but it was silent and missing a lot of what makes a hit feel like a hit. My feedback was essentially: good start, where are the sounds and all the effects? I added Marthā€™s sword trails, Foxā€™s lasers and special-move effects, hit flashes, landing particles, and the animated stage background. For audio, I rendered the same replay through Slippi Playback on the Mac and brought its game soundtrack into the viewer.[6]

Once those pieces were together, I could pause, scrub through the match, slow it down, or change between tabletop, ringside, and on-stage views. A move I would normally watch from the gameā€™s camera could happen right in front of me, with my kitchen still visible behind it.

The replies became the next build

People immediately started suggesting where theyā€™d put it: a coffee table, a shelf, a tournament setup. Several wanted to know how to try it. Others pointed out the audio sync. I liked how quickly the feedback moved from the novelty of seeing Melee in a room to the practical question of where the stage should sit.

That exposed the difference between placing an object somewhere in space and making it belong in a room. I added placement using the Questā€™s scanned room surfaces through Metaā€™s Mixed Reality Utility Kit.[7] The app now selects the closest suitable table, fits the stage to its bounds, and lets me cycle through tables or reposition it with a pinch and drag. Even ā€œclosest tableā€ needed care: the near edge of a long counter can be closer than the center of a small table across the room.

I posted another clip on September 8 showing the tabletop stage, dragging it around, and switching between MR and VR views. This was the next version people could actually see, with changes prompted by the first videoā€™s replies.

There was also a mysterious cylinder sticking through the stage in the VR views. It turned out to be part of Final Destinationā€™s surrounding background, accidentally shrunk to four percent of its intended scale. A tunnel meant to surround the arena had become an obstruction inside it. Restoring its scale fixed the cylinder. Changing the camera makes some very strange mistakes impossible to miss.

The audio complaint took more investigation. I wasnā€™t hearing the lag while wearing the headset, but the native recordings showed audio arriving roughly 140 milliseconds late. I tested audio timing and compared recordings; the captured offset persisted even with a different audio-output path. I could correct the exported video by shifting its audio, but the recording issue remains something to investigate. A clip needs its own check, because thatā€™s the version everyone else gets to experience.

Then I asked to be one of the fighters

Once I could stand beside the match, I wanted to try the fighterā€™s perspective. I knew it might get dizzy quickly, but I wanted to see it anyway. I added three versions: a steadier camera that follows the fighter while keeping the horizon level, a raw version that follows the animated head, and a slow-motion version. I can briefly enter that view and return to the spectator camera.

That evening I posted the fighter-POV experiment too. The follow-ups were getting less attention than the first clip, but they gave me a reason to keep trying things and showing what changed.

Then I noticed that the camera wasnā€™t turning when the character faced the other direction. That seemed like an obvious thing to fix, so I asked for it. I added the turns. It was way too jerky, and I asked to revert it.

I think that was the most useful experiment of the whole build. A fighter reversing direction is ordinary Melee movement. Having the view reverse around my head is a much bigger event. The steadier mode now keeps its initial heading and lets me look around for myself. I still want to explore the perspective, but faithfully following every turn made it harder to watch.

This was also a very concrete version of verification being the bottleneck. An agent can build a camera that follows a character, run checks on its transforms, and show me rendered frames. I still have to put the headset on to find out whether I want that camera attached to my face.

The table became the stage

Watching the replay made me want the fighters to react to the room. I added a separate Furniture Battle mode, where bots fight on the actual tabletop instead of replaying a recorded match. Fox and Marth were joined by Falco and Captain Falcon in a four-way fight, with stocks, respawns, ledge hangs, and signature moves like Falcon Punch.

The first pass looked promising, but its moves and sounds felt generic. I asked the agents to use the original attack data, animations, voices, and sound effects. That made a much bigger difference than adding another visual effect. I wanted to recognize the game in the way the fighters moved and hit each other.

Fox flipping through the air as Marth swings his sword above a wooden table

The furniture battle uses the table itself as the arena. Watch this version.

I kept changing what counted as a platform. A marked book could provide somewhere to land. I added a laptop mode so lifting and tilting the keyboard deck moved the fight surface. That exposed another problem: if the fighters drifted onto different planes, they could be standing near each other and still miss every attack. I had to keep the fight on a shared plane as the surface moved.

Then I tried coasters. The app detects their shapes and turns them into raised platforms, so moving the real objects rearranges the arena. I added tracking for movement, rotation, and overlap; it remains a prototype for this particular kind of object. A scanned table and a coaster sliding under my hand are very different tracking problems.

Becoming Master Hand

I also wanted to reach into the fight. I put the original Master Hand glove over my tracked hands, then added grabbing: pinch near a fighter, pick it up, move it around, and throw it back into the match.

My tracked hand wearing the Master Hand glove and holding Marth above colored coaster platforms

Holding Marth over the coaster arena. Watch the hand and coaster demo.

Getting the glove right took several attempts. At one point it was too small and its fingers were mangled; after that, the fingertips bent away from my palm. The model was the original asset, but fitting it to my hands still needed work. Eventually I had a glove I liked looking at while I grabbed the fighters.

Grabbing led to recognizable Master Hand attacks: finger shots, a palm slap, and a flying hand. The gestures trigger custom attacks using original assets and animation clips. I could watch four bots fight, then interfere with my own hands.

A fly brain, then a controller

One of the stranger suggestions was to let fruit-fly brain models control the fighters. I made it a separate Fox-versus-Falco mode, with two glowing neural displays hovering behind the arena. A local worker simulates two instances of a model built from the male fruit-fly connectome, and an adapter turns its outputs into game controls.[9]

The wiring comes from a real reconstruction; the simulated dynamics and control mapping are choices I made for the experiment. The watchable version also uses explicit combat assistance for facing and spacing. I havenā€™t taught a fly to play Melee, but I can watch modeled neural activity influence a fight on my table. Hereā€™s the footage.

By September 16, I had also added a mode where I could pick one of the four fighters and use a DualSense against the other three bots. This is a custom combat sandbox using Meleeā€™s assets and move data, with a subset of its mechanics. The replay viewer still follows the recorded match; controller play runs separately. Watch the controller demo.

Once the controller worked, I found another consequence of being able to walk around the game: left and right felt reversed when I watched from the opposite side of the table. I added movement and attack directions relative to my view. A fixed game camera had quietly been doing more work than I appreciated.

The decompilation community, Slippi, and the asset-tool authors supplied the foundations for this project. My experiment was to use those foundations, with AI coding agents, to try a way of watching that I could describe much faster than I could implement myself. The game data stays local. It is still a prototype, but it has grown from one replay into several ways to watch, manipulate, and play with the same characters.

Iā€™d still like a replay library I can browse and watch with someone elseā€”pause a scramble, walk around it, then let it continue. But I also like how far the project wandered from that first idea. I started by putting a match in my room. A few days later I was picking up Marth with a giant glove and moving coasters around to change where the fighters could land.

Citations

[1] Melee progress tracker (checked September 9, 2026) and the project's explanation of matching decompilation. ā†©

[2] doldecomp/melee README: what can be done after decompiling Melee. The build targets US version 1.02; portability is a separate project. ā†©

[3] Slippi tournament replays and slippi-js. This prototype uses Game_20210718T092921.slp from the Summit 11 bundle. The older replay lacks player tags; I identified the matchup from the archive's set sequence and outcome. ā†©

[4] Ploaj's HSDLib, whose HSDRaw code is used in the offline extractor. melee-blender also served as an initial pipeline reference. ā†©

[5] Pinned decompilation sources: common motion states, Fox, and Marth. ā†©

[6] Slippi Playback 3.5.2, used locally to render music, voices, and sound effects from the same disc image and replay. ā†©

[7] Meta MRUK: managing and querying scene data. ā†©

[8] Project descriptions checked October 6, 2026: OpenIW4, IW4L, Halo 3 Delta debug recompilation (March 2007 build), and Halo: Combat Evolved decompilation research. Recompilation translates compiled code to another target; it is distinct from reconstructing matching source. ā†©

[9] Google Research: mapping the complete male fruit-fly brain. This is the wiring dataset behind the experiment, not a claim that it supplies a biologically faithful simulation or a trained game-playing agent. ā†©

Related posts
Projects Ā· GitHub Ā· š• Ā· Instagram Ā· TikTok Ā· Spotify Ā· LinkedIn Ā· Buy me a coffee