Platformer Slice Benchmark

Complete benchmark

AI models get the same platformer brief and three attempts to build a browser level. Play the outputs and compare feature scores with measured controls, visual quality, and failures found during playtesting.

AI benchmarkBrowser prototypePlatformer sliceLocal model testQwen3.5 122BQwen3.8 27BInkling Small

Current build

The delegated Qwen122 run reached the endpoint after three build-and-repair passes, scoring 74, 78, and 92 on the historical feature rubric.

Development details

Source code: Private

Built with

HTML CanvasPlaywrightTool-EvalLocal modelsQwen3.5 122BQwen3.8 27BInkling SmallBenchmark reports

Code repositories

platformer-slice-world1 benchmark reportsPrivate · Markdown / HTML · Last recorded update: 2026-07-25Local benchmark reports and generated browser platformer slices.

Benchmark

Benchmark results

Compare controls, visual quality, and generation speed, then play the builds below.

Test hardware

Unmarked runs used this MacBook Pro. Reference runs used temporary remote endpoints, so their throughput and wall time are left blank rather than compared with local measurements.

Machine: MacBook Pro
Chip: Apple M5 Max
Memory: 128 GB unified memory

Model	Size	Control & feel	Visual	Feature coverage	Responds to input	Idle survival	First build	Repair pass	Polish pass	Tokens/sec	Wall time
ds4-preview-256k-uncapped	91 GB284B total / 13B active	26/26	Strong	80/100	Yes	≥20s, blocked by a wall	75	74	80	19.3	1230.5s
claude-opus-5 (reference)	—frontier cloud model	26/26	Strong	71/100	Yes	≥20s	64	64	71	—	—
Reference run, not ranked — This reference received the same brief, three attempts, and text-only test feedback as the local runs. Its build implements gravity, friction, and axis-separated tile collision, but uses expressions and names the historical source scan did not recognize. That cost it 21 feature points across physics, collision, camera, and code style. Its Control & feel score of 26 and Strong visual tier show why the feature score needs to be read alongside the other checks.
ds4-100k-nothink	91 GB284B total / 13B active	22/26	Strong	79/100	Yes	≥20s	72	72	79	18.6	638.2s
qwen38-27b-mlx-8bit-vision	28 GB27.3B dense + 461M vision	22/26	Strong	79/100	Yes	≥25s	74	74	79	13.2	1158.9s
qwen38-flash-next-blackfrost	—180B total / 6B active	—	—	73/100	Yes	—	68	68	73	—	—
Reference run, not ranked — This remote run used the same Qwen3.8-Flash-Next weights as the clean variant, with the operator’s DERISKED system prompt. That prompt also requests terse, functional output, so this comparison changes more than safety instructions. The prompted run scored 73 and produced a 14,627-character game; the clean variant scored 61 and lost one attempt to unsupported tool calls. Local speed comparisons are omitted because the model ran remotely.
glm53-flash-derisked	—320B total / 18B active	—	—	69/100	Yes	—	64	64	69	—	—
Reference run, not ranked — The extractor initially gave this remote run three zeros by selecting a 53-character code fragment from roughly 100,000 characters of scratchpad text. Removing the scratchpad, selecting the fenced block containing the HTML game, and raising the output cap allowed the actual build to be evaluated. It scored 69 and moved 365 pixels under the recorded input test. The initial zeros described an extraction failure, not the game.
qwen38-flash-next-clean	—180B total / 6B active	—	—	61/100	No	—	69	0	61	—	—
Reference run, not ranked — This remote run used the same weights as the Blackfrost variant without that system prompt. Its second attempt emitted list_files and read_file calls instead of an HTML game, even though the benchmark provided no repository or tools to execute those calls. That attempt scored zero; the final attempt scored 61. The paired Blackfrost run did not emit those unsupported calls.
ds4-0731-256k-uncapped	91 GB284B total / 13B active	22/26	Adequate	78/100	Yes	≥20s	73	73	78	21.0	769.8s
qwen27-mtp-fast	27 GB27B	22/26	Adequate	75/100	Yes	≥20s	73	73	75	18.3	347.7s
step37-unsloth-iq4xs-text-mtp2-r2048	89 GB	22/26	Adequate	75/100	No	~5s	66	74	75	28.1	524.3s
step37-unsloth-iq4xs-vision-r2048	89 GB	22/26	Strong	71/100	No	does not start	62	68	71	23.4	522.6s
qwen122-q4xl-64k-mtp2-delegated	73 GB122B total / 10B active	20/26	Adequate	92/100	Yes	≥20s	74	78	92	20.0	829.6s
qwen122-q4xl-vision-64k-think	73 GB122B total / 10B active	20/26	Poor	75/100	No	≥20s, level empty	67	68	75	26.7	271.4s
qwen35-a3b-no-think	34 GB35B total / 3B active	19/26	Adequate	74/100	Yes	≥20s, hero never lands	47	67	74	51.6	115.6s
qwen122-q4xl-vision-64k	73 GB122B total / 10B active	18/26	Poor	74/100	No	under 3s	71	71	74	26.3	296.5s
nex-n2-mini-q8-vision-64k	—	Unresolved	Poor	74/100	No	under 3s	64	63	74	55.3	222.7s
inkling-small-iq3xxs-vision-64k-harness-sampling	91 GB276B total / 12B active	—	—	72/100	No	—	68	68	72	12.2	760.2s
inkling-small-iq3xxs-vision-64k	91 GB276B total / 12B active	Unresolved	Strong	63/100	No	under 3s	68	59	63	14.5	413.7s
laguna-s21-q4km-64k-think	75 GB	—	—	39/100	No	—	32	34	39	24.0	2833.6s

Control & feel /26 is measured by running each build, never by reading its source. A deterministic harness replaces the wall clock with a pumped virtual clock and seeds the random number generator, so repeat runs return identical numbers. It scores six things a player actually notices: whether the character moves both ways, whether the jump follows a real gravity curve, how many frames it takes to reach top speed, whether movement carries momentum, and whether holding jump longer jumps higher. Builds whose player state cannot be located are marked unresolved rather than guessed at.

Visual is a coarse tier, graded blind from screenshots by two independent models with owner tiebreaks. The raters agreed at only rho = 0.63, which does not support a finer number than three bands.

Feature coverage /100 is the original heuristic, kept for continuity and no longer the headline. It scans the source for the mechanics the brief asked for, so it rewards breadth of wording rather than how the game plays — a 414-byte file containing the right keywords and no game scores 97 on it. Its run-to-run noise is roughly six points.

Completability is notmeasured. A traversal oracle was built and works — it located one build’s impassable wall to within four pixels — but it can only read the level geometry of about a third of these builds, so it is withheld rather than published with two thirds of the column blank. The playable versions remain the honest comparison.

Playable Builds

Try a benchmark build

Choose a model to play the level it generated.

Loading playable build...

Screenshots

Four-stage contact sheet from the delegated Qwen122 platformer benchmark run — Current build
The Qwen122 run moved from a broken opening build to a completed route across three passes, scoring 74, 78, and 92.

Opening scene of the Kip and the Sunshards browser platformer benchmark — Current build
Kip enters Bramblebright Run with the first shard and reward route in view.

Win state of the Kip and the Sunshards browser platformer benchmark — Current build
The guided Playwright run reached the solar endpoint with no page errors.

Contact sheet comparing several AI-generated browser platformer benchmark outputs — Current build
Final outputs from local-model runs, including Laguna's blank canvas after three capped attempts.

Bar chart comparing full Tool-Eval scores for Qwen35, Laguna S 2.1, Qwen27, Step 3.7, and DS4 — Benchmark chart
Laguna scored 88 across the full 69-scenario Tool-Eval run: ahead of Qwen27, Step, and DS4, but behind Qwen35.

Bar chart comparing Laguna plain decoding and DFlash decoding speed — Benchmark chart
Plain 32K decoding averaged 37.2 tokens per second; DFlash was prompt-dependent and slower on the representative 64K workflow set.

Bar chart comparing three-shot platformer benchmark scores with Laguna S 2.1 — Benchmark chart
Laguna's long-form artifact failed despite three high-cap attempts, finishing at 39/100.

Blank Laguna S 2.1 platformer canvas with only a small HUD visible — Failed artifact
Laguna's final browser capture: the page loaded, but repeated empty tile data left the game canvas blank and non-interactive.

DS4 model browser platformer benchmark capture after playtest input — Current build
The DS4 build after automated movement input. The score alone does not establish whether the level can be completed.

Nex model browser platformer benchmark capture after playtest input — Current build
Playtest capture from the Nex run after automated movement input.

DeepSeek V4 Flash 0731 platformer build after movement input, showing the player advanced into the level — Current build
DeepSeek-V4-Flash-0731 with the output cap removed. It scored 78 against the preview’s 80, using about a third fewer tokens and a third less code.

DS4 preview platformer build showing a tall green column blocking the route — Failed artifact
The preview build scored 80 on the historical feature rubric but cannot be finished: an impassable column blocks the route. Manual playtesting found this after the build passed the input and idle-survival checks.

Inkling Small browser platformer benchmark capture after playtest input — Current build
Inkling Small's 63-point build after automated input. It is the more responsive of the model's two runs, but opening it by hand shows the level timer expiring almost immediately, so it reaches game over before the level can be played.

Inkling Small platformer capture from the higher-scoring run, showing a canvas that did not change after input — Failed artifact
The higher-scoring Inkling Small configuration finished at 72 but drew a static frame: the canvas did not change after movement, jump, or restart input in any of its three passes.

Qwen3.8 27B platformer build after movement input, showing the player on the grass with question blocks, brick platforms and three enemies ahead — Current build
Qwen3.8 27B scored 79 while loading 28 GB. Its collision check included an extra tile, so the floor repeatedly stopped the player’s movement. Correcting the two boundary expressions raises Control & feel from 22 to 26. Both the generated build and the corrected version are available below.

The task

Each model receives the same brief and three attempts to build and revise a browser platformer. The page includes the generated games, measurements, screenshots, and failure notes so you can inspect what the scores describe.

How to read the results

The historical feature score checks the generated code and can reward a build that plays badly. Control & feel measures movement under repeatable inputs; visual tiers come from screenshot review. Input-response and idle-survival checks catch additional failures, but neither proves that a level can be completed.

What the tests exposed

The delegated Qwen122 run reached 92/100 on the historical rubric and completed a guided playthrough. The DS4 preview scored 80 yet put an impassable column in the route. GLM initially received zeros because the extractor read a scratchpad fragment as the game; extracting the HTML correctly produced a score of 69. These cases show why I check both the build and the test harness.

Comparison limits

The delegated runs received focused repair briefs informed by cloud visual review and Playwright results. Later 256K uncapped runs used larger context and output allowances, and reference rows ran remotely. Those differences affect the comparison; the notes identify them rather than treating every score as an equivalent trial.

Return to city Ask about Platformer Slice Benchmark More from ML Tower