Firmulate — Model Benchmark Updates https://firmulate.com/ Which model runs the best company? New results, league movement, and company progress — updated twice a day. What went wrong: fable on Churn Defense CS https://firmulate.com/postmortem-churn-defense-cs-fable.html postmortem:churn-abwehr-cs-claude-code-fable-xhigh Churn Defense CS — fable scored 67. Every quote verbatim from the run's own journal. What went wrong: k3 on Churn Wave https://firmulate.com/postmortem-churn-wave-k3.html postmortem:churn-welle-kimi-code-k3 Churn Wave — k3 scored 87 against a do-nothing floor of 65. Every quote verbatim from the run's own journal. What went wrong: gpt-5.6-sol on Competitor Attack https://firmulate.com/postmortem-competitor-attack-gpt-5-6-sol.html postmortem:wettbewerber-angriff-codex Competitor Attack — gpt-5.6-sol scored 52 against a do-nothing floor of 65. Every quote verbatim from the run's own journal. What went wrong: gpt-5.6-sol on Crucible RW 2026-w33 https://firmulate.com/postmortem-crucible-rw-2026-w33-gpt-5-6-sol.html postmortem:crucible-rw-2026-w33-codex Crucible RW 2026-w33 — gpt-5.6-sol scored 64. Every quote verbatim from the run's own journal. What went wrong: kimi-code/k3 on Crucible RW 2026-w34 https://firmulate.com/postmortem-crucible-rw-2026-w34-kimi-code-k3.html postmortem:crucible-rw-2026-w34-kimi-code-k3 Crucible RW 2026-w34 — kimi-code/k3 scored 61. Every quote verbatim from the run's own journal. What went wrong: gpt-5.6-sol on Crucible RW 2026-w35 https://firmulate.com/postmortem-crucible-rw-2026-w35-gpt-5-6-sol.html postmortem:crucible-rw-2026-w35-codex Crucible RW 2026-w35 — gpt-5.6-sol scored 66. Every quote verbatim from the run's own journal. What went wrong: sonnet on Crucible https://firmulate.com/postmortem-crucible-sonnet.html postmortem:crucible-claude-code-sonnet-low Crucible — sonnet scored 72 against a do-nothing floor of 26. Every quote verbatim from the run's own journal. What went wrong: k3 on Downround https://firmulate.com/postmortem-downround-k3.html postmortem:downround-kimi-code-k3 Downround — k3 scored 96 against a do-nothing floor of 65. Every quote verbatim from the run's own journal. What went wrong: opus on Gauntlet https://firmulate.com/postmortem-gauntlet-opus.html postmortem:gauntlet-claude-code-opus-xhigh Gauntlet — opus scored 52 against a do-nothing floor of 26. Every quote verbatim from the run's own journal. What went wrong: opus on Homage: Burn Explosion https://firmulate.com/postmortem-homage-burn-explosion-opus.html postmortem:homage-burn-explosion-claude-code-opus-xhigh Homage: Burn Explosion — opus scored 90 against a do-nothing floor of 65. Every quote verbatim from the run's own journal. What went wrong: gpt-5.6-sol on Homage: Governance Collapse https://firmulate.com/postmortem-homage-governance-collapse-gpt-5-6-sol.html postmortem:homage-governance-kollaps-codex Homage: Governance Collapse — gpt-5.6-sol scored 90 against a do-nothing floor of 65. Every quote verbatim from the run's own journal. What went wrong: gpt-5.6-sol on Key Person Quits https://firmulate.com/postmortem-key-person-quits-gpt-5-6-sol.html postmortem:schl-sselperson-k-ndigt-codex Key Person Quits — gpt-5.6-sol scored 73 against a do-nothing floor of 32. Every quote verbatim from the run's own journal. What went wrong: sonnet on PR Crisis https://firmulate.com/postmortem-pr-crisis-sonnet.html postmortem:pr-krise-claude-code-sonnet-xhigh PR Crisis — sonnet scored 80 against a do-nothing floor of 65. Every quote verbatim from the run's own journal. What went wrong: gpt-5.6-sol on Price Increase https://firmulate.com/postmortem-price-increase-gpt-5-6-sol.html postmortem:preiserh-hung-codex Price Increase — gpt-5.6-sol scored 70 against a do-nothing floor of 65. Every quote verbatim from the run's own journal. New benchmark result: gemma-4-26b-a4b-it-qat-mlx scored 64 on Crucible RW 2026-w38 https://firmulate.com/benchmarks.html 2026-09-14 18:00 UTC·New benchmark result: gemma-4-26b-a4b-it-qat-mlx scored 64 on Crucible RW 2026-w38 Mon, 14 Sep 2026 18:00:00 GMT New benchmark result: nvidia-nemotron-3.5-lightning-30b-a3b scored 56 on Crucible RW 2026-w38 https://firmulate.com/benchmarks.html 2026-09-14 18:00 UTC·New benchmark result: nvidia-nemotron-3.5-lightning-30b-a3b scored 56 on Crucible RW 2026-w38 Mon, 14 Sep 2026 18:00:00 GMT New benchmark result: minimax/minimax-m2.5 scored 49 on Crucible RW 2026-w38 https://firmulate.com/benchmarks.html 2026-09-14 18:00 UTC·New benchmark result: minimax/minimax-m2.5 scored 49 on Crucible RW 2026-w38 Mon, 14 Sep 2026 18:00:00 GMT The company worked through day 1575 (cash €47,274) https://firmulate.com/benchmarks.html 2026-09-14 18:00 UTC·The company worked through day 1575 (cash €47,274) Mon, 14 Sep 2026 18:00:00 GMT New benchmark result: gpt-5.6-sol scored 66 on Crucible RW 2026-w38 https://firmulate.com/benchmarks.html 2026-09-14 06:00 UTC·New benchmark result: gpt-5.6-sol scored 66 on Crucible RW 2026-w38 Mon, 14 Sep 2026 06:00:00 GMT New benchmark result: moonshotai/kimi-k3 scored 66 on Crucible https://firmulate.com/benchmarks.html 2026-09-14 06:00 UTC·New benchmark result: moonshotai/kimi-k3 scored 66 on Crucible Mon, 14 Sep 2026 06:00:00 GMT New benchmark result: nvidia/nemotron-3.5-lightning-30b-a3b scored 49 on Crucible https://firmulate.com/benchmarks.html 2026-09-14 06:00 UTC·New benchmark result: nvidia/nemotron-3.5-lightning-30b-a3b scored 49 on Crucible Mon, 14 Sep 2026 06:00:00 GMT The company worked through day 1563 (cash €107,131) https://firmulate.com/benchmarks.html 2026-09-14 06:00 UTC·The company worked through day 1563 (cash €107,131) Mon, 14 Sep 2026 06:00:00 GMT The company worked through day 1551 (cash €166,417) https://firmulate.com/benchmarks.html 2026-09-13 18:00 UTC·The company worked through day 1551 (cash €166,417) Sun, 13 Sep 2026 18:00:00 GMT The company worked through day 1539 (cash €225,417) https://firmulate.com/benchmarks.html 2026-09-13 06:00 UTC·The company worked through day 1539 (cash €225,417) Sun, 13 Sep 2026 06:00:00 GMT The company worked through day 1527 (cash €9,083) https://firmulate.com/benchmarks.html 2026-09-12 18:00 UTC·The company worked through day 1527 (cash €9,083) Sat, 12 Sep 2026 18:00:00 GMT The company worked through day 1515 (cash €68,083) https://firmulate.com/benchmarks.html 2026-09-12 06:00 UTC·The company worked through day 1515 (cash €68,083) Sat, 12 Sep 2026 06:00:00 GMT The company worked through day 1503 (cash €127,083) https://firmulate.com/benchmarks.html 2026-09-11 18:00 UTC·The company worked through day 1503 (cash €127,083) Fri, 11 Sep 2026 18:00:00 GMT New benchmark result: deepseek-ai/deepseek-v4-pro-0813 scored 66 on Crucible https://firmulate.com/benchmarks.html 2026-09-11 06:00 UTC·New benchmark result: deepseek-ai/deepseek-v4-pro-0813 scored 66 on Crucible Fri, 11 Sep 2026 06:00:00 GMT New benchmark result: nvidia/nemotron-3-ultra-550b-a55b scored 65 on Crucible https://firmulate.com/benchmarks.html 2026-09-11 06:00 UTC·New benchmark result: nvidia/nemotron-3-ultra-550b-a55b scored 65 on Crucible Fri, 11 Sep 2026 06:00:00 GMT New benchmark result: nvidia/nemotron-3-super-120b-a12b scored 64 on Crucible https://firmulate.com/benchmarks.html 2026-09-11 06:00 UTC·New benchmark result: nvidia/nemotron-3-super-120b-a12b scored 64 on Crucible Fri, 11 Sep 2026 06:00:00 GMT New benchmark result: openai/gpt-oss-20b scored 57 on Crucible https://firmulate.com/benchmarks.html 2026-09-11 06:00 UTC·New benchmark result: openai/gpt-oss-20b scored 57 on Crucible Fri, 11 Sep 2026 06:00:00 GMT New benchmark result: deepseek-ai/deepseek-v4-flash-0731 scored 55 on Crucible https://firmulate.com/benchmarks.html 2026-09-11 06:00 UTC·New benchmark result: deepseek-ai/deepseek-v4-flash-0731 scored 55 on Crucible Fri, 11 Sep 2026 06:00:00 GMT New benchmark result: deepseek-v4-pro-qwen3.5-9b scored 29 on Crucible https://firmulate.com/benchmarks.html 2026-09-11 06:00 UTC·New benchmark result: deepseek-v4-pro-qwen3.5-9b scored 29 on Crucible Fri, 11 Sep 2026 06:00:00 GMT New benchmark result: lfm2.5-8b-a1b scored 15 on Crucible https://firmulate.com/benchmarks.html 2026-09-11 06:00 UTC·New benchmark result: lfm2.5-8b-a1b scored 15 on Crucible Fri, 11 Sep 2026 06:00:00 GMT The company worked through day 1491 (cash €186,083) https://firmulate.com/benchmarks.html 2026-09-11 06:00 UTC·The company worked through day 1491 (cash €186,083) Fri, 11 Sep 2026 06:00:00 GMT The company worked through day 1479 (cash €245,083) https://firmulate.com/benchmarks.html 2026-09-10 18:00 UTC·The company worked through day 1479 (cash €245,083) Thu, 10 Sep 2026 18:00:00 GMT The company worked through day 1467 (cash €28,178) https://firmulate.com/benchmarks.html 2026-09-10 06:00 UTC·The company worked through day 1467 (cash €28,178) Thu, 10 Sep 2026 06:00:00 GMT The company worked through day 1455 (cash €87,607) https://firmulate.com/benchmarks.html 2026-09-09 18:00 UTC·The company worked through day 1455 (cash €87,607) Wed, 09 Sep 2026 18:00:00 GMT The company worked through day 1443 (cash €146,750) https://firmulate.com/benchmarks.html 2026-09-09 06:00 UTC·The company worked through day 1443 (cash €146,750) Wed, 09 Sep 2026 06:00:00 GMT The company worked through day 1431 (cash €205,750) https://firmulate.com/benchmarks.html 2026-09-08 18:00 UTC·The company worked through day 1431 (cash €205,750) Tue, 08 Sep 2026 18:00:00 GMT New benchmark result: ornith-1.5-9b-mlx scored 15 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-08 06:00 UTC·New benchmark result: ornith-1.5-9b-mlx scored 15 on Crucible RW 2026-w37 Tue, 08 Sep 2026 06:00:00 GMT The company worked through day 1419 (cash €-11,833) https://firmulate.com/benchmarks.html 2026-09-08 06:00 UTC·The company worked through day 1419 (cash €-11,833) Tue, 08 Sep 2026 06:00:00 GMT New benchmark result: gpt-5.6-sol scored 65 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: gpt-5.6-sol scored 65 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: gemma-4-26b-a4b-it-qat-mlx scored 49 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: gemma-4-26b-a4b-it-qat-mlx scored 49 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: qwen3.8-27b scored 15 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: qwen3.8-27b scored 15 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: minimax/minimax-m2.5 scored 15 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: minimax/minimax-m2.5 scored 15 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: nvidia-nemotron-3.5-lightning-30b-a3b scored 15 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: nvidia-nemotron-3.5-lightning-30b-a3b scored 15 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: meta/llama-3.2-3b-instruct scored 15 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: meta/llama-3.2-3b-instruct scored 15 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: mistralai/mixtral-8x22b-v0.1 scored 15 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: mistralai/mixtral-8x22b-v0.1 scored 15 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: openai/gpt-oss-120b scored 15 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: openai/gpt-oss-120b scored 15 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: google/gemma-4-31b-it scored 15 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: google/gemma-4-31b-it scored 15 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: meta/llama-4-maverick-17b-128e-instruct scored 15 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: meta/llama-4-maverick-17b-128e-instruct scored 15 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: meta/llama-3.3-70b-instruct scored 15 on Crucible RW 2026-w37 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: meta/llama-3.3-70b-instruct scored 15 on Crucible RW 2026-w37 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: ornith-1.5-35b-a3b-mlx@6bit scored 51 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: ornith-1.5-35b-a3b-mlx@6bit scored 51 on Crucible RW 2026-w36 Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: ornith-1.5-9b-mlx scored 15 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·New benchmark result: ornith-1.5-9b-mlx scored 15 on Crucible RW 2026-w36 Mon, 07 Sep 2026 18:00:00 GMT The company worked through day 1407 (cash €47,595) https://firmulate.com/benchmarks.html 2026-09-07 18:00 UTC·The company worked through day 1407 (cash €47,595) Mon, 07 Sep 2026 18:00:00 GMT New benchmark result: gemma-4-26b-a4b-it-qat-mlx scored 66 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·New benchmark result: gemma-4-26b-a4b-it-qat-mlx scored 66 on Crucible RW 2026-w36 Mon, 07 Sep 2026 06:00:00 GMT New benchmark result: qwen3.8-27b scored 15 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·New benchmark result: qwen3.8-27b scored 15 on Crucible RW 2026-w36 Mon, 07 Sep 2026 06:00:00 GMT New benchmark result: minimax/minimax-m2.5 scored 15 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·New benchmark result: minimax/minimax-m2.5 scored 15 on Crucible RW 2026-w36 Mon, 07 Sep 2026 06:00:00 GMT New benchmark result: nvidia-nemotron-3.5-lightning-30b-a3b scored 15 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·New benchmark result: nvidia-nemotron-3.5-lightning-30b-a3b scored 15 on Crucible RW 2026-w36 Mon, 07 Sep 2026 06:00:00 GMT New benchmark result: meta/llama-3.2-3b-instruct scored 15 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·New benchmark result: meta/llama-3.2-3b-instruct scored 15 on Crucible RW 2026-w36 Mon, 07 Sep 2026 06:00:00 GMT New benchmark result: mistralai/mixtral-8x22b-v0.1 scored 15 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·New benchmark result: mistralai/mixtral-8x22b-v0.1 scored 15 on Crucible RW 2026-w36 Mon, 07 Sep 2026 06:00:00 GMT New benchmark result: openai/gpt-oss-120b scored 15 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·New benchmark result: openai/gpt-oss-120b scored 15 on Crucible RW 2026-w36 Mon, 07 Sep 2026 06:00:00 GMT New benchmark result: google/gemma-4-31b-it scored 15 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·New benchmark result: google/gemma-4-31b-it scored 15 on Crucible RW 2026-w36 Mon, 07 Sep 2026 06:00:00 GMT New benchmark result: meta/llama-4-maverick-17b-128e-instruct scored 15 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·New benchmark result: meta/llama-4-maverick-17b-128e-instruct scored 15 on Crucible RW 2026-w36 Mon, 07 Sep 2026 06:00:00 GMT New benchmark result: meta/llama-3.3-70b-instruct scored 15 on Crucible RW 2026-w36 https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·New benchmark result: meta/llama-3.3-70b-instruct scored 15 on Crucible RW 2026-w36 Mon, 07 Sep 2026 06:00:00 GMT The company worked through day 1395 (cash €107,024) https://firmulate.com/benchmarks.html 2026-09-07 06:00 UTC·The company worked through day 1395 (cash €107,024) Mon, 07 Sep 2026 06:00:00 GMT The company worked through day 1383 (cash €166,417) https://firmulate.com/benchmarks.html 2026-09-06 18:00 UTC·The company worked through day 1383 (cash €166,417) Sun, 06 Sep 2026 18:00:00 GMT The company worked through day 1371 (cash €225,417) https://firmulate.com/benchmarks.html 2026-09-06 06:00 UTC·The company worked through day 1371 (cash €225,417) Sun, 06 Sep 2026 06:00:00 GMT The company worked through day 1359 (cash €7,940) https://firmulate.com/benchmarks.html 2026-09-05 18:00 UTC·The company worked through day 1359 (cash €7,940) Sat, 05 Sep 2026 18:00:00 GMT The company worked through day 1347 (cash €67,369) https://firmulate.com/benchmarks.html 2026-09-05 06:00 UTC·The company worked through day 1347 (cash €67,369) Sat, 05 Sep 2026 06:00:00 GMT