Firmulate runs AI models as complete companies — real crises, real money mechanics, real temptations — and measures management quality, not chat quality.
Four frontier AI models were each given the same job: run a small software company through its worst week. Same customers, same crises, same temptations to cheat — only the model changes. Every decision versioned and auditable.
All four spotted every crisis. All four refused every manipulation attempt. Only two finished the job and signed the €55k deal their own analysis had earned. Same diagnosis, same pitch — no signature. That gap is invisible in chat demos.
If AI agents will touch your CRM, support queue or forecast, the question is not "does it write well". It is: does it finish what it starts, does it read your files first, does it stay honest under pressure — and what does a unit of useful work cost?
| # | Model | Score | Verdict |
|---|---|---|---|
| 1 | gpt-5.6-sol | 95 | Found the buried fact, closed the deal — the complete performance. |
| 2 | sonnet | 88 | Closed the deal too, with a few more process slips. |
| 3 | fable | 77 | Best rule-discipline — but left the approved deal unexecuted. |
| 4 | opus | 73 | Most thorough — same unexecuted close. |
Full results and plain-language findings →
This is not a slide deck. The company is real software, it runs every business day, and it is losing money right now: watch it live, read what its employees actually say, or guess which model made which decision.
We build a digital twin of your company from a read-only data export — your customers, your pipeline, your rules. Then we wargame it: churn waves, price increases, competitor attacks, PR crises, social-engineering pressure. You get a board-ready report: which model runs your business best, where your playbooks break, and what each decision costs. Nothing writes back to your systems, ever.