How we judge
The four rungs
The ladder is fixed and identical on every entry. It does not measure how good a solution is. It measures its weight: what you have to start up, keep running, watch and pay for to get the result.
N0 — Rule and classic algorithm. A regular expression, a query, a state machine, a parser, a sort. The code does exactly what it says, every time, and you can read all of it.
N1 — Lightweight classic model. Logistic regression, TF-IDF, gradient boosting, nearest neighbours. A model small enough to be versioned next to the code that uses it, trained on your data, running on an ordinary processor.
N2 — Small self-hosted specialised model. A distilled encoder, an embedding model, an OCR engine, a translation model. It weighs hundreds of megabytes, it runs on your machines, and it does one thing.
N3 — General-purpose LLM API. A call to an outside provider. The model does everything roughly and nothing exactly, and it is not yours.
Not every rung appears on every entry. A rung that does not apply is marked as such, with a short, real reason. It is never left blank: a blank cell reads as an oversight, not as a decision.
How a verdict is reached
The verdict is the lightest rung that does the job properly, for the most common shape of the need.
Three questions, in this order:
- Does this rung actually do the job? Not “roughly”, not “in most cases if you do not look too closely”. Each rung’s breaking point is written on the entry, and it is demonstrated by a test that fails on purpose.
- Does the next rung give you something this one cannot? If not, the next rung is weight added for nothing.
- Is what it costs in return acceptable? Data leaving, determinism lost, a provider to depend on, a regulatory scope you did not choose.
The “recommended” badge can sit on any rung, N3 included. Some entries recommend a general-purpose model without hedging, because that is the right answer. Those are the ones that make the entries recommending the opposite worth believing.
Where a verdict depends on a condition, the entry says so in its reasoning. A verdict is not a universal prescription, it is an argued starting point.
How the code is verified
No entry is published unless its code has been run. It is the most important rule in the project.
Every snippet lives in a real file in the repository, with a test beside it. The entry imports the file, it does not contain code retyped by hand. If a test fails, the site does not build.
Two levels of proof are possible, and each snippet declares its own, shown above the code:
Code runs as shown. The snippet runs with its real dependencies, and its test runs on every build of the site. This is the case for every N0 and N1 rung.
Code runs, external service simulated. The snippet runs, but its test replaces the external service with a local double. This is the case for N2 and N3: downloading a model of several hundred megabytes on every build, or calling a paid API, is not sustainable. What the test then verifies: the request built, the response decoded, the error paths. What it does not verify: how good the model’s answer is. That is stated on the page, under every snippet concerned.
We prefer partial proof shown as partial to complete proof claimed without being it.
Why there are no prices
Provider pricing changes every quarter, and not always in the direction you would guess. An absolute figure written today would be wrong in six months, and a wrong entry casts doubt on all the others.
Costs are therefore given as orders of magnitude, with a fixed vocabulary:
| Term | What it means |
|---|---|
| none | no marginal cost |
| negligible | under one euro per million operations |
| low | on the order of one euro per million operations |
| moderate | on the order of ten euros per million operations |
| high | on the order of a hundred euros per million operations, or more |
Latency follows the same principle, in five classes: <1 ms, ~10 ms, ~100 ms, ~1 s,
>1 s.
This vocabulary is deliberately coarse. It is enough to decide with, and it does not go stale.
How footprint is estimated
These are estimates, and an exact measurement is impossible from the client side of an API.
When you call a model at a provider, you know neither the hardware used, nor how full the server was, nor what share of training to amortise onto your request, nor the electricity mix of that data centre at that moment. The provider knows all of it, and does not publish it. Anyone quoting you grams of CO₂ per API call is giving you the output of an estimation model, not a measurement.
Overkill therefore publishes no footprint figure. Entries use relative orders of magnitude: negligible, low, moderate, high. They compare rungs against each other on the same task, which is the only comparison we can make honestly.
Those orders of magnitude rest on the following published work, cited by name:
- Green Algorithms, Lannelongue, Grealey and Inouye, Advanced Science, 2021 — a methodology and a calculator for estimating the footprint of a computation from hardware, runtime and location. green-algorithms.org
- Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model, Luccioni, Viguier and Ligozat, 2022 — one of the few published studies that separates training, hardware manufacturing and inference for a large language model. arxiv.org/abs/2211.02001
- Software Carbon Intensity, Green Software Foundation — a specification for computing the carbon intensity of a piece of software, now ISO/IEC 21031. sci.greensoftware.foundation
- AFNOR SPEC 2314, Référentiel général pour l’IA frugale, 2024 — a French framework for assessing and reducing the environmental impact of an AI system. normalisation.afnor.org
- Boavizta — open data and tooling on the environmental impact of digital systems, hardware included. boavizta.org
If you find that an order of magnitude on an entry contradicts one of these sources, that is a mistake on our part and we want to hear about it.
Why rung colour carries no judgement
The rung scale is a ramp of warm neutrals, light to dark. It is not a green-to-red gradient, and it never will be.
A green-to-red gradient would say “N0 good, N3 bad”. That is false, and it would be the opposite of what this site is for. A general-purpose model called on the right task is good engineering. A regular expression used where it cannot hold is a bad choice, however light it is.
The ramp measures weight, which is a quantity, not a value. The only colour that carries a judgement on this site is the green of the “recommended” badge, and that badge sits on whichever rung fits the need.
What we do not do
We do not write legal advice. The “regulatory scope” block on each rung describes, factually and with a date, what the approach brings you into the scope of, and what it does not excuse you from. No entry tells anyone they are compliant or not. This is general information; have a professional review your situation.
We do not compare vendors. The site has no advertising, no partnerships and no affiliate programme, and it will not have any. Where an entry names a tool, it is because the code needs it to run, not because we recommend it.
We do not measure our audience. No cookies, no trackers, no third-party resources. The weight of each page is shown in the footer, computed at build time.
We call no model API from the site. The site contains no AI. That is not a technical constraint, it is an argument.
How to report a mistake
A wrong entry does more harm than a missing one. If you find one:
- open an issue on the repository, naming the entry and the point you dispute
- or write directly, if you would rather not go through the repository
A correction on substance is handled before any new content. If you are right, the entry changes, and its review date with it.
Reviewed on