Write product descriptions
Write the sales text for every item in a catalogue, starting from its characteristics.
Yes, a general-purpose model earns its place
élevéAbout a secondYour data goes to a third party
The verdict
N3, and no apology for it: variety is what you are buying here, and variety is exactly what the bottom level cannot manufacture — sixteen wordings written into the snippet, which collapse into twelve sentence frames over two hundred products once the item’s own values are blanked out, both measured by the N0 test; and the seventeenth wording is written by hand by somebody who has already written sixteen. N2 demands that you already published the catalogue you are asking it to write, and the E2E challenge found that sequence-to-sequence models without a semantic control often fail to express the attributes they are given correctly: it needs the same check as N3. The price is real — copy metered by the token, one item at a time, your product records in a third party’s hands, and a review to redo on every regeneration, since two calls on the same record do not write the same sentence — while the copy itself is generated once, stored in the catalogue and versioned there like a copywriter’s text: the stored text is what gets published, not a call per page view. What it buys is a shop that does not read like a form. For two hundred items generated in one go, the example provider’s batch API is built for that and costs less than two hundred calls made one at a time. If your catalogue is thirty items a copywriter can write once, no level on this entry is the answer.
Copywriting through a general-purpose model, with a grounding check against the product record
Yes, a general-purpose model earns its place
General-purpose LLM APIN3élevéAbout a second
Python
"""
Write a product description by asking a general-purpose model.
Rung N3. On most entries of this site this rung is the expensive answer to a
question that did not need it. Here it is the one that wins: the template of
rung N0 draws every sentence from a list written in advance, and its test
counts the frames that repeat, while this rung asks for new prose on every
call.
What it costs is visible in the code, and none of it is the model's doing: the
request, the size cap, the retry, an answer that is only probably JSON, and a
temperature above zero — variety is the thing being bought here, and the SDK
documents higher temperatures as making the output more random, so nothing can
be reviewed once and trusted afterwards.
Which is why the last check is the one from the rung below. A model that writes
freely also claims freely. The copy is read back against the record, term by
term, and anything the record does not support is refused rather than
published.
"""
from __future__ import annotations
import json
import re
import unicodedata
# The instructions are written in the language of the shop, like the copy.
PROMPT = (
"Tu rédiges la présentation d'un article pour une boutique en ligne.\n"
"Écris deux phrases en français, sans superlatif, et n'affirme rien qui ne\n"
"figure pas dans les caractéristiques ci-dessous.\n"
"Réponds par un objet JSON et rien d'autre : {\"description\": \"…\"}.\n"
"\n"
"Caractéristiques :"
)
# A model charges by the token. Refusing an oversized record is not an
# optimisation, it is a cost control.
MAX_CHARACTERS = 600
# Shorter than that, the model answered a fragment and not a description.
MIN_CHARACTERS = 40
class DescriptionUnavailable(Exception):
"""The provider failed, or answered something no shop can publish."""
class UngroundedDescription(DescriptionUnavailable):
"""The copy claims an attribute the product record does not carry."""
# The provider named here is an example, not a recommendation: the reasoning
# holds for any general-purpose model API, and the client is swappable. Pass
# any object with a `complete(prompt=..., temperature=...)` method.
MODEL = "gpt-4.1-mini" # an example id: check the parameters your model accepts
class ProviderClient:
"""The one call this snippet makes, on top of the provider's SDK."""
def __init__(self, sdk=None, model: str = MODEL):
if sdk is None: # pragma: no cover - needs a key and a network
from openai import OpenAI
sdk = OpenAI()
self.sdk, self.model = sdk, model
def complete(self, *, prompt: str, temperature: float) -> str:
response = self.sdk.chat.completions.create(
model=self.model,
messages=[{"role": "user", "content": prompt}],
temperature=temperature,
)
return response.choices[0].message.content
def describe(
product: dict,
client=None,
*,
vocabulary=(),
attempts: int = 3,
temperature: float = 0.7,
) -> str:
"""
Write the description of one product.
`client` is injected so this can be tested without a network call; in
production it defaults to a real provider client.
`vocabulary` is the attribute words your catalogue uses — materials,
finishes, features, claims. A word of that list found in the copy and
nowhere in the record is an invention, and refused. An empty vocabulary
switches the check off, which is a decision, not a default to leave alone.
"""
attributes = _attributes(product)
if not attributes:
raise ValueError("empty product record: nothing to describe")
if len(attributes) > MAX_CHARACTERS:
raise ValueError(f"product record longer than {MAX_CHARACTERS} characters")
client = client or ProviderClient()
description = _ask(client, f"{PROMPT}\n{attributes}", attempts, temperature)
if len(description) < MIN_CHARACTERS:
raise DescriptionUnavailable("the model answered a fragment")
invented = [
term for term in vocabulary if _states(description, term) and not _states(attributes, term)
]
if invented:
raise UngroundedDescription(", ".join(invented))
return description
def _attributes(product: dict) -> str:
"""One « field: value » line per attribute, which is what the model reads."""
lines = []
for key, value in product.items():
items = value if isinstance(value, (list, tuple)) else [value]
joined = ", ".join(str(item) for item in items if item is not None) # None: a NULL column
if joined.strip():
lines.append(f"- {key} : {joined.strip()}")
return "\n".join(lines)
def _ask(client, prompt: str, attempts: int, temperature: float) -> str:
"""Call the provider, decode the answer, and retry what can be retried."""
last_error: Exception | None = None
for _ in range(attempts):
try:
answer = _decode(client.complete(prompt=prompt, temperature=temperature))
written = answer.get("description") if isinstance(answer, dict) else None
if isinstance(written, str) and written.strip():
return " ".join(written.split())
last_error = ValueError("the model answered without a description")
except Exception as error: # noqa: BLE001 - any provider failure is retried
last_error = error
raise DescriptionUnavailable(str(last_error))
def _decode(answer):
"""JSON, or JSON wrapped whole in one code fence. No text (a refusal) is unusable."""
if not isinstance(answer, str):
raise ValueError("the model returned no text")
text = answer.strip()
if text.startswith("```") and text.endswith("```") and text.count("```") == 2:
text = text[3:-3].removeprefix("json")
return json.loads(text)
# A term right after one of these words is denied, not stated: « non étanche ».
NEGATIONS = frozenset({"non", "pas", "sans", "ni", "aucun", "aucune"})
def _states(text: str, term: str) -> bool:
"""
Whether the text states the term: whole words, case and accents set aside,
each word allowed the agreement endings e, s and es (« garantie à vie »
states « garanti à vie »), and not right after a negation.
"""
folded = _fold(text)
words = r"\s+".join(re.escape(word) + "(?:e|s|es)?" for word in _fold(term).split())
for found in re.finditer(rf"(?<!\w){words}(?!\w)", folded):
before = re.findall(r"\w+", folded[: found.start()])
if not before or before[-1] not in NEGATIONS:
return True
return False
def _fold(text: str) -> str:
"""Lowercase and drop accents, so « À vie » meets « a vie »."""
letters = unicodedata.normalize("NFD", str(text).lower())
return "".join(char for char in letters if not unicodedata.combining(char))JavaScript
/**
* Write a product description by asking a general-purpose model.
*
* Rung N3. On most entries of this site this rung is the expensive answer to a
* question that did not need it. Here it is the one that wins: the template of
* rung N0 draws every sentence from a list written in advance, and its test
* counts the frames that repeat, while this rung asks for new prose on every
* call.
*
* What it costs is visible in the code, and none of it is the model's doing:
* the request, the size cap, the retry, an answer that is only probably JSON,
* and a temperature above zero — variety is the thing being bought here, and
* the SDK documents higher temperatures as making the output more random, so
* nothing can be reviewed once and trusted afterwards.
*
* Which is why the last check is the one from the rung below. A model that
* writes freely also claims freely. The copy is read back against the record,
* term by term, and anything the record does not support is refused rather
* than published.
*/
// The instructions are written in the language of the shop, like the copy.
const PROMPT = [
"Tu rédiges la présentation d'un article pour une boutique en ligne.",
'Écris deux phrases en français, sans superlatif, et n\'affirme rien qui ne',
'figure pas dans les caractéristiques ci-dessous.',
'Réponds par un objet JSON et rien d\'autre : {"description": "…"}.',
'',
'Caractéristiques :',
].join('\n');
// A model charges by the token. Refusing an oversized record is not an
// optimisation, it is a cost control.
export const MAX_CHARACTERS = 600;
// Shorter than that, the model answered a fragment and not a description.
export const MIN_CHARACTERS = 40;
export class DescriptionUnavailable extends Error {}
export class UngroundedDescription extends DescriptionUnavailable {}
// The provider named here is an example, not a recommendation: the reasoning
// holds for any general-purpose model API, and the client is swappable. Pass
// any object with a `complete({ prompt, temperature })` method.
export const MODEL = 'gpt-4.1-mini'; // an example id: check the parameters your model accepts
export async function providerClient(sdk, model = MODEL) {
if (!sdk) {
const { OpenAI } = await import('openai');
sdk = new OpenAI();
}
return {
async complete({ prompt, temperature }) {
const response = await sdk.chat.completions.create({
model,
messages: [{ role: 'user', content: prompt }],
temperature,
});
return response.choices[0].message.content;
},
};
}
/**
* Write the description of one product.
*
* `client` is injected so this can be tested without a network call; in
* production it defaults to a real provider client.
*
* `vocabulary` is the attribute words your catalogue uses — materials,
* finishes, features, claims. A word of that list found in the copy and
* nowhere in the record is an invention, and refused. An empty vocabulary
* switches the check off, which is a decision, not a default to leave alone.
*
* @param {Record<string, string|string[]>} product
* @param {{complete: Function}} [client]
* @param {{vocabulary?: string[], attempts?: number, temperature?: number}} [options]
*/
export async function describe(
product,
client,
{ vocabulary = [], attempts = 3, temperature = 0.7 } = {},
) {
const attributes = attributeLines(product);
if (attributes === '') throw new RangeError('empty product record: nothing to describe');
// Characters are counted as code points, as in Python: an emoji is one.
if ([...attributes].length > MAX_CHARACTERS) {
throw new RangeError(`product record longer than ${MAX_CHARACTERS} characters`);
}
client ??= await providerClient();
const description = await ask(client, `${PROMPT}\n${attributes}`, attempts, temperature);
if (description.length < MIN_CHARACTERS) {
throw new DescriptionUnavailable('the model answered a fragment');
}
const invented = vocabulary.filter((term) => states(description, term) && !states(attributes, term));
if (invented.length > 0) throw new UngroundedDescription(invented.join(', '));
return description;
}
/** One « field: value » line per attribute, which is what the model reads. */
function attributeLines(product) {
const lines = [];
for (const [key, value] of Object.entries(product)) {
const items = (Array.isArray(value) ? value : [value]).filter((item) => item != null); // null: a NULL column
const joined = items.map(String).join(', ');
if (joined.trim()) lines.push(`- ${key} : ${joined.trim()}`);
}
return lines.join('\n');
}
/** Call the provider, decode the answer, and retry what can be retried. */
async function ask(client, prompt, attempts, temperature) {
let lastError;
for (let i = 0; i < attempts; i += 1) {
try {
const answer = decode(await client.complete({ prompt, temperature }));
const written = answer && typeof answer === 'object' ? answer.description : null;
if (typeof written === 'string' && written.trim()) return written.split(/\s+/).filter(Boolean).join(' ');
lastError = new Error('the model answered without a description');
} catch (error) {
lastError = error;
}
}
throw new DescriptionUnavailable(String(lastError));
}
/** JSON, or JSON wrapped whole in one code fence. No text (a refusal) is unusable. */
function decode(answer) {
if (typeof answer !== 'string') throw new Error('the model returned no text');
let text = answer.trim();
if (text.startsWith('```') && text.endsWith('```') && text.split('```').length === 3) {
text = text.slice(3, -3).replace(/^json/, '');
}
return JSON.parse(text);
}
// A term right after one of these words is denied, not stated: « non étanche ».
const NEGATIONS = new Set(['non', 'pas', 'sans', 'ni', 'aucun', 'aucune']);
/**
* Whether the text states the term: whole words, case and accents set aside,
* each word allowed the agreement endings e, s and es (« garantie à vie »
* states « garanti à vie »), and not right after a negation.
*/
function states(text, term) {
const folded = fold(text);
const words = fold(term).split(/\s+/).filter(Boolean)
.map((word) => `${word.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}(?:e|s|es)?`);
const pattern = new RegExp(`(?<![\\p{L}\\p{N}_])${words.join('\\s+')}(?![\\p{L}\\p{N}_])`, 'gu');
for (const found of folded.matchAll(pattern)) {
const before = folded.slice(0, found.index).match(/[\p{L}\p{N}_]+/gu) ?? [];
if (!NEGATIONS.has(before.at(-1))) return true;
}
return false;
}
/** Lowercase and drop accents, so « À vie » meets « a vie ». */
function fold(text) {
return String(text)
.toLowerCase()
.normalize('NFD')
.replace(/\p{Diacritic}/gu, '');
}Try it
This code cannot run in a browser. The outputs below are computed when the site is built, by running the real code.
The copy the model returns is simulated by the local double the tests use. What is computed here is what the code sends, what it refuses before spending, what it retries, and what its check against the record catches. The temperature sent is not zero, and the provider’s SDK describes that as making the output more random.
Copy that sticks to the record
Given
nom : Aurore 500 catégorie : sac à dos matière : toile recyclée public : les randonneurs à la journée équipements : poche pour ordinateur, sangle ventrale coloris : ardoise, sable garantie : deux ans
What it returns
Aurore 500 tient la journée de marche sans se rappeler à vous. Sa toile recyclée encaisse les ronces, et sa poche pour ordinateur rentre au bureau le lundi.
1 call billed, 490 characters sent to the provider, instructions and record together.
The same record, with the stockroom notes still in it
Given
the same record, importer notes included: 702 characters of attributes, 102 too many
What it returns
Refused before the first call
product record longer than 600 characters
Nothing left for the provider, nothing owed.
The provider fails twice
Given
the same record, but the first two calls fail
What it returns
Aurore 500 tient la journée de marche sans se rappeler à vous. Sa toile recyclée encaisse les ronces, et sa poche pour ordinateur rentre au bureau le lundi.
3 calls billed: the first two failed, the third answered.
The model answers in prose
Given
the same record; the model answers “Bien sûr ! Voici une proposition…”
What it returns
Answer rejected
The model answered in prose where the code asked for JSON. The code raises rather than publishing that on a product page.
3 calls sent and billed, no publishable answer.
The model promises a lifetime guarantee
Given
the same record, which says a two-year warranty; the model writes « garanti à vie »
What it returns
Publication refused: a promise the record does not carry
The term-by-term check found “garanti à vie” in the copy and nowhere in the record.
1 call billed, 490 characters sent to the provider, instructions and record together.
The model promises the same thing, in other words
Given
the same record, which says nothing about waterproofing; the model writes « ne craint pas la pluie battante »
What it returns
Aurore 500 tient la journée de marche sans se rappeler à vous. Sa toile recyclée ne craint ni les ronces ni la pluie battante, et votre dos reste sec sous l’averse.
1 call billed, 490 characters sent to the provider, instructions and record together.
Here it breaksThe copy promises a bag that keeps your back dry in a downpour. The record says nothing of the kind, and the shop has never sold this bag as waterproof. The check sees nothing: it looks for the catalogue’s terms, and « étanche » is not written — the promise is made in other words. That check is worth exactly what the list is worth. The copy goes live.
What this means for you
- Your data goes to a third party
- The result varies from one call to the next
- Hard to test
- Depends on an outside provider
- High footprint
What this adds to your regulatory scope
- Transfer of your product records to a processor, unreleased items included, with the contractual framing that implies
- Processing location to be confirmed with the provider
- Does not excuse you from answering for the claims you publish: a sentence written by the model binds you as one written by you
Breaking point
The model promises what the shop does not sell, with the same confidence as the true sentence before it. The test has the local double write that the bag is « garanti à vie », guaranteed for life, where the record says two years. What catches it is the check from the level below, term by term against the record, and it is worth exactly what your list is worth. On top of that comes the thing you are buying: the snippet asks for a non-zero temperature, which the provider’s SDK describes as making the output more random. The copy itself is generated once per item and stored in the catalogue: that stored text is what you review and what you publish. What costs you is regeneration — correct a product record, and the sentence you reviewed yesterday is no longer the one that will go out.
When to move up a level
There is no level above this one.
The four options
- N0Slot templates, with grammatical agreement and a fallback on the attributes presentFree · Instant
- N1Not worth it here
- N2Small self-hosted generative model (BARThez fine-tuned on the descriptions you have already published)modéré · About a second
- RecommendedN3Copywriting through a general-purpose model, with a grounding check against the product recordélevé · About a second
The other options
N0Slot templates, with grammatical agreement and a fallback on the attributes presentFree · Instant
Slot templates, with grammatical agreement and a fallback on the attributes present
Rule and classic algorithmN0FreeInstant
Python
"""
Write a product description by filling slot templates.
Rung N0. Deterministic, standard library only. A template is not the poor
relation of a model: it never claims a feature the product does not have, it
renders in the time it takes to read a dictionary, and every sentence it can
possibly produce was written and approved by a human being before it shipped.
Three things separate a template that survives a real catalogue from the one
everybody writes in ten minutes and throws away in a week.
First, a missing attribute must not leave a hole in a sentence. Each block of
the description offers several wordings; only those whose slots are all filled
are eligible, and the ones using the most attributes are preferred. A product
with no colour list simply gets no colour sentence, instead of « Disponible
en . ».
Second, agreement is not optional in a sales page. The markers `{un}`, `{e}`
and `{s}` carry the grammatical gender of the category noun and the number of
the enumerated list, so « Garantie deux ans » and « Points forts » come out
right without a second template per case.
Third, an enumeration is written with commas and one conjunction at the end,
never dumped as a comma-separated list.
What this cannot do is the subject of the breaking-point test next to it.
"""
from __future__ import annotations
import re
import unicodedata
# A slot in a wording: an attribute name, or one of the grammar markers.
SLOT = re.compile(r"\{(\w+)\}")
# The description is built block by block, in this order. Inside a block, the
# wordings say the same thing with different attributes and different words:
# the ones that can be filled are kept, the most informative of those win, and
# the product draws one of them.
BLOCKS = (
( # What the product is.
"{name} : {un} {category} en {material}, pensé{e} pour {audience}.",
"{name}, {un} {category} en {material} pour {audience}.",
"{name} : {un} {category} en {material}.",
"{name}, {un} {category} en {material}.",
"{name} : {un} {category} pour {audience}.",
"{name} : {un} {category}.",
"{name}, {un} {category}.",
),
( # What it brings.
"Point{s} fort{s} : {features}.",
"Au programme : {features}.",
"Côté équipement : {features}.",
),
( # What there is to choose.
"Disponible en {colours}.",
"À choisir en {colours}.",
"Existe en {colours}.",
),
( # What you are promised.
"Garanti{e} {warranty}.",
"La garantie court sur {warranty}.",
"Livré{e} avec {warranty} de garantie.",
),
)
# The three markers above that carry grammar rather than an attribute value.
GRAMMAR = ("un", "e", "s")
def describe(product: dict, *, conjunction: str = "et") -> str:
"""
Render the description of one product.
`product` maps an attribute name to a string or to a list of strings. Only
the attributes actually present are used. `gender` holds the grammatical
gender of the category noun and defaults to masculine, which is the only
piece of grammar a product database never stores and a French sentence
always needs.
"""
values, plural = _slots(product, conjunction)
feminine = str(product.get("gender", "m")).lower().startswith("f")
sentences = []
for wordings in BLOCKS:
usable = _usable(wordings, values)
if usable:
drawn = usable[_variant(str(product.get("name") or ""), len(usable))]
sentences.append(_fill(drawn, values, plural, feminine))
return " ".join(sentences)
def _usable(wordings: tuple, values: dict) -> list:
"""
The wordings that can be filled, and among those the most informative.
A wording is dropped as soon as one of its slots has no value. Of those
that remain, only the ones using the most attributes are kept: an attribute
the shop took the trouble to fill in must not be left out because the draw
fell on a shorter sentence. Several wordings usually tie, and that tie is
where the variety of this rung lives.
"""
scored = []
for wording in wordings:
slots = [slot for slot in SLOT.findall(wording) if slot not in GRAMMAR]
if all(slot in values for slot in slots):
scored.append((len(slots), wording))
best = max((count for count, _ in scored), default=0)
return [wording for count, wording in scored if count == best]
def _slots(product: dict, conjunction: str) -> tuple[dict, set]:
"""
Attribute values as insertable text, and the names that are plural.
A value of None, a NULL column, is an attribute the product does not have:
it must not reach a sentence as the word « None ».
"""
values: dict[str, str] = {}
plural: set[str] = set()
for key, value in product.items():
if value is None:
continue
if isinstance(value, (list, tuple)):
items = [str(item).strip() for item in value if item is not None and str(item).strip()]
if len(items) > 1:
plural.add(key)
value = _enumerate(items, conjunction)
if str(value).strip():
values[key] = str(value).strip()
return values, plural
def _enumerate(items: list[str], conjunction: str) -> str:
"""« ardoise, sable et bronze » : commas, then the conjunction once."""
if len(items) < 2:
return items[0] if items else ""
return f"{', '.join(items[:-1])} {conjunction} {items[-1]}"
def _fill(wording: str, values: dict, plural: set, feminine: bool) -> str:
"""Write the attributes and the grammar markers into one wording."""
slots = SLOT.findall(wording)
# The grammar markers are written last, so an attribute called `s` or `e`
# cannot quietly take their place.
filled = {
**values,
"un": "une" if feminine else "un",
"e": "e" if feminine else "",
"s": "s" if any(slot in plural for slot in slots) else "",
}
return SLOT.sub(lambda match: filled[match.group(1)], wording)
def _variant(seed: str, count: int) -> int:
"""
Draw one wording out of `count`, the same one for the same product.
Summing the code points is deliberately crude. It has one job: give the
same answer as the JavaScript version of this snippet, so a catalogue
rendered by either reads identically. The name is composed first (NFC), so
an accent typed as two code points draws the same wording as one.
"""
return sum(ord(char) for char in unicodedata.normalize("NFC", seed)) % countJavaScript
/**
* Write a product description by filling slot templates.
*
* Rung N0. Deterministic, no dependency. A template is not the poor relation
* of a model: it never claims a feature the product does not have, it renders
* in the time it takes to read a dictionary, and every sentence it can
* possibly produce was written and approved by a human being before it
* shipped.
*
* Three things separate a template that survives a real catalogue from the one
* everybody writes in ten minutes and throws away in a week.
*
* First, a missing attribute must not leave a hole in a sentence. Each block
* of the description offers several wordings; only those whose slots are all
* filled are eligible, and the ones using the most attributes are preferred. A
* product with no colour list simply gets no colour sentence, instead of
* « Disponible en . ».
*
* Second, agreement is not optional in a sales page. The markers `{un}`, `{e}`
* and `{s}` carry the grammatical gender of the category noun and the number
* of the enumerated list, so « Garantie deux ans » and « Points forts » come
* out right without a second template per case.
*
* Third, an enumeration is written with commas and one conjunction at the end,
* never dumped as a comma-separated list.
*
* What this cannot do is the subject of the breaking-point test next to it.
*/
// A slot in a wording: an attribute name, or one of the grammar markers.
const SLOT = /\{(\w+)\}/g;
// The description is built block by block, in this order. Inside a block, the
// wordings say the same thing with different attributes and different words:
// the ones that can be filled are kept, the most informative of those win, and
// the product draws one of them.
export const BLOCKS = [
[ // What the product is.
'{name} : {un} {category} en {material}, pensé{e} pour {audience}.',
'{name}, {un} {category} en {material} pour {audience}.',
'{name} : {un} {category} en {material}.',
'{name}, {un} {category} en {material}.',
'{name} : {un} {category} pour {audience}.',
'{name} : {un} {category}.',
'{name}, {un} {category}.',
],
[ // What it brings.
'Point{s} fort{s} : {features}.',
'Au programme : {features}.',
'Côté équipement : {features}.',
],
[ // What there is to choose.
'Disponible en {colours}.',
'À choisir en {colours}.',
'Existe en {colours}.',
],
[ // What you are promised.
'Garanti{e} {warranty}.',
'La garantie court sur {warranty}.',
'Livré{e} avec {warranty} de garantie.',
],
];
// The three markers above that carry grammar rather than an attribute value.
const GRAMMAR = ['un', 'e', 's'];
/**
* Render the description of one product.
*
* `product` maps an attribute name to a string or to an array of strings. Only
* the attributes actually present are used. `gender` holds the grammatical
* gender of the category noun and defaults to masculine, which is the only
* piece of grammar a product database never stores and a French sentence
* always needs.
*
* @param {Record<string, string|string[]>} product
* @param {{conjunction?: string}} [options]
*/
export function describe(product, { conjunction = 'et' } = {}) {
const { values, plural } = slotsOf(product, conjunction);
const feminine = String(product.gender ?? 'm').toLowerCase().startsWith('f');
const sentences = [];
for (const wordings of BLOCKS) {
const usable = usableWordings(wordings, values);
if (usable.length > 0) {
const drawn = usable[variant(String(product.name ?? ''), usable.length)];
sentences.push(fill(drawn, values, plural, feminine));
}
}
return sentences.join(' ');
}
/**
* The wordings that can be filled, and among those the most informative.
*
* A wording is dropped as soon as one of its slots has no value. Of those that
* remain, only the ones using the most attributes are kept: an attribute the
* shop took the trouble to fill in must not be left out because the draw fell
* on a shorter sentence. Several wordings usually tie, and that tie is where
* the variety of this rung lives.
*/
function usableWordings(wordings, values) {
const scored = [];
for (const wording of wordings) {
const slots = slotsIn(wording).filter((slot) => !GRAMMAR.includes(slot));
if (slots.every((slot) => Object.hasOwn(values, slot))) scored.push([slots.length, wording]);
}
const best = Math.max(0, ...scored.map(([count]) => count));
return scored.filter(([count]) => count === best).map(([, wording]) => wording);
}
/** The slot names a wording asks for, in order. */
function slotsIn(wording) {
return [...wording.matchAll(SLOT)].map((match) => match[1]);
}
/**
* Attribute values as insertable text, and the names that are plural.
*
* A value of null, a NULL column, is an attribute the product does not have:
* it must not reach a sentence as the word « null ».
*/
function slotsOf(product, conjunction) {
const values = {};
const plural = new Set();
for (const [key, raw] of Object.entries(product)) {
let value = raw;
if (value === null || value === undefined) continue;
if (Array.isArray(value)) {
const items = value.filter((item) => item !== null && item !== undefined).map((item) => String(item).trim()).filter(Boolean);
if (items.length > 1) plural.add(key);
value = enumerate(items, conjunction);
}
if (String(value).trim()) values[key] = String(value).trim();
}
return { values, plural };
}
/** « ardoise, sable et bronze » : commas, then the conjunction once. */
function enumerate(items, conjunction) {
if (items.length < 2) return items[0] ?? '';
return `${items.slice(0, -1).join(', ')} ${conjunction} ${items.at(-1)}`;
}
/** Write the attributes and the grammar markers into one wording. */
function fill(wording, values, plural, feminine) {
const slots = slotsIn(wording);
// The grammar markers are written last, so an attribute called `s` or `e`
// cannot quietly take their place.
const filled = {
...values,
un: feminine ? 'une' : 'un',
e: feminine ? 'e' : '',
s: slots.some((slot) => plural.has(slot)) ? 's' : '',
};
return wording.replace(SLOT, (_, slot) => filled[slot]);
}
/**
* Draw one wording out of `count`, the same one for the same product.
*
* Summing the code points is deliberately crude. It has one job: give the same
* answer as the Python version of this snippet, so a catalogue rendered by
* either reads identically. The name is composed first (NFC), so an accent
* typed as two code points draws the same wording as one.
*/
function variant(seed, count) {
let total = 0;
for (const char of seed.normalize('NFC')) total += char.codePointAt(0);
return total % count;
}What this means for you
- Your data stays put
- Always the same result
- Unit-testable, case by case
- No provider in the loop
- Negligible footprint
What this adds to your regulatory scope
- No specific scope added: the product attributes never leave your infrastructure
- Does not excuse you from answering for the claims you publish; here, every sentence the code can produce was written and reviewed before it shipped
Breaking point
Repetition, and it is counted. The test renders two hundred complete records, blanks each product’s own values out of its description, and tallies the sentence frames left behind: twelve, across two hundred items, the rarest of them used sixteen times. On the hand-written catalogue, whose records do not all carry the same attributes, the opening sentences fall into nine frames for twenty products and the commonest covers six of them — and that much variety comes from the holes in the data, not from the template.
When to move up a level
You open two pages of your shop in a row and recognise the sentence before you recognise the product.
N1Not worth it here
Not worth it here
Lightweight classic modelN1
A lightweight classic model classifies, scores or tags: writing is not what it is for. None of them produces prose a shop can publish, and for text assembled by recombining fragments of sentences, the N0 template does better — its sentences were at least written by somebody.
N2Small self-hosted generative model (BARThez fine-tuned on the descriptions you have already published)modéré · About a second
Small self-hosted generative model (BARThez fine-tuned on the descriptions you have already published)
Small self-hosted specialised modelN2modéréAbout a second
Python
"""
Write a product description with a small self-hosted generative model.
Rung N2. A sequence-to-sequence checkpoint that lives on your own disk,
fine-tuned on the descriptions your shop has already published. Nothing
leaves your machines and nothing is metered.
The fine-tuning is not in this file, and it is the work of this rung. Start
from a model that already writes French: BARThez, a BART-shaped French
sequence-to-sequence model pretrained on raw French text, 165 M parameters in
its base size, Apache 2.0. The training set is one pair per
description your catalogue has already published — the record on one line, as
`_source` writes it below, and the published description as the target. You
are not buying a corpus; you are using the one your shop wrote. What you get
back is a checkpoint of your own, which is what `CHECKPOINT` points at.
Around the model, everything on this rung is code you now own: the source line
the model was fine-tuned to read, the size cap, the retry, the sentence a small
model leaves half-finished when its token budget runs out, and the check at the
end.
That check is the point of this file. The E2E generation challenge found that
sequence-to-sequence models without a semantic control often fail to express
the attributes they are given correctly. So the copy is read back against the record, term by term, and
anything the record does not support is refused. A shop that promises what it
does not sell has a legal problem, not a style problem.
"""
from __future__ import annotations
import re
import unicodedata
# One product record, written on one line. Longer than that, it is not a
# product record, and the model only wanders further from it.
MAX_CHARACTERS = 600
# Shorter than that, the model handed back a fragment and not a description.
MIN_CHARACTERS = 40
class DescriptionUnavailable(Exception):
"""The model failed, or answered something no shop can publish."""
class UngroundedDescription(DescriptionUnavailable):
"""The copy claims an attribute the product record does not carry."""
# What to fine-tune from, for a French catalogue. Read its card before you
# commit to it: language, licence, size. This one says French, Apache 2.0,
# 165 M parameters.
BASE_CHECKPOINT = "moussaKam/barthez"
# Where your fine-tuned weights are: the output of that training, not a model
# to download.
CHECKPOINT = "./models/catalogue-copy"
class LocalCopywriter:
"""The real model: a fine-tuned checkpoint on your disk, loaded once."""
def __init__(self, checkpoint: str = CHECKPOINT) -> None:
# transformers 5 removed the "text2text-generation" pipeline: the
# tokenizer and the model are called directly.
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer # a large local install
self._tokenizer = AutoTokenizer.from_pretrained(checkpoint)
self._model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)
def generate(self, source: str, **options) -> str:
output = self._model.generate(**self._tokenizer([source], return_tensors="pt"), **options)
return self._tokenizer.decode(output[0], skip_special_tokens=True)
def describe(product: dict, model=None, *, vocabulary=(), attempts: int = 2) -> str:
"""
Write the description of one product.
`model` is injected so this can be tested without the checkpoint; in
production it defaults to the real one above.
`vocabulary` is the attribute words your catalogue uses — materials,
finishes, features, claims. It is what makes the grounding check possible:
a word of that list found in the copy and nowhere in the record is an
invention. An empty vocabulary switches the check off, which is a decision,
not a default to leave alone.
"""
source = _source(product)
if not source:
raise ValueError("empty product record: nothing to describe")
if len(source) > MAX_CHARACTERS:
raise ValueError(f"product record longer than {MAX_CHARACTERS} characters")
model = model or LocalCopywriter()
description = _whole_sentences(_generate(model, source, attempts))
if len(description) < MIN_CHARACTERS:
raise DescriptionUnavailable("the model answered a fragment")
invented = [term for term in vocabulary if _states(description, term) and not _states(source, term)]
if invented:
raise UngroundedDescription(", ".join(invented))
return description
def _source(product: dict) -> str:
"""The shape the model was fine-tuned on: one line of « field: value »."""
fields = []
for key, value in product.items():
items = value if isinstance(value, (list, tuple)) else [value]
joined = ", ".join(str(item) for item in items if item is not None) # None: a NULL column
if joined.strip():
fields.append(f"{key}: {joined.strip()}")
return " | ".join(fields)
def _generate(model, source: str, attempts: int) -> str:
"""Retry: on a machine that also serves the shop, the first call fails."""
last_error: Exception | None = None
for _ in range(attempts):
try:
text = model.generate(source, max_new_tokens=90, num_beams=4)
except Exception as error: # noqa: BLE001 - any model failure is retried
last_error = error
continue
if isinstance(text, str):
return text
last_error = TypeError("the model returned no text")
raise DescriptionUnavailable(str(last_error))
def _whole_sentences(text) -> str:
"""
Keep only what the model finished saying.
A small model stops when its budget runs out, mid-sentence and sometimes
mid-word. Publishing that is worse than publishing nothing at all. A
sentence ends on a mark followed by the end of the text or by a capital,
so « 1.2 kg » is not an end; « M. Dupont » still is.
"""
text = " ".join(text.split())
end = 0
for mark in re.finditer(r"[.!?]", text):
after = text[mark.end():mark.end() + 2]
if after == "" or (after[0] == " " and after[1:].isupper()):
end = mark.end()
return text[:end]
# A term right after one of these words is denied, not stated: « non étanche ».
NEGATIONS = frozenset({"non", "pas", "sans", "ni", "aucun", "aucune"})
def _states(text: str, term: str) -> bool:
"""
Whether the text states the term: whole words, case and accents set aside,
each word allowed the agreement endings e, s and es (« garantie à vie »
states « garanti à vie »), and not right after a negation.
"""
folded = _fold(text)
words = r"\s+".join(re.escape(word) + "(?:e|s|es)?" for word in _fold(term).split())
for found in re.finditer(rf"(?<!\w){words}(?!\w)", folded):
before = re.findall(r"\w+", folded[: found.start()])
if not before or before[-1] not in NEGATIONS:
return True
return False
def _fold(text: str) -> str:
"""Lowercase and drop accents, so « Étanche » meets « etanche »."""
letters = unicodedata.normalize("NFD", str(text).lower())
return "".join(char for char in letters if not unicodedata.combining(char))JavaScript
/**
* Write a product description with a small self-hosted generative model.
*
* Rung N2. A sequence-to-sequence checkpoint that lives on your own disk,
* fine-tuned on the descriptions your shop has already published. Nothing
* leaves your machines and nothing is metered.
*
* The fine-tuning is not in this file, and it is the work of this rung. Start
* from a model that already writes French: BARThez, a BART-shaped French
* sequence-to-sequence model pretrained on raw French text, 165 M parameters
* in its base size, Apache 2.0. The training set is one pair per
* description your catalogue has already published — the record on one line,
* as `source` writes it below, and the published description as the target.
* You are not buying a corpus; you are using the one your shop wrote. What you
* get back is a checkpoint of your own, which is what `CHECKPOINT` points at.
*
* Around the model, everything on this rung is code you now own: the source
* line the model was fine-tuned to read, the size cap, the retry, the sentence
* a small model leaves half-finished when its token budget runs out, and the
* check at the end.
*
* That check is the point of this file. The E2E generation challenge found
* that sequence-to-sequence models without a semantic control often fail to
* express the attributes they are given correctly. So the copy is read back against the record, term by
* term, and anything the record does not support is refused. A shop that
* promises what it does not sell has a legal problem, not a style problem.
*/
// One product record, written on one line. Longer than that, it is not a
// product record, and the model only wanders further from it.
export const MAX_CHARACTERS = 600;
// Shorter than that, the model handed back a fragment and not a description.
export const MIN_CHARACTERS = 40;
export class DescriptionUnavailable extends Error {}
export class UngroundedDescription extends DescriptionUnavailable {}
/**
* The real model: a fine-tuned checkpoint on your disk, loaded once.
* Transformers.js runs ONNX weights: convert the checkpoint first, with Optimum.
*/
// What to fine-tune from, for a French catalogue. Read its card before you
// commit to it: language, licence, size. This one says French, Apache 2.0,
// 165 M parameters.
export const BASE_CHECKPOINT = 'moussaKam/barthez';
// Where your fine-tuned weights are: the output of that training, not a model
// to download.
export const CHECKPOINT = './models/catalogue-copy';
export class LocalCopywriter {
static async load(checkpoint = CHECKPOINT) {
const { pipeline } = await import('@huggingface/transformers'); // a large local install
return new LocalCopywriter(await pipeline('text2text-generation', checkpoint));
}
constructor(write) {
this.write = write;
}
async generate(source, options = {}) {
const [answer] = await this.write(source, options);
return answer.generated_text;
}
}
/**
* Write the description of one product.
*
* `model` is injected so this can be tested without the checkpoint; in
* production it defaults to the real one above.
*
* `vocabulary` is the attribute words your catalogue uses — materials,
* finishes, features, claims. It is what makes the grounding check possible: a
* word of that list found in the copy and nowhere in the record is an
* invention. An empty vocabulary switches the check off, which is a decision,
* not a default to leave alone.
*
* @param {Record<string, string|string[]>} product
* @param {{generate: Function}} [model]
* @param {{vocabulary?: string[], attempts?: number}} [options]
*/
export async function describe(product, model, { vocabulary = [], attempts = 2 } = {}) {
const source = sourceLine(product);
if (source === '') throw new RangeError('empty product record: nothing to describe');
// Characters are counted as code points, as in Python: an emoji is one.
if ([...source].length > MAX_CHARACTERS) {
throw new RangeError(`product record longer than ${MAX_CHARACTERS} characters`);
}
const copywriter = model ?? (await LocalCopywriter.load());
const description = wholeSentences(await generate(copywriter, source, attempts));
if (description.length < MIN_CHARACTERS) {
throw new DescriptionUnavailable('the model answered a fragment');
}
const invented = vocabulary.filter((term) => states(description, term) && !states(source, term));
if (invented.length > 0) throw new UngroundedDescription(invented.join(', '));
return description;
}
/** The shape the model was fine-tuned on: one line of « field: value ». */
function sourceLine(product) {
const fields = [];
for (const [key, value] of Object.entries(product)) {
const items = (Array.isArray(value) ? value : [value]).filter((item) => item != null); // null: a NULL column
const joined = items.map(String).join(', ');
if (joined.trim()) fields.push(`${key}: ${joined.trim()}`);
}
return fields.join(' | ');
}
/** Retry: on a machine that also serves the shop, the first call fails. */
async function generate(model, source, attempts) {
let lastError;
for (let i = 0; i < attempts; i += 1) {
let text;
try {
text = await model.generate(source, { max_new_tokens: 90, num_beams: 4 });
} catch (error) {
lastError = error;
continue;
}
if (typeof text === 'string') return text;
lastError = new TypeError('the model returned no text');
}
throw new DescriptionUnavailable(String(lastError));
}
/**
* Keep only what the model finished saying.
*
* A small model stops when its budget runs out, mid-sentence and sometimes
* mid-word. Publishing that is worse than publishing nothing at all. A
* sentence ends on a mark followed by the end of the text or by a capital, so
* « 1.2 kg » is not an end; « M. Dupont » still is.
*/
function wholeSentences(text) {
const tidy = text.split(/\s+/).filter(Boolean).join(' ');
let end = 0;
for (const mark of tidy.matchAll(/[.!?]/g)) {
const after = tidy.slice(mark.index + 1, mark.index + 3);
if (after === '' || /^ \p{Lu}$/u.test(after)) end = mark.index + 1;
}
return tidy.slice(0, end);
}
// A term right after one of these words is denied, not stated: « non étanche ».
const NEGATIONS = new Set(['non', 'pas', 'sans', 'ni', 'aucun', 'aucune']);
/**
* Whether the text states the term: whole words, case and accents set aside,
* each word allowed the agreement endings e, s and es (« garantie à vie »
* states « garanti à vie »), and not right after a negation.
*/
function states(text, term) {
const folded = fold(text);
const words = fold(term).split(/\s+/).filter(Boolean)
.map((word) => `${word.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}(?:e|s|es)?`);
const pattern = new RegExp(`(?<![\\p{L}\\p{N}_])${words.join('\\s+')}(?![\\p{L}\\p{N}_])`, 'gu');
for (const found of folded.matchAll(pattern)) {
const before = folded.slice(0, found.index).match(/[\p{L}\p{N}_]+/gu) ?? [];
if (!NEGATIONS.has(before.at(-1))) return true;
}
return false;
}
/** Lowercase and drop accents, so « Étanche » meets « etanche ». */
function fold(text) {
return String(text)
.toLowerCase()
.normalize('NFD')
.replace(/\p{Diacritic}/gu, '');
}What this means for you
- Your data stays on your own machines
- Always the same result
- Hard to test
- Depends on a library
- Moderate footprint
What this adds to your regulatory scope
- Processing on your own infrastructure: neither the product record nor the copy leaves it
- The fine-tuning corpus is made of the descriptions you have already published, with the ownership of those texts that implies
- Licence and provenance of the checkpoint to be established before production use: the one this entry starts from, BARThez, is published under the Apache 2.0 licence
- Does not excuse you from answering for the claims you publish: the grounding check only covers the terms on the list you maintain, and that list belongs to whoever answers for the claims — quality lead or lawyer — not to whoever writes the code
Breaking point
The model asserts what the record does not say. The test has the local double hand back a sentence that is right in tone and in grammar — the bag’s recycled canvas is « entièrement étanche », fully waterproof — for an item whose attributes never mention water. The snippet refuses the copy instead of publishing it, including when the record says « non étanche » or the copy writes « étanches », but only for the terms you listed: the next test shows the same sentence going through untouched on an empty vocabulary.
When to move up a level
A new range arrives, and the corpus of published descriptions a fresh fine-tune would call for does not exist yet.
Further reading
- The E2E Dataset: New Challenges For End-to-End Generation — écrire une description à partir d’attributs, posé comme tâche
- Evaluating the state-of-the-art of End-to-End Natural Language Generation — les modèles de séquence à séquence rendent souvent mal les attributs qu’on leur donne, et des systèmes écrits à la main produisent des sorties plus variées
- Hugging Face — la génération de texte avec transformers, generate et ses réglages, appelés par l’extrait N2 Python
- OpenAI — Structured Outputs, sur la réponse JSON que l’extrait N3 doit décoder
- BARThez — le modèle de séquence à séquence français dont part le niveau N2
- OpenAI — Batch API, pour générer un catalogue entier hors ligne plutôt qu’article par article
Where the figures come from
- Fiche de moussaKam/barthez : « A french sequence to sequence pretrained model based on BART », préentraîné sur un corpus de texte français brut, `language: fr`, licence Apache 2.0, taille BASE 12 couches et 165 M de paramètres (mBARThez, LARGE, en compte 458 M) — c’est de cette taille que se lisent `cost: modéré` et `latency: ~1 s`, aucune mesure n’ayant été faite ici
- OpenAI, guide Batch API : « 50% cost discount compared to synchronous APIs », « Each batch completes within 24 hours (and often more quickly) » — d’où la phrase du verdict sur la génération d’un catalogue d’un seul coup
Metadata
General information, kept up to date as best we can. It is not legal advice. Have a professional review your situation.