Route a support ticket to the right team
Send each incoming support ticket to the team that can answer it.
RecommendedN1 Reviewed on
| Rung | Approach | Cost | Latency | Data | Deterministic | Verdict |
|---|---|---|---|---|---|---|
| N0 — Rule and classic algorithm | Keyword rules, with an explicit priority and a default queue | None | <1 ms | Nothing leaves | Yes | |
| N1 — Lightweight classic model | TF-IDF and a linear classifier trained on the archive | Negligible | ~10 ms | Stays on your infrastructure | Yes | Recommended |
| N2 — Small self-hosted specialised model | Self-hosted sentence encoder, vote of the nearest resolved tickets | Low | ~100 ms | Stays on your infrastructure | Yes | |
| N3 — General-purpose LLM API | Classification through a general-purpose model, against a closed list of teams | High | ~1 s | Goes to a third party | No |
N0 — Rule and classic algorithm Rule and classic algorithm
Keyword rules, with an explicit priority and a default queue
- Cost
- None
- Latency
<1 ms
Proof of execution : Code runs as shown
This snippet runs with its real dependencies, and its test runs on every build of the site.
Python
"""
Route a support ticket with keyword rules: ordered, with a default team.
Rung N0. Deterministic, standard library only, and every decision can be
explained to the person who asks why their ticket landed where it did.
Two things turn a keyword list into a rule a support desk can actually run.
First, an explicit priority. Tickets mention several subjects, so two teams
matching the same ticket is the normal case, not the exception. The order of
RULES answers it once and in writing, rather than leaving it to whichever
branch happens to run first.
Second, a default team. Every ticket must land somewhere; a ticket that
matches nothing has to go to a queue a human watches, not to the floor.
"""
import re
import unicodedata
# The queue that gets everything the rules cannot place. Naming it here, next
# to the rules, is what stops a ticket from silently going nowhere.
DEFAULT_TEAM = "general"
# Business knowledge, kept next to the code that applies it.
#
# The order is the priority, and it is a business decision, not a detail of
# implementation: a billing problem has a legal clock on it, an outage blocks
# the customer's work, a parcel is the one that can wait a day. Whoever
# disagrees can reorder this tuple and nothing else.
RULES = (
("billing", ("facture", "remboursement", "prélèvement", "iban", "devis", "paiement")),
("technical", ("bug", "erreur", "panne", "connexion", "mot de passe", "identifiant")),
("shipping", ("livraison", "colis", "transporteur", "expédition", "suivi", "retard")),
)
def _fold(text: str) -> str:
"""Lowercase and drop accents, so « Prélèvement » matches « prelevement »."""
decomposed = unicodedata.normalize("NFD", text.lower())
return "".join(c for c in decomposed if not unicodedata.combining(c))
# A word boundary on the left only. « facture » then also matches « factures »
# and « facturation », which is what French tickets are full of; the price is
# that it would match a longer word starting the same way.
_COMPILED = tuple(
(team, tuple((word, re.compile(rf"\b{re.escape(_fold(word))}")) for word in words))
for team, words in RULES
)
def matches(ticket: str) -> dict[str, list[str]]:
"""
Every team the ticket triggers, with the words that triggered it.
The routing decision needs only the first team, but the reviewer of a
misrouted ticket needs this: it shows what the rules saw, and what they
had to drop.
"""
folded = _fold(ticket)
found = {}
for team, patterns in _COMPILED:
hits = [word for word, pattern in patterns if pattern.search(folded)]
if hits:
found[team] = hits
return found
def route(ticket: str, default_team: str = DEFAULT_TEAM) -> str:
"""The team that gets the ticket. Always one, and always a real queue."""
found = matches(ticket)
for team, _ in RULES:
if team in found:
return team
return default_teamJavaScript
/**
* Route a support ticket with keyword rules: ordered, with a default team.
*
* Rung N0. Deterministic, no dependency, and every decision can be explained
* to the person who asks why their ticket landed where it did.
*
* Two things turn a keyword list into a rule a support desk can actually run.
*
* First, an explicit priority. Tickets mention several subjects, so two teams
* matching the same ticket is the normal case, not the exception. The order of
* RULES answers it once and in writing, rather than leaving it to whichever
* branch happens to run first.
*
* Second, a default team. Every ticket must land somewhere; a ticket that
* matches nothing has to go to a queue a human watches, not to the floor.
*/
// The queue that gets everything the rules cannot place. Naming it here, next
// to the rules, is what stops a ticket from silently going nowhere.
export const DEFAULT_TEAM = 'general';
/**
* Business knowledge, kept next to the code that applies it.
*
* The order is the priority, and it is a business decision, not a detail of
* implementation: a billing problem has a legal clock on it, an outage blocks
* the customer's work, a parcel is the one that can wait a day. Whoever
* disagrees can reorder this list and nothing else.
*/
export const RULES = [
['billing', ['facture', 'remboursement', 'prélèvement', 'iban', 'devis', 'paiement']],
['technical', ['bug', 'erreur', 'panne', 'connexion', 'mot de passe', 'identifiant']],
['shipping', ['livraison', 'colis', 'transporteur', 'expédition', 'suivi', 'retard']],
];
/** Lowercase and drop accents, so « Prélèvement » matches « prelevement ». */
function fold(text) {
return text.toLowerCase().normalize('NFD').replace(/\p{M}/gu, '');
}
// A word boundary on the left only. « facture » then also matches « factures »
// and « facturation », which is what French tickets are full of; the price is
// that it would match a longer word starting the same way.
const COMPILED = RULES.map(([team, words]) => [
team,
words.map((word) => [word, new RegExp(`\\b${fold(word).replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}`)]),
]);
/**
* Every team the ticket triggers, with the words that triggered it.
*
* The routing decision needs only the first team, but the reviewer of a
* misrouted ticket needs this: it shows what the rules saw, and what they had
* to drop.
*/
export function matches(ticket) {
const folded = fold(ticket);
const found = {};
for (const [team, patterns] of COMPILED) {
const hits = patterns.filter(([, pattern]) => pattern.test(folded)).map(([word]) => word);
if (hits.length) found[team] = hits;
}
return found;
}
/** The team that gets the ticket. Always one, and always a real queue. */
export function route(ticket, defaultTeam = DEFAULT_TEAM) {
const found = matches(ticket);
for (const [team] of RULES) {
if (team in found) return team;
}
return defaultTeam;
}Risks
- Data leaving
- Nothing leaves
- Determinism
- Yes
- Testability
- Unit testable
- Vendor dependency
- None
- Footprint
- Negligible
- Regulatory scope
-
- No specific scope added: the ticket is processed where it is already stored
Breaking point
The ticket that belongs to two teams, and the one that triggers none. « Le colis n'est jamais arrivé et le prélèvement est passé quand même » matches both billing and shipping; priority settles on billing, and the parcel half vanishes without a trace in the queue that receives it. At the other end, « je n'arrive plus à faire ce que je faisais avant » matches nothing and goes to the default queue.
When to move up a rung
The default queue becomes the busiest queue you have.
N1 — Lightweight classic model Lightweight classic model Recommended
TF-IDF and a linear classifier trained on the archive
- Cost
- Negligible
- Latency
~10 ms
Proof of execution : Code runs as shown
This snippet runs with its real dependencies, and its test runs on every build of the site.
Python
"""
Route a ticket with TF-IDF and a linear classifier trained on the archive.
Rung N1. The keyword rules of N0 know the words someone thought of. This
knows the words the desk actually received: it is trained on the tickets the
teams have already answered, so the vocabulary of the customers, not the
vocabulary of the rule writer, decides.
What it keeps from N0, deliberately: a default team. A classifier always
returns something, and its most likely class on a ticket it has no opinion
about is still a class. The confidence floor below is what turns that shrug
back into the default queue, instead of a wrong queue.
Training data is the exported archive: the resolved tickets and the team that
resolved each one. The model is a table of weights small enough to keep in the
repository, and retraining it is a step in the nightly export, not a project.
"""
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
DEFAULT_TEAM = "general"
def train(tickets: list[str], teams: list[str]):
"""
`teams` is the team that actually handled each past ticket.
Word pairs as well as single words, because « mot de passe » and « en
retard » carry more than the words they are made of. Classes are weighted
by their rarity: an archive is never balanced, and an unweighted model
learns to answer the busiest team.
"""
model = make_pipeline(
TfidfVectorizer(strip_accents="unicode", ngram_range=(1, 2), sublinear_tf=True),
# C loosens the penalty: the default is set for documents, and on texts
# as short as a ticket it flattens the probabilities so far that no
# ticket ever clears a useful floor.
LogisticRegression(class_weight="balanced", max_iter=1000, C=10),
)
model.fit(tickets, teams)
return model
def rank(model, ticket: str) -> list[tuple[str, float]]:
"""
Every team with the probability the model gives it, best first.
A support desk needs the runner-up: two teams at almost the same score is
exactly the ambiguous ticket of N0, and here it is visible instead of
being silently resolved by a priority order.
"""
probabilities = model.predict_proba([ticket])[0]
ranked = sorted(zip(model.classes_, probabilities), key=lambda pair: -pair[1])
return [(str(team), float(probability)) for team, probability in ranked]
def route(model, ticket: str, min_confidence: float = 0.5,
default_team: str = DEFAULT_TEAM) -> str:
"""
The team that gets the ticket, or the default queue below the floor.
The floor is yours to set. Raise it and more tickets are read by a human
before they move; lower it and more tickets are moved on a hunch. Nothing
in the model can make that choice for you.
"""
team, confidence = rank(model, ticket)[0]
return team if confidence >= min_confidence else default_teamJavaScript
/**
* Route a ticket with TF-IDF and a linear classifier trained on the archive.
*
* Rung N1. The keyword rules of N0 know the words someone thought of. This
* knows the words the desk actually received: it is trained on the tickets the
* teams have already answered, so the vocabulary of the customers, not the
* vocabulary of the rule writer, decides.
*
* Written out rather than pulled from a library, because TF-IDF and a softmax
* regression are forty lines. The model is a table of weights: small enough to
* keep beside the code, retrained while you read this, and every weight can be
* printed and argued about when someone asks why their ticket moved.
*
* What it keeps from N0, deliberately: a default team. A classifier always
* returns something, and its most likely class on a ticket it has no opinion
* about is still a class. The confidence floor is what turns that shrug back
* into the default queue, instead of a wrong queue.
*/
export const DEFAULT_TEAM = 'general';
/** Words of a ticket, lowercased and stripped of accents. */
function tokens(text) {
const folded = text.toLowerCase().normalize('NFD').replace(/\p{M}/gu, '');
const words = folded.match(/[\p{L}\p{N}]+/gu) ?? [];
// Word pairs as well as single words, because « mot de passe » and « en
// retard » carry more than the words they are made of.
return words.concat(words.slice(0, -1).map((w, i) => `${w} ${words[i + 1]}`));
}
/** Vocabulary and inverse document frequency, learnt from the archive alone. */
function fitVocabulary(tickets) {
const seen = new Map();
for (const ticket of tickets) {
for (const term of new Set(tokens(ticket))) seen.set(term, (seen.get(term) ?? 0) + 1);
}
const terms = new Map();
const idf = [];
for (const [term, documents] of seen) {
terms.set(term, idf.length);
idf.push(Math.log((1 + tickets.length) / (1 + documents)) + 1);
}
return { terms, idf };
}
/** One TF-IDF row, brought to length one. Unknown terms are simply dropped. */
function vector(vocabulary, ticket) {
const counts = new Map();
for (const term of tokens(ticket)) {
const j = vocabulary.terms.get(term);
if (j !== undefined) counts.set(j, (counts.get(j) ?? 0) + 1);
}
const row = new Float64Array(vocabulary.idf.length);
// Sublinear term frequency: a word repeated ten times is not ten times the
// signal, and an angry customer repeats words.
for (const [j, count] of counts) row[j] = (1 + Math.log(count)) * vocabulary.idf[j];
const norm = Math.hypot(...row);
if (norm) for (const [j] of counts) row[j] /= norm;
return row;
}
/** Softmax over the teams: one score per team, summing to one. */
function scores(model, row) {
const raw = model.classes.map((_, c) => {
let z = model.bias[c];
for (let j = 0; j < row.length; j += 1) z += model.weights[c][j] * row[j];
return z;
});
const top = Math.max(...raw);
const exponentials = raw.map((z) => Math.exp(z - top));
const total = exponentials.reduce((a, b) => a + b, 0);
return exponentials.map((e) => e / total);
}
/**
* `teams` is the team that actually handled each past ticket.
*
* Each example is weighted by the rarity of its team: an archive is never
* balanced, and an unweighted model learns to answer the busiest team.
*/
export function train(tickets, teams, { epochs = 300, rate = 1 } = {}) {
const vocabulary = fitVocabulary(tickets);
const classes = [...new Set(teams)].sort();
const rows = tickets.map((t) => vector(vocabulary, t));
const share = new Map(classes.map((c) => [c, teams.filter((t) => t === c).length]));
const model = {
vocabulary,
classes,
weights: classes.map(() => new Float64Array(vocabulary.idf.length)),
bias: new Float64Array(classes.length),
};
for (let epoch = 0; epoch < epochs; epoch += 1) {
for (let i = 0; i < rows.length; i += 1) {
const predicted = scores(model, rows[i]);
const step = (rate * teams.length) / (classes.length * share.get(teams[i]));
for (let c = 0; c < classes.length; c += 1) {
const error = predicted[c] - (teams[i] === classes[c] ? 1 : 0);
for (let j = 0; j < rows[i].length; j += 1) model.weights[c][j] -= step * error * rows[i][j];
model.bias[c] -= step * error;
}
}
}
return model;
}
/**
* Every team with the probability the model gives it, best first.
*
* A support desk needs the runner-up: two teams at almost the same score is
* exactly the ambiguous ticket of N0, and here it is visible instead of being
* silently resolved by a priority order.
*/
export function rank(model, ticket) {
const probabilities = scores(model, vector(model.vocabulary, ticket));
return model.classes
.map((team, c) => [team, probabilities[c]])
.sort((a, b) => b[1] - a[1]);
}
/**
* The team that gets the ticket, or the default queue below the floor.
*
* The floor is yours to set. Raise it and more tickets are read by a human
* before they move; lower it and more tickets are moved on a hunch. Nothing in
* the model can make that choice for you.
*/
export function route(model, ticket, { minConfidence = 0.5, defaultTeam = DEFAULT_TEAM } = {}) {
const [team, confidence] = rank(model, ticket)[0];
return confidence >= minConfidence ? team : defaultTeam;
}Risks
- Data leaving
- Stays on your infrastructure
- Determinism
- Yes
- Testability
- Statistically testable
- Vendor dependency
- Library
- Footprint
- Low
- Regulatory scope
-
- Creation of a training corpus from real customer tickets, kept apart from the tickets it was drawn from
- The corpus belongs in your record of processing activities, where you keep one
Breaking point
The model knows the archive and nothing else. « Votre entrepôt accepte-t-il les visites scolaires le mercredi » overlaps none of the words in the resolved tickets: the three teams come out nearly tied, the confidence floor is not cleared, and the ticket returns to the default queue. N1 did not remove that queue, it made it smaller.
When to move up a rung
Agents keep pulling tickets out of the default queue that turn out to be ordinary known problems, described in words the archive does not contain.
N2 — Small self-hosted specialised model Small self-hosted specialised model
Self-hosted sentence encoder, vote of the nearest resolved tickets
- Cost
- Low
- Latency
~100 ms
Proof of execution : Code runs, external service simulated
This snippet runs on every build of the site, but its test replaces the external service with a local double. What is verified: the request sent, the response decoded, and the error paths. What is not: how good the model’s answer is.
Python
"""
Route a ticket by its nearest resolved tickets, with a self-hosted encoder.
Rung N2. N1 learns the words of the archive; an encoder maps a ticket to a
vector by meaning, so a customer who says « je n'arrive plus à entrer dans mon
espace » lands next to the archived tickets about a lost password even though
they share no word with them.
There is no training step here, and that is the point of the rung: the index
is the archive itself. A team that changes scope is a re-encoding, not a
retraining, and the neighbours are shown to the agent as the reason for the
routing — which is more than N1's weights ever explain.
What it costs: a model file to ship and keep in sync, a warm process to hold
it, and a routing whose answers change the day you upgrade the model. The
archive also has to be re-encoded then, and the old scores are not comparable
to the new.
"""
from __future__ import annotations
MODEL_NAME = "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"
DEFAULT_TEAM = "general"
def load_encoder(name: str = MODEL_NAME):
"""The real encoder: fetched once, then held in memory and run locally."""
from sentence_transformers import SentenceTransformer # pragma: no cover
return SentenceTransformer(name)
def build_index(tickets: list[str], teams: list[str], encoder=None) -> dict:
"""
Encode the resolved archive once, and keep the encoder for the queries.
`encoder` is injected so this can be tested without loading a model. Left
alone, it is the real one above.
"""
encoder = load_encoder() if encoder is None else encoder
tickets = list(tickets)
return {"tickets": tickets, "teams": list(teams), "encoder": encoder,
"vectors": [_unit(v) for v in encoder.encode(tickets)]}
def neighbours(index: dict, ticket: str, k: int = 3) -> list[tuple[float, str]]:
"""The k nearest resolved tickets, best first, with their cosine score."""
vector = _unit(index["encoder"].encode([ticket])[0])
scored = [(_dot(known, vector), team)
for known, team in zip(index["vectors"], index["teams"])]
# A stable sort, so two equally close tickets always come back in archive
# order. A routing run has to be replayable.
scored.sort(key=lambda pair: -pair[0])
return scored[:k]
def route(index: dict, ticket: str, k: int = 3, min_similarity: float = 0.25,
default_team: str = DEFAULT_TEAM) -> str:
"""
The team of the nearest resolved tickets, each voting with its similarity.
A vote rather than the single best neighbour: one archived ticket that
happens to be phrased like this one is an accident, three of them are a
pattern.
The floor is what keeps the default queue of N0 alive. Below it, nothing
in the archive resembles this ticket, and the honest answer is that this
rung has never seen the problem.
"""
votes: dict[str, float] = {}
for similarity, team in neighbours(index, ticket, k):
if similarity >= min_similarity:
votes[team] = votes.get(team, 0.0) + similarity
if not votes:
return default_team
# Ties go to the team of the closest neighbour, which is the first key
# inserted above.
return max(votes, key=lambda team: votes[team])
def _unit(vector) -> list[float]:
"""Cosine similarity is a dot product once both sides have length one."""
values = [float(v) for v in vector]
norm = sum(v * v for v in values) ** 0.5
return [v / norm for v in values] if norm else values
def _dot(a: list[float], b: list[float]) -> float:
return sum(x * y for x, y in zip(a, b))JavaScript
/**
* Route a ticket by its nearest resolved tickets, with a self-hosted encoder.
*
* Rung N2. N1 learns the words of the archive; an encoder maps a ticket to a
* vector by meaning, so a customer who says « je n'arrive plus à entrer dans
* mon espace » lands next to the archived tickets about a lost password even
* though they share no word with them.
*
* There is no training step here, and that is the point of the rung: the index
* is the archive itself. A team that changes scope is a re-encoding, not a
* retraining, and the neighbours are shown to the agent as the reason for the
* routing — which is more than N1's weights ever explain.
*
* What it costs: a model file to ship and keep in sync, a warm process to hold
* it, and a routing whose answers change the day you upgrade the model. The
* archive also has to be re-encoded then, and the old scores are not
* comparable to the new.
*/
export const MODEL_NAME = 'Xenova/paraphrase-multilingual-MiniLM-L12-v2';
export const DEFAULT_TEAM = 'general';
/** The real encoder: fetched once, then held in memory and run locally. */
export async function loadEncoder(name = MODEL_NAME) {
const { pipeline } = await import('@xenova/transformers');
const extract = await pipeline('feature-extraction', name);
return { encode: async (texts) => (await extract(texts, { pooling: 'mean' })).tolist() };
}
/**
* Encode the resolved archive once, and keep the encoder for the queries.
*
* `encoder` is injected so this can be tested without loading a model. Left
* alone, it is the real one above.
*/
export async function buildIndex(tickets, teams, encoder) {
const model = encoder ?? (await loadEncoder());
const kept = [...tickets];
return {
tickets: kept,
teams: [...teams],
encoder: model,
vectors: (await model.encode(kept)).map(unit),
};
}
/** The k nearest resolved tickets, best first, with their cosine score. */
export async function neighbours(index, ticket, k = 3) {
const vector = unit((await index.encoder.encode([ticket]))[0]);
const scored = index.vectors.map((known, i) => [dot(known, vector), index.teams[i]]);
// A stable sort, so two equally close tickets always come back in archive
// order. A routing run has to be replayable.
return scored.sort((a, b) => b[0] - a[0]).slice(0, k);
}
/**
* The team of the nearest resolved tickets, each voting with its similarity.
*
* A vote rather than the single best neighbour: one archived ticket that
* happens to be phrased like this one is an accident, three of them are a
* pattern.
*
* The floor is what keeps the default queue of N0 alive. Below it, nothing in
* the archive resembles this ticket, and the honest answer is that this rung
* has never seen the problem.
*/
export async function route(index, ticket, { k = 3, minSimilarity = 0.25, defaultTeam = DEFAULT_TEAM } = {}) {
const votes = new Map();
for (const [similarity, team] of await neighbours(index, ticket, k)) {
if (similarity >= minSimilarity) votes.set(team, (votes.get(team) ?? 0) + similarity);
}
if (votes.size === 0) return defaultTeam;
// Ties go to the team of the closest neighbour, which is the first key
// inserted above.
let best = defaultTeam;
let bestVote = -Infinity;
for (const [team, vote] of votes) {
if (vote > bestVote) [best, bestVote] = [team, vote];
}
return best;
}
/** Cosine similarity is a dot product once both sides have length one. */
function unit(vector) {
let sum = 0;
for (const v of vector) sum += v * v;
const norm = Math.sqrt(sum);
return norm ? vector.map((v) => v / norm) : [...vector];
}
function dot(a, b) {
let total = 0;
for (let i = 0; i < a.length; i += 1) total += a[i] * b[i];
return total;
}Risks
- Data leaving
- Stays on your infrastructure
- Determinism
- Yes
- Testability
- Statistically testable
- Vendor dependency
- Library
- Footprint
- Moderate
- Regulatory scope
-
- An encoded copy of the archive kept as an index on your own infrastructure
- An erasure request on an archived ticket also reaches the index, which is a copy of it
- Model downloaded from a third party: its licence and its provenance are yours to check
Breaking point
There is no model of the teams on this rung, only an archive, and the nearest ticket is not always a relevant one. « Bonjour, merci de me confirmer que vous avez bien reçu mon dossier » is addressed to nobody in particular and lands on billing with a high score: the archive happens to hold one billing ticket written with the same politeness formulas, and the vote reads that as a strong match. A real encoder moves where the accident happens, it does not remove it: whatever the archive is made of is the routing policy, including the parts nobody chose.
When to move up a rung
Tickets arrive in a language the archive does not contain, or the routing rule exists as a written instruction that no resolved ticket illustrates.
N3 — General-purpose LLM API General-purpose LLM API
Classification through a general-purpose model, against a closed list of teams
- Cost
- High
- Latency
~1 s
Proof of execution : Code runs, external service simulated
This snippet runs on every build of the site, but its test replaces the external service with a local double. What is verified: the request sent, the response decoded, and the error paths. What is not: how good the model’s answer is.
Python
"""
Route a ticket by asking a general-purpose model.
Rung N3. This is the option people reach for first, and it is the shortest
piece of routing logic on the entry: no rules to maintain, no archive to
label, and a ticket in any language.
Note what the code has to do that N0 did not: cap the input size, retry a
provider that fails, parse an answer that is only probably JSON, and refuse a
team the model made up. That plumbing is the real cost of this rung, and it is
the part your tests have to cover, because the model itself is not testable.
Read the end of `route` closely. A model answers with words, and
words are not queues. The closed list is the only thing standing between a
confident answer and a ticket sitting in a queue nobody watches.
"""
from __future__ import annotations
import json
# The queues that exist. Business knowledge, and here also a safety rail.
TEAMS = ("billing", "technical", "shipping")
DEFAULT_TEAM = "general"
MAX_CHARACTERS = 4000
PROMPT = (
"You are routing a customer support ticket to one team.\n"
"Answer with JSON only: {{\"team\": \"...\"}} where team is one of: {teams}.\n"
"If the ticket does not clearly belong to one of them, answer {default!r}.\n\n"
"Ticket:\n{ticket}"
)
class RoutingUnavailable(Exception):
"""The provider could not be reached, or answered something unparseable."""
def route(ticket: str, client=None, *, attempts: int = 3) -> str:
"""
The team that gets the ticket.
`client` is injected so this function can be tested without a network
call. In production it defaults to a real provider client.
"""
if client is None: # pragma: no cover - needs a key and a network
from openai import OpenAI
client = OpenAI()
# A model charges by the token, and a ticket with a forwarded thread under
# it is long. Refusing oversized input is not an optimisation, it is a
# cost control.
if len(ticket) > MAX_CHARACTERS:
raise ValueError(f"ticket longer than {MAX_CHARACTERS} characters")
answer = _ask(client, ticket, attempts)
team = answer.get("team") if isinstance(answer, dict) else None
team = str(team).strip().lower() if team is not None else ""
# Two failures, two treatments. A provider that cannot answer is an
# incident, and `_ask` above raises. A model that answers a team nobody
# created is a normal Tuesday, and the ticket goes to the default queue.
return team if team in TEAMS else DEFAULT_TEAM
def _ask(client, ticket: str, attempts: int) -> dict:
prompt = PROMPT.format(teams=", ".join(TEAMS), default=DEFAULT_TEAM, ticket=ticket)
last_error: Exception | None = None
for _ in range(attempts):
try:
# Temperature zero, because a routing decision that changes
# between two identical calls cannot be reviewed.
return json.loads(client.complete(prompt=prompt, temperature=0))
except Exception as error: # noqa: BLE001 - any provider failure is retried
last_error = error
raise RoutingUnavailable(str(last_error))JavaScript
/**
* Route a ticket by asking a general-purpose model.
*
* Rung N3. This is the option people reach for first, and it is the shortest
* piece of routing logic on the entry: no rules to maintain, no archive to
* label, and a ticket in any language.
*
* Note what the code has to do that N0 did not: cap the input size, retry a
* provider that fails, parse an answer that is only probably JSON, and refuse
* a team the model made up. That plumbing is the real cost of this rung, and
* it is the part your tests have to cover, because the model itself is not
* testable.
*
* Read the end of `route` closely. A model answers with words, and words are
* not queues. The closed list is the only thing standing between a confident
* answer and a ticket sitting in a queue nobody watches.
*/
// The queues that exist. Business knowledge, and here also a safety rail.
export const TEAMS = ['billing', 'technical', 'shipping'];
export const DEFAULT_TEAM = 'general';
export const MAX_CHARACTERS = 4000;
const PROMPT = [
'You are routing a customer support ticket to one team.',
`Answer with JSON only: {"team": "..."} where team is one of: ${TEAMS.join(', ')}.`,
`If the ticket does not clearly belong to one of them, answer '${DEFAULT_TEAM}'.`,
'',
'Ticket:',
].join('\n');
export class RoutingUnavailable extends Error {}
/**
* The team that gets the ticket.
*
* @param {string} ticket
* @param {object} options
* @param {{complete: Function}} [options.client] injected so this can be
* tested without a network call; defaults to a real provider client
* @param {number} [options.attempts]
*/
export async function route(ticket, { client, attempts = 3 } = {}) {
if (!client) {
// Needs a key and a network, so it is never reached in the tests.
const { OpenAI } = await import('openai');
client = new OpenAI();
}
// A model charges by the token, and a ticket with a forwarded thread under
// it is long. Refusing oversized input is not an optimisation, it is a cost
// control.
if (ticket.length > MAX_CHARACTERS) {
throw new RangeError(`ticket longer than ${MAX_CHARACTERS} characters`);
}
const answer = await ask(client, ticket, attempts);
const named = answer && typeof answer.team === 'string' ? answer.team.trim().toLowerCase() : '';
// Two failures, two treatments. A provider that cannot answer is an
// incident, and `ask` above throws. A model that answers a team nobody
// created is a normal Tuesday, and the ticket goes to the default queue.
return TEAMS.includes(named) ? named : DEFAULT_TEAM;
}
async function ask(client, ticket, attempts) {
let lastError;
for (let i = 0; i < attempts; i += 1) {
try {
// Temperature zero, because a routing decision that changes between two
// identical calls cannot be reviewed.
const answer = await client.complete({ prompt: `${PROMPT}\n${ticket}`, temperature: 0 });
return JSON.parse(answer);
} catch (error) {
lastError = error;
}
}
throw new RoutingUnavailable(String(lastError));
}Risks
- Data leaving
- Goes to a third party
- Determinism
- No
- Testability
- Hard to test
- Vendor dependency
- External provider
- Footprint
- High
- Regulatory scope
-
- Transfer of ticket content to a processor, with the contractual framing that implies
- A ticket is free text: the transfer also covers whatever the customer wrote in it unprompted
- Processing location to be confirmed with the provider
- Does not excuse you from your own duties of transparency and minimisation
Breaking point
The model answers words, and words are not queues. Asked to pick from three teams, it returns a fourth that sounds right and does not exist: « customer success », as well-formed JSON, with no hedging at all. The closed list catches it and the ticket goes to the default queue; take that check out, as the first version of this kind of code usually does, and the ticket is filed into a team nobody created and nobody watches.
When to move up a rung
There is no rung above this one.
The verdict
RecommendedN1
N1 wins because its training data already exists: the archive of resolved tickets, each labelled with the team that closed it, is one export away, and it routes the phrasings N0's keyword list never anticipated. It also puts the ambiguous ticket back on the table, since the runner-up team is part of the answer, where N0 dropped it silently. N2 buys paraphrase matching at the price of a model file, a warm process and an index to re-encode on every upgrade: that price is worth paying the day your agents start fishing known problems out of the default queue, not before.
Further reading
- scikit-learn — Extraction de caractéristiques sur du texte, TF-IDF
- scikit-learn — Classification de documents texte, exemple complet
- Introduction to Information Retrieval — la pondération tf-idf
- Sentence Transformers — encodeurs de phrases auto-hébergés