AI for triaging inbound requests and email: what to hand the model and where a human is needed

Short answer: a language model does the first pass over an inbound stream well. It determines the topic and the urgency, pulls the company details, the line items and the quantities out of an email, creates a record in the CRM and routes it to the right person. It gets things wrong on rare request types, on poor attachments and wherever a decision has to be made. So at the start the model prepares the record and a person confirms it. Confirmation is removed type by type, once the share of corrections for that type gets small.

What the model does well

Topic and routing. A quote request, a complaint, a payment question, a request for documents, spam. The model tells them apart by meaning, even when the email contains none of the obvious words, and sends each into the right queue.

Extraction. Company name, tax number, contact, the list of items with quantities, the date wanted. Out of free-form email text and out of an attached table, this is done more accurately than rules built on regular expressions can manage.

Matching against reference data. “Cable, three by two and a half, a hundred metres” gets matched to a line in your own product list. Where several candidates fit, it shows them to the person.

A short summary. A long thread folds into three lines: who, what they want, by when.

Where it gets things wrong

Rare requests. A request type that arrives once a month gets confused with a similar frequent one. Those types are better sent straight to a human.

Poor attachments. An invoice scan with a stamp across the text, a photo of a delivery note taken at an angle. The text is recognised with errors, and the errors travel into the record.

Decisions. Whether to give a discount, whether to accept a complaint, whether this customer counts as urgent under your own rules. The model can suggest; the call belongs to a person or to a hard rule in the system.

How to measure quality

Take a few hundred real requests from past months and label them by hand: topic, extracted fields, who it went to. Run the model over them and count how much matched, field by field and request type by request type.

After launch, track the share of records a person corrected, separately by type. Where corrections are rare, confirmation can come off. Where they are frequent, look for the reason: too few examples, poor attachments, or a type that is simply too rare.

What a launch needs

Access to the mailbox or the form the requests arrive through. Reference data for products and customers, ideally exported from the accounting system or the CRM. A list of request types and the rules for who gets which type. And a sample of past requests to measure against.

If the email contains personal data about customers, the model runs in a Russian cloud or on your own server. Why that is so is covered in can you put customer data into ChatGPT.

Common questions

How much time does this triage save? It depends on the volume. If a manager spends several minutes moving a request from an email into the CRM, and there are dozens of requests a day, the saving is visible immediately. Work it out on your own numbers before the project.

Does this need a model of our own? No. An off-the-shelf model with good instructions and examples is enough; fine-tuning is rarely needed. When it is justified is covered in RAG or fine-tuning.

Can the model answer the customer itself? Simple questions with a document to cite, yes; the rest is better left to a person. A mistake in a reply to a customer costs more than a mistake in an internal record.

The rule

Give the model the triage and the draft record, and keep the decisions with a person. Take confirmation off one request type at a time, once corrections on that type have all but stopped.