Transformers and LLMs for NLP

LLMs can do many NLP tasks with no training data. This module shows how to use them for classic tasks — and when a cheaper model is the better choice.

5.1 From word counts to transformers

NLP moved through three broad stages: rules and counts (regex, bag of words, TF-IDF), learned representations (word embeddings and recurrent networks that read text in order) and, since 2017, transformers — the architecture behind today's LLMs. The background is covered in How AI and LLMs Work; here we focus on what it means for practical NLP.

5.2 Attention in one idea

A transformer's key mechanism, attention, lets every token look at every other token and decide which ones matter for its meaning. In "The parcel didn't fit in the box because it was too big", attention helps the model link "it" to "parcel". Because attention handles context directly and runs in parallel, transformers can be trained on enormous amounts of text.

5.3 Tokens, again — and why they cost money

LLMs read subword tokens, not words: common words are one token, rare words and names split into several. API pricing, speed limits and context windows are all measured in tokens, so long inputs cost more and run slower. Non-English text and code often use more tokens per word.

5.4 Zero-shot and few-shot classification

An LLM can classify text with no training data (zero-shot) or with a handful of examples in the prompt (few-shot) — ideal when you have no labels or categories change often.

Python · few-shot classification with an LLM (optional — needs an API key)EXAMPLES = """Message: The kettle stopped working after two days -> product
Message: I was billed twice this month -> billing
Message: Courier left my parcel in the rain -> delivery"""

def llm_classify(message):
    prompt = (f"Classify the customer message as delivery, product or billing. "
              f"Reply with one word.\n\n{EXAMPLES}\nMessage: {message} ->")
    reply = client.messages.create(model=MODEL, max_tokens=5,
                                   messages=[{"role": "user", "content": prompt}])
    return reply.content[0].text.strip().lower()

sample = X_test.head(20)
llm_preds = [llm_classify(m) for m in sample]
print("Agreement with labels:", np.mean(np.array(llm_preds) == y_test.head(20).values).round(2))

Always measure an LLM classifier on labelled examples exactly as you would a classic model — prompts that look fine on three messages can fail on the three-hundredth.

5.5 Classic NLP, LLM, or both?

A common pattern: use an LLM to label a few thousand examples, have humans check a sample, then train a cheap classic model on those labels for high-volume use — and send low-confidence cases back to the LLM or a person.

def route(texts, threshold=0.7):
    probs = clf.predict_proba(texts)
    out = pd.DataFrame({"text": list(texts),
                        "predicted": clf.classes_[probs.argmax(axis=1)],
                        "confidence": probs.max(axis=1).round(2)})
    out["needs_review"] = out["confidence"] < threshold
    return out

r = route(X_test)
print(r["needs_review"].mean().round(2), "of messages need review")
print(r.sort_values("confidence").head())

Key takeaways

• Transformers use attention to model context.
• LLM cost, speed and limits are measured in tokens.
• Few-shot prompts classify without training data — but measure them.
• Classic models are cheap and predictable; LLMs handle nuance.
• Combine them: LLM labels, classic model at scale, humans for edge cases.