Embeddings and vector search

Semantic search is the engine of RAG. In this module you'll build it from scratch and see both its power and its blind spots.

2.1 Embeddings recap

An embedding turns a piece of text into a list of numbers — a vector — so that texts with similar meanings get similar vectors. Embedding models are trained specifically for this. OpenAI's text-embedding-3-small, for example, returns 1,536 numbers per text by default; open-source models from the sentence-transformers family commonly return 384 or 768.

2.2 Measuring similarity

The standard measure is cosine similarity: the cosine of the angle between two vectors. It ranges from −1 to 1; higher means more similar meaning. Many embedding models produce normalised vectors (length 1), in which case cosine similarity is simply the dot product.

Python · cosine similarity from scratchimport numpy as np

def cosine(a, b):
    a, b = np.array(a), np.array(b)
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

print(cosine([1, 0], [1, 0]))   # 1.0  same direction
print(cosine([1, 0], [0, 1]))   # 0.0  unrelated
print(cosine([1, 0], [-1, 0]))  # -1.0 opposite

2.3 Semantic search in a few lines

Python · semantic search with an open-source embedding model# !pip install sentence-transformers
from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("all-MiniLM-L6-v2")   # small, free, runs on CPU

docs = [
    "Damaged items can be returned within 30 days for a full refund.",
    "Standard UK delivery takes 3 to 5 working days.",
    "Our bamboo boards should be hand-washed and oiled monthly.",
    "Gift cards are valid for 12 months from purchase.",
]
doc_vecs = model.encode(docs, normalize_embeddings=True)

def search(query, k=2):
    q = model.encode([query], normalize_embeddings=True)[0]
    scores = doc_vecs @ q                      # cosine, because vectors are normalised
    top = np.argsort(-scores)[:k]
    return [(round(float(scores[i]), 3), docs[i]) for i in top]

print(search("my chopping board arrived cracked, can I get my money back?"))

Notice the query shares almost no words with the refund policy ("cracked" vs "damaged", "money back" vs "refund") — yet semantic search finds it. That's the power of embeddings.

2.4 Vector databases and indexes

Comparing a query with every vector works for thousands of chunks. For millions, systems use approximate nearest neighbour (ANN) indexes, such as HNSW graphs, that find very close matches much faster by trading a little accuracy for speed.

Options range from libraries (FAISS) to dedicated vector databases and vector features in existing databases (e.g. PostgreSQL with pgvector) and cloud search services (Azure AI Search, Amazon OpenSearch, Google Vertex AI Vector Search). Choose based on scale, filtering needs, security and what your team already runs.

2.5 Choosing an embedding model

• Quality on your data — test a few models on real queries (Module 6). Public leaderboards are a starting point, not an answer.
• Language and domain — multilingual content or specialised jargon needs a model that handles it.
• Size and cost — dimensions affect storage and speed; some models let you shorten vectors.
• Privacy — sending documents to an API vs running a model yourself.
• Consistency — queries and documents must be embedded with the same model. Changing models means re-embedding everything.

Key takeaways

• Embeddings map meaning to vectors; similar meaning → similar vectors.
• Cosine similarity measures closeness; with normalised vectors it's the dot product.
• Semantic search handles paraphrases but can miss exact identifiers.
• ANN indexes make search fast at scale.
• Use the same embedding model for documents and queries; test candidates on your data.