Sentence Embeddings & Semantic Search

By the end of this lesson you will be able to build search that understands meaning: a query like "fees in parts" should find "Can I pay the hostel fee in instalments?" even though they share almost no words. You will learn what sentence embeddings are, how vector search ranks results, how to mix keyword and meaning scores, and how to measure whether your search is any good.

In TF-IDF you built a search engine that matched exact words. It failed when the user chose different words from the document. Semantic search compares meanings instead.

Sentence embeddings

A sentence embedding is one vector for a whole sentence or paragraph, produced by a model trained so that sentences with similar meanings get similar vectors. "My parcel never came" and "order not delivered" end up close together; "order not delivered" and "update my email" end up far apart.

It is like a librarian who has read every book and shelves them by subject, not by title. Ask for "books about saving money" and you are taken to the personal finance shelf, even if no title contains "saving".

Popular models come from the sentence-transformers library. This is plain code for your own computer (it needs sentence-transformers, which installs PyTorch):

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
faqs = ["How do I pay the semester fee online?",
        "Can I pay the hostel fee in two instalments?",
        "How can I get a duplicate ID card?"]
embeddings = model.encode(faqs)
print(embeddings.shape)

(3, 384)

This model gives 384 numbers per sentence, so three FAQs become a 3 × 384 table. To compare a query with every FAQ, embed the query the same way and ask the model for cosine similarities:

query = model.encode(["fees in parts"])
print(model.similarity(query, embeddings))      # one score per FAQ

The result is a 1 × 3 table of scores. Run it and check that the instalment FAQ scores highest; we do not show the exact numbers because PyTorch could not run on our test machine. For Hindi, Tamil or mixed-language text, choose a multilingual model and test it on your own queries.

How vector search ranks results

Whatever model you use, search works the same way:

1. Index time: embed every document once and store the vectors.
2. Query time: embed the query.
3. Compute the cosine similarity between the query and every document vector.
4. Return the top k documents.

If all vectors are scaled to length 1 first (normalised), cosine similarity is just a dot product, which is very fast. In the example below, the vectors are small hand-made stand-ins for model output, so you can see every number. Imagine the four positions mean "fees", "documents", "hostel" and "exams".

import numpy as np

faqs = ["How do I pay the semester fee online?",
        "Can I pay the hostel fee in two instalments?",
        "How can I get a duplicate ID card?",
        "Where do I collect the hostel room key?",
        "When is the exam form due?"]

# Stand-in sentence vectors (a real model would give 384 or more numbers)
doc_vecs = np.array([[0.9, 0.1, 0.0, 0.2], [0.8, 0.0, 0.6, 0.0], [0.1, 0.9, 0.1, 0.0],
                     [0.0, 0.1, 0.9, 0.0], [0.2, 0.3, 0.0, 0.9]])
queries = {"fees in parts": [0.9, 0.0, 0.3, 0.0], "lost my college card": [0.0, 0.95, 0.0, 0.1]}

def normalise(m):
    m = np.asarray(m, dtype=float)
    return m / np.linalg.norm(m, axis=-1, keepdims=True)

docs = normalise(doc_vecs)                                   # Step 1: done once
for text, vec in queries.items():
    scores = docs @ normalise(vec)                           # Steps 2-3: dot product = cosine
    top = np.argsort(scores)[::-1][:2]                       # Step 4: top k
    print(f"{text!r}")
    for i in top:
        print(f"   {scores[i]:.2f}  {faqs[i]}")

'fees in parts'
   0.95  Can I pay the hostel fee in two instalments?
   0.92  How do I pay the semester fee online?
'lost my college card'
   0.98  How can I get a duplicate ID card?
   0.40  When is the exam form due?

"fees in parts" ranks the instalment FAQ first, with the online-fee FAQ close behind. "lost my college card" finds the ID card FAQ with no shared words at all. That is the strength of meaning-based search, provided the embedding model is good.

Keyword plus meaning: hybrid search

Embeddings are not perfect. They can miss exact codes, names and numbers: a query for "form 16" or an order ID needs an exact match. Hybrid search combines a keyword score (TF-IDF or BM25, a popular improved version of TF-IDF used by search engines) with the embedding score:

final score = alpha × meaning score + (1 − alpha) × keyword score

import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

docs = ["Order OD458812 delayed at Pune hub",
        "Refund for order OD771203 is processed",
        "Parcels to Pune are delayed due to rain"]
query = "where is OD458812"

vec = TfidfVectorizer(stop_words="english").fit(docs)
keyword = cosine_similarity(vec.transform([query]), vec.transform(docs))[0]
meaning = np.array([0.55, 0.60, 0.50])     # stand-in embedding scores: the IDs look alike to the model

for alpha in (1.0, 0.5, 0.0):
    final = alpha * meaning + (1 - alpha) * keyword
    best = final.argmax()
    print(f"alpha={alpha}: {np.round(final, 2)} -> {docs[best]}")

alpha=1.0: [0.55 0.6  0.5 ] -> Refund for order OD771203 is processed
alpha=0.5: [0.53 0.3  0.25] -> Order OD458812 delayed at Pune hub
alpha=0.0: [0.52 0.   0.  ] -> Order OD458812 delayed at Pune hub

With meaning only (alpha=1.0), the wrong order wins, because order IDs look alike to an embedding model. With keywords included, the exact ID decides. Real systems tune alpha on a set of test queries.

Measuring search quality

You cannot improve search without measuring it. Build a small test set: real queries, each with the IDs of the documents that should come back. Two simple metrics:

• Recall@k: the share of queries where a correct document appears in the top k results.
• MRR (mean reciprocal rank): for each query, take 1 divided by the rank of the first correct result (1 for first place, 0.5 for second, 0.33 for third), then average.

results = {          # ranked document IDs returned by a search system for each query
    "fees in parts":        [2, 1, 5],
    "lost id card":         [3, 7, 1],
    "hostel key":           [6, 4, 2],
    "exam form last date":  [8, 9, 5],
}
relevant = {"fees in parts": {2}, "lost id card": {3}, "hostel key": {4}, "exam form last date": {5}}

def recall_at_k(k):
    hits = sum(1 for q, ranked in results.items() if set(ranked[:k]) & relevant[q])
    return hits / len(results)

def mrr():
    total = 0.0
    for q, ranked in results.items():
        for rank, doc in enumerate(ranked, start=1):
            if doc in relevant[q]:
                total += 1 / rank
                break
    return total / len(results)

print(f"Recall@1 = {recall_at_k(1):.2f}  Recall@3 = {recall_at_k(3):.2f}  MRR = {mrr():.2f}")

Recall@1 = 0.50  Recall@3 = 1.00  MRR = 0.71

Two of four queries got the right answer first; all four found it within the top 3. MRR rewards putting the right answer near the top. Run these numbers before and after every change (new model, new alpha, new chunking) and keep the change only if they improve.

Storing vectors at scale

For a few thousand documents, a NumPy array and a dot product are enough. For millions, you need approximate nearest neighbour (ANN) search, which finds very close vectors without checking every one. Tools include FAISS (a library), pgvector (a PostgreSQL extension) and dedicated vector databases. The Generative AI course covers these in Chunking, Vector Stores & Hybrid Search, and uses semantic search to feed documents to an AI model in Retrieval-Augmented Generation (RAG).

Common mistakes

• Mixing models. Query and document vectors must come from the same model and version.
• Forgetting to normalise. A dot product equals cosine similarity only for vectors of length 1.
• Embedding very long documents whole. Models have an input limit and long texts blur together. Split them into chunks of a few sentences.
• Dropping keyword search. IDs, codes and rare names need exact matching; use hybrid search.
• Never measuring. Keep a test set of queries and track recall@k and MRR.

Key points

• A sentence embedding is one vector per sentence; similar meanings give similar vectors.
• Semantic search embeds documents once, embeds each query, and ranks by cosine similarity.
• Normalised vectors make cosine similarity a fast dot product.
• Hybrid search mixes keyword and meaning scores, which helps with IDs, codes and names.
• Measure search with a test set using recall@k and MRR.

results = {"fees in parts": [2, 1, 5], "lost id card": [3, 7, 1], "hostel key": [6, 4, 2],
           "exam form last date": [8, 9, 5], "change mobile number": [11, 12, 10]}
relevant = {"fees in parts": {2}, "lost id card": {3}, "hostel key": {4},
            "exam form last date": {5}, "change mobile number": {10}}

def recall_at_k(k):
    return sum(1 for q, r in results.items() if set(r[:k]) & relevant[q]) / len(results)

def mrr():
    total = 0.0
    for q, r in results.items():
        for rank, doc in enumerate(r, start=1):
            if doc in relevant[q]:
                total += 1 / rank
                break
    return total / len(results)

def precision_at_k(k):
    return sum(len(set(r[:k]) & relevant[q]) / k for q, r in results.items()) / len(results)

print(f"Recall@1 {recall_at_k(1):.2f}, Recall@3 {recall_at_k(3):.2f}, MRR {mrr():.2f}, P@3 {precision_at_k(3):.2f}")

Recall@1 0.40, Recall@3 1.00, MRR 0.63, P@3 0.33