Embeddings Fundamentals

Today’s milestone

By the end of today, you should be able to:

• Explain what an embedding is.
• Explain why semantic search needs embeddings.
• Generate embeddings for feedback text.
• Compare two embeddings using cosine similarity.
• Build a tiny in-memory semantic search.
• Explain embeddings clearly in an interview.

Do not study pgvector, chunking, RAG, or vector indexes today.

1. The problem embeddings solve

Computers work well with numbers, but users give us text:

"The delivery was late."

Traditional application code cannot directly calculate whether this sentence is similar to:

"My order arrived after the expected date."

The words are different, but the meaning is similar.

Keyword matching may fail because:

delivery != order
late != after expected date

Embeddings solve this by converting text into numerical representations that capture aspects of meaning.

Text
  ↓
Embedding model
  ↓
Vector of numbers

Example:

"The delivery was late."

might become conceptually:

[0.12, -0.43, 0.88, 0.21, ...]

The actual vector may contain hundreds or thousands of numbers.

2. What is an embedding?

An embedding is a dense numerical vector representing an item in a learned semantic space.

The item could be:

• A sentence
• A paragraph
• A document
• An image
• A product
• A user
• A code snippet

For your project:

Customer feedback
       ↓
Embedding model
       ↓
Feedback embedding

Similar feedback should have vectors located closer together.

"Delivery was late"
        ≈
"Order arrived too slowly"

Different feedback should be farther apart:

"Delivery was late"
        ≠
"The payment page is not working"

3. Important mental model

Imagine a multidimensional map of meanings.

In a simplified two-dimensional illustration:

                 Delivery issues

        "Package arrived late"
                 ●

             ● "Slow shipping"


Billing issues                         Positive reviews

● "Charged twice"                     ● "Excellent product"

      ● "Refund missing"                    ● "Loved it"

Real embeddings do not usually have two dimensions. They may have hundreds or thousands.

Each dimension is not necessarily something human-readable like:

dimension 1 = sentiment
dimension 2 = delivery

The dimensions are learned mathematical features distributed across the vector.

Do not try to interpret individual numbers directly.

The useful information comes from comparing the complete vectors.

4. Embedding model versus generative model

This distinction is important.

Generative model

Input:

Summarize this customer feedback.

Output:

The customer is unhappy about delayed shipping.

It generates text.

Embedding model

Input:

The delivery was late.

Output:

[0.12, -0.43, 0.88, ...]

It generates a numerical representation.

A RAG application often uses both:

Embedding model → find relevant information
Generative model → create the final answer

5. What is a vector?

A vector is an ordered list of numbers.

feedback_embedding = [
    0.12,
    -0.43,
    0.88,
    0.21,
]

Each number is a coordinate in a multidimensional space.

A two-dimensional point looks like:

[2, 5]

A three-dimensional point:

[2, 5, 8]

An embedding may look like:

[0.12, -0.43, 0.88, ..., 0.34]

with many dimensions.

Vector dimension

The number of values in the vector is called its dimensionality.

embedding = [0.1, 0.4, -0.2]

dimension = len(embedding)  # 3

In production, the dimension is determined by the embedding model.

You must know it because:

• Database vector columns need a dimension.
• Vectors with different dimensions cannot be compared directly.
• Changing embedding models may require re-embedding stored data.

6. Why similar meanings produce similar vectors

Embedding models are trained on large datasets to learn relationships between language elements.

The model learns that sentences such as:

"The delivery was late."
"My order arrived slowly."
"Shipping took too long."

often appear in related contexts.

It places them in nearby regions of vector space.

It may place:

"The card payment failed."

farther away because the meaning is different.

The model is not storing a dictionary such as:

{
    "late": "delivery problem",
}

It learns statistical relationships and patterns across language.

7. Semantic search versus keyword search

Suppose the user searches:

slow shipping

Your database contains:

My package arrived three days after the promised date.

Keyword search

Looks for exact or related tokens:

slow
shipping

Neither word exists in the stored feedback, so it may not match.

Semantic search

Converts both texts into embeddings:

"slow shipping"
        ↓
Query vector

"My package arrived three days after the promised date"
        ↓
Feedback vector

Then compares the vectors.

Because their meanings are related, the similarity should be high.

Comparison

Production systems often combine both using hybrid search.

You do not need hybrid search today.

8. Similarity measurement

Once you have embeddings, you need a way to compare them.

Common options:

• Cosine similarity
• Dot product
• Euclidean distance

Today, focus on cosine similarity.

Cosine similarity intuition

Cosine similarity measures how similar the directions of two vectors are.

Typical interpretation:

1.0   → highly similar direction
0.0   → unrelated directions
-1.0  → opposite directions

For many text embedding use cases, higher means more semantically similar.

You do not need to manually derive the formula today, but understand the concept:

Query vector
       ↘
        angle between vectors
       ↗
Feedback vector

Smaller angle means greater similarity.

Python implementation

import math


def cosine_similarity(
    vector_a: list[float],
    vector_b: list[float],
) -> float:
    if len(vector_a) != len(vector_b):
        raise ValueError(
            "Vectors must have the same dimensions."
        )

    dot_product = sum(
        a * b
        for a, b in zip(vector_a, vector_b, strict=True)
    )

    magnitude_a = math.sqrt(
        sum(value * value for value in vector_a)
    )
    magnitude_b = math.sqrt(
        sum(value * value for value in vector_b)
    )

    if magnitude_a == 0 or magnitude_b == 0:
        raise ValueError(
            "Cosine similarity is undefined for zero vectors."
        )

    return dot_product / (
        magnitude_a * magnitude_b
    )

Example:

vector_a = [1.0, 0.0]
vector_b = [0.9, 0.1]
vector_c = [0.0, 1.0]

print(cosine_similarity(vector_a, vector_b))
print(cosine_similarity(vector_a, vector_c))

vector_a and vector_b should be more similar than vector_a and vector_c.

9. How embeddings fit into your project

Your project is an AI Feedback Analysis Platform.

You already have structured classification:

Feedback
   ↓
LLM classification
   ↓
Sentiment, category, priority

Now you will add:

Feedback
   ↓
Embedding model
   ↓
Embedding vector

Later, that vector enables:

• Finding similar complaints
• Detecting repeated issues
• Semantic feedback search
• Grouping related feedback
• Retrieving evidence for RAG
• Finding duplicate tickets

Today, implement only:

Feedback → Embedding

and:

Query embedding
     ↓
Compare against feedback embeddings
     ↓
Return top matches

Store the vectors in memory for now.

10. Minimal architecture

Keep it small:

app/
├── providers/
│   └── embedding_provider.py
├── services/
│   └── semantic_search_service.py
├── schemas/
│   └── semantic_search.py
└── api/
    └── semantic_search.py

Do not create a complex abstraction hierarchy today.

11. Define an embedding provider

The exact SDK call depends on the provider you use. Keep your internal contract independent from the SDK.

from typing import Protocol


class EmbeddingProvider(Protocol):
    async def embed_text(
        self,
        text: str,
    ) -> list[float]:
        ...

This is useful because the application should care about:

await provider.embed_text(text)

not whether the provider is OpenAI, Gemini, or a local model.

Since you already understand provider abstraction, this should feel familiar.

12. Fake embedding provider for learning

Before calling a real model, you can verify your service design with a fake provider.

class FakeEmbeddingProvider:
    async def embed_text(
        self,
        text: str,
    ) -> list[float]:
        normalized_text = text.lower()

        return [
            1.0 if "delivery" in normalized_text else 0.0,
            1.0 if "payment" in normalized_text else 0.0,
            1.0 if "product" in normalized_text else 0.0,
            1.0 if "support" in normalized_text else 0.0,
        ]

This is not a real embedding model.

It is only useful for:

• Testing the pipeline
• Understanding vector comparison
• Avoiding API usage during setup

For example:

provider = FakeEmbeddingProvider()

embedding = await provider.embed_text(
    "Delivery was extremely slow."
)

print(embedding)

Output:

[1.0, 0.0, 0.0, 0.0]

13. Define your stored feedback structure

For today, keep records in memory.

from pydantic import BaseModel


class EmbeddedFeedback(BaseModel):
    id: int
    text: str
    embedding: list[float]

Search result:

class SemanticSearchResult(BaseModel):
    id: int
    text: str
    similarity: float

14. Build an in-memory semantic search service

from app.providers.embedding_provider import (
    EmbeddingProvider,
)
from app.schemas.semantic_search import (
    EmbeddedFeedback,
    SemanticSearchResult,
)


class SemanticSearchService:
    def __init__(
        self,
        embedding_provider: EmbeddingProvider,
    ) -> None:
        self.embedding_provider = embedding_provider
        self.records: list[EmbeddedFeedback] = []

    async def add_feedback(
        self,
        feedback_id: int,
        text: str,
    ) -> EmbeddedFeedback:
        embedding = (
            await self.embedding_provider.embed_text(text)
        )

        record = EmbeddedFeedback(
            id=feedback_id,
            text=text,
            embedding=embedding,
        )

        self.records.append(record)
        return record

    async def search(
        self,
        query: str,
        top_k: int = 3,
    ) -> list[SemanticSearchResult]:
        query_embedding = (
            await self.embedding_provider.embed_text(query)
        )

        results = [
            SemanticSearchResult(
                id=record.id,
                text=record.text,
                similarity=cosine_similarity(
                    query_embedding,
                    record.embedding,
                ),
            )
            for record in self.records
        ]

        results.sort(
            key=lambda result: result.similarity,
            reverse=True,
        )

        return results[:top_k]

15. Try the complete experiment

import asyncio


async def main() -> None:
    provider = FakeEmbeddingProvider()

    service = SemanticSearchService(
        embedding_provider=provider,
    )

    await service.add_feedback(
        feedback_id=1,
        text="The delivery was very late.",
    )

    await service.add_feedback(
        feedback_id=2,
        text="My payment was charged twice.",
    )

    await service.add_feedback(
        feedback_id=3,
        text="Customer support never replied.",
    )

    results = await service.search(
        query="delivery problem",
        top_k=2,
    )

    for result in results:
        print(
            result.text,
            result.similarity,
        )


asyncio.run(main())

With the fake provider, the delivery feedback should rank first.

The fake provider is simplistic, but the service flow is the same as with a real embedding model.

16. Replace fake embeddings with a real provider

Your real provider implementation should:

1. Accept text.
2. Call the embedding model.
3. Extract the vector.
4. Return list[float].
5. Translate provider errors into application exceptions.
6. Use timeout and retry configuration.

Conceptually:

class RealEmbeddingProvider:
    async def embed_text(
        self,
        text: str,
    ) -> list[float]:
        response = await self.client.create_embedding(
            model=self.model,
            input=text,
        )

        return response.embedding

Do not copy this exact method blindly—the SDK shape depends on your provider.

Your internal service should not change when you switch from fake to real:

service = SemanticSearchService(
    embedding_provider=real_provider,
)

That proves your provider abstraction is useful.

17. Important production concepts

You only need to understand these today, not implement all of them.

Use the same model for storage and queries

This is critical.

Bad:

Feedback embeddings → Model A
Query embeddings    → Model B

The vectors may exist in incompatible semantic spaces.

Good:

Feedback embeddings → Same model
Query embeddings    → Same model

Store embedding metadata

Later your database record should include:

embedding_model
embedding_dimension
embedding_version
created_at

Why?

Because if you change models later, you need to know which data must be re-embedded.

Embeddings are model-specific

A vector has no universal meaning by itself.

This:

[0.13, -0.82, ...]

is meaningful only in the space produced by its model.

Input quality matters

These produce different embeddings:

"Bad"

and:

"The product arrived damaged and support refused a replacement."

More context often leads to a more useful representation, but unnecessary text can also introduce noise.

18. What embeddings do not solve

Embeddings are powerful, but not magical.

They do not automatically provide:

• Exact factual correctness
• Business-rule validation
• Perfect classification
• Explainable dimensions
• Real-time knowledge
• Authorization filtering
• Guaranteed relevance

Example:

A semantic search may find text that is conceptually similar but not permitted for the current user.

Authorization must happen separately.

Semantic similarity ≠ access permission

19. Common misunderstandings

“An embedding is a summary”

No.

A summary is text.

An embedding is a numerical representation.

“Each dimension has a clear meaning”

Usually not.

Meaning is distributed across the vector.

“High similarity means identical”

No.

It means the model believes the items are semantically related.

“Embeddings understand truth”

No.

They model learned relationships, not guaranteed truth.

“Larger dimensions are always better”

No.

Higher dimensions can provide richer representation, but also increase:

• Storage
• Memory
• Search cost
• Index size

The best model depends on quality, latency, cost, language support, and your domain.

20. Today’s coding tasks

Required task 1

Implement:

cosine_similarity(
    vector_a,
    vector_b,
)

Test:

• Similar vectors
• Different vectors
• Different dimensions
• Zero vectors

Required task 2

Create:

EmbeddingProvider
FakeEmbeddingProvider

Required task 3

Create:

SemanticSearchService

with:

add_feedback()
search()

Required task 4

Add at least five feedback examples:

Delivery was late.
Payment failed.
Product quality is excellent.
Support did not respond.
I received the wrong product.

Search using:

shipping delay
billing issue
customer service problem

Observe which results rank highest.

Optional task

Replace the fake provider with your actual free embedding provider.

21. Tests to write

import pytest


@pytest.mark.asyncio
async def test_delivery_feedback_is_ranked_first():
    provider = FakeEmbeddingProvider()
    service = SemanticSearchService(provider)

    await service.add_feedback(
        1,
        "Delivery was late.",
    )
    await service.add_feedback(
        2,
        "Payment failed.",
    )

    results = await service.search(
        "delivery issue",
        top_k=1,
    )

    assert results[0].id == 1

Also test:

• top_k=2 returns at most two items.
• Results are sorted by descending similarity.
• Empty records return an empty list.
• Dimension mismatch raises an error.
• Empty text is rejected by Pydantic or service validation.

22. Interview preparation

What is an embedding?

An embedding is a dense numerical vector representing an item such as text in a learned semantic space. Items with related meaning tend to have nearby vectors, enabling semantic search, clustering, recommendation, and retrieval.

Why not use keyword search only?

Keyword search relies heavily on exact terms, while embeddings can match semantically related phrases and paraphrases. In production, both can be combined using hybrid search.

What is vector dimensionality?

It is the number of numerical coordinates in an embedding. The dimension is determined by the model and affects storage, compatibility, indexing, and search cost.

Why must query and document embeddings use the same model?

Because each embedding model creates its own vector space. Vectors from different models are generally not directly comparable.

What is cosine similarity?

It measures the directional similarity between two vectors. In semantic search, a higher cosine similarity generally indicates more semantically related text.

Are embeddings deterministic?

A careful answer:

They are generally much more stable than generative text outputs, but exact determinism depends on the model, provider, version, and implementation. Production systems should track the model and version rather than assume vectors remain unchanged forever.

What happens when you change embedding models?

Existing stored vectors should usually be regenerated because the new model creates a different vector space and may use a different dimensionality.

Embeddings versus LLM classification?

Classification returns predefined labels suited to business workflows. Embeddings return reusable vectors that support similarity search, clustering, retrieval, and related tasks. They solve different problems and may be used together.

23. Explain it using your project

A strong project explanation:

In my feedback analysis platform, structured output classifies each feedback item into sentiment, category, and priority. I additionally generate an embedding for the feedback text. The structured labels support deterministic filtering and analytics, while embeddings support semantic search and retrieval of feedback with similar meaning, even when the wording differs.

That is a strong interview answer because it shows you know when to use classification and when to use embeddings.