Embedding Models + Similarity Search
Today’s milestone
By the end of today, you should be able to:
- Understand how embedding models differ from each other.
- Choose an embedding model using practical criteria.
- Understand cosine similarity, dot product, and Euclidean distance at a useful level.
- Build a small in-memory semantic search using real or fake embeddings.
- Explain top-k retrieval and similarity thresholds.
- Understand the main production mistakes when using embedding models.
Today, do not move into pgvector, vector indexing, chunking, or RAG.
1. What is an embedding model?
An embedding model converts input data into a numerical vector.
Text
↓
Embedding model
↓
Vector
Example:
"The delivery was delayed"
becomes something like:
[0.18, -0.42, 0.73, ...]
Different embedding models may produce:
- Different vector dimensions
- Different semantic quality
- Different language support
- Different latency
- Different costs
- Different behavior for short and long text
Two models may accept the same sentence but produce completely different vectors.
That does not mean one is necessarily wrong. Each model creates its own vector space.
2. Why embedding model selection matters
Suppose your feedback platform contains:
"My package arrived three days late."
A user searches:
"slow delivery"
A strong embedding model should place those two texts close together.
A weaker or unsuitable model may fail to capture the relationship.
Model choice affects:
- Search relevance
- Multilingual support
- Storage size
- Query speed
- API cost
- Re-embedding effort
- Production scalability
Choosing the embedding model is an architectural decision, not just an SDK setting.
3. Main criteria for choosing an embedding model
Semantic quality
The most important question is:
Does the model retrieve the correct results for my real data?
Do not select a model only because it is popular.
For your feedback platform, test it using queries such as:
slow shipping
payment issue
support not responding
damaged product
refund delay
Then verify whether the expected feedback appears near the top.
Vector dimension
Dimension means the number of values in the embedding.
embedding = [0.1, 0.2, -0.3]
dimension = 3
Real models usually produce much larger vectors.
Higher-dimensional vectors often require more:
- Storage
- Memory
- Network bandwidth
- Index space
- Search computation
A larger dimension does not automatically mean better results.
Input length
Every embedding model has a maximum supported input size.
If the input is too large, you may need to:
- Reject it
- Truncate it
- Split it into chunks
- Summarize before embedding
For customer feedback, individual feedback records are usually short enough.
For documents, chunking becomes necessary. That is a later topic.
Language support
Some embedding models perform well mainly in English.
Others are designed for multilingual search.
This matters if your platform processes:
- English
- Hindi
- Marathi
- Hinglish
- Other regional languages
A multilingual model is usually more useful when users search in one language and the feedback is written in another.
Domain suitability
A general-purpose model may work well for normal customer feedback.
A domain-specific model may perform better for:
- Medical text
- Legal documents
- Scientific papers
- Source code
- Financial data
Do not assume a general text model is ideal for every domain.
Latency
Embedding generation happens during:
- Record ingestion
- Query execution
- Re-indexing
- Batch processing
Query embedding latency directly affects search response time.
Document embeddings can often be generated asynchronously during ingestion, but the query embedding usually happens during the user request.
Cost
Embedding APIs are generally cheaper than generation APIs, but cost still matters at scale.
Consider:
Number of records
×
Average input size
×
Re-embedding frequency
The biggest cost problem often appears when you change models and must regenerate every stored embedding.
Deployment model
You may use:
- Hosted API
- Open-source model
- Local model
- Self-hosted inference service
Hosted models are easier to start with.
Local or self-hosted models provide more control but add infrastructure and operational complexity.
For your current roadmap, use the simplest provider that lets you learn reliably.
4. Never mix embedding models
This is one of the most important rules.
Bad design:
Stored feedback embeddings → Model A
Search query embedding → Model B
Even if both produce vectors of the same dimension, they may represent meaning differently.
The scores become unreliable because the vectors belong to different spaces.
Good design:
Stored feedback embeddings → Model A
Search query embedding → Model A
When changing the model, re-embed the stored data.
You should store metadata such as:
{
"embedding_model": "model-name",
"embedding_version": "v1",
"embedding_dimension": 768,
}
Later, this metadata helps with migrations.
5. What is similarity search?
Similarity search answers:
Which stored vectors are closest to the query vector?
Flow:
User query
↓
Generate query embedding
↓
Compare with stored embeddings
↓
Calculate similarity scores
↓
Sort results
↓
Return top matches
Example:
Query:
"delivery issue"
Possible results:
1. "My package arrived late" 0.93
2. "Shipping took more than one week" 0.89
3. "Support did not answer" 0.42
4. "Payment failed" 0.17
Higher similarity usually means better semantic relevance.
6. Similarity metrics
The three common metrics are:
- Cosine similarity
- Dot product
- Euclidean distance
Today, understand when they differ. You do not need a deep mathematical derivation.
Cosine similarity
Cosine similarity compares vector direction.
It focuses less on magnitude and more on orientation.
Conceptually:
Same direction → highly similar
Different angle → less similar
Opposite → very dissimilar
Formula:
cosine_similarity = dot_product / (
magnitude_a * magnitude_b
)
Typical interpretation:
Closer to 1 → more similar
Closer to 0 → less related
Cosine similarity is commonly used for semantic search because text meaning is often represented more by direction than magnitude.
Dot product
Dot product multiplies corresponding values and sums them.
dot_product = sum(
a * b
for a, b in zip(vector_a, vector_b, strict=True)
)
The result is affected by:
- Direction
- Magnitude
If the embedding model returns normalized vectors, dot product and cosine similarity may produce equivalent rankings.
Do not assume this without checking the model documentation.
Euclidean distance
Euclidean distance measures straight-line distance between two vectors.
Smaller distance means more similarity.
Cosine similarity:
higher is better
Euclidean distance:
lower is better
It can be useful when vector magnitude is meaningful.
For your first semantic search implementation, use cosine similarity.
7. Implement similarity functions
Cosine similarity
import math
def cosine_similarity(
vector_a: list[float],
vector_b: list[float],
) -> float:
if len(vector_a) != len(vector_b):
raise ValueError(
"Vectors must have the same dimensions."
)
dot_product = sum(
a * b
for a, b in zip(vector_a, vector_b, strict=True)
)
magnitude_a = math.sqrt(
sum(value * value for value in vector_a)
)
magnitude_b = math.sqrt(
sum(value * value for value in vector_b)
)
if magnitude_a == 0 or magnitude_b == 0:
raise ValueError(
"Cosine similarity is undefined for zero vectors."
)
return dot_product / (
magnitude_a * magnitude_b
)
Dot product
def dot_product(
vector_a: list[float],
vector_b: list[float],
) -> float:
if len(vector_a) != len(vector_b):
raise ValueError(
"Vectors must have the same dimensions."
)
return sum(
a * b
for a, b in zip(vector_a, vector_b, strict=True)
)
Euclidean distance
import math
def euclidean_distance(
vector_a: list[float],
vector_b: list[float],
) -> float:
if len(vector_a) != len(vector_b):
raise ValueError(
"Vectors must have the same dimensions."
)
return math.sqrt(
sum(
(a - b) ** 2
for a, b in zip(
vector_a,
vector_b,
strict=True,
)
)
)
8. What is top-k retrieval?
top_k means:
Return the k most similar results.
Example:
top_k = 3
Possible output:
1. Late delivery complaint
2. Shipping delay complaint
3. Courier issue complaint
A larger top_k returns more candidates, but may include irrelevant items.
A smaller top_k is more focused, but may miss useful information.
Trade-off:
Small top_k
→ higher precision
→ may miss relevant records
Large top_k
→ better coverage
→ more noise
For your experiment, start with:
top_k = 3
9. Similarity threshold
A threshold lets you reject results that are not similar enough.
Example:
minimum_similarity = 0.70
Then:
filtered_results = [
result
for result in results
if result.similarity >= minimum_similarity
]
Why use a threshold?
Without one, the search system always returns something—even when nothing is relevant.
Example:
Query:
employee salary policy
But your records contain only product feedback.
The system may still return the “least unrelated” feedback.
A threshold allows:
No sufficiently relevant results found.
Important: there is no universal threshold like 0.7 that works for all models and datasets.
You must evaluate it experimentally.
10. Build a small real search flow
Use this schema:
from pydantic import BaseModel
class EmbeddedFeedback(BaseModel):
id: int
text: str
embedding: list[float]
class SearchResult(BaseModel):
id: int
text: str
similarity: float
Service:
from typing import Protocol
class EmbeddingProvider(Protocol):
async def embed_text(
self,
text: str,
) -> list[float]:
...
class SimilaritySearchService:
def __init__(
self,
provider: EmbeddingProvider,
) -> None:
self.provider = provider
self.records: list[EmbeddedFeedback] = []
async def add_feedback(
self,
feedback_id: int,
text: str,
) -> EmbeddedFeedback:
embedding = await self.provider.embed_text(text)
record = EmbeddedFeedback(
id=feedback_id,
text=text,
embedding=embedding,
)
self.records.append(record)
return record
async def search(
self,
query: str,
*,
top_k: int = 3,
minimum_similarity: float | None = None,
) -> list[SearchResult]:
query_embedding = await self.provider.embed_text(
query
)
results = [
SearchResult(
id=record.id,
text=record.text,
similarity=cosine_similarity(
query_embedding,
record.embedding,
),
)
for record in self.records
]
if minimum_similarity is not None:
results = [
result
for result in results
if result.similarity
>= minimum_similarity
]
results.sort(
key=lambda result: result.similarity,
reverse=True,
)
return results[:top_k]
11. Test with useful feedback examples
Use at least these records:
feedbacks = [
"My package arrived four days late.",
"The payment page keeps failing.",
"Customer support never replied.",
"The product quality is excellent.",
"I received the wrong item.",
"Shipping was much slower than promised.",
"I was charged twice for the same order.",
]
Try these queries:
delivery delay
billing problem
support response issue
wrong product received
positive product review
For each query, inspect:
- Which result ranked first?
- Were the top three useful?
- Were unrelated records ranked too highly?
- Would a threshold improve the result?
This manual evaluation is important.
12. Use batch embedding when possible
Embedding one record at a time is simple:
for text in texts:
await provider.embed_text(text)
But this may require many network calls.
A better provider contract may support:
async def embed_texts(
self,
texts: list[str],
) -> list[list[float]]:
...
Then:
10 feedback items
↓
One batch request
↓
10 embeddings
Benefits:
- Lower network overhead
- Better throughput
- Potentially lower cost
- Faster ingestion
For today, you only need to understand this. Implement batch support only if your current provider makes it easy.
13. Query embedding versus document embedding
Some embedding systems distinguish between:
- Query embeddings
- Document embeddings
Why?
A search query is often short:
delivery issue
A document may be longer:
My package arrived three days after the expected delivery date.
Some models or APIs use different instructions or modes for these two inputs.
Your internal interface may eventually become:
async def embed_query(
self,
text: str,
) -> list[float]:
...
async def embed_document(
self,
text: str,
) -> list[float]:
...
Do not add this abstraction unless your selected model needs it.
14. Model versioning
Suppose you initially use:
Embedding model v1
Later, you switch to:
Embedding model v2
The old and new vectors should not be mixed blindly.
A production migration may look like:
Existing vectors
↓
Create new embedding column/version
↓
Generate new embeddings in batches
↓
Validate search quality
↓
Switch search traffic
↓
Remove old embeddings later
This is why model metadata matters.
For your current in-memory implementation, just add model metadata conceptually.
15. Model selection experiment
Create a small evaluation dataset.
Example:
evaluation_cases = [
{
"query": "delivery delay",
"expected_ids": {1, 6},
},
{
"query": "payment charged twice",
"expected_ids": {2, 7},
},
{
"query": "support not replying",
"expected_ids": {3},
},
]
For each query:
- Run semantic search.
- Retrieve top 3 results.
- Check whether the expected record appears.
- Record failures.
Simple evaluator:
async def evaluate_search(
service: SimilaritySearchService,
cases: list[dict],
) -> float:
successful_cases = 0
for case in cases:
results = await service.search(
case["query"],
top_k=3,
)
returned_ids = {
result.id
for result in results
}
if returned_ids & case["expected_ids"]:
successful_cases += 1
return successful_cases / len(cases)
This is not a complete retrieval metric, but it is much better than selecting a model based on intuition alone.
16. Production decisions for your project
For the AI Feedback Analysis Platform, your first model should optimize for:
- Short and medium-length feedback
- Strong semantic similarity
- Good English support
- Low cost
- Fast query latency
- Easy API integration
- Stable model versioning
Later, if you support multiple Indian languages, multilingual retrieval becomes a stronger requirement.
Your initial decision does not need to be permanent.
However, changing it later requires re-embedding stored feedback.
17. Error handling
Embedding providers can fail for the same reasons as generative providers:
- Authentication errors
- Rate limits
- Timeouts
- Invalid input
- Provider outages
- Input length limits
Reuse your existing reliability architecture.
Embedding SDK error
↓
Application-level LLM/embedding exception
↓
Service or global FastAPI handler
You might either reuse:
LLMRateLimitError
LLMTimeoutError
LLMProviderError
or create broader provider exceptions:
AIProviderRateLimitError
AIProviderTimeoutError
AIProviderError
A broader naming strategy may be cleaner now that the application uses both generation and embedding models.
Do not refactor this today unless it is simple.
18. Tests to write today
Similarity metric tests
Test:
- Identical vectors
- Different vectors
- Dimension mismatch
- Zero vector
- Sorting by descending cosine similarity
Example:
def test_identical_vectors_have_similarity_one() -> None:
vector = [1.0, 2.0, 3.0]
similarity = cosine_similarity(
vector,
vector,
)
assert similarity == pytest.approx(1.0)
Search tests
Test:
- Correct top result
top_klimit- Threshold filtering
- Empty dataset
- Results sorted correctly
Example:
@pytest.mark.asyncio
async def test_threshold_removes_weak_results() -> None:
provider = FakeEmbeddingProvider()
service = SimilaritySearchService(provider)
await service.add_feedback(
1,
"Delivery was delayed.",
)
results = await service.search(
"payment problem",
minimum_similarity=0.8,
)
assert results == []
Your fake provider must generate sufficiently different vectors for this test to be meaningful.
19. Interview preparation
How do you choose an embedding model?
I evaluate semantic quality on domain-specific queries, vector dimension, input limits, language support, latency, cost, deployment model, and model stability. I choose based on retrieval performance on my actual dataset rather than popularity alone.
Why can’t vectors from two embedding models be compared?
Different embedding models create different vector spaces. Even if the vectors have the same dimension, their coordinates do not necessarily represent the same learned relationships.
Cosine similarity versus Euclidean distance?
Cosine similarity compares vector direction, while Euclidean distance measures absolute distance. Cosine similarity is commonly used in semantic search because meaning is often represented by direction, but the best metric depends on how the embedding model was trained and normalized.
What is top-k retrieval?
It returns the k most similar records for a query. A smaller k improves focus, while a larger k improves coverage but can introduce more noise.
Why use a similarity threshold?
Without a threshold, the system always returns the closest records even when none are meaningfully relevant. A threshold allows the system to return no result when confidence is too low.
How would you evaluate an embedding model?
I would build a domain-specific test set containing queries and expected relevant records, run retrieval, and measure whether relevant records appear in the top results. I would also inspect false positives, latency, cost, and multilingual behavior where needed.
What happens when the embedding model changes?
Existing vectors usually need to be regenerated because the new model creates a different semantic space and may produce a different dimension. I would version embeddings and migrate them gradually.