Vector Databases for AI Embeddings: How They Work

Published:

You have a useful library of lessons, but a question phrased differently from a lesson title sends people back to your inbox. The answer exists. Finding it is the problem.

Adding AI search sounds promising, until you face embedding models, indexes, and database plans before a single useful result appears. You can spend plenty of time building the wrong part.

This guide connects those pieces, shows when a separate database earns its place, and walks through a local lesson-search example so you can judge the results before paying for more infrastructure.

How Do Vector Databases Work With AI Embeddings?

An embedding model turns content into a list of numbers called a vector. Those numbers represent patterns the model learned during training.

A vector database stores these vectors and finds records close to a query vector. Your application then uses the matching text, images, or other source content.

For text search, the connected workflows look like this:

Ingestion:
Source documents → passages → embeddings → stored records and index

Retrieval:
Question → compatible query embedding → similarity search → matching passages

The document and question must enter a shared embedding space so their vectors can be compared meaningfully. Sentence Transformers explains this retrieval process.

A vector’s dimensions are the number of values it contains. A 384-dimensional embedding has 384 numbers. Individual coordinates generally do not have dependable labels such as “beginner difficulty” or “painting technique.”

How Do Vector Databases Work With Ai Embeddings?: How Vector Databases Retrieve Lesson Passages; Prepare Passages; Create Embeddings; Store Vectors + Sources; Embed Question; Find Nearby Vectors; Return Passage + Source

Long documents are usually split into chunks: passages that get their own embeddings. This lets search retrieve one useful explanation instead of representing an entire course manual with one vector.

Imagine you sell beginner watercolor courses and PDF guides. Students ask questions in everyday language, while your lessons use terms such as “wet-on-wet” and “hard edges.” You want search to connect those questions with useful lesson passages.

One stored record could contain:

FieldExample value
Chunk IDwb:wet-edges:01
EmbeddingNot supplied in this illustration; generate it from the passage with your chosen model
Passage text“For a defined edge, let the neighboring shape dry before adding another color.”
Course IDwatercolor-basics
Lesson titleControlling Wet Edges
Source URLhttps://example.com/courses/watercolor-basics/wet-edges
Revision1

Keep the original content. Embeddings help you find a passage, but they do not replace the passage you need to display or quote.

Also separate the database from its vector index. The index speeds up neighbor lookup. The database manages stored records, updates, deletion, and filtering. An existing database can provide vector search too, so this does not automatically require another service.

How Does Similarity Search Find Relevant Passages?

Embedding Compatibility Determines What Can Be Compared

Use a compatible embedding model for your documents and questions, and follow its required input formats. Some models expect different prefixes or encoding methods for queries and documents.

Matching dimensions are not enough. Two models can both produce 384 numbers while placing content in unrelated spaces. Comparing those outputs would give you numbers, but little reason to trust them.

Embedding Compatibility Determines What Can Be Compared: Embedding Compatibility Goes Beyond Vector Size; 384 Values; Same Size ≠ Shared Space; Compatible Model + Required Input Formats; Follow Query And Document Encoding Rules

Choose a model suited to your language, content, and retrieval task. The model determines how your material is represented. The database searches that representation.

Three common comparison methods are:

MethodWhat it compares
Cosine similarityHow closely the vectors point in the same direction, ignoring their length
Dot productDirectional alignment influenced by vector length
Euclidean distanceStraight-line separation between vectors
Embedding Compatibility Determines What Can Be Compared: Three Ways To Compare Ai Embeddings; Cosine Similarity; Direction; Dot Product; Direction + Length; Euclidean Distance; Endpoint Separation; Top_K = 5 Requests Neighbors, Not Five Proven

Follow the model’s guidance on comparison and normalization. Normalization rescales vectors to a common length, which changes how these methods relate. Sentence Transformers documents these similarity functions.

You will also encounter top_k, the requested number of nearest matches. Asking for five neighbors means “return the five closest eligible records,” not “find five passages that definitely answer this.”

Similarity scores rank candidates. They are not probabilities that an answer is correct.

Indexes Reduce Search Work

Exact search compares the query with every eligible vector and selects the nearest matches. It gives you a useful baseline because there is no approximate index to blame when results disappoint.

Approximate nearest-neighbor search, usually shortened to ANN, reduces the search work but can miss some neighbors that exact search would return.

HNSW, or Hierarchical Navigable Small World, organizes vectors into a layered graph. Search starts in sparse upper layers, moves toward promising areas, and descends through lower layers to refine the matches. The original HNSW paper describes this structure.

Indexes Reduce Search Work: Exact Search Vs Approximate Vector Search; Exact: Compare Every Eligible Vector; Ann: Less Work, Possible Missed Neighbors; Recall: Recover Exact Neighbors; Usefulness: Answer The Question

Another index, IVFFlat, groups vectors into lists and searches selected lists. Searching more lists generally trades more work for a better chance of recovering the nearest matches.

Compare index settings using:

  • Nearest-neighbor recall: how many of the neighbors returned by exact search the approximate search recovers.
  • Query latency under realistic traffic.
  • Memory and storage use.
  • Index-building time and the work needed to handle updates.

Recall and usefulness answer different questions. An index can recover the exact nearest vectors perfectly while those vectors point to passages that do not answer the question.

A faster route to the wrong paragraph is still the wrong paragraph.

Metadata Filters Restrict Eligible Results

Metadata is information stored alongside a vector, such as course ID, language, and publication status. Filters can limit the watercolor search to published English passages from a course the student has purchased. Vector similarity then orders those eligible matches.

Metadata Filters Restrict Eligible Results: Why Filtered Vector Search Can Return Too Few Results; All Stored Passages; Ann Candidate Search; Course + Language + Published; Eligible Passages Can Remain Unseen; Fewer Than Top_K May Return; Test More

There is an important interaction with ANN: if filtering happens after a limited candidate search, too few candidates may qualify. You can receive fewer than top_k results even when more eligible records exist.

Test restrictive filters with realistic data. The right fix depends on the system and may involve searching more candidates, changing the index layout, or using exact search over the eligible subset. pgvector documents this filtering behavior and its available remedies.

Where Do Semantic Search, RAG, and Agent Memory Fit?

These features can share the same retrieval layer. What changes is how the application uses the retrieved material.

FeatureWhat happens after retrievalWatercolor example
Semantic searchShow passages and source linksA student searches “How do I stop colors running together?” and opens a lesson about controlling wet edges
Retrieval-augmented generation, or RAGGive retrieved passages to a language model so it can compose an answerThe application explains the relevant technique and links to the supporting lesson
Agent memoryLet an agent retrieve selected notes or previous interactions through a search toolAn assistant retrieves a saved note that the student is practicing layered washes
Where Do Semantic Search, Rag, And Agent Memory Fit?: Semantic Search, Rag, And Agent Memory; Search: Show Passages + Links; Rag: Compose From Retrieved Context; Memory: Retrieve Selected Past Notes; Retrieved Context Does Not Guarantee A Correct

Semantic search is the simplest place to begin because you can inspect the retrieved passages directly.

RAG adds a generation step. The language model receives useful source text alongside the question and writes an answer from that context. Retrieval does not train the language model on your course or guarantee a correct response. Check whether its claims are supported by the passages it cites.

Where Do Semantic Search, Rag, And Agent Memory Fit?: What Ai Agent Memory Should Decide; Recall Context; Practicing Layered Washes; Verify Authoritative State; Purchases • Permissions • Task Status; Similarity Is Not Proof Of Access

Agent memory needs another distinction. Searchable notes can help an agent recall a previous discussion, but purchases, permissions, and task status belong in authoritative structured records. A vaguely similar memory is a poor basis for deciding whether someone owns a course.

I would build search first, add generated answers when retrieval works, and give an agent access only when there is a clear task for it.

Do You Need a Dedicated Vector Database?

Start with the kind of question your application must answer.

  • Exact fields need structured queries. “Which courses has this account purchased?” belongs in your application database.
  • Literal words need keyword search. An exact lesson title or product code should be easy to find.
  • Questions phrased differently from the source may benefit from vector search.

These methods can work together. Hybrid search combines keyword and vector results into one ranking. It is useful when students mix conceptual questions with exact lesson names or specialist terms. Weaviate explains this combination of keyword and vector retrieval.

Do You Need A Dedicated Vector Database?: Choose The Right Search For Each Question; Who Purchased This Course?; Structured Query; Choosing Watercolor Paper; Keyword Search; Stop Colors Running Together; Vector Search; Hybrid Search Combines Wording

Before choosing infrastructure, run exact search over a representative set of embeddings. Check whether semantic retrieval finds better passages than your existing search.

If it does, decide what the application needs next:

  1. Durable storage, updates, filters, or shared access justify database storage.
  2. Measured search delays may justify an approximate index.
  3. Hosting, scaling, or operational requirements may justify a dedicated service.
Do You Need A Dedicated Vector Database?: When Vector Search Needs More Infrastructure; Test Exact Retrieval First; Need Durable Records And Shared Access?; Measured Delays Justify An Index; Hosting And Scaling Needs Justify A Service; No Universal

If your application already uses PostgreSQL, I would consider pgvector first. It keeps embeddings beside relational records and supports exact search, HNSW, and IVFFlat. The project documents these capabilities.

There is no useful universal rule such as “move at 100,000 vectors.” Dimensions, filters, hardware, traffic, and latency requirements all affect that decision.

I would skip a separate service until a measured requirement earns it a place.

What Determines Storage and Operating Costs?

Count your chunks before comparing plans. Each indexed chunk normally produces one stored vector.

For an illustrative watercolor library, suppose you have 100 lessons averaging 10 chunks each. That produces 1,000 vectors.

To estimate the numeric values alone, multiply the number of vectors by their dimensions and the bytes used for each value:

Raw vector storage = vectors × dimensions × bytes per value

1,000 × 384 × 4 = 1,536,000 bytes
What Determines Storage And Operating Costs?: Estimate Raw Ai Vector Storage; 100 Lessons × 10 Chunks = 1,000 Vectors; 1,000 × 384 × 4 Bytes; 1,536,000 Bytes ≈ 1.54 Mb; Numeric Values Only; Text, Metadata, Indexes, Overhead, Replicas, Backups: Extra

That is approximately 1.54 MB using four-byte values.

It excludes passage text, metadata, indexes, database overhead, replicas, and backups. It also tells you nothing about the memory needed to run the embedding model.

Embedding costs are a separate calculation. For a hosted model billed by input tokens, count the billable tokens across the work you expect it to perform:

Embedding charge =
billable tokens ÷ tokens per billing unit × price per billing unit
What Determines Storage And Operating Costs?: Budget The Full Cost Of Vector Search; Embedding Workload; Tokens ÷ Billing Unit × Unit Price; Infrastructure + Machine Resources; Optional Processing + Operations; Measure A Representative Workload

Include:

  • The first pass through your source library.
  • Changed or added passages.
  • Incoming search questions.
  • A full rebuild when you change embedding models.

Then budget separately for database hosting, application servers, and generated answers. If you add a reranker to reorder retrieved passages by relevance, include its processing costs too. A local embedding model avoids hosted embedding charges, but it still uses machine resources and processing time.

Before projecting a full workload, index a representative sample and measure storage, memory, ingestion time, and query latency. Include expected traffic and availability needs in that test.

Maintenance belongs in the comparison too. A free server that needs your attention every weekend has found another way to charge you.

Which Starting Option Fits Your Application?

These options can support the same lesson-search feature, but they leave different work on your plate.

OptionBest starting situationFree starting routeOperational responsibilityReason to move beyond it
Exact search in application memoryYou need to test whether embeddings retrieve useful passagesRun locally without a database subscriptionLoad saved vectors, manage updates, and run comparisons in your codeShared access, data management, or measured search performance needs more infrastructure
Local ChromaYou want a compact Python prototype with stored recordsUse the local persistent clientManage the machine, database files, and backupsDeployment, concurrent access, or recovery needs exceed the local setup
PostgreSQL with pgvectorYour application already uses PostgreSQL or needs relational queries beside vector searchInstall the open-source extension on a local PostgreSQL instanceManage PostgreSQL yourself or choose managed hostingMeasured retrieval needs exceed the setup you can reasonably operate
Qdrant CloudYou want hosted vector storage and less database administrationUse its free prototype clusterManage application access, data lifecycle, and recovery needs while the provider runs the serviceTraffic, capacity, or availability requirements exceed the free cluster

Chroma’s persistent client automatically saves local data and loads it again when you reopen the database. Its client documentation describes local persistence.

Qdrant’s free cluster provides one node with 1 GB RAM and 4 GB disk without a credit card. Unused clusters are suspended after one week and deleted after four weeks of inactivity unless reactivated, so I would treat this as a prototype route. Check Qdrant’s current free-cluster terms.

For a first local build, I would use Chroma. For an application already running on PostgreSQL, I would start with pgvector. Managed hosting becomes attractive when operating the database takes time away from improving the feature.

Before paying, compare minimum charges, usage billing, backup and restore provisions, and export options against your workload estimate. A free starting route does not establish the cost of the finished application.

Build a Small Knowledge-base Search

The example below uses Python, Sentence Transformers to create embeddings, and Chroma to store and search them. It runs locally after the model has downloaded and needs no embedding API key.

Use a Python 3.11 virtual environment and a working directory where you can write files. Install the dependencies:

python -m pip install sentence-transformers chromadb

This example has not been execution-tested. It prints Python and dependency versions so you can record the environment used for your run.

Prepare Lesson Chunks and Their Source Metadata

Start with a few lessons and questions they should answer. Extract readable text, retain useful headings, and remove repeated navigation or PDF footers.

On one digital-product site, I found repeated footer text mixed into the passages I was preparing for retrieval. I had been looking at the search settings, but opening the extracted text made the problem obvious. Now I inspect the text before tuning the search.

Prepare Lesson Chunks And Their Source Metadata: Prepare Lesson Chunks That Preserve Answers; 1. Inspect Extracted Text; 2. Keep Conditions With Instructions; For A Defined Edge, Let The Neighboring Shape Dry; 3. Add Overlap When Context Needs It

Split at meaningful section and sentence boundaries. Keep enough surrounding explanation for each chunk to stand on its own, and add overlap when a boundary would separate a condition from its instruction.

For this English-language example, we will use sentence-transformers/all-MiniLM-L6-v2. It produces 384-dimensional vectors and truncates inputs beyond 256 wordpieces by default. A wordpiece can be part of a word, so counting words is not enough. The model card specifies its dimensions and input limit.

Prepare Lesson Chunks And Their Source Metadata: Keep Every Lesson Chunk Traceable; Watercolor-Basics; Controlling Wet Edges; Wb:wet-Edges:01; Lesson Id; Source Url; Position: 1; Revision: 1; Open The Source; Track Updates

The script checks complete inputs with the model’s tokenizer, including lesson headings and special tokens. It raises an error instead of silently accepting an oversized passage.

Give chunks stable IDs and retain their lesson, course, source, position, and revision. The sample source URLs below are placeholders to replace with your real lesson links.

Embed and Store the Passages

The complete script includes ingestion and querying so you can save it as watercolor_search.py and run it as one file.

It supplies every embedding explicitly. Chroma therefore stores and searches the vectors you create rather than choosing an embedding model for you.

import platform
from importlib.metadata import version
from time import perf_counter

import chromadb
from sentence_transformers import SentenceTransformer


MODEL_ID = "sentence-transformers/all-MiniLM-L6-v2"
COURSE_ID = "watercolor-basics"
COLLECTION_NAME = "watercolor-minilm-l6-v2-cosine-v1"

# Illustrative lesson excerpts and placeholder source URLs.
lessons = [
    {
        "id": "wb:wet-edges:01",
        "lesson_id": "wet-edges",
        "title": "Controlling Wet Edges",
        "text": (
            "Wet paint can flow into a neighboring wet shape. "
            "For a defined edge, let the neighboring shape dry "
            "before adding another color. For a soft transition, "
            "work while the adjoining wash is still damp."
        ),
    },
    {
        "id": "wb:layers:01",
        "lesson_id": "layers",
        "title": "Letting Layers Dry",
        "text": (
            "Let the first wash dry before painting a transparent "
            "layer over it. Working over a damp layer can disturb "
            "the earlier paint. Try the sequence on a practice "
            "swatch before using it in your painting."
        ),
    },
    {
        "id": "wb:paper:01",
        "lesson_id": "paper",
        "title": "Choosing Watercolor Paper",
        "text": (
            "Use paper made for watercolor when practicing washes. "
            "Compare a smooth sheet with a textured sheet using "
            "the same brush and wash. Note how the surface changes "
            "the marks you can make."
        ),
    },
    {
        "id": "wb:brushes:01",
        "lesson_id": "brushes",
        "title": "Cleaning Your Brush",
        "text": (
            "Rinse the brush when switching colors and check it "
            "on scrap paper before loading fresh paint. Reshape "
            "the tip gently after cleaning and avoid leaving "
            "the brush standing on its bristles."
        ),
    },
]

print("Python:", platform.python_version())
for package in (
    "sentence-transformers",
    "chromadb",
    "transformers",
    "torch",
    "numpy",
):
    print(f"{package}: {version(package)}")

# The first run downloads the model before any local retrieval.
model = SentenceTransformer(MODEL_ID)
print("Model:", MODEL_ID)
print("Maximum input tokens:", model.max_seq_length)

passages = [
    f"{lesson['title']}\n{lesson['text']}"
    for lesson in lessons
]
question = "How do I stop colors running together?"

# Include headings and special tokens in the length check.
for text in passages + [question]:
    token_ids = model.tokenizer(
        text,
        add_special_tokens=True,
        truncation=False,
    )["input_ids"]
    if len(token_ids) > model.max_seq_length:
        raise ValueError(
            f"Input has {len(token_ids)} tokens; "
            f"limit is {model.max_seq_length}. Split it first."
        )

vectors = model.encode(
    passages,
    normalize_embeddings=True,
    convert_to_numpy=True,
)

if vectors.shape != (len(passages), 384):
    raise ValueError(f"Unexpected embedding shape: {vectors.shape}")

metadatas = [
    {
        "course_id": COURSE_ID,
        "lesson_id": lesson["lesson_id"],
        "lesson_title": lesson["title"],
        "source_url": (
            "https://example.com/courses/"
            f"{COURSE_ID}/{lesson['lesson_id']}"
        ),
        "position": 1,
        "revision": 1,
        "language": "en",
        "status": "published",
    }
    for lesson in lessons
]

client = chromadb.PersistentClient(path="./watercolor-db")
collection = client.get_or_create_collection(
    name=COLLECTION_NAME,
    embedding_function=None,
    configuration={"hnsw": {"space": "cosine"}},
)

collection.upsert(
    ids=[lesson["id"] for lesson in lessons],
    embeddings=vectors.tolist(),
    documents=passages,
    metadatas=metadatas,
)

started = perf_counter()
query_vector = model.encode(
    question,
    normalize_embeddings=True,
    convert_to_numpy=True,
).tolist()

if len(query_vector) != 384:
    raise ValueError("Unexpected query embedding dimensions.")

# Demo scope only. In an application, derive permitted course
# scope from the authenticated account on your backend.
results = collection.query(
    query_embeddings=[query_vector],
    n_results=3,
    where={"course_id": COURSE_ID},
    include=["documents", "metadatas", "distances"],
)
elapsed_ms = (perf_counter() - started) * 1000

print(f"\nQuestion: {question}")
print(f"Embedding + database query: {elapsed_ms:.1f} ms")

# [0] selects the results for our single question.
for chunk_id, passage, metadata, distance in zip(
    results["ids"][0],
    results["documents"][0],
    results["metadatas"][0],
    results["distances"][0],
):
    print(f"\n{metadata['lesson_title']} [{chunk_id}]")
    print(f"Cosine distance: {distance:.4f}")
    print(passage)
    print("Source:", metadata["source_url"])

Run it with:

python watercolor_search.py

The script normalizes the passage embeddings and converts them to lists before storing them. Its upsert operation inserts new IDs or updates records with existing IDs, so rerunning this unchanged sample updates the same four records.

The collection uses cosine distance through Chroma’s HNSW configuration. Keep this collection name tied to this model, preprocessing, and metric; use a separate collection when those choices change. Chroma documents the collection configuration and distance functions.

After a successful run, save the printed versions and capture the installed packages:

python -m pip freeze > requirements.lock.txt

Embed a Question and Retrieve Its Sources

The question uses the same model and normalization as the passages. The database then searches within the course specified by where={"course_id": "watercolor-basics"} and requests three neighbors with n_results=3.

Chroma returns results grouped by input query. That is why the script selects [0], then keeps IDs, passages, metadata, and distances aligned while printing them.

Embed A Question And Retrieve Its Sources: Read Chroma Cosine Distance Correctly; Schematic Similarity View; Distance = 1 − Cosine Similarity; Lower Distance Means Closer Vectors; N_Results = 3; Check Whether The Passage Answers; Distance Is Not

For cosine search, Chroma returns distance calculated as 1 − cosine similarity. Lower values indicate closer matches. They still do not prove that a passage answers the question.

The timing printed by the script covers query embedding and the database request, after model loading and ingestion. It is one observation, so use repeated queries and realistic traffic before making performance decisions.

Embed A Question And Retrieve Its Sources: What The Lesson Search Timer Measures; Model Loading + Ingestion Complete; Query Embedding; Database Query; Reported Elapsed Time; Repeat Under Realistic Traffic; One Run Is One Observation

Before exposing this through a website:

  • Keep database access behind your backend.
  • Derive permitted courses from the authenticated account.
  • Apply those permissions during retrieval.
  • Return source links with the passages.

The hardcoded course in the script is demonstration scope, not an access-control system.

Calibrate any distance cutoff using relevant and irrelevant questions from your application. I would skip a universal threshold copied from another project. If the retrieved material cannot support an answer, say that the lesson library does not contain enough information.

Check Retrieval Before Adding Generated Answers

Write down the passage you expect to find before running each test. Otherwise, it is easy to accept anything that sounds vaguely related.

For this sample, begin with the questions below. The script returns neighbors even when none answers the question, so judge the passages yourself.

TestQuestionWhat to inspect
Everyday phrasing“How do I stop colors running together?”Whether the wet-edge explanation appears near the top
Paraphrase“When can I paint over the first wash?”Whether the drying lesson is retrieved
Exact lesson name“Choosing Watercolor Paper”Whether the matching lesson ranks well
Ambiguous request“Why does this look wrong?”Whether any passage answers this without more detail about what looks wrong
Missing information“Can I get a refund?”None of the four passages explains refunds; record this as unanswerable from the sample

As the library grows, record whether at least one answering passage appears among the requested results. Measure latency separately, and keep questions with no available answer in their own group.

Check Retrieval Before Adding Generated Answers: Five Tests For Lesson Search Retrieval; Everyday Phrasing → Wet Edges; Paraphrase → Drying Lesson; Exact Name → Paper Lesson; Ambiguous → Inspect Answerability; Missing Policy → Unanswerable; Write

When a test fails, investigate in this order:

  1. Does the source library contain the answer?
  2. Did extraction preserve it correctly?
  3. Did chunking keep the necessary context together?
  4. Are document and query embeddings compatible?
  5. Did filters exclude a passage that should have been eligible?
  6. Did the approximate index miss useful candidates?

Add keyword retrieval when exact terms keep failing. If useful candidates are retrieved but rank poorly, test a reranker. It scores question–passage pairs after the initial search and reorders the candidates. Sentence Transformers describes this retrieve-and-rerank workflow.

Once retrieval works, add RAG by sending useful passages and stable source IDs to the language model. Ask it to answer from those passages and resolve its citations to actual lesson records.

Check Retrieval Before Adding Generated Answers: Diagnose Failed Vector Retrieval In Order; 1. Source Contains The Answer?; 2. Extraction Preserved It?; 3. Chunk Retains Context?; 4. Embeddings Compatible?; 5. Filters Allow The Passage?; 6. Ann

Treat retrieved text as source material, separate from application instructions and tool permissions. A sentence inside an uploaded document should not gain authority to change how your application behaves.

Evaluate generated answers separately from retrieved passages. Keep the same test questions when comparing changes so you can tell whether you improved retrieval, writing, or neither.

Finding the right passage is only part of the job once you add generated answers. A small test pack can help you check supported answers, missing information and instructions hidden inside source text.

Lead Magnet Cover
FREE AI SKILL

Free AI Evaluation Studio

Use AI to draft tests for everyday requests and the tricky cases you might overlook.
Send My AI Evaluation Studio ❯
Instantly delivered to your inbox
Customer Avatar 1
Customer Avatar 2
Customer Avatar 3
Customer Avatar 4
Rated highly by 25,000+ users

Keep the Knowledge Base Current and Recoverable

A search feature needs an update process as much as an ingestion process.

When a lesson changes, reprocess its chunks and remove obsolete ones. Upserting the new version does not remove surplus chunks left behind when a lesson becomes shorter.

Keep The Knowledge Base Current And Recoverable: Update Lesson Search Without Leaving Stale Chunks; 1. Reprocess The Changed Lesson; 2. Upsert Current Chunks; 3. Delete Obsolete Chunk Ids; Previous Ids − New Ids = Leftovers; Upsert Alone Does Not

Keep a record of the chunk IDs belonging to each lesson. That makes it possible to compare the previous set with the new set and delete the leftovers.

Version your embedding model and preprocessing configuration, including chunking rules and added headings. For a model change, I would:

  1. Build embeddings in a separate collection.
  2. Run the existing retrieval tests.
  3. Switch document and query encoding together.
  4. Retain a rollback route until the replacement is working.
Keep The Knowledge Base Current And Recoverable: Migrate Embedding Models With A Recovery Path; 1. Build A Separate Collection; 2. Run The Same Retrieval Tests; 3. Switch Both Encoders Together; 4. Keep Rollback Until Verified; Preserve Portable

Monitor ingestion failures, delays between source changes and search updates, query latency, retrieval quality, and spending. Use bounded retries and query timeouts where failures affect the application.

Finally, preserve source documents, chunk IDs, metadata, and embedding settings independently of the provider. Test restoration rather than assuming a backup is usable.

Vector databases for AI embeddings should make useful content easier to retrieve. Keeping your sources and configuration portable also makes the database easier to replace when your needs change.

What Next?

Choose five source passages and write ten questions they should help answer, including two that your material cannot answer. Run the local example with your own content and inspect the retrieved passages before adding generated responses.

Use the social buttons to share this guide when the database decision is holding up a useful search feature.

Which part of your retrieval setup feels least certain: preparing chunks, choosing storage, or judging the results? Leave your answer in the comments, and bring any other questions too.

Share this post with your friends & followers:
Abhishek is a data scientist by day & an online entrepreneur by night. He is known for his ability to simplify complex concepts and make them accessible to a wider audience. He started Passive Book to share his insights and experiences on how to effectively build an online business, which has quickly become a go-to resource for anyone looking to bootstrap an online business from scratch.

Leave a Comment

Build an Online Business 
Without Quitting Your Job

Join 10,000+ solopreneurs building six-figure online businesses from scratch with limited time & tight budgets using smart systems & AI automations.
Optin Form - Main

Free Bonus: Get access to our AI workflows & cheat sheets
Person sitting with laptop in home office environment with launching rocket, coffee mug, sleeping dog, bookshelf, and house plant illustrating work-from-home online business lifestyle. Decorative image.
Thank you for sharing

Follow Us On: