Software & Guides/Guide/Explainer

What Is a Vector Database? How Embeddings and RAG Search Work

Vector databases index embeddings for similarity search, making them the retrieval backbone behind RAG pipelines and AI agents.

Slonik, the official PostgreSQL elephant logo, representing the open-source database behind the pgvector extension
Slonik, the PostgreSQL project mascot. Image: PostgreSQL.

A vector database is a database built to store, index and search embeddings — long lists of numbers that represent the meaning of text, images, audio or other data — so that an application can find the items whose meaning is closest to a query, rather than the items whose fields match it exactly. That single difference, searching by closeness instead of by equality, is why vector databases have become a standard piece of infrastructure for retrieval-augmented generation (RAG), AI agents, recommendation systems and semantic search, and why relational databases like PostgreSQL or MySQL generally cannot do the same job well on their own without an extension or add-on built for the purpose.

Quick facts
  • A vector database stores numeric vectors (embeddings) and finds the nearest ones to a query vector, using a similarity metric such as cosine distance, dot product or Euclidean (L2) distance.
  • Most vector databases use approximate nearest neighbor (ANN) search, commonly a graph-based method called HNSW, to stay fast as the number of vectors grows into the millions or billions.
  • RAG pipelines use a vector database as the retrieval step: a user's question is embedded, the nearest stored chunks are fetched, and those chunks are added to the prompt sent to the language model.
  • Options range from extensions bolted onto an existing database, such as pgvector for PostgreSQL, to purpose-built systems such as Pinecone, Weaviate, Qdrant and Milvus.

What is a vector database?

A vector database is a system designed around one core operation: given a query vector, return the stored vectors that are most similar to it, ranked by a distance or similarity score. Every item — a paragraph of text, a product photo, a snippet of audio — is first converted into an embedding, a fixed-length array of floating-point numbers produced by a machine learning model. The database then indexes those embeddings so that a nearest-neighbor search can run in milliseconds instead of comparing the query to every single vector in the collection.

As Pinecone's own explainer on the concept puts it, a proper vector database adds the operational features a raw in-memory index lacks: inserts, updates and deletes, metadata storage and filtering, durability, backups and access control, on top of the similarity search itself. That combination — a specialized index plus ordinary database plumbing — is what separates a vector database from a standalone library such as FAISS or hnswlib, which can perform the nearest-neighbor math but leaves persistence, concurrency and data management to the application.

How is a vector database different from a relational (SQL) database?

A relational database such as PostgreSQL or MySQL organizes data into rows and columns and is optimized for exact and range-based matches: WHERE user_id = 42, WHERE price < 20, a join across two tables on a shared key. That model is extremely good at structured, precise lookups, but it has no native notion of "meaning" — it cannot tell you that the phrase "lightweight laptop for travel" is conceptually close to a product named "13-inch ultrabook" unless the two share literal keywords or someone has manually tagged them.

A vector database is built for the opposite case: approximate, semantic closeness across many dimensions at once, typically 100 to a few thousand per vector. There is no single "correct" row to return, only a ranked list of the nearest matches. The table below summarizes the practical differences.

CharacteristicRelational (SQL) databaseVector database
Core queryExact match, range, joinNearest-neighbor similarity search
Data shapeRows and typed columnsHigh-dimensional float vectors (plus optional metadata)
Typical indexB-tree, hash, GiSTHNSW, IVF, or other ANN graph/cluster structures
ResultDeterministic — rows that satisfy the conditionRanked, approximate — the closest matches by distance score
Handles raw text/image meaning?Only via separate full-text search or manual tagsYes, via embeddings from a machine learning model
Common home for the dataStandalone RDBMSPurpose-built system, or an extension inside an RDBMS

In practice the line has blurred: PostgreSQL can gain vector search through the open-source pgvector extension rather than requiring a separate system, which is one reason teams already running Postgres — including on platforms built around it, such as Supabase's Postgres-based backend — often add vector search instead of standing up a second database. Readers tracking the underlying engine's own roadmap can see how that extension story keeps evolving in Pandromeda's PostgreSQL 19 vs. PostgreSQL 18 comparison.

What are embeddings, exactly?

An embedding is the output of a machine learning model that reads a piece of content and produces a vector — an ordered list of numbers — positioned in a space where distance corresponds to meaning. Two sentences that mean similar things end up as vectors that sit close together in that space, even if they do not share a single word in common; two unrelated sentences end up far apart. The same idea extends to images, audio clips, product catalogs or lines of code, as long as there is a model trained to embed that type of content.

Wikipedia's entry on retrieval-augmented generation describes the underlying split as dense versus sparse vectors: dense embeddings are compact, fixed-length vectors that capture semantic meaning, while sparse representations (such as classic keyword weighting or newer sparse neural methods) track which specific terms appear. Many production systems, including Qdrant, now combine both — dense vectors for semantic recall and sparse or keyword scoring for lexical precision — in what is usually called hybrid search.

How does similarity search actually work?

Once content is embedded, finding "similar" items means finding the vectors whose distance from the query vector is smallest, or whose similarity score is highest. Three measures show up repeatedly across vector database documentation:

  • Cosine similarity / cosine distance — compares the angle between two vectors, ignoring their magnitude. It is the most common default for text embeddings.
  • Dot product / inner product — a raw multiplication-and-sum of the two vectors' components; pgvector exposes it as the <#> operator, returning the negative inner product so that smaller values still mean "closer."
  • Euclidean (L2) distance — straight-line distance between the two points in vector space, exposed in pgvector as the <-> operator.

The brute-force approach — computing the distance from the query to every stored vector — gives an exact answer but does not scale: comparing one query against tens of millions of vectors, each with hundreds of dimensions, is too slow to run on every request. That is the problem approximate nearest neighbor search exists to solve.

Approximate nearest neighbor search and HNSW

Approximate nearest neighbor (ANN) search trades a small amount of accuracy for a large amount of speed, by organizing vectors into a structure that lets a query skip most of the collection instead of scanning all of it. The most widely deployed ANN method today is HNSW (Hierarchical Navigable Small World), described in a 2020 paper by Yury Malkov and Dmitry Yashunin and now built into most major vector databases.

HNSW, per Wikipedia's technical summary, arranges vectors as nodes in a multi-layer graph. Every vector sits in the bottom layer; progressively fewer vectors appear in each layer above it. A search starts at an entry point in the top (sparsest) layer and greedily hops toward whichever neighbor is closest to the query, dropping down a layer whenever no closer neighbor remains at the current one — using the upper layers for long jumps across the space and the bottom layer for fine-grained comparison. The structure is built incrementally: each new vector is assigned a random maximum layer, finds nearby existing nodes, and gets wired into the graph at every layer it occupies.

HNSW is not the only option. pgvector's own GitHub documentation also supports IVFFlat, which divides vectors into lists and searches only the lists nearest the query; it builds faster and uses less memory than HNSW but delivers lower recall at a given speed, and generally performs best when built after the table already has data, whereas HNSW can be built before any rows are loaded. Milvus's documentation lists several further index families — IVF, flat (exhaustive) search, DiskANN, and GPU-accelerated indexes such as NVIDIA's CAGRA — each trading build time, memory and accuracy differently; which one is "best" depends on data size, hardware and the recall a given application needs, not a single universal answer.

Why do RAG and AI agents need a vector database?

Retrieval-augmented generation (RAG) is, per Wikipedia's definition, a technique that lets a large language model pull in outside information at the moment it answers a question, rather than relying only on what was baked into it during training. The retrieval half of that pipeline is almost always a vector database, for a straightforward reason: embedding a company's documents, a codebase, or a support-ticket history once and storing the vectors is far cheaper than retraining or fine-tuning a model every time that underlying knowledge changes.

The flow is consistent across vendors: a document is split into chunks, each chunk is embedded and stored with a pointer back to the source text; when a user asks a question, the question itself is embedded with the same model, the vector database returns the nearest stored chunks, and those chunks are inserted into the prompt the language model actually sees. Wikipedia's RAG entry lists the resulting benefits plainly — access to current or specialized information the model was never trained on, cheaper updates than retraining, answers that can cite a source, and (though not a complete fix) fewer unsupported, fabricated claims, since the model is working from retrieved text rather than from memory alone.

AI agents extend the same pattern to memory and tool use: an agent that needs to recall earlier steps of a long task, past conversations, or a large private knowledge base generally stores that history as embeddings and queries a vector database before deciding what to do next, rather than keeping every past token in the model's limited context window. As language models themselves keep changing — see Pandromeda's rundown of current frontier model pricing and specifications — the retrieval layer underneath them has stayed a fairly stable architectural piece, because swapping which model generates an answer does not remove the need to first find the right source material for it to work from.

There is no single "winner" among vector databases — the right pick depends on whether a team wants to add vector search to infrastructure it already runs, or stand up a dedicated system. This is not an endorsement of any one product; the summary below reflects only what each project states in its own public documentation at the time of writing.

ProjectWhat it isPrimary indexTypical deployment
pgvectorOpen-source extension adding vector search to PostgreSQLHNSW or IVFFlatInside an existing Postgres database
PineconeManaged vector database serviceProprietary ANN index (serverless architecture)Fully managed cloud service
WeaviateOpen-source vector database, also offered as a managed cloudHNSW, flat, or an experimental dynamic/cluster indexSelf-hosted or managed
QdrantOpen-source vector search database with a managed cloud optionHNSW, plus sparse and hybrid searchSelf-hosted or managed
MilvusOpen-source vector database (Apache 2.0), with Zilliz Cloud as a managed optionHNSW, IVF, flat, DiskANN, GPU indexesLite (embedded), standalone server, or distributed cluster

pgvector's project page on GitHub describes it as open-source vector similarity search for Postgres that keeps vectors in the same ACID-compliant database as everything else, which means ordinary SQL joins and transactions still work against the embedded data — a meaningful convenience for teams that would rather not synchronize two separate stores. Pinecone's own explainer frames its product the opposite way: a dedicated system built specifically around the vector workload, with serverless infrastructure that separates storage from compute. Qdrant's documentation emphasizes combining dense vector search with BM25-style lexical search and payload-based filtering in a single pass over its HNSW graph. Weaviate's documentation describes HNSW as its primary index alongside a lighter-weight flat index for small, per-tenant collections, with an experimental dynamic index that starts flat and promotes itself to HNSW once a collection grows past a size threshold. Milvus's documentation positions the project as the retrieval layer for RAG, general semantic search and agent memory, with three deployment modes — an embedded Python library for prototyping, a single-machine Docker server, and a Kubernetes-based distributed cluster for very large datasets.

How to think about choosing one

None of the sources above publish a universal benchmark that settles which system is fastest or cheapest for every workload, and figures like that vary enormously with hardware, dataset size, dimensionality and how an index is tuned — so this guide will not invent one. Instead, the practical questions teams tend to ask are:

  • Where does the rest of the data already live? If an application already runs on PostgreSQL, adding pgvector avoids operating a second database just for embeddings.
  • Self-hosted or managed? Weaviate, Qdrant and Milvus can all run self-hosted or as a managed cloud offering from their maker; Pinecone is managed-only.
  • Scale and filtering needs. Collections ranging from thousands to a few million vectors behave very differently from collections in the billions, and filtering search results by metadata (category, date, user ID) adds its own indexing considerations on top of the vector index itself.
  • Hybrid search. Applications that need both keyword precision and semantic recall should check whether a candidate database supports combining sparse and dense retrieval natively, as Qdrant and others document, rather than bolting it on separately.

Limitations and common mistakes

Vector search is approximate by design, so it is possible for the single best match to be missed in favor of a very close second — a tradeoff every ANN index documented above makes explicitly in exchange for speed. Embeddings are also only as good as the model that produced them: a query embedded with one model and compared against vectors embedded with a different model will generally not return meaningful results, since different models place meaning in different vector spaces. And because RAG and agent systems depend on retrieval finding the right chunk, a vector database does not remove the underlying risk that a language model's answer is only as accurate as the material it was handed — it reduces, rather than eliminates, the kind of unsupported answers that come from a model working off memory alone.

Bottom line

A vector database exists to answer one question well: "what else in this collection is most like this?" — a question relational databases were never designed to answer efficiently at scale. Embeddings turn text, images and other content into comparable numeric vectors; approximate nearest neighbor methods like HNSW make searching millions of those vectors fast enough for a live request; and that combination is why RAG pipelines and AI agents lean on a vector database as their retrieval layer rather than scanning raw documents on every query. Whether that means adding pgvector to an existing Postgres instance or standing up a dedicated system like Pinecone, Weaviate, Qdrant or Milvus depends on what a team already runs and how far its data needs to scale — not on which product has the loudest marketing.

Frequently asked questions

What is a vector database in simple terms?

It is a database that stores data as numeric vectors (embeddings) and finds the stored items most similar to a query vector, instead of finding rows that exactly match a condition.

How is a vector database different from a SQL database?

A SQL database like PostgreSQL or MySQL returns rows that exactly satisfy a condition using indexes such as B-trees. A vector database returns the nearest matches to a query by distance or similarity score, using approximate nearest neighbor indexes such as HNSW.

What is a vector database used for in AI?

It is most commonly used as the retrieval layer in retrieval-augmented generation (RAG) pipelines, AI agent memory, semantic search and recommendation systems, where results need to be ranked by meaning rather than by exact keyword match.

What is the difference between a vector database and an embedding model?

An embedding model converts content such as text or images into a numeric vector. A vector database stores those vectors and searches them. The two are used together: a model creates the embeddings, and the database indexes and retrieves them.

Does RAG need a vector database?

Most RAG systems use one, because it lets a system store embedded documents once and retrieve the closest matches quickly when a query comes in, rather than re-embedding and comparing against the entire document set on every request.

What is HNSW and why does it matter?

HNSW (Hierarchical Navigable Small World) is a graph-based approximate nearest neighbor algorithm, described in a 2020 paper by Yury Malkov and Dmitry Yashunin, that most major vector databases use as their primary index because it offers a strong balance of search speed and accuracy.

Sources

More on Vector Database →Vector DatabaseEmbeddingsRAGAI AgentspgvectorSemantic Search
Felix Moreau
Written byFelix Moreau

Felix Moreau writes Pandromeda's software coverage and how-to guides. He covers Windows, macOS and Linux updates, the apps people rely on, emulators and developer tools, and turns official documentation into clear, numbered steps that work on the current version.

More from Software & Guides

See all