The moment semantic search or RAG lands on a roadmap, somebody reaches for a dedicated vector database. Often before anyone has counted the vectors.
If you already run Postgres, pgvector stores and queries embeddings next to your data, with no new service and no new bill. This is how to ship it, and an honest account of when a dedicated vector database is genuinely the right call.
pgvector in practice
It is a Postgres extension adding a vector column type and similarity operators. You store an embedding per row and query by nearest-neighbour distance, inside the database you already back up, secure and join against.
That last part is the one people underrate. Your embeddings live in the same transaction as the row they describe, which means they cannot drift out of sync with it.
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE documents (
id bigserial PRIMARY KEY,
content text NOT NULL,
embedding vector(1536)
);
-- Approximate index for fast nearest-neighbour search
CREATE INDEX ON documents
USING hnsw (embedding vector_cosine_ops);Build that HNSW index after the first bulk load rather than before it. Inserting into an existing index is slower than building one over populated data, and on a first ingest of any size the difference is minutes.
Storing and querying
Generate the embedding when you write the row. The vector is just an array of numbers.
const { data } = await openai.embeddings.create({
model: 'text-embedding-3-small',
input: content,
});
const embedding = data[0].embedding; // number[]
await sql`
INSERT INTO documents (content, embedding)
VALUES (${content}, ${JSON.stringify(embedding)})
`;Then embed the query and sort by cosine distance with <=>, where smaller means more similar.
SELECT content, 1 - (embedding <=> $1) AS similarity
FROM documents
ORDER BY embedding <=> $1 -- $1 = query embedding
LIMIT 5;That is the entire retrieval half of a RAG pipeline. Feed those chunks to the model as context and you have the other half, which I went through in building a production RAG chatbot.
Add a relevance floor before you ship it. If the best similarity is weak, the corpus does not contain the answer, and knowing that is worth more than returning five bad chunks with confidence.
The actual decision
pgvector fits when you already run Postgres and are comfortably under roughly ten million vectors, when you want to filter and join embeddings against your relational data in one query, and when you value transactions and having one thing to back up.
That middle point is the one that decides it more often than scale does. "Find me similar documents, but only ones this user can see, created this quarter, from an active account" is one query with a WHERE clause in Postgres. In a separate vector store it is a metadata filter, or two round trips and an intersection in application code, and that gets worse as the filters get more interesting.
A dedicated vector database earns its place at tens of millions to billions of vectors or at very high QPS, when you want managed sharding and scaling handled for you, and when vector search is a core product surface rather than a feature inside one.
One operational note if you run Postgres on AWS. Embedding traffic and index traffic leaving a private subnet is exactly the kind of volume that routes through a NAT gateway without anyone deciding it should, which is how AWS bills jump while nothing breaks.
Start with what you already operate
pgvector takes you considerably further than its reputation suggests, and "we outgrew Postgres for vectors" is a good problem to have later. It is a much better problem than running a second stateful system you did not need yet, with its own backups, its own failure modes and its own consistency story against the database that holds the actual data.
If you reach the ceiling, migrating is a batch re-index and a changed query path, and you will do it with real usage numbers in hand rather than a guess made before launch.
Optimise for the smallest architecture that solves the problem in front of you.