What's a Vector, and Why Does It Matter?

A vector is a list of numbers. Sounds trivial, but in the context of AI, it's how meaning gets represented mathematically.

When you run a piece of text through an embedding model (like OpenAI's text-embedding-3-small or a local Sentence Transformers model), the model converts that text into a vector — typically 768 to 3,000 numbers. The key property: semantically similar text produces similar vectors. "Dog" and "canine" end up close together in vector space. "Dog" and "quarterly revenue" end up far apart.

This is why traditional keyword search fails for AI applications. A user searching for "how do I reset my account" won't find a document titled "password recovery steps" unless both phrases are treated as semantically related. Vector search solves this.

What a Vector Database Does

A vector database stores these embedding vectors and lets you search them by similarity — not exact match. When a query comes in, you embed it, then ask the database: "What stored vectors are closest to this?"

The algorithm behind this is called Approximate Nearest Neighbor (ANN) search. It's optimized for high-dimensional spaces where traditional database indexes break down.

Vector databases also handle:

• Metadata filtering (find similar documents, but only from a specific customer's data)
• Hybrid search (combine vector similarity with keyword search)
• Namespacing and multi-tenancy
• Incremental updates (add new documents without re-indexing everything)

The Main Options

Pinecone — Managed, fully hosted, easy to start with. Good default for teams that want to ship fast without managing infrastructure. Pricing scales with index size and query volume.

Weaviate — Open-source with a managed cloud option. More flexible than Pinecone. Supports hybrid search natively. Good choice if you want control or need to self-host for compliance reasons.

Qdrant — Open-source, Rust-based, very fast. Strong filtering capabilities. Good for teams that want to self-host and care about performance.

pgvector — A PostgreSQL extension that adds vector search to your existing Postgres database. Not as fast as dedicated vector DBs at scale, but may be all you need if your dataset is under a few million vectors and you're already on Postgres.

Chroma — Popular for local development and prototyping. Easy to run in-memory. Not production-grade at scale, but great for getting started quickly.

Do You Actually Need a Dedicated Vector Database?

Probably not at first. This is the honest answer most vendors won't give you.

If you have under 100,000 vectors, pgvector on your existing Postgres instance will handle it fine. The query latency difference between pgvector and Pinecone becomes meaningful only at significant scale or with very high query volume.

The right progression:

1. Prototype: Use Chroma or pgvector locally
2. MVP: Add pgvector to your Postgres instance, or use Pinecone's free tier
3. Growth: Migrate to a dedicated vector DB when pgvector performance degrades or you need advanced filtering

Don't over-architect this early. We've seen teams spend two weeks setting up Weaviate clusters for products with 500 users.

Chunking and Embedding Quality Matter More Than the Database

Here's what founders miss: the vector database is the easy part. The hard part is:

Chunking strategy — How you split your documents before embedding dramatically affects retrieval quality. Chunk too large: the retrieved context overwhelms the LLM. Chunk too small: you lose context needed to answer questions. Sentence-level chunking with overlap usually works better than fixed token windows.

Embedding model choice — Different embedding models excel at different tasks. OpenAI's embeddings are good general-purpose. For code retrieval, specialized models outperform them. For multilingual products, use a multilingual embedding model.

Reranking — After vector retrieval, a reranking step (using a cross-encoder model) significantly improves which documents actually get passed to the LLM. It's one of the highest-ROI improvements you can make to a RAG pipeline.

The Bottom Line

Vector databases are a core infrastructure component for AI products that need to retrieve knowledge — but they're not magic, and the choice of vendor matters less than your data pipeline. Start simple, validate that your retrieval quality is good before optimizing for scale, and don't let infrastructure decisions block you from shipping.