Self-Hosted Vector Databases: Full Control Over Your AI Data

Why Self-Host

Three factors drive the self-hosting decision, and most teams are motivated primarily by one of them.

Data sovereignty. Regulated industries (healthcare, finance, legal, government) often require that data stays within specific jurisdictions or on controlled infrastructure. Managed vector database services cannot guarantee the level of data isolation that self-hosting provides. When your embedding vectors encode sensitive information, patient records, financial transactions, legal documents, keeping them on your own servers eliminates an entire category of compliance risk.

Cost at scale. Managed vector databases charge per query, per GB stored, or per compute hour. These per-unit costs are low at small scale but compound quickly. At 500,000 queries per day, Pinecone serverless costs approximately $1,200 to $2,400 per month. Self-hosted Qdrant on a $80 per month dedicated server handles the same workload with lower latency. The monthly savings fund the engineering time needed to maintain the infrastructure many times over at this scale.

Performance control. Self-hosting lets you choose the exact hardware, network topology, and configuration that your workload requires. You can colocate the vector database on the same machine or network as your application for sub-millisecond network latency. You can pin memory, tune kernel parameters, and choose storage hardware specifically for vector workloads. Managed services abstract these decisions, which is convenient but prevents optimization.

Which Database to Self-Host

Qdrant is the strongest choice for most self-hosted deployments. Its Rust implementation delivers the lowest latency (2 to 5ms at million-scale), its memory-mapped storage handles datasets larger than RAM gracefully, and its Docker deployment requires minimal configuration. A single Docker container with a mounted volume is the entire deployment. Qdrant's resource efficiency means you need less hardware than alternatives, a 4-core VPS with 16GB RAM handles 1 million vectors comfortably.

Weaviate is the right self-hosted choice when you need hybrid search or built-in vectorization. Its resource requirements are higher (roughly 40 to 50% more RAM than Qdrant for the same dataset), but the features justify the cost when you actually need them. Deploy via Docker Compose with the Weaviate container plus any vectorizer module containers you need.

Milvus targets the extreme end of scale. If you need to store and search billions of vectors with GPU acceleration, Milvus is purpose-built for it. The trade-off is deployment complexity: Milvus requires etcd, MinIO (or S3), and Pulsar (or Kafka) as dependencies, making it a multi-service deployment even for development environments. Only choose Milvus when your scale genuinely demands it.

pgvector is the simplest self-hosted option for teams already running PostgreSQL. Adding the extension to your existing database eliminates the need for any new infrastructure. The performance ceiling is lower (adequate up to 5 to 10 million vectors), but the operational simplicity is unmatched.

Hardware Sizing Guide

Vector database hardware requirements are dominated by RAM, not CPU. The HNSW index must fit in memory (or at least in the OS page cache) for fast queries. CPU requirements are modest because vector search is memory-bound, not compute-bound.

Small deployment (up to 1 million vectors, 1536 dimensions): 4 vCPU, 8 to 16GB RAM, 50GB NVMe SSD. Suitable VPS providers: Hetzner CX32 (~$15/month), DigitalOcean regular droplet (~$48/month), AWS t3.large (~$60/month). Qdrant uses approximately 8GB for 1 million vectors with HNSW index and metadata.

Medium deployment (1 to 10 million vectors): 4 to 8 vCPU, 32 to 64GB RAM, 200GB NVMe SSD. Hetzner CCX33 (~$40/month), DigitalOcean memory-optimized (~$84/month), AWS r6g.xlarge (~$150/month). Enable scalar quantization in Qdrant to reduce memory consumption by 4x, fitting 10 million vectors in 32GB instead of 80GB.

Large deployment (10 to 100 million vectors): Distributed cluster with 3 to 5 nodes, each with 8 to 16 vCPU and 64GB RAM. Total infrastructure cost $200 to $600 per month on Hetzner or $500 to $1,500 on AWS. Scalar quantization is essential at this scale. Plan for 2x replication for redundancy.

Storage considerations. NVMe SSDs are strongly preferred over SATA SSDs or spinning disks. Vector databases make random read patterns when the index does not fit entirely in RAM, and NVMe's IOPS performance (100,000+ random reads per second vs 10,000 for SATA SSD) directly translates to query latency. On Hetzner's dedicated servers, NVMe storage comes standard. On cloud providers, ensure you are using provisioned IOPS or NVMe-backed instance storage.

Docker Deployment

Docker is the standard deployment method for self-hosted vector databases. It provides consistent environments, easy updates, and clean isolation from the host system.

For Qdrant, a production-ready Docker run command maps port 6333 for the HTTP API, port 6334 for gRPC, and mounts a host directory for persistent storage. Set memory limits to prevent the container from consuming all host RAM. Qdrant's configuration file can be mounted as a volume or passed through environment variables. The most important configuration settings are the storage path, WAL capacity, and optimizer thresholds.

For Weaviate with Docker Compose, the compose file includes the Weaviate container, any vectorizer module containers (text2vec-openai, text2vec-transformers), and volume mounts for persistent storage. Environment variables configure the enabled modules, authentication settings, and cluster configuration. A minimal compose file with OpenAI vectorization has two services: Weaviate itself and the configuration through environment variables.

For pgvector, no separate container is needed. If you already run PostgreSQL in Docker, add pgvector to the image by using a pgvector-enabled base image (ankane/pgvector or the official PostgreSQL image with pgvector installed). If you run PostgreSQL on bare metal, install pgvector from your package manager.

Docker volume management is critical for data safety. Always use named volumes or bind mounts to host directories, never rely on container-internal storage. Configure the host directory permissions correctly, and ensure the storage directory is on your fastest available disk.

Backup and Recovery

Self-hosted databases require explicit backup strategies. Managed services handle this automatically, but self-hosting means you are responsible for preventing data loss.

Qdrant snapshots: Qdrant supports collection-level snapshots through its REST API. A POST to /collections/{name}/snapshots creates a point-in-time snapshot that can be downloaded as a file or uploaded to S3. Schedule snapshots with a cron job, running daily or hourly depending on your data change rate. Restoration is a single API call that uploads the snapshot file and rebuilds the collection.

Weaviate backups: Weaviate supports backup to filesystem, S3, or GCS through its backup API. Backups capture all collections, their data, and their configurations. Schedule with cron or use Weaviate's built-in backup scheduler if available in your version. Restoring a backup replaces the current state of all included collections.

pgvector: Use standard PostgreSQL backup tools. pg_dump for logical backups, pg_basebackup for physical backups, and WAL archiving for point-in-time recovery. All these tools work unchanged with pgvector tables and indexes. This is one of pgvector's strongest advantages, mature backup infrastructure that has been battle-tested for decades.

Test your backups regularly. A backup that you have never restored is not a backup, it is a hope. Schedule quarterly restore tests to a separate environment to verify that your backup and recovery process works end to end.

Monitoring and Alerting

Production self-hosted deployments need monitoring to catch problems before they affect users.

Qdrant exposes Prometheus metrics at /metrics on port 6333. Key metrics: search latency histogram (alert when p99 exceeds your SLA), points_total (track data growth), memory usage (alert at 80% of available RAM), WAL size (alert if it grows unexpectedly, indicating write pressure). Set up Grafana dashboards with these metrics and configure PagerDuty or Slack alerts for threshold violations.

Weaviate exposes Prometheus metrics similarly. Key metrics: query latency by class, object count by class, memory usage per module, and garbage collection pause times (a Go-specific concern). Monitor GC pause frequency and duration, if pauses exceed 50ms regularly, your instance may need more memory or fewer concurrent queries.

pgvector uses PostgreSQL's standard monitoring. pg_stat_statements tracks query performance including vector operations. pg_stat_user_tables shows table-level access patterns. auto_explain logs slow queries with their execution plans, which is invaluable for diagnosing cases where the query planner chooses a sequential scan over the HNSW index.

At minimum, monitor these five things: disk space (vector databases fill disk faster than you expect), memory usage (index expansion can cause OOM kills), query latency p99 (performance degradation is gradual until it is sudden), backup success (failed backups are invisible until you need a restore), and service uptime (use an external health check service to catch network-level outages).

Security Considerations

Self-hosted databases do not come with security configured by default. You are responsible for network isolation, authentication, encryption, and access control.

Network isolation. Never expose vector database ports to the public internet. Use firewall rules (iptables, security groups, or cloud VPC settings) to restrict access to only your application servers. If remote access is needed for monitoring or management, use SSH tunnels or a VPN.

Authentication. Qdrant supports API key authentication, configure it in the configuration file and pass the key in request headers. Weaviate supports API key and OIDC authentication. pgvector inherits PostgreSQL's comprehensive authentication system including password, certificate, and LDAP authentication.

Encryption in transit. Enable TLS for all database connections. Qdrant supports TLS with certificate configuration. Weaviate supports TLS through reverse proxy (nginx or Traefik) or built-in TLS in newer versions. pgvector uses PostgreSQL's native SSL support.

Encryption at rest. Use filesystem-level encryption (LUKS on Linux, EBS encryption on AWS) to protect data at rest. This encrypts the entire storage volume transparently without any database configuration changes. All major cloud providers offer encrypted block storage at no additional cost.

Self-Hosted vs Managed: The Real Cost Comparison

The total cost of self-hosting includes infrastructure (servers, storage, networking), engineering time (setup, maintenance, troubleshooting, upgrades), and risk (downtime cost if something breaks). Managed services fold all of these into a single bill.

For a team with fewer than 100,000 vectors and no data sovereignty requirements, managed services are cheaper because Pinecone's free tier costs nothing. For a team with 1 to 5 million vectors and moderate query volume, the decision depends on whether you have infrastructure engineering capacity. If you do, self-hosting saves 50 to 70% on monthly costs. If you do not, the engineering time to learn and maintain the infrastructure can exceed the managed service premium.

The crossover point where self-hosting is almost always cheaper, even accounting for engineering time, is approximately 200,000 to 500,000 queries per day. At this scale, managed services cost $500 to $2,000 per month while self-hosted infrastructure costs $40 to $200 per month, a savings that covers significant engineering effort.