跪拜 Guibai
← All articles
Backend

Vector Search Breaks When It Lives in a Separate Database

By 一只牛博 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Standalone vector databases create a synchronization tax that grows with every business filter added to a query. When permissions, document versions, and embeddings drift out of alignment, the recall results are silently wrong, and debugging requires manually correlating logs across three systems. A unified transactional boundary turns that class of failure into a straightforward database constraint violation.

Summary

Enterprise vector search falls apart when permissions live in one database, documents in another, and vectors in a third. A document retraction that doesn't sync to the vector index, a permission change that misses the recall step, or a model upgrade that leaves old and new vectors mixed together all produce answers that look plausible but are wrong in ways no one can trace. The fix is not to cram everything into one column but to manage relational metadata, document bodies, vector embeddings, and temporal conditions inside the same transactional and query boundary.

KingbaseES's multi-model approach keeps each data type in its own table structure while letting a single SQL statement combine business filters with vector distance ordering. A query can check department membership, security clearance, and effective dates before ranking by cosine similarity, and the whole thing runs in one transaction. When something goes wrong, the audit trail points to a specific condition, not a mystery across three systems.

Model upgrades and document versioning become manageable because the database records which embedding model produced which vector batch. Regression testing can replay the same questions against old and new vectors to measure recall changes, rather than relying on gut feel. The architecture doesn't eliminate the hard work of setting thresholds, annotating test sets, or isolating workloads, but it stops data shuffling from being the thing that breaks first.

Takeaways
Vector search in production breaks when permissions, document metadata, and embeddings live in separate systems with independent update cycles.
KingbaseES stores relational columns, document fields, vector columns, and temporal data in the same database, letting one SQL statement combine business filters with similarity ranking.
Document publishing and vector writing should share a transaction; a half-committed state where the file is published but vectors are missing produces silent recall gaps.
Every vector record must carry the embedding model name and batch so that model upgrades can be regression-tested before switching the active flag.
An audit table that captures query hash, filter snapshot, and returned chunk IDs makes it possible to replay a failed recall and identify whether the cause was a permission change, a version change, or a model change.
Keyword search on exact fields like department codes and policy numbers should run alongside vector similarity, not replace it; the two retrieval modes complement each other when combined in one query.
Multi-model fusion does not mean all workloads go on one instance; vector index builds, bulk imports, and spatial queries still need separate resource planning and tablespace design.
Vectors can contain sensitive information derived from personal data or contracts; access controls must apply to vector columns the same way they apply to readable text.
Conclusions

The standard vector-database demo, chunk, embed, search, top-K, ignores the reality that most enterprise queries need five or six WHERE clauses before similarity ranking even matters. The industry's fixation on ANN benchmark numbers has obscured the fact that recall quality is dominated by stale metadata, not index recall speed.

Treating a model upgrade as an in-place column overwrite is a data-loss event in disguise. Without a model_name column and a batch-switching pattern, an organization permanently loses the ability to explain why a recall result changed, which makes vector search un-auditable for any regulated use case.

The argument for a separate vector database has always been about specialized indexing, but the operational cost of keeping permissions, versions, and vectors in sync across systems is higher than most teams estimate. A converged database that supports HNSW indexes inside the same transaction scope eliminates an entire class of distributed consistency bugs.

Keyword search and vector search are often pitched as competitors, but the article's SQL examples show they are complementary filters inside the same query. Exact-match fields handle the structured half of a question; vectors handle the fuzzy half. Splitting them into separate services forces the application to merge result sets, which is strictly worse than letting the query planner do it.

Concepts & terms
Multi-model fusion architecture
A database design where relational, document, vector, spatial, and time-series data models coexist under one management system, sharing transactions, permissions, and query access, rather than running as separate database instances.
HNSW index (Hierarchical Navigable Small World)
A graph-based approximate nearest neighbor index used for high-dimensional vector similarity search. It builds a layered graph structure that allows fast traversal to near neighbors without scanning every vector.
Vector distance operator (<->)
In PostgreSQL-compatible vector extensions, the <-> operator computes the distance between two vectors, typically cosine distance or Euclidean distance, and is used in ORDER BY clauses to rank results by similarity.
Embedding model versioning
The practice of recording which embedding model and batch produced each vector row, so that when a model is upgraded, old and new vectors are not mixed, and recall changes can be attributed and regression-tested.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗