ublo
bogdan's (micro)blog

bogdan » Stop Searching by Coincidence: Portability and the Storage Decoupling Layer

09:47 pm on Jul 6, 2026 | read the article | tags: medium

Most architecture diagrams are much easier to understand after the system has already been built. You look at the boxes, see an API gateway connected to an embedding model, a vector database, a message broker, and a relational database, and the design appears almost inevitable. Of course the query goes there. Of course ingestion goes through a queue. Of course the model is separated from the application.

The problem is that real systems rarely emerge that cleanly. They grow from constraints. One choice creates pressure somewhere else. Solving that pressure introduces another requirement. Eventually, the architecture starts to look deliberate, but the interesting part is not the final diagram. The interesting part is understanding why the boundaries exist in the first place.

The system I have been building powers SearchPixel, but the infrastructure is intentionally broader than a WordPress search plugin. At its core, it is infrastructure for multi-tenant semantic search and retrieval workloads. It needs to accept documents, transform them into representations suitable for retrieval, store those representations, execute low-latency hybrid searches, and remain flexible enough that neither the vector database nor the machine learning model becomes a permanent architectural dependency.

The system uses Kubernetes as its infrastructure backbone, a Go API gateway as its coordination layer, Snowflake Arctic as the primary embedding model, Qdrant as the primary vector database, MySQL for tenant and credential configuration, and NATS for the asynchronous ingestion path.

Kubernetes Was the First Architectural Boundary

I started with Kubernetes before choosing most of the application components. That may sound backwards, as a common approach is to build the application first and decide how to deploy it later. For this system, infrastructure portability was already one of the product requirements, so deployment could not be treated as an implementation detail. I wanted to preserve two very different operating modes.

At one end, I wanted the ability to run on a public cloud using a managed control plane. At the other end, I wanted to be able to move the same workloads onto cheaper infrastructure and operate the cluster myself when cost became more important than convenience. Those two environments are not identical, and I do not pretend Kubernetes magically erases every difference between providers. Storage, networking, load balancers, GPU availability, and node provisioning remain provider-specific concerns. What Kubernetes gives me is a stable operational boundary above those differences.

My model servers are pods. The API gateway is a deployment. Qdrant is stateful infrastructure with persistent storage. NATS is another network service. Configuration is declarative. Service discovery follows the same conceptual model regardless of whether the nodes underneath happen to come from a large public cloud or from a cluster I manage myself.

That distinction matters because cost curves change as systems grow. A managed service can be an excellent choice when operational simplicity is more valuable than infrastructure cost. The calculation changes when traffic becomes predictable, when workloads are large enough to justify dedicated nodes, or when expensive inference hardware dominates the bill. I did not want a future decision to move infrastructure to require redesigning the application at the same time. Kubernetes therefore became less of a deployment tool and more of an architectural portability layer.

There was another reason for making that choice early. Machine learning infrastructure is unusually heterogeneous. An API service, an embedding model, and a vector database do not scale in the same way. They do not have the same memory profile. They do not start at the same speed. They do not fail for the same reasons. A stateless network service may scale horizontally in seconds. A model server may need a large artifact before it can answer its first request. A vector database may need stable local storage and careful placement. I wanted an infrastructure substrate where those workloads could coexist without pretending they were the same.

This decision also explains my preference for relatively plain Kubernetes manifests, Kustomize, and FluxCD. I am not opposed to Helm in principle. I simply prefer to keep the deployment model visible when I am responsible for operating the system, especially because I pay for this infrastructure out of my own pocket. I want to be able to inspect a StatefulSet and understand what is running, how storage is mounted, how a node starts, and where configuration comes from.

Isolating Storage via the Adapter Pattern

Semantic search systems have a natural tendency to organize themselves around the vector database. It makes sense, because the vector store sits directly on the critical query path. It contains expensive derived data. Its indexing strategy affects latency, recall, memory consumption, and operational cost. Once enough data accumulates, migrating away from it can become painful. That is exactly why I did not want the rest of the system to become structurally dependent on one database.

Today, Qdrant is my primary vector engine. I wrote a separate article about why I chose it, so I will not repeat that evaluation here. The more important architectural decision is that Qdrant is a routing target, not the identity of the platform.

In practice, the storage decoupling layer is not a universal database API. It is the combination of tenant-aware placement, backend-specific adapters, and a gateway that keeps those decisions away from clients. To achieve this, the gateway isolates backend-specific behavior behind small storage adapters. The request path resolves the tenant’s configured backend and dispatches the operation to the corresponding implementation. The goal is not to pretend every vector database has identical semantics. It is to keep backend placement and backend-specific query construction out of the external API and the higher-level request flow. When a request reaches the gateway, the system does not simply assume a collection or index structure. Instead, tenant configuration identifies both the backend type and the target indexing node. The gateway resolves that placement at request time and sends the operation through the corresponding backend adapter.

My current production gateway code still contains both Qdrant and RediSearch paths operating behind this abstraction layer. This design choice is larger than merely preserving backward compatibility with Redis. I do not believe a single vector database will necessarily remain the correct answer for every retrieval workload, because the underlying ecosystem is changing too quickly for that assumption to be comfortable. Databases are constantly adding sparse vectors, new quantization techniques, disk-backed indexes, late interaction support, GPU acceleration, and different approaches to distributed search. A design that is optimal today can become an operational bottleneck surprisingly quickly.

There is also a more fundamental distributed systems reason for application-level database routing. Horizontal scalability is not infinite in the way abstract cloud marketing benchmarks suggest. It is easy to say that a database scales horizontally, but that phrase hides the cost of coordination. As a distributed dataset grows, queries may touch more partitions, replication consumes more bandwidth, rebalancing becomes more expensive, coordination surfaces expand, and tail latency becomes increasingly sensitive to the slowest participant in a distributed operation. At some point, depending on the workload, adding more capacity to a single logical database cluster produces diminishing returns.

For a multi-tenant system, this is especially important because tenants already provide a natural partitioning dimension. There is no inherent reason every independent website must live inside one globally scaled vector database cluster. A collection of tenants can be assigned to one Qdrant deployment, another collection to a second deployment, and a particularly heavy tenant can be completely isolated on its own dedicated database node if its workload justifies it.

Consider a deployment serving thousands of relatively small websites. Most of them may fit comfortably on a shared Qdrant cluster. Then one customer arrives with a catalogue larger than hundreds of existing tenants combined and a sustained query profile that changes the capacity model. I do not want the only response to be scaling the shared cluster for everyone. The routing layer gives me another option: place that tenant elsewhere without changing the client API or forcing unrelated tenants to absorb the operational consequences.

Independent deployments also create a useful failure boundary. Scaling is not the only reason to separate workloads; sometimes the goal is simply to prevent one pathological tenant or one overloaded cluster from becoming everybody’s incident.

Database sharding asks how to distribute data inside a database system, whereas application-level routing asks whether all data needed to be inside the same database system in the first place. For a strongly multi-tenant workload, I want both options. That is why the indexing node and indexing backend belong to tenant configuration rather than being hardcoded into application logic. It means that introducing another vector engine later can remain primarily a local adapter problem rather than becoming a platform-wide migration.

Decoupling storage solved one class of future migration, but it exposed the next assumption in the system. A hybrid search request does not begin at the database. It begins with a machine learning model, and those models evolve even faster than storage engines. The moment I accepted that the vector database could not define the platform, I had to admit that the embedding model could not define it either. That is where the next part begins.