For years, our obsession has been the big app, small team model: flattening the software complexity curve so small teams can build production cloud applications with an expertise-first and scale-first mantra. Those lessons, that expertise, and those workflows have become even more relevant in the era of Agentic Production Coding. Now we can build a big app with small models.

Why do small models matter?

Because AI costs have become the new bottleneck, and a hybrid approach that uses smaller models for most tasks while bringing in larger ones only when needed can be a major advantage in most AI production environments.

So, while engineering has been reshaped, good engineering is still what matters for quality and scale, and by avoiding common traps, applying the right techniques, and practicing good engineering discipline, we can make AI a force multiplier, not a wallet crusher.

1. The trap: buying your way out of expertise

There’s a quiet wish underneath a lot of AI enthusiasm right now. It goes something like:

Maybe we don’t need the expensive expertise anymore. Maybe we don’t need people who understand architecture and what makes applications scale. The AI will figure it out, and if it can’t today, it’s just a not-yet problem.

If we take that train of thought, we reach for the biggest model we can find, and the biggest-model chase starts. The focus moves from augmenting expertise to delegating expertise. The bigger the gap we ask the model to fill, the bigger the model we think we need.

For demos, this works extremely well now. We one-shot an MVP, add a couple of iterations, and feel like we’re at production level. When issues show up, we call it a not-yet problem and wait for the next new-awesome-frontier model.

But production has never been about one-shotting anything. It is about architectural discipline, tradeoffs, and understanding which track we want the system to follow and why. That requires expertise, plus a huge amount of unwritten experience and context.

The real opportunity in a production system is managing complexity as it grows, then taking the steps necessary to keep that complexity from becoming unmanaged. AI can absolutely force multiply that work, but only when there is a strong foundation to multiply from.

That is where a robust, scalable blueprint from the start becomes the game changer. If we skip it, we accumulate a photo collage of technology: a little framework here, a database there, a shiny tool from LinkedIn, all easy in the moment and none chosen because it belonged. Complexity does not grow linearly here. It grows exponentially.

In other words, scaling engineering with AI is still about flattening the complexity curve as the system and requirements scale.

Look at the red line, easy first. It starts low and cheap, then curves violently upward. The blue line, architecture first, starts a little higher because we invested up front, but it stays flat as we scale. And the green line beneath it? That’s the blueprint doing its job, pulling the whole system down toward manageable.

The choice isn’t “big model versus small model.” It’s managed complexity versus unmanaged complexity. And that’s a choice we make with architecture, not with a bigger GPU bill.

2. What managed complexity actually looks like

Let’s make this concrete. Say we want to build a video annotation platform, the kind of thing where surgeons review procedure footage, mark moments, and have the system run AI recognition on individual frames. At a high level, we need a few services:

• downscaler - Downscales video from HD to 480p, or even smaller, for fast scrubbing.

• framer - Extracts images at 4fps or lower for AI analysis, UI annotation painting, and related workflows.

• ai-oob - Identifies out-of-body frames for PHI redaction when the camera is outside the body.

• ai-in-body - Segments entities when the camera is inside the body, for example tools and organs.

Then we need one or more user-facing web services, whether API or UI:

• main-ui - UI annotation tools that allow clinicians to annotate and assess surgical procedures.

• devops-ui - Internal DevOps web UI to manage cloud services and related operational workflows.

From a high-level standpoint, there are two very different ways to tackle this.

1) Spider Web Architecture - Unmanaged Complexity

In this approach:

• all services talk to each other as needed

• communication happens through gRPC for maximum performance

• the data pipeline is fixed in some format with a cloud-dependent executor

It feels natural, each service just talks to whoever it needs. With three services, it’s fine. By the time we have fifteen, we’ve built a spider web: every service has to know about every other service, and each new one we add potentially wires into all the rest. Connections grow with the square of the number of services. The fifteenth service isn’t one more connection, it’s fourteen more. That’s the red curve, live.

And because this, or some form of it, is relatively common in smaller applications, AI will probably gravitate toward it. Then it doubles down, triples down, and unmanaged complexity skyrockets.

2) Bus Architecture - Managed Complexity

• All services talk to the bus only.

• Communication can be as simple as valkey-stream, a lean Redis/Valkey-style stream.

• Separation of concerns is handled through app, data, and system event types.

• Message payloads stay small, mostly actions and S3/DB UUIDs.

• The data pipeline is event based, not map based, allowing us to independently scale job workers without changing other services or hardcoding pipeline rules.

With this architecture, every communication path points straight down into one thing: the bus. Web servers, the command pod, services 1, 2, and 3, none of them talk to each other. Messages are kept very light by design, and even the data pipeline is event based rather than map based, giving the whole system more scale and flexibility.

Now complexity is managed from the start, and AI can do its magic without adding unnecessary complexity to the system.

When we push this further, we can have a full blueprint that frames the work for AI, no matter which model or model generation we use, so it builds inside a robust and scalable application architecture.

Here’s the same idea grown into a real system. Web, mobile, and API access up top. Front-facing web servers and a fleet of worker pods. The message bus in the middle. Kubernetes underneath, auto-scaling those workers because they’re decoupled and stateless. Prod DB is RDS Postgres. Blob storage is S3. Nothing exotic. Every box earns its place.

That’s managed complexity: decoupled communication and small data. Two decisions, made deliberately, that keep the curve flat.

Software is a game of orders of magnitude. Shrink the payload by 1000x and we’re not optimizing, we’ve changed the entire frame of reference for what the system needs to be.

3. Why this is exactly what AI wants

Now bring the small model back in.

When we manage complexity, we relentlessly drive unnecessary complexity toward zero. When we do not manage it, we do not even know the ratio. We cannot tell which complexity is real and which is self-inflicted.

And this is precisely where a smaller, cheaper model thrives.

When the architecture is simple, small messages, one communication pattern, clear contracts, boring proven tools, the model is not being asked to hold a spider web in its head. It is writing a worker that reads an ID from a stream, fetches an object from S3, and writes a row to Postgres. That’s a bounded problem. We do not need a frontier model to nail it. We need a competent one working inside good constraints.

The reason we often believe we need the biggest model is that we are asking it to operate inside an unmanaged system, to reason about a codebase where everything is coupled to everything and any change ripples everywhere. Of course that needs raw horsepower. We made the problem exponentially hard.

Flatten the complexity curve and we shrink the problem the model has to solve. A simple architecture is the great equalizer for models. The blueprint does the heavy lifting so the model does not have to.

That’s the whole thesis in one line:

The blueprint is what lets a smaller models build a big production application.

And best of all, the same discipline makes bigger models perform better too. At the highest level, this is not really about small models versus big models. It is about getting the most out of AI for production coding and deployment.

4. The takeaway for people who build

For a CTO, an architect, or an engineer thinking about where AI actually fits, a few things are worth holding onto.

• Code architecture, not PowerPoint architecture. The blueprint that matters is not the PowerPoint diagram. It is the real structure of the running system: what talks to what, and what moves between them. Crystallize that, and everything downstream gets easier.

• Lame is the new cool. Choose fewer technologies, and choose boring ones. Default to Postgres for metadata and relational state. Use specialized storage only for specific needs, for example true big-data workloads. Use S3 for blobs. Use one bus. valkey-stream is simple and scalable. Every additional technology is a permanent tax on complexity. The goal is the right tech in the right place, and as little of it as possible.

• Dumb is the new smart. Each service should be smart at one thing and dumb at everything else. Services know how to feed themselves, do their job, save their state, and emit events back to the bus.

• Keep it small. In the bus, storage, contracts, and everywhere else, keep payloads small. Bus events should carry actions, metadata, and storage UUIDs, not large objects. Adding large payloads to the bus creates a whole slew of unnecessary complexity.

Then, and only then, bring in AI. A small model inside a clean architecture will out-build a big model drowning in an unmanaged one, every time.

The industry is chasing bigger models to one-shot its way past the need for expertise. It is the wrong race. The teams that win will not be the ones with the biggest model. They will be the ones with the flattest curve.

Big app, small model. It starts with the blueprint. Simple always scales better.

BriteSnow builds production cloud applications with small teams and the right architecture. More on flattening the complexity curve at stack.britesnow.com.

_{(Originally published 2019 January 24, on Medium)}

Over the last couple of years, Kubernetes has been one of the few new technologies that established itself as the de facto standard in its field. To understand why Kubernetes is such a transformative evolution, let’s take a little journey down virtualization memory lane. For the sake of brevity, we will over-simplify the various architecture types to highlight their core characteristics and evolutions.

One to One

At the beginning of microcomputers, physical hardware was mostly used with one and only one operating system, and most applications such as databases, web servers, and websites usually had a single end-user application purpose, at least in production.

This meant that when a business needed to deliver or host a “pet.com” and a “food.com” end-user applications, the common production architecture was to have a dedicated server(s) for each application. As the end-application load requirement increased, horizontal layers were split apart in their own servers; but overall, the one-to-one relationships still remained.

Dedicated ServersWhile this worked relatively well for bigger, simple three-tier architecture applications, the low service granularity (i.e., dedicated server(s) per application) made resource utilization sub-optimal and scaling slow and costly, as adding new resources could take days and oversizing hardware resources were common practice to mitigate handling peak demand. In short, those were the costly days of building internet services.

“Application Virtualization” (e.g., Virtual Host)

To increase service granularity, most infrastructure applications, such as web servers and databases, offered some sort application compartmentalization capabilities, allowing one infrastructure application process to support multiple front-end applications. In the web server world, this is called virtual hosts, where one web server can serve multiple websites based on domain names, whereas in the database realm, a single database server could offer completely isolated logical database instances on the same server/port.

Those application virtualization capabilities allowed many businesses to deploy logically isolated applications, or at least provide the illusion of isolation while utilizing the same physical resources.

Anecdote: In my early Netscape days - the late 90s - Apache Virtual Hosts support was one of its killer features compared to Netscape Enterprise Server, which could serve only one site per port. Apache ate our lunch in the web site service providers’ space.

As in the previous model, when throughput requirements increased, each of the horizontal layers were split out in their own server while still serving multiple applications. Some applications, such as databases, started to offer more advanced horizontal scaling with their clustering capabilities, allowing one logical database to span across multiple hardware resources, enabling further hardware specialization and more optimum resource utilization.

While these techniques increased service granularity and improved resource utilization, it came at the cost of software architecture and deployment complexity. That added complexity and inter-dependencies made scaling service more of an art exercise than a good engineering one. Although early hosting companies did manage to offer web hosting as a service, it did not come without some serious elbow grease.

In other words, these were the days in which the internet was complex and brittle.

Virtual Machine

As Linux gained more popularity on the server, so did operating system virtualization, which allowed a single hardware to host multiple operating systems. With this new model, each system or application could have its own operating system and environment while still sharing the same hardware resources.

Note: Technically, virtual machine techniques were pioneered in the 60s by IBM; but in this context, it is fair to state that Linux (and Windows, in some respect) popularized the concept of full OS virtualization.

This new model maximized resource utilization and simplified application architecture back to their original form by removing inter-system dependency, but the most transformative consequence of these new capabilities was the formation of new IT operations driven by the Amazon Web Service and the like, which abstracted all of the hardware provisioning and networking complexity to offer a connected internet operating system as a service. With these new hosted or on-premise services, software organizations could then focus on what they knew best; software, and let the few hardware and networking experts build and scale the backbone of those services.

However, as hosted infrastructure services got more sophisticated, so did applications, which started to move from their traditional three-tier monolith architectures to multi-service ones (e.g., micro-service), and designing, deploying, and managing scalable multi-service applications in a cost-effective manner became relatively complex and highly cloud-specific.

Containers

The industry resolved the virtual machine model limitation for micro-service architecture with a rather ancient Linux capability that allowed operating system processes to be boxed into a subset of the operating system resources on a single operating system; and so, the next chapter in virtualization, containerization, was born and mostly won by the now ubiquitous Docker container format.

With containerization, applications could now be architected with the optimum set of services for their needs while trusting that the technology will compartmentalize those services in the most efficient manner. What used to take days in the old physical hardware era and minutes in the virtual machine days, are now only taking sub-seconds with a minimal (i.e. reasonable) memory footprint.

While containerization dramatically improved service granularity (i.e., resource utilization), optimized sophisticated application architectures, and normalized service runtime across development and production, the sheer volume of those new parts added significant deployment management complexity and configuration challenges. Scaling a production system still required cloud-specific techniques to scale the underlying virtual machines with the added complexity of deciding where and when to create new service containers.

In summary, containerization was instrumental but needed something else to realize the true value proposition.

Kubernetes

This is where Kubernetes came in and completed the puzzle. Based on containers, Kubernetes is a service (sometimes called “engine”) that provides a way to describe, manage, and run a complete system with all of its parts (i.e. micro-services), interactions, and scaling rules in a standard and environment-portable manner.

Whereas previously the development, staging, and production environments heavily differed, Kubernetes’ new approach normalized and streamlined how a complete system could be described, ran, and scaled which dramatically reduced the cost of building and managing multi-service system.

Note: Google Cloud Platform, creator of Kubernetes, has a state-of-the-art implementation of Kubernetes, and while Kubernetes has become a very well-supported de facto standard on all major cloud platforms such as Amazon Web Service, Azure, and IBM, getting started with Google Cloud often provides the best learning experience. For example, Google Kubernetes Engine provides excellent support for node auto-scaling.

It is important to understand that all of those technological milestones have been evolutive and additive and typically used in concert with one another. From a purely technical point of view, the revolution is in the evolution and not in the disruption. Containers typically run on virtual machines (although they can run on bare metal as well), virtual machines still run on a main operating system (called a host) running on a physical server, and many infrastructure applications such as databases, web servers, and caches still provide useful and sometimes critical application-specific clustering and compartmentalization capabilities.

Putting all of the scorecards together, we can see how all the virtualization technologies stack up.

While Kubernetes has been the result of an evolution rather than disruption, its business values are nonetheless transformative.

Here are the key business values of Kubernetes:

• Normalize Dev to Ops: Since Kubernetes is infrastructure-independent (i.e. it can run on the cloud, on-premise, or even on a laptop), it provides a very robust and flexible way to normalize the development and operations environment, which is probably one of the most important attributes in a high-velocity development methodology.

• Eliminate multi-service overhead: Just a couple of years ago, moving from monolith architecture to the micro-service one, while the right architectural move, did come with significant development and operational setup frictions. Kubernetes is designed from the ground up to tackle these requirements in an exhaustive and relatively simple way, and used correctly, completely eliminates that overhead. For example, at BriteSnow, all our new architecture are now multi-service from the get-go, even though we often start a new “big app” with a relatively “small team.”

• Maximize Cloud portability: One of the most impressive accomplishments of Kubernetes is that in just a few years, it took the industry by storm and made everybody, from the major cloud providers to hybrid cloud technology providers, race to add their support in the new standard. This has put Kubernetes as the cornerstone of cloud portability and allows most of the modern cloud applications to express 80% to 90% of their system and scalability topology in a cloud-portable manner.

• Trivialize and standardize scalability: Part of the value of cloud portability is the standardization of common and custom scaling rules that make scaling a multi-service system exponentially simpler while still providing the customization hook when needed.

I am usually very cautious about jumping onto the latest “trendy” technology bandwagons, as more often than not, while exhilarating, they tend to be relatively ephemeral. However, once a while, a technology becomes “trendy” for of all of the right reasons, and Kubernetes is one of the few that is being cemented as foundational technology like Linux, OS virtualization, and Git once were in their own field.

In short, Kubernetes is probably the safest investment any cloud application development organization could make today and one that, if applied right, can dramatically increase the speed and quality of development and delivery.