Code as Code – Architecture as Code? – The Inevitable Slide Towards Everything as Code!

2022/07/19

An abstract thinking test. Memories from high-school physics. Speed-running through programming mistakes and anti-patterns: again and again, each time somewhat faster.

Remember from what is programming that a program is something that, very generally, takes som input and gives you an output. Today we’re going to take that idea further.

I’ve had this post on my mind for a while now, and finally found the motivation to get it out. This was triggered by a recent chat with Lucas Fiévet from logicflow.ai, where he shared some thoughts about Architecture as Code which nicely complemented my own observations.

Abstract Thinking – A Detour Through Physics

Before we proceed, let me share some high-school memories. At the time, during my first physics courses, something that was at first pretty hard for me when solving a problem, was to keep things general and abstract until the very, very last moment.

The exercise might describe a situation with precise physical quantities (eg: how long until a liter of water at room temperature that sits on a 900W stove evaporates), but the best way to solve it – or at least to understand what is going on – is to stitch the relevant equations together, arrange them in a way that they give you the answer you are looking for and only then start replacing the variables with actual numbers.

In software, and for the sake of this argument, such variables (or let’s call them meta-variables) represent programming language, frameworks, specific technologies or even particular architectures.

What I’m hinting at: it is interesting to wait for as long as possible before filling them in when reasoning about a problem, because like in physics (or math), it often happens that variables cancel out and simply vanish from your problem!

Hold this thought for a while.

Code is Data is Code

Here’s the rough history of my own discoveries in the field of computing and how some of my perceptions evolved. This happens to loosely map to certain hype cycles the field went through:

  1. Code, Data, Configuration and Hardware are four entirely different and separate things. Cloud does not exist as a concept. Code exists to manipulate data and doing so is annoyingly cumbersome. Developper and SysAdmin describe very different kinds of jobs and people.
  2. Data and Code are still two strictly separate things, but manipulating Data becomes much easier. Configuration starts to look a little bit like data: you can transform and maybe even generate it, but it’s mostly basic string interpolation. Some clouds appear, but they’re basically just a 1-1 mapping of the Hardware you know: a VM in someone else’s data center that looks and feels like a real computer.
  3. Cloud stops to be synonym with VM: it offers smarter primitives (Blob stores, message queues, etc.). Configuring it is still manual ClickOps, albeit some APIs appear so the cool kids start to talk about Infrastructure as Code1.
  4. Some lazy (but smart!) folks hate writing boiler-plate code so much that they start generating it with some other code: think JooQ, Thrift, GRPC, … Interestingly these have to do with the transfer and storage of Data. Some code looks at some data and spits out… more code to work with that data. Slowly, Code as Code has more to it than meets the eye at first.
  5. You can now spawn vast cloud infrastructure by writing simple configuration files: Infrastructure as Text is an accurate description for most users. Thus it quickly evolves into Infrastructure as Copy-Paste. Hardware, services and any computational resource have become pure Configuration.
  6. Tools like Bazel come along, letting you reliably pipe any executable’s output into any other exec’s input. Recursively and ad-infinitum, if you so wish. The difference between code, data and configuration becomes more a matter of syntax and definition. Code as Data gets familiar.
  7. "But," you say, “there’s still real data out there that’s not code or anything like it, like pictures or sensor measurements or…”. To which I’d reply: “Yup, and that data is being fed through machine learning which will output some models. The models will be run to process some input and produce an output.2” Data as Code is here already.
  8. Slowly but surely, everything is becoming code. The distinctions betwen Developer, DevOps, Frontend, Backend, … are slowly but surely melting away. Or they could, given the tooling that is available.3. Data as Code, Code as Data … We’ve come full circle to Code as Code.

What about architecture which the title mentions? Remember that code is a description of how inputs are processed and turned into outputs. What we commonly describe as architecture is how multiple executable components are assembled together into a higher-level system. That system will be processing inputs and output something. Thus architecture already is code.

What Was It With Physics

Let’s get to one of the reasons we live in interesting and exciting times with respect to software and computing: so many things can now be manipulated and controlled through code that the limit on what you can do with the possible combinations really becomes a matter of imagination.

However, here comes the proverbial but: similar or identical problems are being solved in slightly different ways depending on the context and who is encountering them. This is fine as our goal should always be to get things done, but when taking a step back we quickly see that inefficiences abound.

Here’s where the math or physics analogy comes in: in today’s stack we often start by filling in our meta-variables first, and then move on to solve our problem. In the process, we may miss some opportunities for massive simplifications.

A Modern Example

Say you’re building a modern web app that will need a frontend, complemented with a backend for some data processing and storage. Obviously it needs to run in the cloud, but that’s about it for the requirements as long as the app does what it needs to do. Also, the project is sufficiently big that you can hire multiple people for it – a frontend dev, a backend dev and why not a third person for handling the infrastructure and other operational aspects.

You might be a laissez-faire kind of person: bring the team together and let them figure things out. Here are two ways this could go:

Everyone brings their favourite tools

The team is diverse and everyone has a strong command of their own tools:

Because everyone’s strongly opinionated and you allow them to, they end up using each of their favourite tools. The final setup looks like this:

There is no reason this should not work, and for sufficiently large projects it would even be reasonable.

However, let’s take a step back and count the various things we have here:

Can A Single Language Be Enough?

Here’s another way things could go. For this, let’s say you’re still in charge and a quite laissez-faire kind of person, but because you’re afraid of the curse of cardinality you kindly ask your team to try to keep things simple in that regard.

Assuming a mature and open-minded team, their discussion could go like this:

Ok, what we’re building will need some things to run in the browser, at least one backend component that requires a database and we want to rely on modern practices with respect to operations and configuration.

In short:

And here’s one train of thought they could follow while tackling this:

Our frontend needs would be met beautifully with Typescript, so that language is a given. Where else can we leverage it? Some of the business logic is simple enough that the backend dev is willing to tackle it in Typescript too. Plus, we might find a framework that takes care of the boiler-plate around user and schema management. Regarding orchestration and infrastructure configuration, do we really need to add two technologies with their own specific quirks?

And they could end up with this stack:

This might not necessarily be better than the first option, but is a valid alternative as it might well reduce the barriers between frontend, backend, operations and infrastructure – ultimately allowing everyone on the team to easily contribute to areas beyond their direct specialty. This can be valuable in an environment where any kind of hiring is hard, and be beneficial with regard to organizational fragmentation.

Oh, and of course, you can take it even further with services like Vercel and supabase, which let you forget about the infrastructure part entirely5.

A Speed-Run Through Anti-Patterns

An interesting side effect of the whole X as Code story is that we get to relive both the good and bad aspects of the history of programming.

I already joked about copy-paste build engineering and how, once we don’t feel like we’re coding anymore, a lot of big-no-no’s suddenly become acceptable.

There’s something similar happening in the infrastructure world, were many definitions are simply copy-pasted. Sure, it beats manual clicky-clicky operations, but still causes many of the same issues you run into when you do this with your regular code:

Oops, messed up my search-and-replace and the reviewer did not catch that production-breaking change among the thousands of changed lines…

So remember: Everything as Code in turn implies Coding good practices apply to Everything.

Which makes me want to write a post about how functional programming can better your everyday life.

But possibly it’s just me overthinking again… so I’ll just drop in two illustrations about silos and trees, as they are kinda relevant to the issue of Everything as Code:

Because, as code tends to make things more efficient, organisational structures that are slowing you down or otherwise introduce friction become more and more apparent.

Happy coding!

  1. I guess the early IaC was much closer to as Code, at least if you wrote the scripts calling the APIs yourself. ↩︎

  2. If you want to go on a tangent, read Jaron Lanier’s description of the rainstorm thought experiment ↩︎

  3. Whether the people who need to understand this actually do is another question… I’d still avoid labeling myself as a DevOps for pure career reasons. ↩︎

  4. Before you start thinking that I’ve become a Typescript absolutist, please note that these can also allow you to use other programming languages. ↩︎

  5. These let us build the app behind optifocus in less than two days – the only challenge was the business logic, the rest is taken care of automagically. ↩︎