Infrastructure as a Code: Everything you ever wanted to know but were afraid to ask

Infrastructure deployment automation is such a nice mantra – everyone talks about it, but few customers really have a fully automated deployment of their entire application infrastructure. So in today’s article we will look at how to achieve it. We’ll describe the reasons when and why to automate in this way, and show some examples.

Martin Gavanda

 

When (not) to consider automated infrastructure

As our colleague Lukáš Klášterský likes to say, if you are starting out with the cloud, “don’t think big” and get to know it thoroughly first. On the other hand, if the cloud is already a standard for you, you run several larger applications there and everything “runs as it should”, then maybe it’s time to think about Infrastructure as a Code (IaaC).

So what is the main goal of IaaC? As the name suggests, the idea is to have the infrastructure defined in a similar way to the application source code. The result is an IaaC blueprint (or artifact) that describes “what this part of the infrastructure should look like”. Simply put, we can say that each part of the infrastructure has a clear blueprint that defines, “This is what my database should look like,” or,“This is what my Redis Cluster looks like.“

Of course, you can argue that each component of each of your applications is different, individual. In that case, IaaC is probably not for you. The most interesting synergistic effect across the entire environment is achieved when you have clearly defined architectural standards (and adhere to them), which you then implement using IaaC.

However, if you have no standard or it is difficult to define one, there may be specific use-cases where IaaC makes sense.

 

 

I’ll be a bit ahead of the curve: this example (simplified) shows how such an IaaC blueprint for a database can look like (in an AWS environment, blueprint defined using AWS CloudFormation):

 

 

Now we won’t go into detail about what each part means, but I wanted to show you “what it really looks like” right from the start.

 

Why automate infrastructure or use cases

Okay, so we know roughly what IaaC is, and we have a rough idea of what it might be good for. Let’s look at specific uses of IaaC.

For those of you who prefer brevity to detailed examples, the summary is roughly as follows:

 

When to consider Infrastructure as a Code

• I have clearly defined infrastructure standards.
• I need to dynamically create individual application environments.
• There are frequent changes in the environment that I want to manage centrally.

 

When for me Infrastructure as a Code is rather not

• I am just getting acquainted with the cloud, I have rather basic experience with running a cloud environment.
• My infrastructure is primarily “static”, without many changes.
• Application infrastructure is highly individual and it is not possible to define a general binding standard.

 

In general, we can say that Infrastructure as a Code is an integral part of cloud automation and automation itself is one of the main motivations for using the cloud.

 

Use case 1: Dynamic environment

This scenario typically applies to development or testing environments. Why should I run a development environment permanently (and pay for it) when development needs it “only when it is deploying and testing something”? Or maybe I have multiple development teams working on the same app. To avoid “clashes” between them, I want to be able to give each team an identical environment (and then delete it again when not needed).

If you define such an environment using IaaC blueprints, you can dynamically deploy the entire environment and delete it when you no longer need it. And it’s all completely automated within your CI/CD pipeline.

 

Use case 2: Infrastructure as an integral part of the application

This scenario supports one important aspect of cloud-native application development, which is parity between environments. Ideally, I want my test environment to be the same as the production environment (maybe smaller in terms of performance), but identical in terms of architecture.

This approach greatly simplifies my search for potential bugs and the overall troubleshooting environment. The infrastructure description is therefore an integral part of the application delivery and I am able to deploy such an application into the target environment immediately.

In a moment we will say that there are different types of blueprints and one of them is blueprints for configuring the landing zone (cloud environment) itself. In this use case, the goal is to have all environments (from a general perspective, not in the context of the application itself) set up exactly the same – to force a standard.

For example, all environments always require a specific tagging policy, all environments have an audit trail configured, and all environments “by default” have the backbone configured in some (identical) way.

 

Use case 3: Automated infrastructure lifecycle

Over time, standards are changed, expanded or otherwise modified. Thanks to IaaC, I can easily make modifications or other changes over all infrastructure and thus manage the entire infrastructure lifecycle. From its creation through all its changes to its final cancellation.

For example, I defined at the beginning that I want to have 14 daily backups for all databases and that each database can be accessed by administrators from a specific network range. Over time, the decision was made that it was necessary from a business perspective to maintain 30 backups, and at the same time the organisation expanded to include another branch office whose staff also needed access to all the databases.

I can easily modify my original blueprint for the database, incorporate these requirements into it and then automate the modification of all databases created based on this blueprint. Similarly, I can, for example, remove all components created on the basis of this bluepint in bulk at any time.

 

Blueprints everywhere you look

As I indicated a moment ago, there are basically three basic categories of IaaC blueprints:

• Infrastructure
• Configuration environment (landing zone)
• Security

 

Blueprints for application infrastructure

These blueprints are then further divided into:

• Generic
• Application-specific

Under generic application blueprints we can imagine, for example, the aforementioned database. It is represented by a single IaaC blueprint and all applications use only it. Another example could be the deployment of a containerization infrastructure or a virtual server.

Application-specific blueprints are part of one specific application and are not used by another application. This can be, for example, application parameters stored in the AWS System Manager Parameter Store or Azure AppConfig) or specific application secrets in AWS Secret Manager or Azure Vault.

 

Blueprints for environment configuration (landing zone)

I mentioned these blueprints a while ago. They are designed to configure the entire cloud environment and are always deployed, regardless of what application is running within the environment.

These include, for example, components such as Budgets (so that each environment has some mechanism for controlling and reporting costs), backbone network configuration (for example, standard routing tables for my Hub network, as Jakub Procházka describes in his article We untangle cloud networks) or it can be for example some serverless function responsible for forwarding security logs to a central SIEM environment.

 

Safety blueprints

This is a specific category of blueprints that are also deployed in every environment but are the responsibility of the security department. The purpose of security blueprints is to define security policies that control or directly enforce that my security requirements are met (typically using components like AWS Config or Azure Policy).

Examples include compliance with the tagging policy, whether all components are really encrypted, whether specific parameters of individual components are set in a particular way, etc.

You could argue that security blueprints are unnecessary because all parameters are set “as they should be” thanks to application blueprints. This is true, but only when such a blueprint is deployed. Over time, individual cloud resources may be modified (human error, unconscious change, etc.). The purpose of security blueprints is therefore to continuously monitor the entire environment. They can therefore be seen as “control blueprints”.

 

How it all plays together

So if you decide (and you should) to implement all three types of IaaC blueprints, the result will look something like this:

 

 

First, individual blueprints for landing zone configurations are deployed to the environment, then security blueprints, and finally individual infrastructure blueprints.

This will achieve a state where:

• each environment (in this case, the AWS environment) is deployed according to a global standard,
• colleagues in the security department know that all requirements for a compliance environment are implemented,
• the application itself is deployed in a clearly defined standard.

 

Life cycle of blueprints

We should certainly not forget that individual blueprints change and evolve over time as your internal architectural or security requirements change. It is therefore important to have a defined procedure for changing the blueprints over time.

We treat them similarly to other program code.

 

 

Typically, the resulting blueprints are stored in a central repository (such as AWS S3 or Azure Storage Account) from which they are deployed. Under no circumstances should this repository allow direct modification of these blueprints!

A typical scenario for controlled and audited modification of a blueprint is that individual versions are stored in some central source-code repository (e.g. GIT). When a new version of the blueprint is changed (commuted), the CI/CD pipeline is started, the first step of which is to validate the changes made (review process).

If the review is successful, the new version of the blueprint is deployed to the test environment, where it is tested that “everything is as it should be”, and then the new version of the blueprint is finally made available in the central blueprint repository.

 

Desired state and why it is so important

Let’s describe what a desired state is and why you should care about it. The desired state approach describes the “target state of an object“, i.e. that my (for example) database should look like “so and so”.

It therefore uses a declarative approach. I declare “as it should be”. Which is a different approach than you may be used to – the imperative approach, where I define “what should be configured how”.

An example of a declarative approach:

• My database should be called MyDb.
• The size of the database should be 20 GB.
• The performance of the database is 2 CPUs and 4 GB RAM.
• The database disk is to be encrypted.

An example of the imperative approach:

• Create a database named MyDb.
• Configure the MyDb database to be 20 GB in size.
• Configure the MyDb database to have 2 CPUs and 4 GB of RAM.
• Encrypt the MyDb database disk.

 

 

The great advantage of declarative notation is that I have the desired state – i.e. the information how it should be. If I have information about the target state, I am able to easily check the current state against the target state at any time.

For example, in the beginning the database was encrypted, but over time someone (for example due to human error) decrypted it. If I know the current and target state, I am able to detect this situation, including detailed information “what is different than it should be”.

 

Example of using desired state

An example of AWS CloudFormation Drift functionality, a technology that allows us to detect deviations against a target state:

 

 

Here, at first glance, I can see that the ManagementSecurityGroup resource has been modified and does not currently match its target state. Based on this information, I have the opportunity to take corrective action if necessary.

 

 

AWS Cloud Formation Drifts is one of the tools that fall under the category of continuous cloud compliance, which is a slightly broader area dedicated to the continuous evaluation of defined rules. This includes a plethora of tools – both native, such as Azure Policy, AWS Config or the aforementioned Cloud Formation Drifts, and external, such as CheckPoint CloudGuard.

The topic of continuous cloud compliance is rather for a separate article – don’t worry, it will be soon!

 

So what tools can I use?

If you’ve read this far, congratulations! So the concept of Infrastructure as a Code has probably intrigued you and now you are probably wondering how to actually implement it.

The first decision you need to make is whether you want to focus on one specific cloud environment or whether you want to manage everything from on-premise across multiple clouds with one tool.

If your goal is to be truly hybrid, our recommendation is to use HashiCorp’sTerraform tool. This tool is extremely powerful, however, from my perspective, its complex deployment can be a relatively complicated task. On the other hand, it will offer you a truly hybrid approach to IaaC across any environment.

 

 

If, on the other hand, you only want to focus on one specific cloud environment, I would recommend using the tools of your chosen cloud provider.

In the Amazon Web Services environment, this is the CloudFormation tool. The astute reader will have noticed that this article uses samples primarily from this tool, which I personally prefer.

 

 

What may be interesting is the ability to use the associated Cloud Development Kit (CDK) tool to create CloudFormation blueprints. The CDK will probably be used by those of you whose cloud environment is mainly dedicated to the development department. CDK allows the creation of CloudFormation blueprints using individual programming languages such as JavaScript, TypeScript, Python, Java, C#, and Go.

If your cloud environment is Azure, you will definitely use Azure Resource Manager (ARM) templates.

 

 

ARM is a native and integral part of Azure. If you’re observant, you’ll have noticed that Azure translates all your operations (performed in the Azure graphical console) related to the creation of any resource into an ARM template, which it then applies.

 

 

If you want to get some inspiration on what these “internal” ARM templates look like, you can try creating any source and download this generated ARM template.

 

Conclusion on Infrastructure as a Code

If you’re serious about cloud automation, Infrastructure as a Code is definitely the way to go in any tool. And frankly – if you’re thinking about how to use the cloud to its full potential in your environment, you should automatically be thinking about automation.

At first glance, Infrastructure as a Code may look complicated, but trust me, once you try deploying a few components into your environment and get familiar with the whole concept of IaaC, you probably won’t want to build your environment “by hand” anymore.

With this article, we bid you farewell for this year, but don’t despair – our Cloud Encyclopedia will continue in 2022.

On behalf of the whole ORBIT I wish you a pleasant Christmas holiday.

 

 

This is a machine translation. Please excuse any possible errors.