Automating Cloud Infrastructure Provisioning with Terraform
One of the core benefits of the cloud is the ability to get access to resources like servers, databases, etc, when you need them (on demand) and pay for what you use. One of the responsibilities of a DevOps engineer is to manage the provisioning of these resources' lifecycle.
Infrastructure Provisioning can be referred to as the steps required to manage access to data and resources, and make them available for use [1]. Before an application is built there are certain requirements that are known beforehand that determine the type of application, the software stack it should be built on amongst other things, these requirements also determine how the infrastructure would be provisioned i.e server size, database requirements, DNS management, etc.
During development and even after there is a tendency for some of these requirements to be modified, a factor that should also be accommodated while planning. Automating the process makes it easy to manage these changes when they happen while maintaining the integrity of the infrastructure for continuous integration and delivery.
What is Terraform
Terraform is an open-source infrastructure as code (IAC) software tool that provides a consistent CLI workflow to manage hundreds of cloud services. Terraform codifies cloud APIs into declarative configuration files.[2]
With Terraform several cloud resources such as servers, databases, VPCs can be created, updated, and deleted from a single point of entry, making maintenance easy. Terraform requires plugins called providers to interact with cloud providers, SaaS providers, and other APIs. A full list of providers that can be used on Terraform can be found on the Terraform Provider Registry.
Terraform can be downloaded from the terraform website as a binary file and installed on the major operating systems and multiple Linux distributions.
Terraform language:
The terraform language is Terraform’s primary user interface its syntax is declarative in nature and consists of only a few basic elements. It also has a code-like syntax as an Infrastructure as Code tool
<BLOCK TYPE> "<BLOCK LABEL>" "<BLOCK LABEL>" {
# Block body
<IDENTIFIER> = <EXPRESSION> # Argument
}Terraform configuration
A Terraform configuration is a complete document in the Terraform language that tells Terraform how to manage a given collection of infrastructure. A configuration can consist of multiple files and directories.[4]
Terraform files and Directory
Code in the Terraform language is stored in plain text files with the .tf file extension. There is also a JSON-based variant of the language that is named with the .tf.json file extension.
Files containing Terraform code are often called configuration files.
Terraform State
Terraform must store state about your managed infrastructure and configuration. This state is used by Terraform to map real-world resources to your configuration, keep track of metadata, and improve performance for large infrastructures. This state is stored by default in a local file named “terraform.tfstate”, but it can also be stored remotely (ideal in a team environment).
Resources
These are the most important element in the Terraform language. Each resource block describes one or more infrastructure objects, such as compute instances, virtual networks, or higher-level components such as DNS records.
Variables and Outputs
Variables can be used within the Terraform configuration to pass user values around resources and save outputs from created resources.
Example: Simple Infrastructure Setup with Terraform
This section aims to explain the steps involved in automating the setup of an AWS EC2 Instance.
While working with Terraform, the Terraform Provider Registry is a very valuable tool as it contains a full description of how to set up a cloud provider( from this point we would refer to this as “provider”). The Terraform documentation also comes with examples to get you started quickly.
Prerequisites
- Basic Knowledge of Cloud Computing
- An AWS Account. Sign Up here to get one with up to 12months free tier on some resources.
- Terraform Installed on your PC — follow the instructions here to set up your environment
- Visual Studio Code Installed with the Terraform extension(optional)
Once you have successfully completed the steps above the next step is to create a project file locally to work from with a name of your choice. Within this directory create a terraform file — a file with any name of your choice with an extension .tf, for instance main.tf
By the end of this article, we would have defined resources and infrastructure in human-readable, declarative configuration files to do the following: define the required providers, configure the AWS Provider, create the required resources( AWS EC2 Instance, and an AWS Key Pair).
Define the Required Providers
As stated above Terraform connects to the provider API and would need to download some files to enable the connection so in a terraform file it’s custom to first define the required providers to be used. When the file is executed terraform ensures that the provider is available and downloads it if it is not found.
terraform {required_providers { aws = { source = “hashicorp/aws” version = “~> 3.0” } }}
Configure The AWS Provider.
A provider is configured using the provider keyword followed by the name of the provider and a block with some configuration settings. AWS requires authentication credentials to be provided to allow communication with your account resources. An access_key and secret_key which can be generated from your AWS console are required to allow access. These credentials can be set statically within the provider block as seen below ( this method is not recommended as it exposes your credentials ), or by setting them as environmental variables, or as shared credentials file see AWS Terraform Config Documentation for more information.
# Configure the AWS Providerprovider “aws” {region = “us-east-1”access_key = “my-access-key”secret_key = “my-secret-key”}
Create the Necessary Resources
A resource might be a physical or virtual component such as an EC2 instance, or it can be a logical resource such as a Heroku application.[4] A resource is created with the resource keyword followed by the <resource_type> and the <resource_name> Individual resource configuration values can be gotten from the Terraform Provider Registry a search for the resource name would navigate you to its dedicated page, VPC configuration instructions can be found on Setting Up a VPC Resource via Terraform
- Create an AWS key pair: This creates a key pair on AWS to allow SSH access to the console from the origin ( ideally the computer where you require access from) of the public key set as the <public_key>.
resource "aws_key_pair" "deployer-pub-key" { key_name = "deployer-pub-key" public_key = "ssh-rsa xxxxxxxxxxxxx}
2. Create AWS EC2 Instance:
An AWS EC2 instance is a server instance resource that can be created by defining the AMI(Amazon Machine Image that the server should be built off on)server instance_type(AWS server family), availability_zone, key_name(aws_key_pair to be saved on the server at creation to allow remote access) and an optional tags block.
resource "aws_instance" "my-project-webserver" { ami = "ami-00399ec92321828f5" instance_type = "t2.micro" availability_zone = "us-east-2a"
key_name = aws_key_pair.deployer-pub-key.key_name tags = {
Name = "my_app_server" }
}
At this point we can test out Terraform’s connection to the AWS account and that resources are getting created. On the command line/ Terminal navigate to the directory that holds the Terraform configuration file main.tf. A new configuration file needs to be initialized to download and install the providers defined in the configuration file. terraform init is used to initialize a configuration file.
Terraform downloads the AWS provider and installs it in a hidden subdirectory of your current working directory, named .terraform. The terraform init command prints out which version of the provider was installed. Terraform also creates a lock file named .terraform.lock.hcl which specifies the exact provider versions used, so that you can control when you want to update the providers used for your project. [4]
For consistency, the Terraform file can be formatted using the terraform fmt command can be used to update configurations in the current directory for readability and consistency. Configuration can also be validated with the use of the terraform validate command.
Once the configuration file is formatted and validated successfully, a preview of what would be created, modified, destroyed can be gotten by running terraform plan. To create the infrastructure terraform apply command is used.
Before it applies any changes, Terraform prints out the execution plan which describes the actions Terraform will take in order to change your infrastructure to match the configuration similar to the output gotten from terraform plan[4]. Creation is then paused with a prompt for you to approve the changes about to be made. After successful execution, you can confirm the creation of your specified resources by checking on your AWS console.
The public Ip of the AWS server can be passed as an output can be used to establish an SSH connection with the server (if you have port 22 open to incoming traffic on your console)
output "instance_ip" { value = aws_instance.my-project-webserver.public_ip}
Adding more resources to this infrastructure would entail updating the main.tf Terraform configuration file with that resource block, if no changes are made to the configuration of existing resources their state is maintained and only the new resources are created, deleting an existing resource from the configuration file would cause it to be deleted when we run terraform apply.
Conclusion
Terraform, an Infrastructure as a Code tool offers a lot of benefits in maintaining consistency across IT infrastructure. Used together with a version control solution like git it has support for teams to collaborate together on a configuration file in provisioning resources across their infrastructure. It helps save time as the file can be written in minutes and resources created in a short time, the configuration file can also be reused for several projects by creating a copy and making updates unique to the project. It also makes Continuous infrastructure delivery possible.
References and Resources
