Infrastructure as Code: Serverless deployment through the Terraform template module
Before cloud computing, virtualization, DevOps, and agile practices became part of the IT lexicon, the management of IT infrastructure was completely manual. System administrators had to physically set up servers on racks, install operating systems, and connect and configure networks. Only then would applications be ready and accessible.
This type of manual infrastructure management had a number of shortcomings. First and foremost, the cost factor; organizations had to employ many IT professionals to execute different tasks at every stage of the process, right from hardware engineers, network engineers to hardware maintenance technicians.
Secondly, scalability and availability of applications. As the entire process was manual and time-consuming, system administrators would have to run helter-skelter to set up additional servers, in case of spikes in application access. Also, if the organization did not have the necessary backup servers in place, application unavailability or downtime was inevitable.
Last but not the least, the large number of technicians involved in the manual deployment of different configurations would lead to serious technical discrepancies.
However, today things have changed drastically and for the better. The rapid development of cloud computing, along with the emergence of infrastructure as code, has considerably reduced the software development cycles. This has led to the need for faster and improved infrastructure management capabilities. Organizations have to be nimble and quickly scale applications, without much risk and at a diminished cost.
Making the complex simple
Infrastructure as Code (IaC) has helped organizations to elevate their infrastructure management and the time taken to deploy applications. Infrastructure as Code refers to the management of IT infrastructure through the use of configuration files. By using a blend of tools, processes, protocols, and languages, IaC is capable of securely building and configuring various infrastructure components in record time, deploying similar infrastructure universally, bringing DevOps like agility to infrastructure management, ability to create reusable modules, and much more. The four primary IaC methods used by developers are:
- Scripting: This is the most direct method of implementing IaC. Scripts are ideal for carrying out simple, precise, one-time tasks.
- Configuration management tools: These are specialized tools designed to manage software applications. These tools typically focus on the installation and configuration of servers. CFEngine, Puppet, and Ansible are examples of configuration management tools.
- Provisioning tools: These tools are used by developers to create specific infrastructure components. For example, AWS CloudFormation, Terraform, and OpenStack Heat.
- Containers and templating tools: These tools are used by developers to generate templates that have all the libraries and components needed to run a software application. It is considerably easier to distribute containerized workloads, when compared to running a full-fledged server. For example, Docker, Kubernetes, and Google Container Engine.
Zeroing in on the best approach
The specific needs of a project should determine the right approach to IaC. Typically, we implement the Jenkins pipeline for deployment, but Jenkin pipelines have their own set of constraints. However, for one of our customer service automation projects, we decided to take a different route and dive right into uncharted waters. We zeroed in on Terraform for deployment.
Out of our comfort zone, we did what any other competent dev team would do; we got our hands dirty with Terraform. At first, it seemed like a daunting task, as our team consists of only development and test engineers and Terraform was like Greek and Latin to us. As we proceeded with Terraform, we discovered that it offered a number of advantages, when compared to Jenkins. Some of the advantages include:
- Orchestratization: Terraform focuses more on server provisioning, than just server configuration. When the entire cloud infrastructure is considered as code and all parameters are merged in a configuration file, team members can collaborate on them with ease, as with any other code.
- Cloud agnostic: Terraform is very versatile, as it supports multiple cloud platforms, such as AWS, OpenStack, and Azure, along with on-premises deployments.
- Extensive portability: With Terraform, only one tool and language are needed to define the infrastructure for multiple cloud providers.
We started with the fundamentals of Terraform. While going through such resources, the first couple of points in the Modules Overview section stood out and actually was a cause of alarm for us (due to the simple fact that we had just started learning about Terraform). The points were:
- As we manage the infrastructure we will create increasingly complex configurations
- No intrinsic limit to the complexity of a single Terraform configuration file or directory
As alarming as the above points may sound, the first and foremost question in our collective minds was ‘what do we do to manage such complexities in our project?’ To a certain extent, our fears were allayed with the points mentioned here. But things were made much clearer with Terraform’s Standard Module Structure. We uncovered a few out-of-the-box features like:
- Execution Plans
- Resource Graphs
- Change Automation
Post learning curve?
This is how we proceeded:
- Terraform Google Cloud Provider? Check
- Cloud Function resource? Check
- First, we created a Service Account[SA] in GCP that could be used for deployments.
- We added the required policy bindings for the newly created SA to deploy and access storage buckets (where the source archive of cloud functions are present).
- Structured Terraform scripts in Standard Module Structure.
- Ensured that the dependencies are mentioned appropriately, as the deployment of one cloud function relies on the other to a small extent.
- Finally test and demo the scripts.
PS: We used the Jenkin pipeline to publish the source archives, after the necessary preliminary stages.
All set up…..Good to go? Not yet
There were a few niggling doubts, such as where is the documentation for the written modules? How can we use these scripts?, and so on. At this stage, we already had a large amount of variables for each and every cloud function that had to be deployed and they could be deployed or used independently. Quite the herculean task, isn’t it? Well not really. We were able to incorporate all of the Terraform features, such as generate docs, format code, lint code, and much more.
The most important part of the story? All of the above learning and experimentation in less than a week’s time. We now use Terraform with ease. The true outcome of this journey? Nothing ventured, nothing gained. IaC is a crucial cog in the DevOps wheel. It takes cloud computing to the next level and can help you exploit its potential to the fullest.