Terraform

Today, more and more companies are using the cloud to implement work en­vir­on­ments or even complete IT struc­tures. In­fra­struc­ture as a Service (IaaS) is often the most simple and in­ex­pens­ive way to create a basis for planned projects. Cloud solutions also enable you to react precisely to current re­quire­ments: While the un­der­ly­ing com­pon­ents, such as servers, firewalls or load balancers are static in the provider’s data centre, cloud solutions are not, they can be changed dy­nam­ic­ally in a vir­tu­al­ised cloud en­vir­on­ment – giving customers the ability to increase or decrease resources as needed at any time.

To ensure this flex­ib­il­ity, providers offer APIs that allow the leased IaaS en­vir­on­ment to be scaled at any time with the ap­pro­pri­ate software – freedoms that are at­tract­ive, but also involve a big ad­min­is­trat­ive effort. With the software Terraform, the US company HashiCorp is providing the right answer to minimise this effort in the long run. What this tool for In­fra­struc­ture as Code (IaC) is all about is explained below.

When it comes to un­der­ly­ing in­fra­struc­ture in software de­vel­op­ment, ad­min­is­trat­ors always face a challenge: The tendency towards agile de­vel­op­ment leads to shorter and shorter de­vel­op­ment cycles and an even faster im­ple­ment­a­tion of changes. In order to meet the high demands of this fast pace, IT managers need to automate resource man­age­ment as much as possible. This involves writing machine-readable con­fig­ur­a­tion code, also known as In­fra­struc­ture-as-Code (IaC).

In order to simplify this process, IT managers can fall back on the services of the Terraform software, which on the one hand enables a uniform de­scrip­tion of the target in­fra­struc­ture and on the other ensures that this is im­ple­men­ted to the standards of supported IaaS providers.

Terraform is basically Open Source (Mozilla Public License 2.0) and can therefore be down­loaded and used free of charge. However, several team col­lab­or­a­tion features as well as features for public in­sti­tu­tions and or­gan­isa­tions are only available in the paid editions “En­ter­prise Pro” and “En­ter­prise Premium”. The IaC Service 2012 was developed by San Francisco-based HashiCorp.

Since usually it’s not just one, but many cloud services that are used to implement IT in­fra­struc­ture or the necessary resources for software de­vel­op­ment, you as the re­spons­ible person have to adapt to a number of different in­ter­faces (APIs) by default, as well as fa­mil­i­ar­ise yourself with the nuances of the re­spect­ive platform. This applies re­gard­less of whether you make changes to the in­fra­struc­ture, or whether you want to write your own IaC in­struc­tions for automated access.

As a Terraform user, you won’t have this problem. Instead of the interface languages, you can either use the sim­pli­fied JavaS­cript markup language JSON or the JSON-based HashiCorp Con­fig­ur­a­tion Language (HCL) to describe the desired target in­fra­struc­ture. The latter is the option HashiCorp recommend, as it makes writing and reading code through comments and other features even easier. Terraform works with the re­spect­ive con­fig­ur­a­tion files as ordinary text files, whereas the IaC tool uses the extension .tf. Using an ap­pro­pri­ate tool such as Git or SVN, you can easily version these files if you wish.

The main function of Terraform is the user-friendly con­fig­ur­a­tion of in­fra­struc­ture resources, in which HCL or JSON play a central role. However, lin­guist­ic sim­pli­city is def­in­itely not all that dis­tin­guishes the IaC tool: Created con­fig­ur­a­tion files can only achieve their desired effect with the co­oper­a­tion of all important software and hardware providers with in­di­vidu­ally ex­pand­able basic ar­chi­tec­ture, as well as the easy-to-use command line interface.

The HashiCorp Con­fig­ur­a­tion Language mentioned above is primarily intended to describe the desired result, not the steps to achieve it. The most important element of the Terraform language is resources, because these describe the in­fra­struc­ture com­pon­ents whose man­age­ment is to be automated with Terraform. All other language features exist ex­clus­ively to specify the re­spect­ive resources. The following three basic elements play an important role:

• Blocks: Blocks are con­tain­ers for content. In this function, they provide space for con­fig­ur­a­tions of any objects, es­pe­cially for con­figured resources. Each block has a block type, one or more labels and a body area. The latter contains any number of arguments and sub-blocks.
• Arguments: To assign a value to a name, use arguments in HCL. These are always declared within blocks.
• Ex­pres­sions: Ex­pres­sions represent a value. It can be a concrete value or a reference to one or more other values. An ex­pres­sion occurs either as a value for an argument or within other arguments.

With HCL as a de­clar­at­ive language, the sequence of the in­di­vidu­al blocks is not relevant. Terraform auto­mat­ic­ally processes the described resources in the correct sequence based on pre­defined re­la­tion­ships. Only the so-called pro­vi­sion­er blocks which are needed to execute scripts, play a role.

Terraform provides its own command line interface so that the created con­fig­ur­a­tion files can be evaluated and used. This interface defines the syntax and overall structure of the Terraform language and co­ordin­ates all change steps required to implement the con­figured in­fra­struc­ture. The CLI itself has no knowledge of specific types of in­fra­struc­ture resources, but obtains this in­form­a­tion via special “provider” ex­ten­sions. In this way, the interface knows how to define and manage the in­di­vidu­al resource types at the supported cloud providers – and therefore also how to pass on the uni­ver­sally for­mu­lated in­struc­tions in the Terraform con­fig­ur­a­tion files to the various cloud APIs.

In order to allocate real resources to the con­fig­ur­a­tion, keep track of metadata, and con­tinu­ously improve the per­form­ance of larger in­fra­struc­tures, Terraform stores the status of resources and the IaC con­fig­ur­a­tion created. By default, the software stores the file terraform.tfstate in a local directory – al­tern­at­ively, a different location can be selected, which is par­tic­u­larly re­com­men­ded for team work. In order to compare the status with the current in­fra­struc­ture each time, Terraform auto­mat­ic­ally updates the CLI before each program operation.

If you have created an in­fra­struc­ture by other means, you can import this existing setup thanks to the in­teg­rated function to manage the cor­res­pond­ing resources in the future with Terraform. Currently only the resources, but not their con­fig­ur­a­tion, are added to the current status. The latter, however, should be possible in future versions of the IaC tool.

Terraform allows you to create a de­pend­ency graph of the defined in­fra­struc­ture con­fig­ur­a­tion or execution plan. All you need to do is use the CLI command “terraform graph” and define the desired graph type, and Terraform will auto­mat­ic­ally generate a cor­res­pond­ing output file in DOT format. This can be re­pro­duced using the visu­al­isa­tion software Graphviz, which HashiCorp also re­com­mends in the official online doc­u­ment­a­tion. The command line can also be used to easily convert the file into an image format like SVG.

Terraform is based on a plugin-based ar­chi­tec­ture that makes it possible to extend the func­tion­al­ity of the IaC suite at any time. For example, the com­mis­sion­ers for executing the scripts are among the plugins, as are the provider ex­ten­sions that provide the CLI with all relevant in­form­a­tion about the supported web services. De­velopers who are pro­fi­cient in the pro­gram­ming language Go(lang) and have knowledge of the cor­res­pond­ing provider APIs can even write their own ex­ten­sions to link Terraform with cloud services that are not of­fi­cially supported. Newly released plugins from HashiCorp are auto­mat­ic­ally down­loaded and included, while all third-party ex­ten­sions require you to do this manually.

Thanks to the range of supported cloud service providers which HashiCorp is con­tinu­ously expanding and which can also be in­de­pend­ently expanded with the cor­res­pond­ing know-how, Terraform is suitable for a wide variety of ap­plic­a­tions. Among the most popular use cases of the In­fra­struc­ture-as-Code tool are the following three:

Heroku is one of the best known PaaS providers for hosting your own web apps. All tools and services required for the app can be included in the form of add-ons and monitored via the platform’s own dashboard. Terraform does a lot of the work for you by not just writing the code for your app’s setup, but also ensuring that all necessary add-ons are available at all times.

The layer ar­chi­tec­ture has proven itself as a struc­tur­ing concept for software systems – the two-layer ar­chi­tec­ture with web server and sub­or­din­ate database level is par­tic­u­larly popular. The ability to scale the layers in­de­pend­ently is ad­vant­age­ous for many projects. Terraform proves to be an ideal tool: Each layer can be described as a col­lec­tion of resources. In addition, the IaC software regulates de­pend­en­cies, e.g. in the case of the model mentioned, it ensures that the database level is available before the web server starts.

Software Defined Net­work­ing is becoming more and more important in data centres. Software-con­trolled man­age­ment of vir­tu­al­ised network resources offers ad­min­is­trat­ors and de­velopers a high degree of control and flex­ib­il­ity. Terraform allows you to write a complete con­fig­ur­a­tion for these SDN services and use it to auto­mat­ic­ally set up and customise the software-defined network struc­tures.

For many managers, dis­trib­ut­ing in­fra­struc­ture across multiple clouds is an extremely at­tract­ive model. Dis­trib­ut­ing across multiple providers increases fault tolerance and re­si­li­ence, and makes it easier to recover lost resources than relying on a single provider. Thanks to its universal syntax, Terraform sim­pli­fies the creation of such multi-cloud struc­tures. In addition, the ap­plic­a­tion can manage multiple providers with a single con­fig­ur­a­tion and even manage cross-cloud de­pend­en­cies.

In­fra­struc­ture as Code helps you stay com­pet­it­ive in a con­stantly changing IT en­vir­on­ment. A high level of auto­ma­tion helps you keep track of the resources you need and make the necessary in­fra­struc­ture changes as quickly as possible. Terraform proves to be an efficient tool for both the creation and execution of “pro­gram­mable” in­fra­struc­ture – not least thanks to the support of cloud services like AWS, Azure and Google Cloud. One of the IaC software’s greatest strengths is its uniform syntax for de­scrib­ing the in­fra­struc­ture. However, you always have the choice between the HashiCorp con­fig­ur­a­tion language HCL and the JSON award.

A further advantage of Terraform is the plugin-based structure, which enables both the man­u­fac­turer and third-party providers to extend the func­tion­al­ity of the software at any time. In the case of man­u­fac­turer ex­ten­sions, Terraform even downloads them auto­mat­ic­ally without you having to initiate the update. In addition, the open source solution convinces with features like a storage function, thanks to which you can call up and con­veni­ently adapt in­fra­struc­ture setups once they have been set up at any time, as well as the pos­sib­il­ity of gen­er­at­ing a de­pend­ency graph for all steps of the IaC creation and execution.

In terms of costs, Terraform also cuts a good figure due to the open source approach: The use of the IaC software is basically free of charge. However, if you need col­lab­or­a­tion features such as a link to the version man­age­ment of your choice or the pos­sib­il­ity to manage resources with Terraform via remote access, you are dependent on the re­l­at­ively expensive en­ter­prise plans. Small busi­nesses can quickly reach their financial limits at this point. What’s more, re­gard­less of the edition you choose, you have to be prepared for a demanding, time-consuming start. Once you’ve fa­mil­i­ar­ised yourself with how Terraform works and the HCL language you designed, you can use the tool to make tedious routine work and laborious cross-service resource man­age­ment a thing of the past.