Terraform for Multi-Cloud, Multi-Environment Development

Terraform is a fantastic infrastructure as code tool that is cloud agnostic. Cloud-agnostic means that much of the same terraform scripts can be run on AWS cloud as well as Azure cloud. It is very powerful in that you can setup and tear down environments in a few minutes. There are also strategies that deploy an environment based on a branch in source control. This is the holy grail for many developers and a CI/CD pipeline as it helps eliminate the environmental factors which plague development and take time away from producing quality software.

A challenge when dealing with multi-environments is the situation of multi-environments in a single AWS or Azure cloud account. There are many ways of handling this, and one is with different Terraform state files for each environment. This is ideal as it segregates terraform deployments. Another way to have different environments in the same cloud account is to utilize something called Terraform workspaces. Simply changing the workspace based on the environment gives you another avenue for tackling the many environments in a single account issue.

For large scale projects, however, terraform advises you against using workspaces. It is better to use separate state files. From the terraform website:

“In particular, organizations commonly want to create a strong separation between multiple deployments of the same infrastructure serving different development stages (e.g. staging vs. production) or different internal teams. In this case, the backend used for each deployment often belongs to that deployment, with different credentials and access controls. Named workspaces are not a suitable isolation mechanism for this scenario.”

Utilizing python to generate separate terraform state files during and Azure DevOps deployment is ultimately what my team adopted.