Migrating Azure CAF landing zones to Terraform 0.13

More than a year ago we announced at Ignite 2019 that Cloud Adoption Framework proposes an implementation on Terraform. Since then, we have seen a tremendous feedbacks from customers and partners architects.

In the meantime lot of great things have happened: an enterprise-scale reference architecture and it’s ARM-based implementation, alongside with an incredible acceleration on the release rhythm of the Azure Terraform provider reducing the delay between features appearing in Azure and time it is available in native Terraform.

Terraform 0.13

Terraform 0.13 has been released mid August, and you can read the full details here. For us, the major improvements are:

  • Registry-based provider sourcing: No more need to side-load our custom provider, you can leverage it directly from the Terraform registry just like we for modules: our CAF provider is published here
  • Variables validation: Only allow variable to be within a certain range or validating a regex.
  • Iterations and conditions on modules: more later in this article.

Testing Terraform 0.13 with rover

After weeks of testing reliability of Terraform 0.13 during this summer’s beta, we were very happy with the stability and performance so we decided to push further our way of operating.

rover is part of the DevOps toolset we provide you with and recommend using for development teams and pipelines deployment. During this testing stage, it played full role allowing us to switch fast from one codebase to the other and also to one version of Terraform to the other.

What’s new in CAF landing zones

Leveraging Terraform 0.13, we were able to introduce new concepts in landing zones on Azure:

One module to rule them all

We have been curating 20+ modules during the last year, all published on the Terraform registry and some of them being consumed more than 26,000 times.Screenshot 2020-10-06 at 11.12.02 AM

While it was important to have different modules to maintain a very fine-grained versioning of the components we use, we found it to be a lot of work, for little added value.

We then decided to explore a model where we use only one module, that would call and iterate on different submodules – that permitted by Terraform 0.13.

We must admit we have been impressed by the results on how the graph performs, and the speed at which it works.

So the new module is still in preview but already available available here https://registry.terraform.io/modules/aztfmod/caf/azurerm/latest

Iteration-based objects deployment

To deploy core features, landing zones are now calling one module. This master module has all the major components we need to deploy within landing zones, and it iterates on every objects in the following way:

Screenshot 2020-10-06 at 11.05.16 AM

So from the configuration prospective, you define your network configuration with a complex object like this one:

Screenshot 2020-10-06 at 11.08.44 AM

So wether you need one or ten virtual networks, we iterate automatically for you.
By default the vnets like all the other variables are set to {}. So it means nothing will be created, only the variables you populate will be iterated on.

Key-based configuration and customization

All configuration objects will call each other based on the object keys. As we just seen, we use complex objects that are iterated and referenced via their keys.

It means for instance we create all the resource groups using the following object:

Screenshot 2020-10-06 at 11.14.03 AM

In that example, we create two resource groups which keys are vnet_region1 and vnet_region2. We we create the virtual network called hub (and which key is hub_sg we reference the resource group via the resource_group_key property.

Under the hood, the module is mapping back the resource group to the key with the following logic:
referencetokey

No-code environment composition

Overall this makes writing the modules code slightly more complicated, but what it means for users is the possibility to compose complex environment without changing anything in the code.

A good example for that is the networking landing zone where we are able to compose all kind of Azure network topologies (like Hub-and-Spokes virtual networks, Virtual WAN-based hub-and-spokes, etc.).

We can establish peering between on object in the current landing zone and a remote landing zones (this is very common to have separate states for the hub and the spokes)

Screenshot 2020-10-06 at 11.26.12 AM

Overall we have tested it internally and externally and have found:

  • We have been able to compose any kind of architecture using this pattern.
  • It is very fast to compose a new environment.
  • It is very fast to process by Terraform and Terraform graph.

DevOps from the ground-up

At the root of deployments, we have our launchpad which sets the Azure backend and collaboration requirements.
launchpad-200

We want to offer our customers choices and options so the new launchpad is modular: allowing add-ons to complement the configuration like:

For an example of landing zones on Azure DevOps, you can refer to the following demo:
For an example of landing zones on Terraform Cloud, you can refer to the following demo:

Coding everywhere

Using Visual Studio Codespaces or GitHub Codespaces, you can develop and deploy landing zones without installing anything on your laptop, with full access to the rover and development environment.
Screenshot 2020-10-02 at 5.02.46 PM

You can try it now: VScodespaces or on GitHub Codespaces

The journey ahead

Enterprise-scale landing zones

Enterprise-scale is a reference architecture, set of design guidelines, and reference implementation for an Azure enterprise environment.
You can find all the details here.ns-vwan.

We are working on it with the team and are integrating the module on the foundations layers. You can also try out the module here.

Delivering solutions

We found a lot of interest from customers and partners to leverage the DevOps and everything-as-code environment we setup in order to accelerate the development and the deployment of ready-to-use solutions.

We are currently working to publish solutions that you can stack-up on top of the lower levels of landing zones, amongst them, you will be able to find:
* AKS topologies including layer 4 applications
* Data and analytics, to deploy Azure Synapse, Databricks, Machine Learning, etc..
* More to come…

On the way to Terraform 0.14

Just announced, Terraform 0.14 will also bring some good features, you will be able to test it out soon in a new rover. You can read the news here

Contribute

All of that is still work in progress and you can contribute by opening an issue, a pull request or just testing it!

The repo is here: https://github.com/Azure/caf-terraform-landingzones and (most) of the things described are currently in the 0.4 branch, and will be available on this month release (2010 Update)

We’d love to hear from you, so don’t hesitate to give us feedback here or on Gitter

Arnaud

Understanding landing zones for Azure Cloud Adoption Framework – Terraform edition

Microsoft Cloud Adoption Framework (http://aka.ms/caf) is the one Microsoft guidance to adopt Azure in entreprise. One of the fastest way to put the framework in action and deploy an environment fast and good, is to use CAF landing zones for
Terraform.

CAF landing zones are an open source project living on GitHub and we welcome any contribution: https://aka.ms/tf-landingzones

In this article, you will find a set of videos and articles that should help you understanding the landing zones architectures and how to start crafting your own.

Cloud Adoption Framework: Introduction landing zones for Terraform


Cloud Adoption Framework: Deep-dive on landing zones for Terraform


First Deployment:


Setup Environment:

Cloud Adoption Framework landing zones with Terraform

Official Cloud Adoption Framework landing zones:

https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/expanded-scope/terraform-landing-zone

If you are getting started with Terraform on Azure:

Operating governance on cloud native technologies:

Cloud Adoption Framework landing zones with Terraform

At Microsoft Ignite 2019, we announced that Cloud Adoption Framework for Azure now has a Terraform edition of it’s landing zones. We see landing zones as a great way to enable value quicker on Azure, and as we see tremendous excitment and growth in this area, we are evolving the experience to make it simpler and more powerful.

Adopting CAF landing zones with Terraform, customer consistenly realize they can:

  • deploy foundations of cloud environments fast
  • easily deploy governance and policies as code (with ongoing compliance enforcement and remediation)
  • have consistent ways to deploy innovation (infra as code, policies as code or application code are using the same mechanisms).
  • and many more.

This overall enables them to: optimize their operations, innovate faster and cheaper, but we will come back to that in another article.

For now, let’s get you started with landing zones development and deployment!

 

Setting up development environment

When deploying Infra-As-Code environments, we found it extremely important to have:

  • strong versioning (accross dev environment, DevOps pipelines, etc.).
  • short feedback loops: ability to innovate fast on your development, test new code without having to go through the whole pull-request and so mechanisms.
  • accomodate diversity of environments (running Windows, Mac, Linux).
  • strong identity consistency and no secret in any code.

In order to enable that, we use Docker containers as development (on your laptop) and deployment (CI/CD pipelines) environement. The toolset is packaged inside what we called the rover and in order to get started – whichever your platform is, just:

Once its done, make sure Docker is up and running, then Open Visual Studio Code:

We need to install “Remote Development” extension, so click the Extensions icon and then find and install “Remote Development” from Microsoft:

Screenshot 2019-11-15 at 10.10.05 AM.png

To make your life easier, we are going to leverage Visual Studio Development Containers, which allows you to seamlessly work on your machine and execute your code in the rover container.:

68747470733a2f2f636f64652e76697375616c73747564696f2e636f6d2f6173736574732f646f63732f72656d6f74652f636f6e7461696e6572732f6172636869746563747572652d636f6e7461696e6572732e706e67

So on your machine, you will have your Terraform files, with your Git connection and other tools, but when you run the code, its seemlessly integrated with the Rover running in the container, so you can develop, test, run the complete environment.

 

Deploying your first landing zone

Once you have deployed the prerequisites, you can just clone (with Git, or GitHub Desktop) from our repository: http://aka.ms/tf-landingzones and open it in Visual Studio.

If you look at the files, it’s classic Terraform scripts except for the  .devcontainer folder where we have two files:

  • devcontainer.json: describes how to connect to the container, the mounting points and extensions.
  • docker-compose.yml: describes the container configuration needed, especially where to get the rover image (tip: we put it on the Docker registry so you dont have to manage it)

Screenshot 2019-11-15 at 10.18.53 AM.png

You can then go to the bottom left, green part of Visual Studio Code and select Screenshot 2019-11-15 at 10.21.51 AM.png

Open Remote-Containers: Open Workspace in Container, then select the default workspace we created for you (workspace.code-workspace). You will see a new window opening and you can see the details of the Docker container operations:

  • downloading the container from registry
  • mounting the volumes
  • creating the network
  • installing VSCode drivers in the containers

Screenshot 2019-11-15 at 10.24.33 AM.png

You are now ready to start hacking, you can start by launching:

  1. rover login 
  2. rover /tf/caf/landingzones/landingzone_caf_foundations plan 
  3. rover /tf/caf/landingzones/landingzone_caf_foundations apply Screenshot 2019-11-15 at 10.29.00 AM

So let the fun begin!

 

Demo

TLDR; Here is a video with the different steps I described:

 

Happy hacking, let me know your feedbacks and lets connect!

@arnaudlheureux

Azure Sentinel – on the fast track

Azure Sentinel is Microsoft cloud-native SIEM and SOAR. Say goodbye to 6 months SIEM solution setup and architecture – get started with visibility on you environement just now, and use the rich ecosystem of connectors to extend intelligence to your complete security suite.

On-demand session

Here is the link to attend the full session, on-demand: https://info.microsoft.com/AP-AzureSec-WBNR-FY20-10Oct-24-AzureSentinel-SRDEM10204_LP02OnDemandRegistration-ForminBody.html 

Slides

Demos

Onboarding Sentinel and dahsboards exploration

Security Investigations with Azure Sentinel

Additionnal resources:

As usual, happy to hear your feedbacks!

Arnaud

Azure DDoS Protection Standard

Azure DDoS Protection Standard is a service that enables AI-based protection of your cloud applications. Below is the list of resources of our webinar aired on 19th September 2019.

As a quick reminder, below is the set of network protections in front of your Azure instances. DDoS protection exists in a free tier which is present by default on all resources you publish using a public IP address. In this session we discuss about the addition of the “Standard” edition of the Azure DDos protection.

azuresecurityarchitecture

On demand session

https://info.microsoft.com/AP-AzureINFRA-WBNR-FY20-09Sep-19-AzureDDoSProtectionStandard-SRDEM4832_LP01Registration-ForminBody.html \

Slides

Demo

Additionnal links:

As usual, feel free to send your feedbacks and suggestions!

Arnaud