Infrastructure Provisioners

Updated 1 month ago by Michael Cretzman

Infrastructure Provisioners define blueprints from known Infrastructure-as-Code technologies (Terraform, Cloud Formation) and map the output (such as load balancers, VPCs, etc). They enable Workflows to provision infrastructure on the fly when deploying Services.

When defining the entities of your Application, there is an assumption that you already have an infrastructure in place where you want to deploy your services. In some cases, you will want to use an Infrastructure Provisioner to define this infrastructure on the fly.

Add an Infrastructure Provisioner such as CloudFormation or Terraform as a blueprint for the system, networking, and security infrastructure for the service(s) you deploy. Define components such as load balancer, VPC, subnets, firewall rules, security groups and so on. When Harness deploys your microservice, it will build your infrastructure according to this blueprint.

After you set up an Infrastructure Provisioner in your Harness Application, you can add it as a Pre-Deployment step in a Workflow or as a Service Infrastructure in an Environment. Once configured, your Workflow can deploy multiple Services, each using the same Infrastructure Provisioner as a blueprint, and passing in Service-specific input variables to create Service-specific deployments using the single Infrastructure Provisioner.

You can create any resource using Terraform and output values can be published for use in Harness Workflow and Pipelines. Harness supports first class Service Mapping for AWS-based infrastructures (SSH, ASG, ECS, Lambda) and Google Kubernetes (GKE). Google Kubernetes is not supported for CloudFormation as CloudFormation is for provisioning infrastructure on AWS only.

See also Application Checklist, Add a Service, Add an Environment, Add a Workflow.

Provisioning Process with Harness Variables

Infrastructure provisioner templates include inputs and outputs that Harness uses for provisioning:

  • Inputs - You can write values from Harness to template inputs to be used at runtime (such as access keys).
  • Outputs - You can map template outputs (VPCs, regions, subnets, etc) to a Harness service to be used when the service is deployed.

The process is as follows:

  1. You set up the services and the workflows that will use your infrastructure provisioner. You configure variables that they will pass to the infrastructure provisioner as inputs.
  2. You configure your infrastructure provisioner and define the variables it will receive from the service and workflows as inputs.
  3. When Harness runs the infrastructure provisioner (Terraform or CloudFormation) to provision, the variables in the services and workflows are passed to the infrastructure provisioner for use as template inputs.
  4. Once provisioning has occurred, the outputs from the infrastructure provisioner (VPCs, regions, subnets, etc) are used to deploy the service (they are mapped to the service).

Intended Audience

  • DevOps

Before You Begin

Add an Infrastructure Provisioner

To add an infrastructure provisioner, do the following:

  1. Click Setup.
  2. Click the application where you want to add an infrastructure provisioner.
  3. Click Infrastructure Provisioners.
    The Infrastructure Provisioners page appears.
  4. Click Add Infrastructure Provisioner, and select Terraform or CloudFormation.
    Below are the options for each infrastructure provisioner type.

CloudFormation

You can link to or add CloudFormation templates to the CloudFormation Infrastructure Provisioner. For information on CloudFormation templates, see AWS CloudFormation Templates. Another good resource for AWS CloudFormation templates is AWS Quick Starts.

The CloudFormation Provisioner dialog has several fields where you enter the source type and template.

The Add CloudFormation Provisioner dialog has the following fields.

Field

Description

Name

Enter a name for this infrastructure that describes when it should be used, such as QA Infrastructure.

Description

Enter a description for this infrastructure that tells users the purpose of this infrastructure.

Source Type

Select Amazon S3 or Template Body. If you select Amazon S3, you can enter the URL to the S3 bucket and filename for the template. If you select Template Body, you can paste in the CloudFormation template. For examples of CloudFormation templates, see AWS CloudFormation Templates.

Template File Path

URL to the S3 bucket and filename for the template in that bucket.

Input Variables

See Input Variables in Workflows below.

Terraform

Terraform can use a single template file or multiple files and folders called modules. For information on how Terraform creates a reusable infrastructure with modules, see the articles How to create reusable infrastructure with Terraform modules by Yevgeniy Brikman and Creating Modules from Terraform.

The Terraform Provisioner dialog has several fields where you set the Git repo and root directory.

The Add Terraform Provisioner dialog has the following fields.

Field

Description

Name

Enter a name for this infrastructure that describes when it should be used, such as QA Infrastructure.

Description

Enter a description for this infrastructure that tells users the purpose of this infrastructure.

Git Repository

The repo where the Terraform module for this infrastructure is located. This is a repo you have set up in your Account under Connectors, and then SourceRepo Providers.

Git Repository Branch

The repo branch where your Terraform configuration files are located.

Terraform Configuration Root Directory

Modules in Terraform are folders with Terraform files (any set of Terraform templates is a reusable module). The directory holding the Terraform files you're applying comprise what is called the root module. It will contain a file named main.tf. If it is in a folder named main, you enter main in Terraform Configuration Root Directory. If it is in the root directory, you enter a period (.).

Variables

See Input Variables in Workflows below.

Backend Configuration (Remote state)

See Terraform Remote State below.

Terraform Execution and the Harness Delegate

Terraform must be installed on a Harness Delegate in order for the Terraform Provisioner script to be executed. Terraform can be installed on all Delegates, one Delegate, or as many as necessary for the deployment workload. Terraform validation ensures dispatching of the execution at the right node.

The simplest method for installing Terraform on a Delegate is to use a Delegate Profile. Delegate Profiles are explained in detail in Delegate Profiles, but here is an example of a Delegate Profile that installs Terraform:

Here is the script:

curl -O -L https://releases.hashicorp.com/terraform/0.11.13/terraform_0.11.13_linux_amd64.zip
unzip terraform_0.11.11_linux_amd64.zip
sudo mv terraform /usr/local/bin/
terraform --version

Here is the Delegate Profile using the script:

Input Variables in Workflows

When you set up your infrastructure provisioner you can set input variables that will be used when workflows deploy a service to that infrastructure provisioner. The infrastructure provisioner will receive these inputs from Harness at runtime. These input variables are typically access keys.

For example, Terraform inputs in a template are variables defined just like any other variables.

variable "key" {
type = "string"
}

Input Variable Example

Use the Input Variables section in the Infrastructure Provisioner dialog to add the input variables that are provided at runtime to execute the Terraform plan. A very common example is to pass in the access keys or secret keys.

If you select the Encrypted Text type for a variable, when you use that variable in a Workflow (for example, as part of a Terraform Provision pre-deployment step), you will use an Encrypted Text entry from the Harness vault to supply a value for that variable. Consequently, you must add the encrypted text value for the variable to Harness Secrets Management. You can add and see encrypted text entries in Harness Secrets Management, under Encrypted Text. For more information, see Secrets Management.

For more information about Terraform input variables, see Input Variable Configuration from Terraform. For information about AWS input variables (called parameters), see Template Anatomy from Amazon.

When you create a Workflow, you can select a Terraform Provision or CloudFormation Create Stack step in the Pre-Deployment Steps. These steps are available in a Canary workflow.

In the step, you select the infrastructure provisioner you added, and the input variables are displayed.

Now, for each input variable, click in Value, and provide the input values for the variables that will be passed to the Infrastructure Provisioner. For an Encrypted Text value, select a secret from Harness Secrets Management. For information on using secrets with Harness, see Secret Management.

Terraform Template Example

To experiment with input variables, here is a section from a Terraform main.tf template you can use. Simply replace some of the values with your own infrastructure information.

variable "global_access_key" {}
variable "global_secret_key" {}


provider "aws" {
region = "ap-southeast-2"
access_key = "${var.global_access_key}"
secret_key = "${var.global_secret_key}"
}

resource "aws_instance" "example" {
ami = "ami-0c9d48b5db609ad6e"
instance_type = "t2.micro"

user_data = <<-EOF
#!/bin/bash
echo "Hello, World" > index.html
nohup busybox httpd -f -p 8080 &
EOF
tags {
Name = "terraform-example"
}
}

Terraform Remote State

In the default Terraform configuration, Terraform stores state in a file in the current working directory where Terraform is run. But when using Terraform in a team, everyone must be working with the same state so that operations will be applied to the same remote objects.

Remote state solves this problem with a fully-featured state backend. Terraform uses remote locking as a measure to avoid two or more different users accidentally running Terraform at the same time, and thus ensure that each Terraform run begins with the most recent updated state.

Instead of having stage or copy the state for all teams, where it could go out of sync, you can use remote state so that all teams are working with the same state.

You can map the variables used by the remote_state command in your configuration file into the Backend Configuration (Remote state) section of the Terraform Provisioner dialog.

Terraform remote state writes the state data to a remote data store that can be shared between all members of a team. This allows you to have each team store variables in Variables section of the Terraform Provisioner dialog, and then enter the variables needed for the shared, remote state in Backend Configuration.

For example when you set up remote state in your Terraform main.tf file, it will look something like this:

data "terraform_remote_state" "network" {
backend = "s3"
config {
bucket = "terraform-state-dev"
key = "network/terraform.tfstate"
region = "us-east-1"
access_key = “${var.aws_access_key}”
secret_key = “${var.aws_secret_key}”
}
}

In the Terraform Provisioner dialog, the variables are added in the Backend Configuration (Remote state) section.

If there are matching variable names in Harness and the Terraform backend configuration, the variable used in Harness will overwrite the variable in the backend configuration. For example, if the backend configuration has a variable named bucket and you add a variable named bucket in Harness Backend Configuration, the value given the bucket in Harness will overwrite the bucket variable in the backend configuration file.

Terraform Workflow Remote State Variables

In your Terraform Provision workflow step, you can select the Terraform Infrastructure Provisioner you created, and the Backend Configuration variables are displayed.

Then, you can enter values for the Backend Configuration remote state variables using text or encrypted text from your Harness secrets, configured following the steps in Encrypted Text in Secrets Management.

Service Mappings

Service Mappings enable Harness to map service-specific values from multiple services to a single Infrastructure Provisioner. Service Mappings are optional.

You can create any resource using Terraform and output values can be published for use in Harness Workflow and Pipelines. Harness supports first class Service Mapping for AWS-based infrastructures (SSH, ASG, ECS, Lambda) and Google Kubernetes (GKE). Google Kubernetes is not supported for CloudFormation as CloudFormation is for provisioning infrastructure on AWS only.

Most infrastructure resources have attributes associated with them, and outputs are a way to easily extract and query that information. For example, in a Terraform template, outputs specify values that will be provided to the user when Terraform applies the template, such as region, VPCs, and subnets.

If you have been running your deployments manually, you might not have outputs configured in your template files. To configure Service Mappings, you will need to add these output variables to your template.

When you use Terraform or CloudFormation with Harness, you can map the template outputs with Harness service settings so that a single Infrastructure Provisioner can be used by many of the services you have configured in Harness.

For example, your Terraform main.tf file contains an output for the AWS region:

output "region" { 
value = "${var_region}"
}

CloudFormation uses a similar outputs format:

"Outputs" : {
"Logical ID" : {
"Description" : "Information about the value",
"Value" : "Value to return",
"Export" : {
"Name" : "Value to export"
}
}
}

You can map the region variable to the Harness services that use this Terraform or CloudFormation template. When the services are deployed, they will use the mapped region.

Mapping is set up in the Infrastructure Provisioning Service Mapping dialog.

To setup service mapping, do the following:

  1. In the Terraform or CloudFormation infrastructure provisioner you configured, click Add Service Mapping. The Service Mapping dialog appears. Here you will specify a service to map to this infrastructure provisioner.

Once you select a service, you can map the Terraform template outputs using this syntax:

${terrafrom.exact_name}

You can map the CloudFormation outputs using this syntax:

${cloudformation.exact_name}

Here is an example of a mapping:

The Service Mapping dialog has the following fields.

Field

Description

Service

Select the service that you want to use this infrastructure provisioner.

Deployment Type

Select the deployment type for the service.

Cloud Provider Type

Select the Cloud Provider type for the deployment environment you will use.

Region

Enter the output variable name for the region output. If you do not have a region output in your template, please configure one.

VPCs

Enter the output variable name for the VPC output. If you do not have a VPC output in your template, please configure one.

Subnets

Enter the output variable name for the subnet output. If you do not have a subnet output in your template, please configure one.

Security Groups

Enter the output variable name for the security group output. If you do not have a security group output in your template, please configure one.

Tags

Enter the output variable name for the output tags. If you do not have a resource tag output in your template, please configure one. Tag variable type must be map.

For more information on Terraform outputs, see Output Variables from Terraform, and Introduction to AWS With Terraform from Medium. For more information on CloudFormation outputs, see Outputs and Amazon.

As an alternative to using variables, you can simply use constants. For example, you could tag nodes with foo and then you would not need to propagate this value through variables.

Set Up with YAML

You can set up an infrastructure provisioner quickly using the Harness code editor.

For information about syncing the Harness code editor with Git, see Configuration as Code.

To set up an infrastructure provisioner using code, do the following:

  1. In the Git repo for Harness, open your application folder.
  2. Inside your application folder, add a new folder named Provisioners.
  3. Add a YAML file with the name that describes your provisioner, like DEV, and save it.
  4. Open the YAML file and define your provisioner. Below are examples for CloudFormation and Terraform. Once you have added the YAML, sync your Git repo with Harness via Configuration As Code. The code editor appears and displays your provisioner.

CloudFormation Example

harnessApiVersion: '1.0'
type: CLOUD_FORMATION
mappingBlueprints:
- cloudProviderType: AWS
deploymentType: SSH
nodeFilteringType: AWS_INSTANCE_FILTER
properties:
- name: region
- name: region
- name: securityGroups
value: ${cloudformation.security_group}
- name: subnets
value: ${cloudformation.subnet}
- name: tags
value: ${cloudformation.aws_ssh_tags}
- name: vpcs
value: ${cloudformation.vpc}
serviceName: TAR-file
name: DEV-CF
sourceType: TEMPLATE_URL
templateFilePath: https://s3.amazon.aws.com/path
variables:
- name: access key
valueType: ENCRYPTED_TEXT
- name: secret key
valueType: ENCRYPTED_TEXT

Terraform Example

harnessApiVersion: '1.0'
type: TERRAFORM
description: module for front-end deployment infra
mappingBlueprints:
- cloudProviderType: AWS
deploymentType: SSH
nodeFilteringType: AWS_INSTANCE_FILTER
properties:
- name: region
value: ${terraform.region}
- name: securityGroups
value: ${terraform.security_group}
- name: subnets
value: ${terraform.Subnet}
- name: tags
value: ${terraform.aws_ssh_tags}
- name: vpcs
value: ${terraform.VPCs}
serviceName: TAR-file
name: front-end
path: terraform-CV
sourceRepoSettingName: git
variables:
- name: access key
valueType: ENCRYPTED_TEXT
- name: secret key
valueType: ENCRYPTED_TEXT


How did we do?