CloudFormation Provisioner

Harness has first-class support for AWS CloudFormation as an infrastructure provisioner. This document contains the following information on using the Harness CloudFormation Infrastructure Provisioner to provision infrastructure on-the-fly as you deploy your applications using Harness:

• Overview
• Permissions
• Delegate and Cloud Provider Setup
• Artifact Server Setup
• Application and Service Setup
• CloudFormation Provisioner Setup
  • Service Mappings
    • SSH
    • ECS
    • AMI
    • Lambda
• Environment Setup
  • Service Infrastructure
• Workflow Setup
  • Workflow Creation
  • CloudFormation Provisioner Step
    • Input Values
• Troubleshooting

Overview

When creating your Harness Application, there is an assumption that you already have an infrastructure in place where you want to deploy your services. In some cases, you will want to use an Infrastructure Provisioner to define this infrastructure on the fly.

You use a CloudFormation Infrastructure Provisioner in the following ways:

• CloudFormation Infrastructure Provisioner - Add a CloudFormation Provisioner as a blueprint for the infrastructure where you will deploy your application. You add the provisioner template by pasting it in or by connecting to an AWS S3 bucket where the CloudFormation templates are kept. You simply need to map some of the output variables in the template to the required fields in Harness. When Harness deploys your microservice, it will build your infrastructure according to this blueprint.

  

• Service Infrastructure - Select the CloudFormation Provisioner you created as the Service Infrastructure of a Harness Environment. The provisioned environment is now the deployment target environment for the Workflow. You can use this Service Infrastructure in any Workflow where you want to deploy to that provisioned deployment infrastructure.

  

• Workflow Step - Add a CloudFormation Provisioner step to a Workflow to build the infrastructure according to your CloudFormation Provisioner and its template.

  

Permissions

The permissions required for Harness to use your provisioner and successfully deploy to the provisioned instances depends on the deployment platform you provision. The permissions are discussed in this topic in the configuration steps where they are applied, but, as a summary, you will need to manage the following permissions:

• Delegate - The Delegate will require permissions according to the deployment platform. It will use the access, secret, and SSH keys you configure in Harness Secrets Management to perform deployment operations. For ECS Delegates, you can add an IAM role to the ECS Delegate task definition. For more information, see Trust Relationships and Roles.
• Cloud Provider - The AWS Cloud Provider must have create permissions for the resources you are planning to create in the CloudFormation template. For Harness AWS Cloud Providers, you can install the Delegate in your AWS VPC and have the Cloud Provider assume the permissions used by the Delegate.

• S3 Bucket - You can use an AWS S3 bucket to point to the provisioner template. The AWS Cloud Provider can be used to access S3 also. The IAM role used by the Cloud Provider simply needs the S3 Bucket policy, described in Add Cloud Providers.
• Access and Secret Keys - These are set up in Harness Secrets Management and then used as inout values when you add a CloudFormation Provisioner step to a Workflow.

Delegate and Cloud Provider Setup

This section describes the Harness Delegate and Cloud Provider setup for CloudFormation provisioning.

There are many types of Delegates, but for CloudFormation, the Shell Script and ECS Delegates are used. The AWS Cloud Provider can connect Harness to your AWS account using these Delegates. For more information, see Delegate Installation and Management and Add Cloud Providers.

Delegate and CloudFormation Setup

The Delegate should be installed where it can connect to the provisioned environment so that it can deploy once it has provisioned the environment. Ideally, this is the same subnet as the instances your will provision, but if you are provisioning the subnet then you can put the Delegate in the same VPC and ensure that it can connect to the provisioned subnet using security groups.

To set up the Delegate, do the following:

1. Install the Delegate on a host where it will have connectivity to your provisioned instances. To install a Delegate, follow the steps in Delegate Installation and Management using a Shell Script or ECS Delegate. Once the Delegate is installed, it will be listed on the Harness Delegates page.

   

2. When you add a Harness AWS Cloud Provider, you will set up the Cloud Provider to assume the IAM used by the Delegate. This is done using a Delegate Tag. For steps on installing a Delegate Tag, see Delegate Tags.

When you are done, the Delegate listing will look something like this:

AWS Cloud Provider Setup

For a CloudFormation deployment, Harness can use a single AWS Cloud Provider to connect to your AWS account and do the following:

• Obtain artifacts from Elastic Container Registry (ECR) or S3.
• Obtain the CloudFormation template from S3.
• Provision infrastructure in AWS.
• Deploy to the provisioned infrastructure in AWS.

When you create the AWS Cloud Provider, you can enter the platform account information for the Cloud Provider to use as credentials, or you can use a Delegate running in your AWS infrastructure to provide the IAM role for the Cloud Provider.

With CloudFormation, you are building an infrastructure on a platform that requires specific permissions, and so the account used by the AWS Cloud Provider (either by username and password or Delegate IAM role) needs the required policies. For example, to create AWS EC2 AMI instances, the account/role needs the AmazonEC2FullAccess policy. See the list of policies in Add Cloud Providers. For steps on adding an AWS Cloud Provider, see Amazon Web Services (AWS) Cloud.

When the AWS Cloud Provider uses the installed Delegate for credentials via the Delegate's tag, it assumes the IAM role used to add the Delegate. For example, here is an AWS Cloud Provider assuming the IAM role from the Delegate you installed in AWS.

AWS S3 Access

CloudFormation templates are added to Harness by either pasting them into a text field or by using an AWS S3 URL that points to the template.

You can use the same AWS Cloud Provider to provision your AWS deployment environment and access the S3 bucket URL.

The AWS Cloud Provider will need credentials to access the S3 bucket. The policy needed by Harness for Amazon S3 access is AmazonS3ReadOnlyAccess, described in Add Cloud Providers.

The policy can be added to the AWS account you use to set up the AWS Cloud Provider. If the AWS Cloud Provider is using the Delegate for credentials, then the policy used to install the Delegate must have the policy.

For information on the AWS policies used by Harness, see Add Cloud Providers.

The following links provide useful information for ensuring access between EC2 instances and S3 buckets:

• Verify Resource-Based Permissions Using the IAM Policy Simulator
• How can I grant my Amazon EC2 instance access to an Amazon S3 bucket in another AWS account?

Artifact Server Setup

There are no Artifact Server settings specific to setting up a Harness Infrastructure Provisioner. When you create a Workflow that uses a provisioner step, you can add an Artifact Check step. Harness will provision the infrastructure and then perform an artifact check of the artifact associated with a Service. Next, Harness deploys the artifact to the provisioned infrastructure.

You will need to set up an Artifact Server to add an artifact to your Harness Service. For steps on setting up an Artifact Server, see Add Artifact Servers.

Application and Service Setup

Infrastructure Provisioners are added to a Harness Application and then incorporated into the Environment and Workflow components.

Any Harness Service can be deployed using the Infrastructure Provisioner. Here, we use a SSH type Service with a TAR File artifact type:

In the Service, we add a TAR file of Apache Tomcat from an AWS S3 bucket.

In this example, an AWS Cloud Provider is used to associate the artifact instead of a Harness Artifact Server because we are using AWS S3. AWS is always integrated as a Harness Cloud Provider.

CloudFormation Provisioner Setup

This section will walk you through a detailed setup of a CloudFormation Provisioner for a deployment to AWS, and provide examples of the supported platforms.

At a high level, setting up the CloudFormation Provisioner involves the following steps:

1. Add your CloudFormation template via its S3 bucket or simply paste it into Harness.
2. Map the relevant CloudFormation output variables from the script to the required Harness fields for the AWS deployment environment.

Below are the detailed steps and examples for each deployment platform.

To set up a CloudFormation Infrastructure Provisioner, do the following:

1. In your Harness Application, click Infrastructure Provisioners.
2. Click Add Infrastructure Provisioner, and then click CloudFormation. The Add CloudFormation Provisioner dialog appears.

   

3. In Display Name, enter the name for this provisioner. You will use this name to select this provisioner in Harness Environments and Workflows.
4. In Source Type, select Template Body, Amazon S3, or Git Repository.

1. If you select Template Body, then paste in the CloudFormation template JSON or YAML.
2. If you select Git Repository, do the following:
   1. In Source Repository, select a SourceRepo Provider for the Git repo you added to your Harness account. For more information, see Add SourceRepo Providers.
   2. In Commit ID, select Latest from Branch or Specific Commit ID.
   3. In Branch/Commit ID, enter the branch or commit ID for the remote repo.
   4. In File Path, enter the repo file and folder path.
      
      For example, if the full path to your script is http://github.com/johnsmith/harness/branch1/scripts/foo.yaml, and you selected Branch and entered branch1, in File Path you can enter scripts/foo.yaml or even ./scripts/foo.yaml.
      
      Using the same example, if you selected Specific Commit ID and enter a commit ID, in File Path you can enter scripts/foo.yaml or even ./scripts/foo.yaml.
3. If you select Amazon S3, in Template File Path, enter the URL for the template in its S3 bucket. Ensure that the AWS Cloud Provider has permissions to read the bucket contents. The required policy is AmazonS3ReadOnlyAccess.

   

Likely, your template contains input parameters that require specific values be passed in when Harness creates a stack from the template. For example, here is an input parameter for a key pair named KeyName:

Parameters:
  KeyName:
    Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
    Type: 'AWS::EC2::KeyPair::KeyName'
    ConstraintDescription: must be the name of an existing EC2 KeyPair.

You can add these input parameters to your Harness CloudFormation provisioner and specify the values for the inputs when you apply this provisioner to a Workflow.

1. In Input Variables, click Add if you added your template manually or Populate Variables if you added a URL to your template in an S3 bucket. We will cover the Populate Variables scenario. The Populate from Example assistant appears.

   

2. In AWS Cloud Provider, select the AWS Cloud Provider you added that has permission to access the template in the S3 bucket.
3. In Region, select the AWS region where the AWS Cloud Provider should connect. AWS S3 is global, but AWS connections require a region.
4. Click Submit. The input parameters from your template are added automatically:

   

5. For each input, select the type of value required: Text or Encrypted Text. When this provisioner is added to a Workflow, the user will have to provide a value for the input that matches the type. Encrypted Text values use secrets you set up in Harness Secrets Management.
6. Click SUBMIT. The CloudFormation Provisioner is created.

Next you will create Service Mappings to map specific template outputs to the fields Harness requires for provisioning.

Service Mappings

In the CloudFormation Provisioner Service Mappings, you map CloudFormation outputs from your CloudFormation template to the fields Harness requires for provisioning. Service Mappings provide Harness with the minimum settings needed to provision using your template.

CloudFormation templates may be in JSON or YAML and both formats are used in this section's examples.

To create a Service Mapping, do the following:

1. In your CloudFormation Provisioner, in Service Mappings, click Add Service Mapping. The Service Mapping dialog appears.

   

   As an example, we will map a ECS type Service with an AWS Cloud Provider.
2. In Service, select the Harness Service you are deploying using the provisioner.
3. In Deployment Type, select the type of deployment, such as SSH, AMI, ECS, or Lambda.
4. In Cloud Provider Type, select the type of Cloud Provider you set up to connect to your deployment environment.
   
   For some Service and Cloud Provider type combinations, there will be additional options. For example, an ECS Service with an Amazon Web Services Cloud Provider Type will also display an AWS Node Type option.
   
   When you are done, the Select Cloud Provider Type section will look something like this:

   

5. Click NEXT. The Configuration settings appear.
6. In Configuration, map the required fields to your CloudFormation template outputs.

You map the CloudFormation template outputs using this syntax, where exact_name is the name of the output:

${cloudformation.exact_name}

The following sections provide examples for the common deployment types. When you are finished, click NEXT and then SUBMIT. The Service Mapping is added to the CloudFormation Provisioner and you can move on to using the provisioner in an Environment.

SSH

The Secure Shell (SSH) deployment type has two AWS Node Type options: AWS Instance and AWS Autoscaling Group.

• AWS Instance - For AWS Instance, only Tags are required. Here is an example mapping both VPCs and Tags:

• AWS Autoscaling Group - For AWS Autoscaling Group, only an Auto Scaling Group is required:

ECS

The ECS deployment type has two AWS Node Type options:

• AWS ECS EC2 - Region and Cluster are required. Here is an example mapping both Region and Cluster:

• AWS ECS Fargate - Region, Cluster, Task Execution Role, VPC, Subnets, Security Group are required. Here is an example mapping all of these:

AMI

The AMI deployment type uses a AWS Auto Scaling Group and only requires that you provide a region and Auto Scaling Group. The Auto Scaling Group output is the Auto Scaling Group you want Harness to mirror when it provisions a new Auto Scaling Group as part of the AMI deployment.

In the following example, we show the following:

• Required outputs.
• The outputs used for the optional Target Group and Application Load Balancer.
• The stage Target Group and Application Load Balancer used for Blue/Green deployments.

Lambda

The Lambda deployment type has two AWS Node Type options: AWS Instance and AWS Autoscaling Group. Both require IAM role and region outputs:

• AWS Instance:

• AWS Autoscaling Group:

Environment Setup

A Harness Environment represents your production and non-production deployment environments and contains Service Infrastructure settings to specify a deployment infrastructure, using a Cloud Provider, a deployment type, such as ECS, and the specific infrastructure details for the deployment, like VPC settings.

Typically, when you add an Environment, you specify the Service Infrastructure for an existing infrastructure. To use your CloudFormation Provisioner, you add the CloudFormation Provisioner to the Service Infrastructure to identify a dynamically provisioned infrastructure that will exist.

The following image shows two Service Infrastructures for an Environment. One infrastructure is for an already-provisioned infrastructure and one is using a CloudFormation Provisioner to dynamically provision the infrastructure.

Later, when you create a Workflow, you will use a CloudFormation Provisioner step to provision the infrastructure using the template in the CloudFormation Provisioner you set up. During deployment, the CloudFormation Provisioner step will provision the infrastructure and then the Workflow will deploy to it via the Environment Service Infrastructure.

To use a CloudFormation Provisioner in an Environment Service Infrastructure, do the following:

1. In your Harness Application, click Environments. The Environments page appears.
2. Click Add Environment. The Environment dialog appears.
3. In the Environment dialog, enter a Name and select an Environment Type, such as Non-Production. When you are done, it will look something like this:

   

4. Click SUBMIT. The new Environment appears. Next, you will add a Service Infrastructure using the CloudFormation Infrastructure Provisioner.

Service Infrastructure

Service Infrastructure is where you specify a deployment infrastructure, using a Cloud Provider, a deployment type, such as ECS or SSH, and the specific infrastructure details for the deployment, like VPC settings. For a deployment using a CloudFormation Infrastructure Provisioner, you select the CloudFormation Infrastructure Provisioner you created and Harness uses its Service Mappings for deployment settings.

1. In your CloudFormation Infrastructure Provisioner, click Add Service Infrastructure. The Service Infrastructure dialog appears.
2. For the Select Cloud Provider section, enter the same Service and Cloud Provider you are using in the CloudFormation Infrastructure Provisioner.

   

3. Click Next. The Configuration settings appear.
4. In Provision Type, select Dynamically Provisioned. The settings change for the provisioner.

   

5. In Provisioner, select your CloudFormation Infrastructure Provisioner. If you do not see it listed, then the Select Cloud Provider section is not using the same Service or Cloud Provider as your CloudFormation Infrastructure Provisioner.
6. In Connection Attributes, select the SSH credentials to use when connecting to the provisioned resources.
   
   For example, if a CloudFormation Provisioner will create an AMI instance, it will likely have a KeyName entry in its template. A SSH secret in Harness using the same pem file associated with the KeyName value will be needed for the SSH connection to the instance. The following image shows how the SSH secret in Harness Secrets Management is used in Connection Attributes and how the CloudFormation template includes the KeyName entry that uses the same key as the pem file.

Now that you have set up a Service Infrastructure, you can use it in a Workflow along with a CloudFormation Provisioner step that provisions the infrastructure.

Workflow Setup

Once you have a CloudFormation Provisioner and an Environment Service Infrastructure that uses it, you can use the CloudFormation Provisioner in a Workflow's pre-deployment steps. The CloudFormation Provisioner step will provision the infrastructure and then the Workflow will deploy to it via the Environment Service Infrastructure.

The following image shows how the Workflow uses the CloudFormation Provisioner step to create the infrastructure, then uses the same CloudFormation Provisioner in the Service Infrastructure to select the node(s) for deployment, and finally deploys to AWS EC2.

Workflow Creation

To use a CloudFormation Provisioner in your Workflow, do the following:

1. In your Harness Application, click Workflows.
2. Click Add Workflow. The Workflow dialog appears.
3. Enter a name and description for the Workflow.
4. In Workflow Type, select Canary.

1. In Environment, select the Environment that has the CloudFormation Provisioner set up in its Service Infrastructure.

   

2. Click SUBMIT. The new Workflows is created.

By default, the Workflow includes a Pre-deployment Steps section. This is where you will add a step that uses your CloudFormation Provisioner.

CloudFormation Provisioner Step

To add the CloudFormation Provisioner Step, do the following:

1. In your Workflow, in Pre-deployment Steps, click Add Step. A dialog containing the available steps appears.

1. In Provisioners, click CloudFormation Create Stack. The CloudFormation Provision dialog appears.

1. In Provisioner, select a Harness CloudFormation Provisioner.
2. In AWS Cloud Provider, typically, you will select the same Cloud Provider you used when setting up the Service Infrastructure used by this Workflow.
3. In Region, select the region where you will be provisioning your resources.
4. Select Use Custom Stack Name, enter a name for your stack. If you do not select this option, Harness will automatically generate a unique name for your stack prefixed with HarnessStack, such as HarnessStack-7HklGe0N6AvviJmZ.
5. In Timeout, enter how long Harness should wait for the successful CloudFormation Provisioner set up before failing the Workflow.
6. Click NEXT. The Input Values settings appear.

Input Values

The Input Values are the same variables from the CloudFormation Provisioner Variables section.

Enter or select a value for each variable in Input Values. For encrypted text values, select an Encrypted Text secret from Harness Secrets Management.

For more information, see Secrets Management.

Click NEXT and then SUBMIT. The CloudFormation Create Stack step is added to your Workflow.

Now your Workflow is set up to provision an infrastructure using your CloudFormation template using the CloudFormation Provisioner, and then deploy to the provisioned infrastructure.

Troubleshooting

This section lists some of the common exceptions that can occur and provides steps to fix the errors.

Delegate Could Not Reach the Resource

In order for the Delegate to reach the provisioned instances it must have connectivity and permission to SSH into the instance and perform deployment tasks. Depending on the platform where you deploy, this can involve different settings.

Ensure that the SSH key selected in the Service Infrastructure Connection Attributes setting will enable the Delegate to connect to the provisioned instances.

In cases where a security group requires hosts to use public DNS to resolve other host names, the Service Infrastructure setting Use Public DNS for connection option should be selected.

Key Changed

The variables populated in the CloudFormation Provisioner Input Variables section are assigned a type.

If the type changes from Encrypted Text to Text, which can happen if you re-populate the variable, you might receive an error during deployment saying the type of key has changed.

Invalid request: The type of variable access_key has changed. 
Please correct it in the workflow step.

Simply correct the variable type in Input Variables.

Type is also used when you add a provisioner step to the Workflow. You should ensure that the type selected in the Provisioner Input Variables section is preserved in the Workflow provisioner Input Values section.

Provisioner mapping region was not resolved

If you have mapped provisioner settings in Service Mappings but your provisioner configuration does not have outputs for those variables, you will receive the following error:

Invalid request: The infrastructure provisioner mapping region 
was not resolved from the provisioner outputs

Error Validating Provider Credentials

If you are using access or secret keys in your provisioner configuration and AWS is unable to validate them, you will receive the following error:

provider.aws: error validating provider credentials: 
error calling sts:GetCallerIdentity: 
InvalidClientTokenId: The security token included in the request is invalid.

Ensure that your access or secret keys are valid. If you are suing Harness Encrypted Text secrets to manage the keys, then update the Encrypted Text secrets with the correct values. For more information, see Secrets Management.