AWS Elastic Container Service (ECS) Deployment

Updated 6 days ago by Michael Cretzman

This guide explains how to use AWS Amazon Elastic Container Service (ECS) with Harness. In this guide, we will set up Harness for ECS, create a Harness Application, and deploy a public Docker image from Docker Hub to an existing ECS cluster using Harness. This deployment scenario is very popular and a walkthrough of all the steps involved will help you set up this scenario in Harness for your own microservices and apps.

Component Overview

The following table lists the ECS components and where they are set up in Harness, as well as the related Harness components that perform ECS deployment operations. For detailed explanations of ECS, see the ECS Developer Guide from Amazon.

Component

Description

Harness Location

Harness Delegate

A software service you run in the same VPC as the ECS cluster to enable Harness to perform deployment operations. The Delegate does not need root privileges, and it only makes an outbound HTTPS connection to the Harness platform.

This guide will describe how to set up the Harness Delegate for ECS deployment.

For general information, see Delegate Installation.

Harness Cloud Provider

A Cloud Provider is a logical representation of your AWS infrastructure. Typically, a Cloud Provider is mapped to a AWS account, Kubernetes cluster, Google service account, Azure subscription, or a data center.

This guide will describe how to set up the AWS Cloud Provider for ECS deployment.

For general information, see Add Cloud Providers.

ECS Task Definition

Describes the Docker containers to run (CPU, memory, environment variables, ports, etc) and represents your application.

Specified in the Harness Service, in Container Specification.

ECS Task

Instance of a Task Definition. Multiple Tasks can be created by one Task Definition, as demand requires.

ECS Service

Defines the minimum and maximum Tasks from one Task Definition to run at any given time, autoscaling, and load balancing.

This is specified in the Harness Service, in Service Specification.

ECS Cluster

A Cluster is a group of ECS Container Instances where you run your service tasks in order for them to be accessible. The container management service handles the cluster across one or more ECS Container Instance(s), including the scheduling, maintaining, and scaling requests to these instances.

ECS Clusters are selected in two Harness components:

  • The AWS Cloud Provider, via the IAM role for Delegate option.
  • Harness application Environment, where you select the AWS Cloud provider, and your ECS cluster name.

Launch Types

There are two types:

  • Fargate - Run containers without having to manage servers or clusters of Amazon EC2 instances.
  • EC2 - Run containers on a cluster of Amazon EC2 instances that you manage.

You specify when launch type to use when adding a Service Infrastructure to a Harness Environment.

Replica Scheduling Strategy

Places and maintains the desired number of tasks across your cluster.

This is specified in the Harness Service, in Service Specification.

Daemon Scheduling Strategy

As of July 2018, ECS has a daemon scheduling strategy that deploys exactly one task on each active container instance that meets all of the task placement constraints that you specify in your cluster.

With a daemon strategy, a task is deployed onto each instance in a cluster to provide common supporting functionality.

This is specified in the Harness Service, in Service Specification.

awsvpc Network Mode

Provides each task with its own elastic network interface. Fargate task definitions require the awsvpc network mode.

Service Discovery

An ECS service can use the ECS Service Discovery to manage HTTP and DNS namespaces for ECS services via the AWS Cloud Map API actions.

This is specified in the Harness Service, in Service Specification.

Auto Scaling

Auto Scaling adjusts the ECS desired count up or down in response to CloudWatch alarms.

This is specified in the Harness Workflow ECS Service Setup command.

Prerequisites

  • Existing ECS cluster.
  • IAM Role for the Harness Cloud Provider connection to AWS. The policies are listed in ECS (Existing Cluster) and also described in this document.
  • EC2 instance for the Harness Delegate that is running in the same VPC as the ECS cluster and meets the Harness Delegate Requirements. Setting up this instance is described in this document.

Deployment Overview

This guide takes you through setting up ECS Deployment using the following steps:

  1. Install and run the Harness ECS Delegate in an ECS cluster in your VPC.
  2. Add an AWS Cloud Provider that uses the IAM role of the Harness ECS Delegate. You can also create a Cloud Provider that uses another AWS account with the required ECS permissions, but using the Delegate is the easiest method.
  3. Create a Harness Application for ECS.
  4. Add a Harness Service. We will cover the following ECS features when we add a service:
    1. Replica Strategy.
    2. Daemon Strategy.
    3. awsvpc Network Mode.
    4. Service Discovery.
  5. Add an Environment and ECS Service Infrastructure.
  6. Add Workflows:
    1. Workflow 1: Canary Deployment with Replica Scheduling.
    2. Workflow 2: Basic Deployment with Daemon Scheduling.
  7. Deploy an ECS Workflow.

Install and Run the Harness ECS Delegate

The ECS cluster for the ECS Delegate is run in the same VPC as the ECS cluster(s) where you will deploy your ECS services. You can run the Delegate in the same subnet as a cluster, which is often the easiest way to manage the Delegate.

Most importantly, in order for Harness to obtain metadata about ECS containers and use that in its Continuous Verification features (using verification providers and Harness machine learning algorithms), the Harness Delegate must be running in the same VPC as the target ECS cluster. This is a restriction of AWS.

For steps on installing and running the Harness ECS Delegate, see the following:

  • ECS (Existing Cluster) - This covers the IAM policies needed for the role assigned to the ECS cluster where the ECS Delegate will be install.
  • Installation Example: Amazon Web Services and ECS - Follow the steps here to install the ECS Delegate in a ECS cluster and apply a Delegate Tag to the ECS Delegate. Ensure that you add a Delegate Tag to the ECS Delegate you install, as you will use that Delegate Tag when you set up the AWS Cloud Provider.

Add an AWS Cloud Provider

Add an AWS Cloud Provider that uses the IAM role of the Harness ECS Delegate by using the Delegate tag.

  1. In Harness, click Setup.
  2. Click Cloud Providers. The Cloud Providers page appears.
  3. Click Add Cloud Provider. The Cloud Provider dialog appears.
  4. In Type, select AWS.
  5. In Display Name, enter a name for the Cloud Provider, such as aws-ecs.
  6. Enable the Assume IAM Role on Delegate option.
  7. In Delegate Tag, enter the tag you gave the ECS Delegate listed in the Installations page.
  8. Click SUBMIT. The Cloud Provider is added.

Create the Harness ECS Application

The Harness Application represents a logical group of the ECS setup and release process, including the ECS service and task definitions, ECS cluster environment, and deployment workflow steps particular to each service you are deploying. For more information on Harness Applications, see Application Checklist.

To create a new Application, do the following:

  1. In Harness, click Setup, and then click Add Application. The Application dialog appears.
  2. Enter the name for your application, such as ECS Demo Application, and click SUBMIT. Your new Application appears.
  3. Click the Application name. The Application entities appear.

Add a Harness ECS Service

A Harness Service represents your microservice as the artifact source (for example, Docker image), ECS container and service specifications, and any runtime variables used for deployment. You define where the artifact comes from, and you define the container and service specs for the ECS cluster. In addition, you can use configuration variables and files for the service.

Harness Services are different from ECS services. Where a Harness Service describes your microservice, an ECS service is a specified number of task definition instances run and maintained simultaneously in an Amazon ECS cluster. For a detailed description of ECS services, see Services from AWS.

In this guide, we will cover the how the following common ECS features are implemented in Harness Services:

  • Replica Strategy.
  • Daemon Strategy.
  • awsvpc Network Mode.
  • Service Discovery.

Configurations for these features are also discussed in the Harness Environment and Workflows, later in this guide. For the Harness Service artifact example, we will use a Docker image publicly hosted on Docker Hub.

To create a Harness Service for ECS, do the following:

  1. In Harness, click Setup. The list of Applications appears.
  2. Click the name of the ECS Application you created. The Application appears.
  3. Click Services. The Services page appears.
  4. Click Add Service. The Service dialog appears.
  5. In Name, enter a name for the service.
  6. In Artifact Type, select Docker Image.
  7. Click SUBMIT. The new service is listed.








    Next, we will add the artifact source for the service, a sample app publicly hosted on Docker Hub.
  8. Click Add Artifact Source, and click Docker Registry.

  1. The Artifact Source dialog appears.
  2. In Source Server, select the Harness Artifact Server for the Docker Registry. For information on setting up a Harness Artifact Server, see Add Artifact Servers.
  3. In Docker Image Name, enter the name of the image.
  4. Click SUBMIT. The Artifact Source is added.

Next, we will define the ECS task definition and service specification.

ECS private registry authentication for tasks using AWS Secrets Manager enables you to store your credentials securely and then reference them in your container definition. For information on using private registry authentication for tasks, see Private Registry Authentication for Tasks from AWS.

Container Specification

You specify the ECS Task Definition in the Harness Service. To specify the ECS Task Definition, do the following:

  1. In the Harness Service, in Deployment Specification, expand ECS (if necessary). The Container Specification appears.
  2. Click Container Specification. The Container Specification dialog appears.








    You can specify the Task Definition using the fields in the dialog or click Advanced Settings to add or edit the JSON.

For a description of all available Task Definition parameters, see Task Definition Parameters from AWS.

The Container Definition JSON uses the following placeholders.

Placeholder

Description

${DOCKER_IMAGE_NAME}

Required. This placeholder is used with the image label in the JSON:

"image" : "${DOCKER_IMAGE_NAME}"

The placeholder is replaced with the Docker image name and tag at runtime.

${CONTAINER_NAME}

This placeholder is used with the name label in the JSON:

"name" : "${CONTAINER_NAME}"

The placeholder is replaced with a container name based on the Docker image name at runtime.

${EXECUTION_ROLE}

Required for Fargate. This placeholder is used with the executionRoleArn label in the JSON.

"executionRoleArn" : "${EXECUTION_ROLE}"

Replaced with executionRoleArn. When you register a task definition, you can provide a task execution role that allows the containers in the task to pull container images and publish container logs to CloudWatch on your behalf. For more information, see Amazon ECS Task Execution IAM Role from AWS.

If you have an existing Task Definition, you can paste it into the JSON. You can obtain the Task Definition from the ECS console:

You can also obtain the Task Definition using the AWS CLI (describe-task-definition):

aws ecs describe-task-definition --task-definition ecsTaskDefinitionName

Ensure that the required placeholders ${DOCKER_IMAGE_NAME} and ${EXECUTION_ROLE} (for Fargate) are used.

For some example Task Definitions, see Example Task Definitions from AWS.

Once Harness deploys the ECS application, you can see the placeholders replaced in the Task Definition JSON:

...
    "volumesFrom": [],

     "image": "registry.hub.docker.com/library/nginx:stable-perl",      ...

     "name": "library_nginx_stable-perl"

   }

For Fargate, you will see the executionRoleArn placeholder replaced:

{

 "ipcMode": null,

 "executionRoleArn": "arn:aws:iam::4XXX0225XX7:role/ecsTaskExecutionRole",

 "containerDefinitions": [

   {

...

awsvpc Network Mode

When configuring the Task Definition via the Harness Service Container Specification, you can set the awsvpc network mode by simply adding the networkMode parameter. For details Network Mode, see networkMode in the AWS docs.

Example for awsvpc Network Mode

The following example shows the networkMode parameter with the awsvpc value.

...

 "networkMode" : "awsvpc"

}

When you look at the Task Definition created by Harness, you can see the awsvpc network mode at the bottom of the definition JSON:

 ...

"pidMode": null,

 "requiresCompatibilities": [],

 "networkMode": "awsvpc",

 "cpu": null,

 "revision": 2,

 "status": "ACTIVE",

 "volumes": []

}

Task definitions that use the awsvpc network mode use the AWSServiceRoleForECS service-linked role, which is created for you automatically. For more information, see Using Service-Linked Roles for Amazon ECS from AWS.

Service Specification

You can specify the ECS service configuration in the Harness Service Service Specification. To specify the service configuration, do the following:

  1. In the Harness Service, in Deployment Specification, expand ECS (if necessary). The Service Specification appears.
By default, the Service Specification uses a Replica strategy.

If you have an existing service and you want to use its JSON in Service Specification, you can enter the JSON in Service Specification. You can enter in any parameter that is specified by the aws ecs create-service command.

You can obtain the JSON using the AWS CLI using describe-services:

aws ecs describe-services --cluster clusterName --service ecsServiceName

For information on all ECS Service definition parameters, see Service Definition Parameters from AWS.

The following sections describe how to configure the Service Specification for different ECS features.

Replica Strategy

You specify the Replica strategy using the schedulingStrategy parameter. By default, when you create a Harness Service using the Docker Image type, the Service Specification will generate the JSON for the Replica strategy. There are no changes that you need to make.

Example Service Specification for Replica Strategy

The following example is the default JSON generated for the Service Specification, setting the scheduling strategy for Replica:

{

"placementConstraints":[ ],

"placementStrategy":[ ],

"healthCheckGracePeriodSeconds":null,

"schedulingStrategy":"REPLICA"

}

Daemon Strategy

You specify the Daemon strategy using the schedulingStrategy parameter. By default, when you create a Harness Service using the Docker Image type, the Service Specification will generate the JSON for the Replica strategy. To set a Daemon strategy, you simply need to change the schedulingStrategy parameter to DAEMON.

Example Service Specification for Daemon Strategy

Here is an example of how to specify the Daemon scheduling strategy in Service Specification:

{

"placementConstraints":[ ],

"placementStrategy":[ ],

"healthCheckGracePeriodSeconds":null,

"schedulingStrategy":"DAEMON"

}

Service Discovery

Harness does not create an ECS Service Discovery Service, but Harness registers the ECS services it creates with the Service Discovery Service.

If you have configured Service Discovery for an ECS service, Harness can deploy to that service, registering its DNS SRV records as needed. During rollback or in the case of an ECS task failing, ECS manages the DNS resolution, replacing A records, etc.

For a detailed description of Service Discovery concepts, see Service Discovery from AWS. In you are new to Service Discovery, see Tutorial: Creating a Service Using Service Discovery from AWS.

Here is what the ECS Service Discovery configuration looks like in AWS:

When you create the Service Discovery Service in ECS, you will specify a namespace and ECS will generate the DNS records (SRV and A records) for the ECS namespace in AWS Route 53. DNS Queries for the namespace are resolved by Route 53 and traffic is routed to the instances supporting the ECS cluster.

To specify the Service Discovery Service in the Harness Service Specification, add the serviceRegistries parameter to the Harness Service Specification. The serviceRegistries parameter is defined like this:

"serviceRegistries": [

 {

   "registryArn": "arn:aws:servicediscovery:us-east-1:52516162:service/srv-jwyz7x4igkxckqno",

   "containerName": "${CONTAINER_NAME}",

   "containerPort": ${serviceVariable.containerPort}

   # "port":

 }

],

In this example, the Harness variable ${serviceVariable.containerPort} is used for the containerPort value. You can simply enter the port number instead, such as 8080. The ${serviceVariable.containerPort} variable is created in the Config Variables section of the Service as containerPort, and referenced as ${serviceVariable.containerPort} in the Service Specification. Using a Config Variable allows you to override the variable value when configuring the Harness Workflow that deploys the Service. For more information, see Workflow Phases.

The following list describes the fields and values needed for a Service Discovery Service in the Harness Service Specification:

  • registryArn - The Amazon Resource Name (ARN) of the service registry. The currently supported service registry is Amazon Route 53 Auto Naming. To obtain the registryArn value, use the aws ecs describe-services command.
  • containerName - The containerName field is the container name value to be used for your Service Discovery Service, and already specified in the task definition in Container Specification. Typically, you simply use the variable ${CONTAINER_NAME}. Harness verifies that the container name is specified in Container Specification.
  • containerPort - The port value to be used for your Service Discovery Service.

You can override Services variables in the Harness Environment and Workflow. For more information see Override a Service Configuration and Workflow Phases.

Which Service Parameters Do I Use?

Here are the guidelines for when to use the different service parameters in serviceRegistries:

  • If the task definition for your service task uses the awsvpc network mode and a SRV DNS record is used, you must specify either a containerName and containerPort combination or a port value, but not both.
  • If you use SRV DNS records, but not the awsvpc network mode, a containerName and containerPort combination is required.
  • If you use awsvpc network mode only (no SRV record), you do not need the containerName and containerPort, but can use port. The port field is used only if both the awsvpc network mode and SRV records are used.

Here is an example where the service task does not use the awsvpc network mode but a SRV DNS record is used:

"serviceRegistries": [

{

"registryArn": "arn:aws:servicediscovery:us-east-1:448640225317:service/srv-jwyz2x4igkxckqno",

"containerName": "${CONTAINER_NAME}",

"containerPort": ${serviceVariable.containerPort}

}

],

The value for containerName is ${CONTAINER_NAME}. This maps to the name field in the Container Specification, itself replaced with a container name based on the image name:

{

 "containerDefinitions" : [ {

   "name" : "${CONTAINER_NAME}",

   "image" : "${DOCKER_IMAGE_NAME}",

...

The value for containerPort is 8080.

    "portMappings" : [ {

     "containerPort" : 8080,

     "protocol" : "tcp"

   } ],
You can use Harness Environment variables to override the Service variables used in the Service Specification, thereby using the same Harness Service in multiple deployment environments. For more information, see Override a Service Configuration.

Using Private Docker Registry Authentication

In the Harness Service, you can add the RepositoryCredentials property type in the Container Specification to specify the repository credentials for private registry authentication.

This process has the following steps:

  1. Add the secretsmanager:GetSecretValue policy to the ECS task execution role. Here is the policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"secretsmanager:GetSecretValue"
],
"Resource": [
"arn:aws:secretsmanager:region:aws_account_id:secret:secret_name",
"arn:aws:kms:region:aws_account_id:key:key_id"
]
}
]
}
The action kms:Decrypt is required only if your key uses a custom KMS key and not the default key. The ARN for your custom key should be added as a resource. For more information, and details about ECS platform versions that support this feature, see Private Registry Authentication for Tasks from AWS.
  1. Add the RepositoryCredentials property type to the Harness Service as a part of the Task Definition in Container Specification like this:
"containerDefinitions": [

{

"name" : "${CONTAINER_NAME}",

  "image" : "${DOCKER_IMAGE_NAME}",...

  "repositoryCredentials": {

    "credentialsParameter": "arn:aws:secretsmanager:region:aws_account_id:secret:secret_name"

  }

...

}

]
  1. In addition to specifying the repositoryCredentials, you must also specify the Task execution role in the Service Container Specification for the Task Definition using the property executionRoleArn. This role authorizes Amazon ECS to pull private images for your task. For more information, see Private Registry Authentication Required IAM Permissions. For example:

"executionRoleArn" : "arn:aws:iam::448000000317:role/ecsTaskExecutionRole",

The Task execution role is specified when the Task Definition is created in ECS, or in AWS IAM (see Amazon ECS Task Execution Role). If you are creating the ECS Task Definition for the first time using Harness, create the role in IAM, and then add it in executionRoleArn in Container Specification.

Add an ECS Environment

Creating a Harness Environment for ECS is a simple process where you specify the ECS Deployment Type and the ECS cluster where you want to deploy the ECS Task and Service you defined in your Harness Services.

To create the Harness Environment for ECS, do the following:

  1. In your Harness Application, click Environments.
  2. Click Add Environment. The Environment dialog appears.
  3. In the Environment dialog, enter a name, such as Stage, select the Non-Production type (you can add your production environment later), and click SUBMIT. The new Environment appears.

    Next, you define one or more Service Infrastructures for the environment. For ECS, a Service Infrastructure specifies the ECS cluster, launch type, and related VPC information.
  4. Click Add Service Infrastructure. The Service Infrastructure dialog appears.
  5. In Select Cloud Provider, specify the following:
    1. Service - Select the Harness Service to deploy in the Service Infrastructure.
    2. Deployment Type - Select Amazon EC2 Container Services (ECS).
    3. Cloud Provider - Select the AWS Cloud Provider you set up for your ECS deployment.
    4. Click Next. The Configuration section appears.
  6. In Configuration, specify the following:
    1. Provision Type - Select Already Provisioned. If you are using a configured Harness Infrastructure Provisioner, select Dynamically Provisioned and then select the Harness Infrastructure Provisioner you set up and click SUBMIT.
    2. Region - Select the AWS region where your ECS cluster is located.
    3. Cluster Name - Select the ECS cluster where Harness will deploy the Task Definition and Service defined in the Harness Service you selected for this Environment.  
    4. Launch Type - Select Fargate LaunchType or EC2 Instances. The only difference when configuring these launch types is Fargate LaunchType requires that you specify the Target Execution Role.
    5. VPC - Select the VPC where the ECS Cluster is located.
    6. SecurityGroups - Select the AWS security group(s) you want to use when creating instances. When selecting the SecurityGroups, the Group ID for the security groups are listed. You can locate the Group ID in the ECS Dashboard, under Security Groups.
    7. Subnets - Select the VPC subnet(s) where the EC2 instances will be located.
    8. Assign Public IP - Select this checkbox to have external public IP addresses assigned to the deployed container tasks.
  7. Click SUBMIT. The new Service Infrastructure is added to your Environment. You will select this Environment and Service Infrastructure when you create your Harness Workflow.

Add ECS Workflows

Workflows are the deployment steps for your ECS services, including deployment types such as Canary and Blue/Green. Workflows can involve a few steps or multiple phases each composed of several steps.

ECS Harness Workflows differ according to the ECS Service Scheduler (Replica and Daemon) used by the Harness Service in the Workflow. In this section, we will set up one Workflow for each Replica and Daemon Scheduler:

  • Workflow 1: Replica Scheduling using Canary Deployment.
    • Phase 1 - 50% Deployment:
      • Setup Container: ECS Service Setup.
      • Deploy Containers: Upgrade Containers.
    • Phase 2 - 100% Deployment:
      • Deploy Containers: Upgrade Containers.
  • Workflow 2: Daemon Scheduling using Basic Deployment.
    • ECS Daemon Service Setup Step.

Replica Scheduling using Canary Deployment

In this procedure, we will create a Workflow to deploy a Harness Service configured with a Replica Scheduling Strategy, as described in Replica Strategy.

To create a Workflow using a Service configured with a Replica Scheduling Strategy, do the following:

  1. In your Harness Application, click Workflows. The Workflows page appears.
  2. Click Add Workflow. The Workflow dialog appears.
  3. Complete the following fields.
    1. Name - Give the Workflow a name that describes its deployment goals, such as ECS Replica Strategy.
    2. Description - Provide details about the Workflow so other users understand its deployment goals.
    3. Workflow Type - Select Canary Deployment.
    4. Environment - Select the Environment you created for ECS. This is the Environment containing a Service Infrastructure for the Harness Service you are deploying with this Workflow. You will select the Service and Service Infrastructure when you set up the stages of the Canary deployment.
    5. Click SUBMIT. The new Workflow is displayed.








      Next, you will add two phases for the Canary deployment. The first phase will set up your ECS service and then upgrade ECS service instances to 50% of the available ECS service instances.
  4. In the Workflow, in Deployment Phases, click Add Phase. The Workflow Phase dialog appears.
  5. Complete the following fields.
    1. Service - Select the Harness Service that uses the Replica Strategy.
    2. Service Infrastructure - This list is populated using the Environment you selected when creating the Workflow. Select the Service Infrastructure describing the cluster where the Amazon ECS service defined in the Harness Service will be deployed.
    3. Service Variable Overrides - If the Harness Service uses variables that you want to override for this Workflow phase, such as those described in Service Discovery, you can override the variable values here.
  6. Click SUBMIT. The new Phase 1 page appears.
  7. Click ECS Service Setup. The ECS Service Setup dialog appears.
  8. Complete the following fields.
    1. ECS Service Name - By default, the ECS service will be named using a concatenation of the Harness Application, Service, and Environment names. You can change the name here using text or a variable. Enter ${ in the field to see a list of all of the variables available.
    2. Instances for the first time - This field displays the number of desired ECS service instances for this stage. By default, the ECS service will be set up using 2 ECS service instances even if the field contains 0.
    3. Fixed - Click this option to fix the specific number of ECS service instances to use for this stage. The Fixed Instances Count field will appear, where you can enter the value.
    4. Resize Strategy - Specify how you want the new ECS service instances added and downsized.
    5. Service Steady State Wait Timeout - Specify how many minutes Harness should wait for the ECS service instances to reach Steady State before failing the set up.
    6. AWS Auto Scaler Configuration - See AWS Auto Scaling with ECS.
    7. Use Load Balancer - See Using ELB Load Balancers During Deployment.
    8. Close or Submit the ECS Service Setup dialog to return to the Phase 1 page.
  9. Click Upgrade Containers. The Upgrade Containers dialog appears.
  10. In Desired Instances, set the number or percentage of ECS service instances to use for this stage. As this is Phase 1 of a Canary deployment, enter 50 Percent.

    The value in Desired Instances relates to the number of ECS service instances set in the ECS Service Setup dialog. For example, if you entered 2 as the Fixed Instances Count in ECS Service Setup and then enter 50 Percent in Upgrade Containers, that means, for this phase, Harness will deploy 1 ECS service instance.
  11. Click SUBMIT.
  12. Click the name of the Workflow in the breadcrumb links to return to the Workflow page and add the second Phase of this Canary deployment.
  13. To add Phase 2, click Add Phase.
  14. In the Workflow Phase dialog, complete the following fields.
    1. Service - Select the same Harness Service that uses the Replica Strategy.
    2. Service Infrastructure - Select the Service Infrastructure describing the cluster where the Amazon ECS service defined in the Harness Service will be deployed.
    3. Service Variable Overrides - If the Harness Service uses variables that you want to override for this Workflow phase, such as those described in Service Discovery, you can override the variable values here.
  15. Click SUBMIT. The Phase 2 page appears.








    As this is the second phase in the Canary deployment, and will only run if Phase 1 deployed successfully, it will upgrade the number of containers to 100%.
  16. Click Upgrade Containers. The Upgrade Containers dialog appears.
  17. In Desired Instances, enter 100, choose Percent, and click SUBMIT. This will deploy the full count of ECS service instances.

The Workflow is complete. You can run the Workflow to deploy the ECS service with the Replica strategy to your ECS cluster.

Daemon Scheduling using Basic Deployment

In this procedure, we will create a Workflow to deploy a Harness Service configured with a Daemon Scheduling Strategy, as described in Daemon Strategy.

To deploy a Harness Service configured with a Daemon Scheduling Strategy, do the following:

  1. In your Harness Application, click Workflows. The Workflows page appears.
  2. Click Add Workflow. The Workflow dialog appears.








    We will be creating a Basic Deployment Workflow using the Harness Service configured with a Daemon Scheduling Strategy.
  3. Complete the following fields.
    1. Name - Give the Workflow a name that describes its deployment goals, such as ECS Daemon Strategy.
    2. Description - Provide details about the Workflow so other users understand its deployment goals.
    3. Workflow Type - Select Basic Deployment.
    4. Environment - Select the Environment you created for ECS. This is the Environment containing a Service Infrastructure for the Harness Service you are deploying with this Workflow. You will select the Service and Service Infrastructure when you set up the Basic deployment.
  4. Click SUBMIT. The new Workflow is displayed.







    This Workflow will simply set up the ECS service using a Daemon strategy.
  5. Click the ECS Daemon Service Setup step. The ECS Daemon Service Setup dialog appears.
  6. Complete the following fields.
    1. ECS Service Name - The step will create the ECS service using the names of the Harness Application, Service, and Environment.
    2. Service Steady State Wait Timeout - Specify how many minutes Harness should wait for the instances to reach Steady State before failing the set up.
    3. Click SUBMIT.

The Workflow is complete. You can run the Workflow to deploy the ECS service with the Daemon strategy to your ECS cluster.

Using ELB Load Balancers During Deployment

Harness can use an AWS Elastic Load Balancer (ELB) for your Amazon ECS service to distribute traffic evenly across the tasks in your service.

When you set up the ELB configuration in Harness, you specify the target group for the ECS service. When each task for your ECS service is started, the container and port combination specified in the Service Specification in the Harness Service is registered with that target group and traffic is routed from the load balancer to that task.

For information about using ELB and ECS, see Service Load Balancing from AWS.

In a Harness Workflow, the ECS Service Setup and ECS Daemon Service Setup steps allow you to use an ELB when deploying your ECS service.

To use an ELB in the ECS Service Setup or ECS Daemon Service Setup steps, do the following:

  1. In the ECS Service Setup or ECS Daemon Service Setup step, enable the Use Load Balancer option. The ELB settings appear.
  2. Complete the following ELB settings.
    1. Elastic Load Balancer - Select the ELB that you want to use. The list is populated using the Service Infrastructure in the Workflow setup. Once you select an ELB, Harness will fetch the list of target groups.
    2. Target Group - Select the target group for the load balancer. You associate a target group to an ECS service. Each target group is used to route requests to one or more registered targets.
    3. IAM Role - Select the IAM role to use when using the ELB. The role must have the AmazonEC2ContainerServiceRole policy.
    4. Target Container Name and Target Port - You can leave these fields blank. They are used if the container specification has multiple container definitions, which is not common. When you deploy your ECS service with Harness, Harness uses the container name and port from the Service Specification in the Harness Service. If you choose to use these fields, note that as an ECS requirement Target Container cannot be empty if Target Port is defined.
  3. Click SUBMIT.

The ELB configuration is set. When Harness deploys the ECS service, traffic will be routed from the load balancer to the service task.

AWS Auto Scaling with ECS

The ECS service(s) you deploy with Harness can be configured to use AWS Service Auto Scaling to adjust its desired ECS service count up or down in response to CloudWatch alarms. For more information on using Auto Scaling with ECS, see Target Tracking Scaling Policies from AWS.

This is what the AWS Auto Scaling setting looks like in the ECS console:

In Harness, you configure Auto Scaling in the ECS Service Setup step of a Workflow (for example, Canary Deployment).

There are two AWS Auto Scaling resource types that you must be set up in Harness to use Auto Scaling with ECS:

  • Scalable Target - Specifies a resource that AWS Application Auto Scaling can scale. For more information, see ScalableTarget from AWS.
  • Scalable Policy - Defines a scaling policy that Application Auto Scaling uses to adjust your application resources. For more information, see ScalingPolicy from AWS.

Before you set up Auto Scaling for the ECS service in Harness, you need to obtain the JSON for the Scalable Target and Scalable Policy resources from AWS.

To obtain the Scalable Target, connect to an EC2 instance in your VPC and enter the following:

aws application-autoscaling describe-scalable-targets --service-namespace ecs

For more information, see describe-scalable-targets from AWS.

To obtain the Scalable Policy, enter the following:

aws application-autoscaling describe-scaling-policies --service-namespace ecs

For more information, see describe-scaling-policies from AWS.

To create the Scalable Target and Scalable Policy resources, see the register-scalable-target and put-scaling-policy commands from AWS.

To set up Auto Scaling for the ECS service in the Harness Workflow, do the following:

  1. In a Workflow with the ECS Service Setup step, open the ECS Service Setup step.
  2. Next to AWS Auto Scaler Configuration, click Add. The Auto Scaling property fields appear.
  3. In Scalable Target, paste the JSON for the property. For example:
{

"ServiceNamespace": "ecs",

"ScalableDimension": "ecs:service:DesiredCount",

"MinCapacity": 2,

"MaxCapacity": 5,

"RoleARN": "arn:aws:iam::448XXXXXXX7:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService"

}

  1. In Scaling Policy, paste the JSON for the property. For example:
{

"ScalableDimension": "ecs:service:DesiredCount",

"ServiceNamespace": "ecs",

"PolicyName": "P1",

"PolicyType": "TargetTrackingScaling",

"TargetTrackingScalingPolicyConfiguration": {

"TargetValue": 60.0,

"PredefinedMetricSpecification": {

"PredefinedMetricType": "ECSServiceAverageCPUUtilization"

},

"ScaleOutCooldown": 300,

   "ScaleInCooldown": 300

 }

}

When Harness deploys your ECS service, it will register the service as an application AWS Auto Scaling can scale and apply the scaling policy, scaling out (and in) using CloudWatch target tracking.

ECS Steady State Check Command

You can use the ECS Steady State Check command in an ECS Workflow to check for the steady state of a service you have deployed using a method other than the default ECS Service Setup or ECS Daemon Service Setup commands, such as a Shell Script command.

The ECS Steady State Check command may be added to the Deploy Containers section of a Workflow. The ECS Steady State Check command dialog looks like this:

In ECS Service Name, enter the name of the ECS service you are deploying.

In Timeout, enter how long Harness should wait for Steady State to be reached before failing the deployment. The default is 600000 ms (10 minutes).

Rollback Command

When you created a Workflow for an ECS service a Rollback command is automatically added to the Rollback Steps section of the Workflow (and its Phases). If Harness needs to rollback and restore the ECS setup to its previous working version, the first step is to rollback the ECS services.

You can enter the following information in the rollback command:

  • Desired Instance Count - Enter the desired number of ECS services to restore.
  • Fixed Instances - Enter a fixed number of ECS services to restore.
  • Max Instances - Enter the max number of ECS services to restore.
  • Resize Strategy - Select whether to resize new ECS services before downsizing old services, or downsize first.
  • Service SteadyState Timeout - Enter how long to wait for restores ECS services (in minutes) before timing out.

Deploy ECS Workflows

Once your ECS Workflow is complete, you can deploy it to your ECS cluster. For more information about deploying Workflows, see Deploy a Workflow.

Let’s look at the deployment of an ECS Workflow that deploys an ECS service using the Replica Strategy as part of a Canary deployment. Here is what the completed Phase 1 looks like in Harness.

The ECS Service Setup step displays the steps executed by the Harness Delegate installed in the same AWS VPC as the ECS cluster named example. Here’s the output with comments explaining the deployment:

INFO   2019-01-07 13:32:54    Begin execution of command: Setup ECS Service

# Cluster named example is selected

INFO   2019-01-07 13:32:54    Cluster Name: example

# Artifact source is identified

INFO   2019-01-07 13:32:54    Docker Image Name: library/nginx:stable-perl

# Container name is created

INFO   2019-01-07 13:32:54    Container Name: library_nginx_stable-perl

# Task definition is created using the Harness Service Container Specification

INFO   2019-01-07 13:32:54    Creating task definition ECS__Example__Default_Replica__stage__ecs with container image library/nginx:stable-perl

INFO   2019-01-07 13:32:54    Creating ECS service ECS__Example__Default_Replica__stage__ecs__4 in cluster example

INFO   2019-01-07 13:32:54    

INFO   2019-01-07 13:32:54    Cleaning versions with no tasks

# No Auto Scaling Configured for this deployment

INFO   2019-01-07 13:32:54    Checking for Auto-Scalar config for existing services

INFO   2019-01-07 13:32:54    No Auto-scalar config found for existing services

# Deployment Success

INFO   2019-01-07 13:32:54    Command execution finished with status SUCCESS

Next, the Upgrade Containers Step updates the desired count of 2 services:

INFO   2019-01-07 13:32:56    Begin execution of command: Resize ECS Service

# Resize instances to 50% of 2 (1 instance)

INFO   2019-01-07 13:32:56    Resize service [ECS__Example__Default_Replica__stage__ecs__4] in cluster [example] from 0 to 1 instances

INFO   2019-01-07 13:32:56    Waiting for service: ECS__Example__Default_Replica__stage__ecs__4 to reflect updated desired count: 1

# Resize Complete

INFO   2019-01-07 13:32:56    Current service desired count return from aws for Service: ECS__Example__Default_Replica__stage__ecs__4 is: 1

INFO   2019-01-07 13:32:56    Service update request successfully submitted.

INFO   2019-01-07 13:32:56    Waiting for pending tasks to finish. 0/1 running ...

INFO   2019-01-07 13:33:37    Waiting for service to be in steady state...

# Service reached Steady State

INFO   2019-01-07 13:33:37    Service has reached a steady state

# Pull dockerId

INFO   2019-01-07 13:33:37    Fetching container meta data from http://10.0.0.53:51678/v1/tasks

INFO   2019-01-07 13:33:37    Successfully fetched dockerId

INFO   2019-01-07 13:33:37    

INFO   2019-01-07 13:33:37    Container IDs:

INFO   2019-01-07 13:33:37      6452d4b0ef39 - 10.0.0.53 (new)

INFO   2019-01-07 13:33:37    

INFO   2019-01-07 13:33:37    Completed operation

INFO   2019-01-07 13:33:37    ----------

# Deployment Success

INFO   2019-01-07 13:33:48    Command execution finished with status SUCCESS

This example is for Phase 1 of a Canary deployment where 50% of 2 services are deployed. Once the Canary deployment is complete and both services are deployed, you can see the deployed services in the AWS ECS console:

ECS infrastructure Provisioners

You can add a Harness Infrastructure Provisioner for an ECS CloudFormation template to your Harness Application and use the Infrastructure Provisioner to define the ECS infrastructure on the fly.

After you set up an Infrastructure Provisioner in your Harness Application, you can add it as a Pre-Deployment Step in a Workflow (most commonly, Canary) or Service Infrastructure in an Environment. Once configured, your Workflow can deploy multiple services, each using the same Infrastructure Provisioner as a blueprint, and passing in Service-specific input variables to create Service-specific deployments using the single Infrastructure Provisioner. For more information about Harness Infrastructure Provisioners, see Add an Infra Provisioner.

To create an Harness Infrastructure Provisioner for ECS, do the following:

  1. In your Harness Application, click Infrastructure Provisioners. The Infrastructure Provisioners page appears.
  2. Click Add Infrastructure Provisioner, and select your provisioner. For ECS, you will likely use CloudFormation. The Add CloudFormation Provisioner dialog appears.
  3. In Name, enter a name for the provisioner.
  4. In Source Type, select Template Body and enter the JSON for the provisioner, or select Amazon S3 and enter the Template File Path for the template (S3 bucket and file URL).
  5. In Input Variables, click Add to specify variables that will need to be filled when this Infrastructure Provisioner is used, or click Populate Variables to have Harness pull variables from the Template file.
  6. Click SUBMIT. The new Infrastructure Provisioner page appears.
  7. Click Add Service Mapping. The Service Mapping dialog appears.
  8. In Service, select the Harness Service to use with this Infrastructure Provisioner.
  9. In Deployment Type, select Amazon EC2 Container Services (ECS).
  10. In Cloud Provider Type, select Amazon Web Services.
  11. Click NEXT.
  12. In ECS Cluster, enter the name of the ECS cluster to use, and click SUBMIT. The Service Mapping is added to the new Infrastructure Provisioner.

To add the Infrastructure Provisioner to an Environment, when you are adding a Service Infrastructure, simply choose Dynamically Provisioned and choose the Infrastructure Provisioner.

To add the Infrastructure Provisioner to a Workflow, in the Workflow Pre-deployment Steps, click Add Step, and select CloudFormation Create Stack. In the CloudFormation Create Stack dialog, in Provisioner, select the Infrastructure Provisioner you added.

Do it All in YAML

All of the Harness configuration steps in this guide can be performed using code instead of the Harness user interface. You can view or edit the YAML for any Harness configuration by clicking the </> button on any page.

When you click the button, the Harness code editor appears.

For example, here is the YAML for the Daemon Scheduler, Basic Workflow we set up in this guide.

harnessApiVersion: '1.0'
type: BASIC
envName: stage-ecs
failureStrategies:
- executionScope: WORKFLOW
failureTypes:
- APPLICATION_ERROR
repairActionCode: ROLLBACK_WORKFLOW
retryCount: 0
notificationRules:
- conditions:
- FAILED
executionScope: WORKFLOW
notificationGroupAsExpression: false
notificationGroups:
- Account Administrator
phases:
- type: ECS
computeProviderName: aws-ecs
daemonSet: false
infraMappingName: example -AWS_ECS--Amazon Web Services- aws-ecs- us-west-1
name: Phase 1
phaseSteps:
- type: CONTAINER_SETUP
name: Setup Container
steps:
- type: ECS_DAEMON_SERVICE_SETUP
name: ECS Daemon Service Setup
stepsInParallel: false
- type: VERIFY_SERVICE
name: Verify Service
stepsInParallel: false
- type: WRAP_UP
name: Wrap Up
stepsInParallel: false
provisionNodes: false
serviceName: Default_Daemon
statefulSet: false
rollbackPhases:
- type: ECS
computeProviderName: aws-ecs
daemonSet: false
infraMappingName: example -AWS_ECS--Amazon Web Services- aws-ecs- us-west-1
name: Rollback Phase 1
phaseNameForRollback: Phase 1
phaseSteps:
- type: CONTAINER_SETUP
name: Setup Container
phaseStepNameForRollback: Setup Container
statusForRollback: SUCCESS
steps:
- type: ECS_SERVICE_SETUP_ROLLBACK
name: Rollback Containers
stepsInParallel: false
- type: VERIFY_SERVICE
name: Verify Service
phaseStepNameForRollback: Deploy Containers
statusForRollback: SUCCESS
stepsInParallel: false
- type: WRAP_UP
name: Wrap Up
stepsInParallel: false
provisionNodes: false
serviceName: Default_Daemon
statefulSet: false
templatized: false

For more information, see Configuration as Code and Harness GitOps.

Troubleshooting

For information on ECS troubleshooting, see Amazon ECS Troubleshooting from AWS.

The following errors might occur when setting up and deploying ECS in Harness.

Invalid CPU or Memory Value Specified

See the required settings in Invalid CPU or Memory Value Specified from AWS.

ClientException: Fargate requires that 'cpu' be defined at the task level

Enure you add the CPU and Memory settings in the Harness Service Container Specification section, for example:

"cpu" : "1",

"memory" : "512"

ClientException: The 'memory' setting for container is greater than for the task

In the Harness Service Container Specification JSON, there are two settings for memory. The memory setting for the container must not be greater than the memory setting for the task:

{

 "containerDefinitions" : [ {

   "name" : "${CONTAINER_NAME}",

   "image" : "${DOCKER_IMAGE_NAME}",

   "memory" : 512,

   ...

 } ],

 "executionRoleArn" : "${EXECUTION_ROLE}",

 ...

 "cpu" : "1",

 "memory" : "512",

 "networkMode" : "awsvpc"

}

Could Not Reach Http://<IP Address>:<Port>/V1/Tasks to Fetch Container Meta Data

The ECS container agent provides an API operation for gathering details about the container instance on which the agent is running and the associated tasks running on that instance. Harness uses the cURL command from within the container instance to query the Amazon ECS container agent on port 51678 and return container instance metadata or task information.

Ensure that the security group(s) used by AWS EC2 instances permits inbound traffic over TCP port 51678.

For more information, see Amazon ECS Container Agent Introspection from AWS.

Next Steps

Learn about adding Continuous Verification to your ECS deployment and running services. For more information, see Continuous Verification and 24x7 Service Guard.


How did we do?