All Articles

Advanced Serverless CICD - Part 2 - CircleCI

Photo by https://unsplash.com/@er1end
Photo by https://unsplash.com/@er1end

In my last post I presented a step-by-step guide on setting up an advanced Serverless CICD using AWS CodeBuild.

To recap from part 1, using CodeBuild has some advantages:

  • Can be configured using CloudFormation
  • Easier to integrate with other AWS services
  • Easier permissions management

But it lacks important features:

  • Smart Email notifications
  • Slack integration
  • Knowledge base (compared to other services)

As mentioned in part 1 any missing feature in CodeBuild can probably be implemented using a Lambda function, but implementing features that are available out of the box when working with CircleCI is redundant

Introduction

In this post I’ll guide you on how to achieve the same CICD process as in part 1 using CircleCI instead of AWS CodeBuild.

If you don’t have a CircleCI account you should sign up here.

There are a few notable differences between CircleCI and CodeBuild that will affect our setup:

  • We can’t use CloudFormation to describe our deployment setup
  • There are no regions in CircleCI so we can’t have a CICD setup per region per stage as in part 1
  • We will need to give CircleCI access to our AWS account
  • We don’t need to setup email notifications or GitHub access tokens as CircleCI handles it for us

Let’s Get Started

As in part 1 we’ll be using the following repository as a baseline for setting up the deployment process, so you should fork and clone it.

In order to create a reproducible CICD setup, I wrote a small CLI that utilizes the CircleCI API to follow a GitHub project, connect it to AWS, set up all the required environment variables and relevant project settings.

In order to use the CLI you’ll need to create a Personal API Token.

Since I’ve already described our requirements, tool stack and prerequisites I won’t repeat them here, but refer back to part 1 for all the details.

CICD Setup

Run the following command from the cicd directory to set up the process (replace with the correct values):

yarn setup:circleci --token ****************** --stagingAdminEmail admin@email.com --prodAdminEmail admin@email.com

The setup command will:

  1. Follow a GitHub project (it takes the project information based on git metadata)
  2. Create an IAM user and API key with admin access and set the relevant environment variables to access your AWS account
  3. Set the relevant environment variables required for our Serverless application
  4. Update CircleCI to only build on pull requests, not to build on forks and not to pass secrets to forks (I found theses settings better than the defaults)

Here is a snippet from the setup command:

cicd/scripts/circleci.js
cicd/scripts/circleci.js

You can see the full command in cicd/package.json:

cicd/package.json
cicd/package.json

Note the remove:circleci and trigger:circleci commands to unfollow a project and to trigger a manual build.

After running the setup command Circle you can visit you CircleCI dashboard to verify the project has been added.

Build Setup

CircleCI config file is under .circleci/config.yml .

The file is very similar to our buildspec.yml from part 1 with the notable difference of the need to handle different deploy stages in the file:

.circleci/config.yml
.circleci/config.yml

.circleci/config.yml
.circleci/config.yml

.circleci/config.yml
.circleci/config.yml

.circleci/config.yml
.circleci/config.yml

Note that I’m using yaml aliases to avoid code repetition

The build filters are defined as follows:

.circleci/config.yml
.circleci/config.yml

The test job will run on pull requests, a staging deploy will run on merge to master and a prod deploy will run on tag push (as in part 1).

Build phases, environment variables, build scripts and end to end tests are similar to the ones described in part 1.

Summary

Using CircleCI we’ve set up an advanced CICD process for our Serverless application, in a very similar way as we did using CodeBuild in part 1.

While losing the option to use CloudFormation to describe our CICD process in a reproducible way, the CircleCI API is a decent replacement, and by using the AWS SDK we can easily set the proper permissions for our CICD jobs.

As you can see, for the basic functionality of having a container to run your build scripts the differences between CodeBuild and CircleCI are negligible.

But the usefulness of a CICD process comes from the ability to monitor errors (e.g. relevant notifications), integrate with other development tools (e.g. Slack), the ability to make changes and resolve issues quickly.
These are all missing from CodeBuild (or require development time and effort) and are included in CircleCI.

I originally wrote this post as a guest post for Lumigo. The original version is available here