Using CodeBuild in Spinnaker for continuous integration

Continuous integration is a DevOps software development practice in which developers regularly merge their code changes into a central repository, then run automated builds and tests. Continuous integration (CI) most often refers to the build or integration stage of the software release process and entails both an automation component (such as a CI or build service) and a cultural component (such as learning to integrate frequently). This example configures AWS CodeBuild to provide CI capabilities in Spinnaker.

Overview of Concepts

AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy. Because CodeBuild is a managed service, you don’t need to provision any resources such as build servers. As a start a build process, CodeBuild automatically allocates resources for you.
Spinnaker is an open-source tool built by Netflix for continuous integration/continuous deployment (CI/CD). The two core features of Spinnaker are application management and application deployment. Application management manages the state of your application and application deployment is used to build continuous delivery workflows.
Spinnaker defines CI/CD workflows as pipelines. A pipeline consists of one or more stages. A stage defines part of the workflow. A pipeline is usually part of an application in Spinnaker. Multiple applications can be logically grouped together as a project..

Prerequisites

In order to configure CodeBuild in Spinnaker, you need the following:

  • An AWS account
  • A CodeBuild project in your AWS Account.
  • Spinnaker installed and running on an Amazon EC2 Instance in AWS.

Using CodeBuild in Spinnaker

This section walks you through the process of creating a new project, application, and pipeline and adding CodeBuild as one of the stages. Before you start using CodeBuild in Spinnaker, you need to enable support for CodeBuild.

Enable AWS in Spinnaker

Give the Amazon EC2 instance additional IAM permissions to CodeBuild projects via the EC2 instance profile.
We will give the EC2 Instance additional permissions via EC2 Instance Profile to AWS CodeBuild projects.
When you install Spinnaker in AWS, you configure two roles:

  1. Spinnaker managing role: Spinnaker authenticates itself as this role. This role is assigned to the Amazon EC2 instance on which Spinnaker is running.
  2. Spinnaker managed role: Instead of giving permissions directly to Spinnaker, define policies in a managed role and enable trust between the roles so that the managing role can assume the managed role and perform the necessary AWS SDK API calls. The managed role requires additional permissions so that it can call CodeBuild via the AWS SDK. This is done by adding the following inline policy to the managed role in IAM:
    {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Action": [
              "codebuild:StopBuild",
              "codebuild:ListProjects",
              "codebuild:StartBuild",
              "codebuild:BatchGetBuilds"
            ],
            "Resource": "*"
          }
        ]
      }

Use SSH to connect to the Amazon EC2 instance on which you have installed Spinnaker.
Validate whether you can assume the role by running the following command:

aws sts get-caller-identity

The output should match the following:

{
    "Account": "111222333444",
    "UserId": "AAAAABBBBBCCCCCDDDDDD:i-01aa01aa01aa091aa0",
    "Arn": "arn:aws:sts::111222333444:assumed-role/Spinnaker-Managing-Role/i-01aa01aa01aa091aa0"
}

The above output indicates that the Amazon EC2 instance can assume the Spinnaker managing role. This assumed role contains the actual permissions to interact with CodeBuild.
List the CodeBuild project by running the following command:

aws codebuild list-projects --region us-east-1 --output table

The output should list all of your CodeBuild Projects. An example output will look like the following:

--------------------------------
|         ListProjects         |
+------------------------------+
||          projects          ||
|+----------------------------+|
||  MyFirstProject            ||
||  EKSBuildProject           ||
||  ServelessBuildProject     ||
||  ................          ||
|+----------------------------+|

If you are not able to list the projects or get any kind of authentication, check the trust between the managing and managed role.

Spinnaker uses Halyard to configure, install, and update itself. To configure AWS as one of the providers in Spinnaker, use the Halyard Command Line Interface (CLI). Run the following commands in the instance on which you have installed Halyard.

  1. Start by defining your AWS account credentials in the terminal:
    export AWS_ACCOUNT=my-aws-account
    export AWS_ACCOUNT_ID=[YOUR_AWS_ACCOUNT_ID]
    export AWS_ROLE_NAME=role/Spinnaker-Managed-Role
    
  2. Add AWS as a cloud provider using the Halyard CLI:
    hal config provider aws account add ${AWS_ACCOUNT} 
      --account-id ${AWS_ACCOUNT_ID} 
      --assume-role ${AWS_ROLE_NAME} 
      --regions us-east-1
    
  3. Enable AWS as a cloud provider:
    hal config provider aws enable
  4. Add CodeBuild as a cloud provider:
    hal config ci codebuild account add ${AWS_ACCOUNT} 
    --account-id ${AWS_ACCOUNT_ID} 
    --assume-role ${AWS_ROLE_NAME} 
    --regions us-east-1
  5. Enable CodeBuild in Spinnaker:
    hal config ci codebuild enable
  6. Apply the new configuration and re-deploy Spinnaker:
    hal deploy apply

If you don’t have a CodeBuild project to use as a stage in Spinnaker, you can follow these instructions to create a new CodeBuild project. It must have a source provider local to the CodeBuild project: it should not be using source from a previous stage in AWS CodePipeline.

Creating an application in Spinnaker

Start by logging in to your Spinnaker instance and then we will creating a new application in Spinnaker.

  1. From the top navigation bar choose Applications, then Create Application. Enter the name of the application and owner email and select aws from the list of the Cloud Providers, as shown in the following screenshot:Creating a new application in Spinnaker
  2. Once you have created a new application, Spinnaker takes you to the Infrastructure section of the application. From this screen, choose Pipelines, as shown in the following screenshot:Select Pipelines after you create the application
  3. Choose Configure a new pipeline to create a new pipeline and give it a name (such as My First Pipeline), then choose Create, as shown in the following screenshot:Select “Configure a new pipeline”
  4. When prompted, enter a value as the Pipeline Name, as shown in the following screenshot: Enter the name of the Pipeline
  5. Once the pipeline is created, Spinnaker takes you to the newly created pipeline. From this new screen, choose Add Stage, as shown in the following screenshot:Adding a new stage to the Pipeline.
  6. Select AWS CodeBuild and assign this stage a name by entering a value in the Stage Name field.Select AWS CodeBuild from the drop down of Type.
  7. Configure additional details:Basic Settings:
    • Account: Select the CodeBuild CI account that you configured.
    • Project Name: Select the CodeBuild Project that you want Spinnaker to trigger when this stage is executed.

    Source Configuration:

    • Source: (Optional) Select the source of the build to override the source artifact already defined in your CodeBuild project.
    • Source Version: (Optional) If a source version for the build is not specified, the artifact version is used. If the artifact doesn’t have a version, the latest version is used. See the CodeBuild reference for more information.
    • Buildspec: (Optional) If an inline buildspec definition is not specified, the buildspec configured in the CodeBuild project is used.
    • Secondary Sources: (Optional) Selecting the secondary sources of the build allows you to override the secondary source artifact already defined in your CodeBuild project. If not specified, secondary sources configured in CodeBuild project are used.

    Environment Configuration:

    • Image: (Optional) Select the image in which the build runs if you want to override the image defined in the CodeBuild project. If not specified, the image configured in the CodeBuild project is used.
  8. Choose Save Change to save this stage and then choose PIPELINES to go to the pipelines. You can see the pipeline you just created, as shown in the following screenshot:After pipeline has been saved.

Testing the pipeline

Congratulations! You have successfully integrated CodeBuild as one of the stages in Spinnaker. Let’s test this pipeline.

  1. On the same page on which you can see the list of the pipelines, choose Start Manual Execution and select the newly created pipeline as shown in the following screenshot.Prompt to run a pipeline.
  2. Once you confirm, Spinnaker starts executing your pipeline. You can check the progress of the pipeline by selecting the pipeline, then choosing Execution Details, as shown in the following screenshot.Pipeline running in progress.
  3. Once the pipeline has finished executing, you can see that the status of the task is SUCCEEDED, as shown in the following screenshot:After Pipeline has finished.You can click on the Build Link and CloudWatch Logs from the above screen

Congratulations! You have now successfully integrated (and executed) CodeBuild in Spinnaker.

Further Reading

Cleanup

If you created a new CodeBuild project navigate to the CodeBuild section of AWS Console and delete the CodeBuild project that you created.

Conclusion

In the above post, we went through the concepts of Spinnaker and walked you through the process of using CodeBuild as a stage in Spinnaker Pipeline.

Integration is just one part of a well-defined CI/CD pipeline. In addition to CodeBuild, you can also use Spinnaker to deploy to AWS EKS.

Happy building!