Automating AWS IoT Greengrass Setup With AWS CloudFormation

In this blog post, I will show you how you can use AWS CloudFormation to set up AWS IoT Greengrass quickly and in a repeatable manner. I will discuss the AWS IoT Greengrass resource types and how they are related to a Greengrass group deployment. I will also demonstrate a deployment that uses an AWS CloudFormation template to:

  • Launch an EC2 instance running the AWS IoT Greengrass Core software.
  • Deploy a Lambda function at the edge.
  • Set up subscriptions to communicate between AWS IoT Core and AWS IoT Greengrass resources.

Use cases for AWS CloudFormation with AWS IoT Greengrass

AWS CloudFormation makes it possible for you to describe and provision all of the infrastructure resources in your cloud environment. Although Greengrass or edge devices are not cloud resources, AWS CloudFormation still provides support for a number of use cases.

  • Production deployment of complete workloads that include cloud and physical device components. For example, if you provide a smart building management solution as a software-as-a-service (SaaS) offering, one of the components might be a Greengrass core provisioned and deployed at the customer’s location. With AWS CloudFormation support, the Greengrass group can be created as part of the AWS CloudFormation deployment. This type of use case is suitable for deployments where a discrete number of Greengrass cores are part of the solution.
  • Development and test tasks. When you use AWS CloudFormation as part of a CI/CD process, you can quickly create, test against, and tear down Greengrass resources and groups. For example, you can use AWS CloudFormation to automate testing new versions of AWS Lambda functions or AWS IoT Greengrass connectors. The tests can run with the Greengrass group, and when they are complete, the entire stack can be deleted.

AWS IoT Greengrass architecture overview

The following diagram shows the collection of settings and components known as a Greengrass group.

A Greengrass group definition is a collection of information about the Greengrass group, which in turn maps to a Greengrass deployment, AWS IoT Core, and other AWS resources, such as AWS Lambda functions. AWS CloudFormation can be used to create a Greengrass core, Lambda functions, and subscriptions resources. Although connectors, devices, and resources are not used in the AWS CloudFormation template for this post, they can be defined and created by AWS CloudFormation, too.

For each resource type, there is a definition and a definition version. Based on the resource type, there are certain conditions for associations, such as one-to-one or one-to-many. For more information, see the AWS IoT Greengrass Resource Types Reference in the AWS CloudFormation User Guide.

An AWS::Greengrass::CoreDefinitionVersion resource referenced from an AWS::Greengrass::GroupVersion resource is required to create a Greengrass group. Each GroupVersion can refer to different resource version ARNs which, in turn, can have one or more resources.

The following diagram shows how the GroupVersion and GroupVersionDefinition are defined. The CoreDefinitionVersion is one-to-one while the definitions can be one-to-many.

This shows how the associations for AWS IoT Greengrass look in AWS CloudFormation:

Type: "AWS::Greengrass::Group"
Properties:
  # Required properties
  RoleArn: String
  Name: MyNewGroup
  # "InitialVersion" is Optional property, not used in this example
  # If used, the AWS::Greengrass::GroupVersion would not be required
  InitialVersion:
    LoggerDefinitionVersionArn: String
    DeviceDefinitionVersionArn: String
    FunctionDefinitionVersionArn: String
    CoreDefinitionVersionArn: String
    ResourceDefinitionVersionArn: String
    ConnectorDefinitionVersionArn: String
    SubscriptionDefinitionVersionArn: String 

Type: "AWS::Greengrass::GroupVersion"
Properties:
  # Required properties
  GroupId: String
  CoreDefinitionVersionArn: String
  # Optional properties
  LoggerDefinitionVersionArn: String
  DeviceDefinitionVersionArn: String
  FunctionDefinitionVersionArn: String
  ResourceDefinitionVersionArn: String
  ConnectorDefinitionVersionArn: String
  SubscriptionDefinitionVersionArn: String

When deployed, the group definition, Lambda functions, connectors, resources, and subscription table are copied to a Greengrass core device. In the example used in this post, the Greengrass deployment process is manual so you can see what happens from an AWS CloudFormation stack perspective.

Prerequisites

All of the steps for creating and deploying a Greengrass group and core are included in the AWS CloudFormation template.

AWS CloudFormation must be able to work with other AWS services, including AWS Identity and Access Management (IAM), so your AWS account must have administrative access.

To review how the AWS IoT Greengrass software runs on the instance, I will use SSH to connect to the instance. If you do not have an SSH key pair or don’t recognize any of the key names when you enter parameters during the launch of the template, you must create a key pair in your AWS Region. For instructions, see Amazon EC2 Key Pairs in the Amazon EC2 User Guide for Linux Instances.

The Oregon (us-west-2) Region is used in the deployment example, but it can run in all AWS Regions where AWS IoT Greengrass is supported.

AWS CloudFormation template details

There are two main sections defined by comment headers in the AWS CloudFormation template.

In the GREENGRASS RESOURCES SECTION:

  • A group needs, at minimum, a Core definition version. In the template, the Function and Subscription definition versions are also used. Other definitions are included, but commented out.
  • A Core definition version requires a ThingArn and certificateId (associated with a policy and the thing). In the GreengrassCoreDefinitionVersion resource, the ThingArn and CertificateId are created from the execution of the custom resource IoTThing that creates the thing, certificate, and policy.
  • The general Definition type of resource requires either a DefinitionVersion or all of the values that make up an InitialVersion of the resource. For example, the GreengrassCoreDefinition references the GreengrassCoreDefinitionVersion resource, while the FunctionDefinition uses the InitialVersion to create the Lambda function resources.

The SUPPORTING RESOURCES SECTION of the template includes the resources used to create the environment in which the Greengrass software executes and to help with resource creation and cleanup.

Deploying the the AWS IoT Greengrass stack using AWS CloudFormation

Now you are ready to use AWS CloudFormation to create and deploy the AWS IoT Greengrass stack. Based on the input parameters, the template will perform:

  1. Cloud resource setup
    1. Deploy an EC2 instance with a public IP address into a newly created VPC and:
      1. Download and install AWS IoT Greengrass Core 1.8.0.
      2. Create all AWS IoT Core and AWS IoT Greengrass config.json settings and certificate files.
      3. Reboot the instance, which starts the Greengrass core in a not deployed state.
    2. Create a Lambda function that will be deployed to the Greengrass group.
  2. AWS IoT Greengrass setup
    1. Create the subscription resources for publish/subscribe to AWS IoT Core.
    2. Configure the Lambda function to be long-lived (that is, to run continuously).
    3. Create the Greengrass group.
    4. Create a Greengrass core.

This example shows a stack named gg-test in the AWS CloudFormation console:

The following parameters are required:

  • Stack name: This is the AWS CloudFormation stack name. Other resources might include this stack name as part of their names.
  • CoreName: This is the Greengrass core name. The CoreName, with _Core appended (for example, gg_cfn_Core), will appear as the AWS IoT thing name. The AWS IoT Thing name will also be created during execution of the AWS CloudFormation stack.
  • SecurityAccessCIDR: This is the IP address or addresses that can SSH into the EC2 instance. The default is to allow access from any public IP address.
  • myKeyPair: This is the SSH key pair used to to authenticate the Ubuntu user who connects to the instance. If there is no drop-down list in this field, you need to create a key pair.

To complete the stack creation process, choose Next, and then choose the Create Stack button. This will create all of the resources. It should take less than five minutes.

Note: The resources in this AWS CloudFormation stack might result in charges to your AWS account. To avoid ongoing charges, be sure to delete the stack and any Amazon CloudWatch log files when you are done.

From the Output tab on the AWS CloudFormation stack, copy the value of EC2IPAddress. This is the publicly accessible IP address of the EC2 instance. You connect to that instance using SSH later on.

Troubleshooting errors

If you encounter errors when you deploy the stack, review the Status reason column in the Events section.

Here are some common errors and their resolution:

  • GroupDeploymentReset: This error usually results when an IAM service role is not associated with AWS IoT Greengrass in the AWS Region where the stack is being deployed. Check the CloudWatch stream logs for errors. For information about associating an IAM service role, see Greengrass Service Role in the AWS IoT Greengrass Developer Guide.
  • Template format error: Unrecognized resource types. This error is displayed when the template is launched and indicates that AWS IoT Greengrass is not available in the AWS Region. Be sure to use a region where AWS IoT Greengrass is supported. For information, see AWS Regions and Endpoints in the AWS General Reference.
  • The maximum number of VPCs has been reached. This error occurs when no additional VPCs can be created in the AWS region. You can either request a service limit increase in the same AWS region, or you can deploy the stack in a different AWS region where AWS IoT Greengrass is supported.

Deployment results

The AWS resources created during deployment of the AWS CloudFormation stack will look similar to this:

  1. The IoTThing custom resource is executed to create and return a thing, certificate, and policy.
  2. The returned values of certificate and private key are passed to the instance creation, where the UserData command performs the following:
    • Installs dependencies and the AWS IoT Greengrass Core software.
    • Saves the certificate and private key.
    • Generates a config.json file.
    • Configures AWS IoT Greengrass to start at boot time.
    • Restarts the instance.
  3. A second custom resource is used to reset the deployment status before the resources are deleted. This is to ensure the deletion process does not leave artifacts behind.

A VPC with an instance is created along with all of the resources required to create a Greengrass group definition.

Verifying and testing the stack deployment

In the AWS IoT console, choose Greengrass, choose Groups, and then verify that the Greengrass group and resources have been created:

You should see a group with the name you provided when you launched the AWS CloudFormation template. To verify the subscription and Lambda function definitions were created, choose the Greengrass group, and then choose each entry. It should look similar to this:

and:

You’ll see the subscriptions and Lambda function definitions were created by the AWS CloudFormation stack. You’ll also see that the Greengrass group is not yet deployed. You can use the console or the AWS IoT Greengrass APIs to perform a deployment. In this post, I use the console.

To see what a new Greengrass core looks like, and the changes performed by a deployment, use the IP address and key pair to connect using SSH to the Ubuntu instance. (The user name is ubuntu.) Then check for the greengrass process and connection to AWS IoT Core on TCP port 8883:

laptop$ ssh -i EC2_KEY_FILE ubuntu@EC2IPAddress
ubuntu@ip-172-31-0-231:~$ ps ax | grep greengrass
 1186 ?        Sl     0:00 /greengrass/ggc/packages/1.8.0/bin/daemon -core-dir /greengrass/ggc/packages/1.8.0 -greengrassdPid 1143
ubuntu@ip-172-31-0-231:~$ netstat -na | grep 8883
tcp        0      0 172.31.0.231:35878      34.213.234.177:8883     ESTABLISHED

Run ps ax | grep greengrass to verify that the Greengrass daemon is running. Next, run netstat -na | grep 8883, which will show the single persistent Greengrass connection to AWS IoT Core waiting for the deployment action.

In the AWS IoT Greengrass console, from Actions, choose Deploy, and then choose Automatic Detection:

After the deployment is complete, a “Successfully completed” message is displayed in the console, and the Greengrass core is now operating with our deployed resources. You can verify by running the ps -ef command. This command will also show the process owner and additional processes that are now running:

ubuntu@ip-172-31-0-231:~$ ps -ef | grep "greengrass|lambda"
root      1186     1  0 13:49 ?        00:00:02 /greengrass/ggc/packages/1.8.0/bin/daemon -core-dir=/greengrass/ggc/packages/1.8.0 -port=8000 -connectionManager=true -cloudSpooler=true -shadow=true -shadowSync=true -tes=true -deviceCertificateManager=true -secretManager=true
ggc_user  1825  1186  0 14:23 ?        00:00:00 /lambda/greengrassSystemComponents -runAs=connectionManager
ggc_user  1841  1186  0 14:23 ?        00:00:00 /lambda/greengrassSystemComponents -runAs=shadowSync
ggc_user  1865  1186  0 14:23 ?        00:00:00 /lambda/ipdetector
ggc_user  1895  1186  0 14:23 ?        00:00:00 /lambda/greengrassSystemComponents -runAs=cloudSpooler
ggc_user  1911  1186  0 14:23 ?        00:00:00 /lambda/greengrassSystemComponents -runAs=shadow
ggc_user  1925  1186  0 14:23 ?        00:00:00 /lambda/greengrassSystemComponents -runAs=tes
ggc_user  1941  1186  0 14:23 ?        00:00:00 /lambda/greengrassSystemComponents -runAs=deviceCertificateManager
ggc_user  1958  1186  0 14:23 ?        00:00:00 /lambda/greengrassSystemComponents -runAs=secretManager
daemon    1983  1186  0 14:23 ?        00:00:00 python2.7 -u /runtime/python2.7/lambda_runtime.py --handler=index.function_handler

In the last line of the output, the Lambda function is a long-running process that performs two functions:

  1. Every five seconds, the Lambda function publishes an incrementing value on core_name/telem topic (gg_cfn/telem is the default). This demonstrates the setting of a long-running function using the Pinned: 'true' parameter.
  2. The Lambda function also subscribes to the gg_cfn/in topic, and any messages received are republished intact (echoed) to the gg_cfn/out topic. This shows how multiple subscriptions can be assigned to a SubscriptionDefinitionVersion. You can test by subscribing to gg_cfn/out and publishing a message to gg_cfn/in and see the echoed message.

At this point, you have tested the functionality deployed in this stack, namely that the defined resources were created and operate on the subscriptions. After you review the deployment of the AWS CloudFormation stack, you can delete the resources by navigating to the stack and deleting it. This will delete the Greengrass group, its resources, the EC2 instance, and all VPC resources.

Summary

In this post, I covered the architecture of a Greengrass group and how the Greengrass resources are defined as AWS CloudFormation resources. I used an AWS CloudFormation template that automated the deployment of the Greengrass resources and supporting components. Finally, I deployed the Greengrass group definition to our Greengrass core and tested the functionality through MQTT messages published to and from different topics.

I hope this blog post was helpful in understanding how AWS CloudFormation resources can be used to automate the setup and launch of AWS IoT Greengrass in your workloads.