AWS App Mesh Walkthrough

5. Shape traffic

Currently, the the app equally distributes traffic among blue, red, and white color teller virtual nodes through the default virtual router configuration, so if you run the curl command a few times, you might see something similar to this:

$ curl $colorapp/color
{"color":"red", "stats": {"blue":0.33,"red":0.36,"white":0.31}}

In the following section, we’ll walk through how to modify traffic according to rules we set.

Apply traffic rules

Open up examples/apps/colorapp/servicemesh/appmesh-colorapp.yaml in an editor. In the definition for ColorTellerRoute, you will see the spec for an HttpRoute (around line 123):

ColorTellerRoute:
Type: AWS::AppMesh::Route
DependsOn:
- ColorTellerVirtualRouter
- ColorTellerWhiteVirtualNode
- ColorTellerRedVirtualNode
- ColorTellerBlueVirtualNode
Properties:
MeshName: !Ref AppMeshMeshName
VirtualRouterName: colorteller-vr
RouteName: colorteller-route
Spec:
HttpRoute:
Action:
WeightedTargets:
- VirtualNode: colorteller-white-vn
Weight: 1
- VirtualNode: colorteller-blue-vn
Weight: 1
- VirtualNode: colorteller-red-vn
Weight: 1
Match:
Prefix: "/"

Modify the HttpRoute block of code to look like this:

HttpRoute:
Action:
WeightedTargets:
- VirtualNode: colorteller-black-vn
Weight: 1
Match:
Prefix: "/"

Apply the update:

./examples/apps/colorapp/servicemesh/appmesh-colorapp.sh
...
+ aws --profile default --region us-west-2 cloudformation deploy --stack-name DEMO-appmesh-colorapp
--capabilities CAPABILITY_IAM --template-file /ho me/ec2-user/projects/aws/aws-app-mesh-examples/examples/apps/colorapp/servicemesh/appmesh-colorapp.yaml --parameter-overrides EnvironmentName=DEMO Se
rvicesDomain=demo.local AppMeshMeshName=appmesh-mesh
...
Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - DEMO-appmesh-colorapp

Now, when you curl the app, you will see a responses like the following:

$ curl $colorapp/color
{"color":"black", "stats": {"black":0.19,"blue":0.28,"red":0.27,"white":0.26}}
...
# repeated calls will increase the stats for black since it's the only color response now
{"color":"black", "stats": {"black":0.21,"blue":0.28,"red":0.26,"white":0.25}}

The following query will clear the stats history:

$ curl $colorapp/color/clear
cleared
# now requery
$ curl $colorapp/color
{"color":"black", "stats": {"black":1}}

Since there are no other colors for the histogram, that’s all you will see no matter how many times you repeat the query.

Simulate A/B tests with a 50/50 split between red and blue:

Edit examples/apps/colorapp/servicemesh/appmesh-colorapp.yaml

WeightedTargets:
- VirtualNode: colorteller-red-vn
Weight: 1
- VirtualNode: colorteller-blue-vn
Weight: 1

Any integer proportion will work for the weights (as long as the sum doesn’t exceed 100), so you could have used 1 or 5 or 50 for each to reflect the 1:1 ratio that distributes traffic equally between the two colortellers. App Mesh will use the ratio to compute the actual percentage of traffic to distribute along each route.

You can see this in the App Mesh console when you inspect the route:

Figure 5. Route weighted targets.

In a similar manner, you can perform canary tests or automate rolling updates based on healthchecks or other criteria using weighted targets to have fine-grained control over how you shape traffic for your application.

To prepare for the next section, go ahead and update the HttpRoute to send all traffic only to the blue colorteller.

examples/apps/colorapp/servicemesh/appmesh-colorapp.yaml

WeightedTargets:
- VirtualNode: colorteller-blue-vn
Weight: 1

Then deploy the update and then clear the color history for fresh histograms:

$ ./examples/apps/colorapp/servicemesh/appmesh-colorapp.sh
...
$ curl $colorapp/color/clear
cleared

In the next section we’ll experiment with updating the route using the App Mesh console and analyze results visually with AWS X-Ray.

Monitor with AWS X-Ray

AWS X-Ray helps us to monitor and analyze distributed microservice applications through request tracing, providing an end-to-end view of requests traveling through the application so we can identify the root cause of errors and performance issues. We’ll use X-Ray to provide a visual map of how App Mesh is distributing traffic and inspect traffic latency through our routes.

When you open the AWS X-Ray console the view might appear busier than you expected due to traffic from automated healthchecks. We’ll create a filter to focus on the traffic we’re sending to the application frontend (color gateway) when we request a color on the /color route.

The Color App has already been instrumented for X-Ray support and has created a Segment called “Default” to provide X-Ray with request context as it flows through the gateway service. Click on the “Default” button (shown in the figure below) to create a group to filter the visual map:

Figure 6. Creating a group for the X-Ray service map.

Choose “Create group”, name the group “color”, and enter an expression that filters on requests to the /color route going through the colorgateway-vn node:

(service("appmesh-mesh/colorgateway-vn")) AND http.url ENDSWITH "/color"
Figure 7. Adding a group filter expression.

After creating the group, make sure to select it from the dropdown to apply it as the active filter. You should see something similar to the following:

Figure 8. Analyzing the X-Ray service map.

What the map reveals is that

  1. Our color request first flows through an Envoy proxy for ingress to the gateway service.
  2. Envoy passes the request to the gateway service, which makes a request to a colorteller.
  3. The gateway service makes a request to a colorteller service to fetch a color. Egress traffic also flows through the Envoy proxy, which has been configured by App Mesh to route 100% of traffic for the colorteller to colorteller-blue.
  4. Traffic flows through another Envoy proxy for ingress to the colorteller-blue service.
  5. Envoy passes the request to the colorteller-blue service.

Click on the colorgateway-vn node to display Service details:

Figure 9. Tracing the colorgateway virtual node.

We can see an overview on latency and that 100% of the requests are “OK”.

Click on the “Traces” button:

This provides us with a detailed view about how traffic flowed for the request.

Figure 9. Analyzing a request trace

If we log into the console for AWS App Mesh and drill down into “Virtual routers” for our mesh, we’ll see that currently the HTTP route is configured to send 100% of traffic to the colorteller-blue virtual node.

Figure 10. Routes in the App Mesh console.

Click the “Edit” button to modify the route configuration:

Figure 11. Editing a route.

Click the “Add target” button, choose “colorteller-red-vn”, and set the weight to 1.

Figure 12. Adding another virtual node to a route.

After saving the updated route configuration, you should see:

Figure 13. The updated route for splitting traffic across two virtual nodes.

Now when you fetch a color, you should start to see “red” responses. Over time, the histogram (stats) field will show the distribution approaching 50% for each:

$ curl $colorapp/color
{"color":"red", "stats": {"blue":0.75,"red":0.25}}

And if you refresh the X-Ray Service map, you should see something like this:

Figure 14. The updated service map with split traffic.

AWS X-Ray is a valuable tool for providing insight into your application request traffic. See the AWS X-Ray docs to learn more instrumenting your own microservice applications to analyze their performance and the effects of traffic shaping with App Mesh.